Photogrammetry: 5 Best Software for Scanning Objects

Contents:

• Photogrammetry: Basics and Applications
• Innovative Photogrammetry-Based Games: Examples and Achievements
• The Best Photogrammetry Software: A Review of Current Programs
• Affordable Photogrammetry Solutions on Mobile Devices
• How Polycam Works: A Step-by-Step Guide
• Processing a 3D Scan After Photogrammetry in Blender
• Alternative Programs for Processing 3D Scans
• Advantages and Disadvantages of Photogrammetry in 3D Graphics

Photogrammetry: Basics and Applications

Photogrammetry is a 3D scanning method that allows you to obtain detailed data about the sizes and shapes of real-world objects based on photographs. The process involves photographing an object from different angles, which makes it possible to create multiple images. These images are then processed using specialized software that analyzes them, forming a point cloud and creating a 3D model that takes into account the terrain.
While photogrammetry is a powerful tool, its use for scanning people is more challenging. A person is a living being, and even when stationary, they can breathe or blink, which complicates the capture process. To achieve high accuracy, multiple cameras are used. For example, a report on creating facial animation for the game Marvel’s Guardians of the Galaxy details the technologies used to capture facial expressions and their subsequent animation.
Despite its modernity, photogrammetry is not a new method. It began to develop almost simultaneously with the invention of photography and has long been used in surveying and cartography. With the advancement of technology and increased computing power, its applications have become more diverse, including visual effects creation and video game development.

Innovative Games Based on Photogrammetry: Examples and Achievements

The release of The Vanishing of Ethan Carter in 2014 was a landmark moment in the video game industry, as the project used photogrammetry to create its environment. This solution provided amazing realism and atmosphere, which, combined with an engaging storyline and well-thought-out puzzles, led to numerous positive reviews and high marks from critics. The game has become a true role model among indie developers.

Since the release of The Vanishing of Ethan Carter, photogrammetry has gained popularity among both indie studios and big-budget projects. According to a study by the Digital Games Research Association (DIGRA), photogrammetry was used in the creation of at least 26 games from 2014 to 2019, among which The Beast Inside and Serious Sam 4 stand out.
When creating maps for Battlefield V, DICE studio used photogrammetry, capturing real natural landscapes and architectural objects. The process included not only traditional photography, but also the use of drones to obtain more detailed images.

Difficulties with weather conditions during outdoor filming led to Towards the use of photometric stereo. This allowed the developers to illuminate objects from different angles, which significantly improved the quality of normal and albedo maps, thereby increasing the realism of in-game surfaces. To create large-scale 3D objects in Forza Motorsport 7, an integrated approach combining photogrammetry and laser scanning was used. During the development of the Dubai track, the Turn 10 Studios team faced a lack of photographs and reference images, which made it difficult to reconstruct the landscape’s dimensions. To solve this problem, the developers used laser scanning using a Leica system and LiDAR technology. This allowed them to create detailed previz images, which were then used to recreate realistic in-game assets. As a result, LiDAR ensured the accuracy of scale and relief, and photogrammetry completed the finalization process.

The Best Photogrammetry Software: A Review of Current Programs

Photogrammetry is a powerful tool that allows you to create 3D models of objects from photographs. The process of working with photogrammetry software usually includes several key stages:

• analysis of uploaded photographs;
• generation of a point cloud;
• generation of a 3D model;
• primary processing and export of the finished model.

Quality The final 3D model's quality depends on many factors, including resolution and the number of photographs taken from different angles. While the process is similar, each program offers unique features, such as mesh optimization, polygon reduction, and support for various export formats.

• Meshroom

Developer: AliceVision
License: Free
Meshroom is a great choice for those wanting to get started with photogrammetry. It's free and works well for scanning static objects. There are plenty of video tutorials available to help you get up to speed. But it's worth noting that it was initially optimized for Nvidia graphics cards, but this can be fixed by installing the MeshroomCL extension to support other GPUs.

• Metashape

Developer: Agisoft
LicensePrice: from $179 to $3,499; a 30-day trial period is available.
Metashape offers extensive capabilities for creating 3D models with photorealistic textures and allows you to process photos from various cameras. It also supports the creation of 360° panoramic images. But users note the complexity of the interface and long data processing times.

• 3DF Zephyr

Developer: 3Dflow
License cost: from 0 to 4200 euros.
3DF Zephyr offers a free version that allows you to upload up to 50 photos in a single session, which is ideal for creating simple 3D objects. For more complex tasks, you can consider paid versions that do not limit the number of photos you can upload. It is important to note that the program is optimized for use with Nvidia graphics cards.

• RealityCapture

Developer: Capturing Reality, Epic Games
License cost: from $10 to $3,750.
RealityCapture differs from other programs in that the user pays for the number of shots and their resolution. This allows for more flexible cost management. Previewing and optimizing photogrammetry results are free, and you only pay for exporting the finished model. An Nvidia graphics card is also required for optimal performance.
RealityScan, a free iOS app released in 2022, allows users to take photos of objects from different angles and generate 3D models for integration into Unreal Engine. While easy to use, the app is still in beta, and users have reported issues with data analysis.
Choosing the right program depends on your needs and skill level. These tools open up new horizons in the creation of 3D models and can significantly simplify the workflow for designers and developers.

Affordable solutions for photogrammetry on mobile devices

The popularity of mobile photogrammetry continues to grow every year, thanks to the improvement of smartphone cameras and the introduction of advanced technologies. For example, with the release of the iPhone 12 Pro and iPad Pro models, LiDAR technology has entered the market, significantly improving the quality of 3D scanning. Today, there are many both free and paid applications for Android and iOS that make it easy to create high-quality 3D models of objects for various projects.
One ​​of the most versatile and popular applications in this area is Polycam. It offers users free photo processing and integration with the renowned Sketchfab portal, allowing them to share their creations with a wider audience. Polycam's LiDAR feature allows for the capture of large spaces, which has led to the emergence of numerous 3D models of user-created interiors online, turning them into something of an internet meme.

How Polycam Works: A Step-by-Step Guide

To get started, you'll need to download the app and create an account. Polycam currently offers three 3D scanning modes that allow users to easily capture objects and create digital copies of them.

• LiDAR (for iPhone/iPad devices) — designed to scan large spaces with high detail;
• Room (also for iPhone/iPad) — captures the layout of rooms, including corners, windows, and doorways while recording video;
• Photo (available on iOS and Android) — a photogrammetry mode known as photo capture, where the app automatically takes pictures of an object from different angles.

Before you start working in Photo mode, it's important to prepare. The basic requirements for photographing objects for photogrammetry are the same as those used for creating PBR textures.

• For outdoor subjects, it's best to use cloudy weather, and indoor subjects, use diffused light.
• Move your smartphone slowly and steadily around the subject to avoid distortion. Make sure the focus is set correctly: 1x or 2.5x depending on the size of the object.
• Avoid reflective and transparent surfaces, as the program will not be able to recognize them.
• Take more frames from different angles. The greater the overlap (at least 50%), the more accurate the processing will be.

After you finish shooting, click the "Done" button. Before sending to the cloud, select the processing quality (usually Raw). On average, the processing process takes 5 to 10 minutes.
The free version of Polycam has a limited number of photo captures, but you can earn additional ones by completing tasks. If you subscribe, the restrictions will be lifted.

When processing is complete, you will see a 3D object created from your photos.

Polycam has a built-in mini-editor that allows you to perform basic manipulations with the model: delete unnecessary fragments, add additional frames, change sizes and record video. AR mode is also available for viewing the model in a real environment.

After editing, save the object (the icon in the right corner) and go to the export settings. In the Export to section, select the software you want to integrate your model into. For example, we use Blender for further processing.

Only the .GLTF format is available in the free version. Then you can choose where to upload the object: to a messenger or to the cloud for subsequent download to your hard drive.

Processing a 3D scan after photogrammetry in Blender

Photogrammetry allows you to create detailed 3D models, but even the highest quality scans require further refinement. At the first stage, it is important to correct the topology, as there are always areas that the program could not process correctly, which leads to defects in the polygonal mesh.
Scanned objects often have too high a polygon count, which makes them unsuitable for use in games. Some programs (Metashape) allow you to set the polygon count at the scanning stage, but even in this case, manual refinement is necessary.
1. Launch Blender to import the scanned model. After centering, you need to analyze what actions to take. You may need to delete unnecessary parts, clean the polygonal mesh, work on textures.
2. The first step is removing unnecessary geometric elements. Blender offers many tools for this task. Depending on the source mesh, you can trim vertices manually or use Box Trim in Sculpting mode.
Let's try using the Boolean modifier for more precise removal. We create a cube and adjust its shape to the model to exclude unnecessary parts.

After selecting the model and adding the Logical modifier, we specify the cube as an object for clipping, using the Intersect option. The process can be repeated, changing the size and position of the cube to achieve the best result.

3. After modifying, check the shaders in the viewport. Artifacts in the form of normal distortions may appear on the surface. To correct them, use the Weighted Normal modifier.
Small elements can be corrected using Boolean or manual editing. Although this method is more time-consuming, it provides a more accurate correction.
4. In Sculpting mode, improve details, remove noise resulting from photogrammetry, and adjust the shape. After completing all manipulations, apply the Weighted Normal modifier again.

5. After finishing the mesh, some extra edges may remain. To clean up the mesh, use the edit mode: Mesh - Cleanup - Dissolve Degenerate Elements. Then apply Merge by Distance.
6. In object mode, duplicate the mesh (Shift + D) and rename both models for convenience. Hide the high-poly model to avoid confusion.
7. Now let's create a new model topology using automatic generation. It's important to achieve a balance between polygon count and mesh quality.

• Voxel Remeshing. Enable the Remesh parameter, select Voxel, and adjust the Voxel Size. This will significantly reduce the polygon count and create a new square mesh.

• QuadriFlow. This mode also allows you to create a new mesh while preserving the sharpness and boundaries of the model. We set the expected Number of Facets and look at the results.

The disadvantage of these methods is that they create a square grid, which can cause problems in some game engines.

• Using the Decimate modifier to reduce the polygon count while maintaining volume.
• An integrated approach. First, we apply Quad, and then add triangles using the Triangulation modifier.

8. Having finished with the mesh, we move on to an important stage - correcting the UV unwrapping.
In the UV Editing section, you can see the chaotic distribution of UV islands. To improve the structure, a new UV map will be required.

Simple models can be processed using Smart UV mapping, but more complex objects will require manual seam placement.
9. After creating the UV mapping, we proceed to baking the textures.
In Shading mode, we check the textures that may lose quality due to the new UV mapping. We delete the old material and create a new one for the low-poly mesh, adding textures for diffusion and normals.

Set the renderer to Cycles, specify GPU computations and minimum samples for baking.
Transfer normals to the low-poly model using the Multi-Size and Shrink modifiers. Make sure the details are transferred correctly, and apply Smooth Shading.

10. Switch to baking mode, select the desired parameters, and bake the normals and diffuse maps. It is recommended to save the resulting textures for later use.

11. After baking, enable the maps in the shader and adjust their parameters.
12. Switch to Texture Paint mode to correct graphical artifacts. We use the Clone tool and add the necessary details with a brush.

13. Comparing the final model with the original 3D scanned one. The resulting low-poly model can be used in various projects, such as visualizations and animations, subject to copyright.

Alternative Programs for Processing 3D Scans

Blender is a great tool, but it is not the only one. There are many alternative programs that can help at various stages of processing 3D scans.

• Mesh Cleaning. Various 3D modeling programs can be used for the initial processing of 3D models, including Autodesk Meshmixer. Although this tool's development was discontinued in favor of Fusion 360, its functionality and free downloadability make it still popular.
• Relief Smoothing. ZBrush, Autodesk Mudbox, and 3DCoat are excellent tools for removing unwanted details and noise. These programs offer a Sculpting mode that allows you to fine-tune the surface of a model.
• Retopology. Instant Meshes is a popular tool that can significantly simplify the retopology process. Also, many 3D modeling programs have built-in functions for this task.
• Baking normals. Substance Designer and Marmoset Toolbag, which provide high-quality baking, are suitable for this task.
• Texturing. Substance Painter and 3DCoat are excellent solutions for working with textures in real time, allowing you to create detailed textures with ease.

Advantages and Disadvantages of Photogrammetry in 3D Graphics

Photogrammetry is a revolutionary technology that not only adds a level of realism to 3D graphics but also brings stylized objects (toys, handicrafts) to life. These elements can be easily integrated into games and visualizations, giving them a unique touch.

• Accessibility of technology. Thanks to the improvement in the quality of cameras in modern smartphones, photogrammetry has become accessible to a wider audience.
• Accurate replication of dimensions. This technology allows for precise measurement of objects, which is especially important for architectural elements. For example, if you scan a staircase, it will correspond to real-life proportions.
• Creation of unique content. Using handicrafts or cultural objects as prototypes can significantly increase interest in your project.
• Possibility of additional income. You can find a lot of assets created using photogrammetry on marketplaces, which can be quite expensive.

• Not perfect accuracy. Compared to laser scanning, photogrammetry can produce less accurate results.
• Investment required. To obtain a high-quality 3D model, you will need a good camera and appropriate software, which is associated with additional costs.
• Complex processing process. The original scan often requires refinement, removal of unnecessary elements, and geometry optimization.
• Artistic skills are required. The texturing process can be complex, as artifacts may appear on the model that require manual refinement.

Despite its drawbacks, photogrammetry remains one of the most effective ways to create realistic graphics, especially on a limited budget. This technology also finds application in non-gaming projects, such as visualizations and interactive exhibitions, as it allows for the preservation of even the most fragile objects. With the development of technology, photogrammetry can become an everyday practice in various fields.