3D Photogrammetry
3D photogrammetry is the technology that allows us to construct virtual 3-dimensional models of an object from 2-dimensional images. This is the same technology used by Google Earth to re-create virtual models of real places from satellite imagery. Though photogrammetry is traditionally performed on objects from scales as large as entire cities to as small as buildings, advancements in computational models have allowed for close-range photogrammetry to become possible.
When a photograph is taken of a 3D object, it is effectively projected onto a flat 2D surface. This is known as a perspective projection.
Figure 1. Perspective projection of a 3D cube onto a 2D canvas
As seen from Figure 1, the single image generated on the canvas is insufficient to fully re-create the 3D cube. With multiple images from different angles, a better description of how different points on the object are in 3D space can be generated. Using 3D photogrammetry requires many images of the same object from various camera angles to accurately determine the coordinates of certain well-defined points on the object.
These points, known as control points, are anchors that computer programs can use to determine the position of all other points on the surface of the object. The output of these programs is the coordinates of thousands of different points on the surface of the 3D object called a point cloud. A mesh is then generated from these point clouds by connecting them to form numerous polygons, where each point from the point cloud makes up a vertex of a polygon (Figure 2).
Figure 2. A 3D virtual model of a bunny as a point cloud and a polygon mesh
Finally, a layer of texture is placed over the mesh. This texture is input as a 2D image, which can simply be taken from the sample pictures that were fed to the computer program.
Figure 3. A 3D polygon mesh of a bunny with a 2D set of textures to be overlaid, giving the final output of the textured virtual model on the right
3 Degrees of Freedom
Given the large sample size of photos from unique angles needed to produce an accurate virtual replica of a 3D object, our 3D scanner will have an optical camera that has the freedom to be positioned anywhere in a hemispherical region of space around the object. This is achieved by a set of motors and actuators, which allow for a spherical range of motion centred on the object (Figure 4).
Figure 4. Systems A to C with their associated range of motion allow for hemispherical coverage of an object located at the centre
System A comprises of a stepper motor that rotates the central cylindrical platform through a full rotation of 360°. In doing so, the optical camera can take pictures of the object from a circular band around it.
System C also comprises of a stepper motor, which rotates the camera arm from a position where it is level with the central platform, to a position directly above the central platform. Combined with system A, this effectively allows the optical camera to take pictures of the object from a hemispherical shell centred on the object.
In an attempt to improve the accuracy of the final result, we have also implemented a third degree of freedom in the radial direction, allowing our camera to position itself closer to the object. This allows the camera to move in and capture finer details of smaller objects, while still being able to accommodate the size of large objects. Radial movement is achieved through a linear actuator, which converts rotational motion into forward and backward motion. For our scanner, we have chosen to use a rack and pinion actuator (Figure 5).
Figure 5. The rack (bottom) and pinion (top right) actuators used in our 3D scanner, including the back shell (top left) and front shell (top middle)
The Camera
As photos are the basis for the whole concept of 3D Photogrammetry, an optical camera is a key component of our design. As mentioned above, we want to incorporate the radial distance of the camera from the object as a degree of freedom, so an important aspect of the camera we choose is its ability to adjust its focus so that we do to get blurry photos. Having decided that we wanted an Arduino and Raspberry Pi to be the brains behind this project, the Arducam V3 16MP IMX519 was the obvious choice.
This sensor has both sufficient quality through its 16MP lens and a built-in autofocus feature that we could incorporate in our code.
The Web App
Originally, we thought of stopping the project at a stage where the user could export all the necessary photos, but we felt that we could still make it even more user-friendly and all-encompassing. We wanted the Raspberry Pi to actually generate the model for the user, but this proved unfeasible due to the lack of horsepower from a single Raspberry Pi. To get around this, we decided to design a WebApp that links our device to a cloud-based 3D Photogrammetry model generator. This was something that came up halfway through our project as a solution to the problem of the Raspberry Pi being too weak to do everything we wanted. We still wanted to stick to our vision of making the whole process of 3D Photogrammetry as easy as possible for the user, so designing a Web App was the best solution.
