HIGH-ACCURACY, HIGH-SPEED 3D OPTICAL SENSING IN UNSTRUCTURED ENVIRONMENTS

Over the last few decades, as many companies have released low-cost commercialized 3D sensors, vision-based 3D sensing has been more accessible and ubiquitous. As a result, the range of applications for 3D-sensing technology has been extended to medicine, entertainment, and manufacturing, as well as other industries. However, unlike with well-controlled industries such as manufacturing factories, commercial sensors and resolutions are not yet accurate enough to be applied in unstructured environments, such as construction sites. For example, to inspect the inside of large infrastructures such as steel bridges, robots need high-accuracy 3D maps for inspection and path planning, and robot sensors should be robust enough to withstand harsh weather. To achieve the goal of scanning and inspecting surrounding environments, the 3D imaging system needs to reconstruct 3D images with high accuracy, high speed, and robustness to noise.

The first challenge in realizing a high-accuracy 3D imaging system in unstructured environments is noise in captured images. To improve the robustness of 3D images, we developed a computational framework by using geometric constraints for high-accuracy 3D sensing with only two-frequency patterns. A previously existing two-frequency phase unwrapping method has a limitation in accuracy because the scaling factor, which is calculated by the difference in fringe width between low-frequency and high-frequency patterns, significantly amplifies the noise signal. The framework suggested to use the relationship of optical devices for 3D sensing inversely. We can dramatically decrease the scaling factor required to reconstruct 3D images. Without additional patterns, we can measure the geometry of objects within a certain depth range accurately.

The second challenge is mainly caused by the dynamic motion of moving platforms. If the sampling rate of 3D sensing is low, it is difficult for robots to localize the platform, generate 3D maps for surrounding environment, and make a right decision in planning a path or inspecting sites based on the information. To increase the speed of 3D sensing, we can reduce the number of patterns used for generating one 3D image. The number of patterns is an important factor in determining the speed of 3D reconstruction because a camera captures the patterns sequentially, which means that the number of patterns is proportional to the time taken to capture a set of images for one 3D image. We developed a method to reduce the number of patterns by using geometric constraints. In addition, by integrating texture image of the object with a phase-coding method, we used a total of five binary patterns to get absolute phase map for 3D reconstruction. By doing experiments with a high-speed camera, the sensing system captures 2D images at 3,333 Hz, and 3D images at 667 Hz.

Although the speed of 3D sensing has increased through reducing the number of patterns, the system has fundamental limitations in speed and spectrum of light. The system typically includes at least one Digital Light Processing (DLP) projector because of its accuracy and flexibility. However, the mechanism of the DLP projector, which flips a set of micro mirrors inside the projector for determining whether each pixel is turned on or off, slows down the speed of 3D sensing. To overcome the speed limitation, we designed a custom-made mechanical projector that rotates a wheel with evenly spaced spokes. By using the rotating wheel, the projector generates fringe patterns for phase retrieval, which is the same as that which the DLP projector generates. With the mechanism, we realize a speed of up to 10 kHz for 3D sensing. In addition, we can overcome another limitation the DLP projector has, which is a limited spectrum of light. The micro mirrors can reflect only a specific light spectrum, and the light emitter inside the projector is not replaceable. The mechanical projector places the source of light independently, so, a broad-light spectrum—including visible, infrared (IR), near infrared (NIR), and ultra-violet light—can be used for 3D sensing.

In summary, this dissertation research has contributed methods for realizing high-accuracy and high-speed 3D shape measurement: (1) using geometric constraints in phase-retrieval procedures to reduce noise on the captured images; (2) reducing the number of images for one 3D image to realize high-speed 3D shape measurement; and (3) developing a new type of projector to avoid the limitations in light spectrum and achieve high-speed 3D data. These contributions enable engineers, workers, and even robots to monitor unstructured environments with 3D sensor accurately and quickly analyze the situations they face in the fields of infrastructure maintenance, homeland security, and construction.