3D Imaging
Industrial 2D image processing works very well for inspecting flat surfaces. However, 3D imaging is gaining increasing popularity, opening up new possibilities for automatic control in applications such as person tracking, robotic industry, the film industry, and logistics automation. On the other hand, 3D vision solutions are often too complex and expensive for many typical applications.
One device capable of performing three-dimensional environmental analysis is the Time-of-Flight (ToF) camera, which operates by measuring the time it takes for a light beam to travel. ToF cameras are an ideal tool for logistics and industrial applications, offering a convenient way to simplify inspection and volumetric tasks. ToF cameras can scan a room in three dimensions, simultaneously capturing 2D images. This device records up to 4 million distance measurements per second, each with centimeter accuracy. Like any technology, ToF has many advantages but also its limitations. How exactly does a Time-of-Flight camera work? What benefits does it offer, and what affects the measurement?
3D Imaging – Principle of Operation
Simply put, a ToF camera measures the distance to an object by measuring the time it takes for a light beam to travel. A precise light source sends appropriately modulated light pulses, lasting only nanoseconds. Such stringent requirements for pulse duration necessitate proper illumination and control electronics. Both LEDs and laser sources are permissible as light sources. Not only the intensity but also the rise and fall times of the signal must be controlled; even a one-nanosecond error can falsify the result by tens of centimeters. Control systems precisely synchronize the shutter opening sequence with the light pulsing to minimize stray light reaching the sensor. After charge accumulation, it is read by an analog-to-digital converter and sent to the analysis unit. Each pixel provides information about how long it took for the light beam to return from that particular part of the image. The longer the time, the further away the point from which it reflected. A single readout of the entire matrix provides complete information about the observed object and allows for generating a point cloud based on it.
An example of a task where a ToF camera excels. A modulated light wave reflects off objects and then returns to the camera, providing information about the spatial distribution of the scene.
It’s worth noting that ToF cameras often also have the ability to record regular, colored images, just like standard 2D cameras do. This eliminates the need for a second camera dedicated to this task. This is an additional benefit in terms of device compactness and functionality, and consequently, for the entire potential vision system.
Comparison of images obtained with a ToF camera (left) and a regular 2D monochrome camera (right), based on the diagram above. The ToF camera image, in addition to information about object position, also provides data on their height.
3D Imaging – Limitations
The technology relies on measuring the path of a precisely modulated light wave. It’s easy to deduce that any other light present in the camera’s working space will negatively affect measurement accuracy. Therefore, ToF technology guarantees the highest operational precision in complete darkness. Stray light from another source, propagating along a different path, can be read by the camera’s sensor as information about the observed object and distort the measurement. Newer ToF device models are characterized by increasing precision and ensure efficient operation even in bright lighting conditions.
Another issue is the presence of specular (mirror-like) surfaces in the measurement area. Such a situation can also negatively affect the results. Rays that were traveling outside the system may reflect off these surfaces and head towards the camera sensor. This beam will return to the device, but having traveled a longer distance than what actually separates the observed object and the sensor. Therefore, measurements of matte surfaces tend to be the most favorable.
The highest accuracy is achieved by measurements performed in the center of the camera’s working area, because at the periphery, less and less light emitted by the camera returns to it, and parasitic light also enters. It’s also important to maintain a constant camera temperature.
The advantages of ToF technology include:
- Large measurement field: Compared to other 3D imaging technologies, ToF cameras can instantly provide spatial information about the entire observed area in real-time.
- Fast integration and ease of use: ToF cameras are very often calibrated by the manufacturer, so the entire set is ready to operate right out of the box. An additional plus is the presence of standard interfaces, such as Gigabit Ethernet, for communication with the managing medium.
- No contrast requirement: ToF technology bases its measurements on analyzing dimensions.
- Compactness: The entire system is just one device.
- No moving parts: An additional benefit for compactness and safety.
- High operating speed: A ToF camera can accelerate and streamline many production processes.
On the other hand, you should be aware of:
- Strong external light: Can affect the detail of the information received.
- Multiply reflected light: Provides inaccurate information about the distance of the observed object.
- Applications where centimeter-level accuracy is insufficient: A comprehensive analysis of the given problem is required to select the best solution.
2D Imaging or 3D Imaging?
There are many possible paths to solve a specific vision task. Sometimes the choice between 2D and 3D measurements is obvious, but in some cases, both technologies could handle the task, bringing different benefits. It’s important to understand these benefits and how they apply to a given application. All of this is to ensure a reliable machine vision solution. Typical 2D applications involve situations where color or texture is a key element, as well as the dimensions, position, and shape of flat components. 3D imaging, on the other hand, works well where we want to analyze volume, the shape of an entire component, or its position in 3D coordinates.
Instead of intensity or color, 3D cameras measure geometric information. They create point clouds, where each point represents coordinates on the surface of an object. This makes it possible to determine the shape, position, and orientation of objects.
Which 3D Technology to Choose?
ToF technology stands out among other 3D technologies due to its high measurement range, its ability to work in both darkness and daylight, and above all, its compactness (unlike interferometric methods or structured light illumination). Even stereovision involves larger device dimensions. ToF allows for capturing the spatial dimensions of a scene and objects instantly, without the need for moving components, e.g., laser scanners. The most precise information is provided by measurements performed using interferometric methods. These are characterized by accuracy on the order of the wavelength of the light used. Unfortunately, for this reason, they also have a very small measurement range.
Example Applications
ToF cameras are widely used in many industries. They allow for a different approach to many machine vision issues and significantly streamline numerous processes. Time-of-Flight technology is used, for example, in “bin picking.” A robot arm picks up consecutive items randomly stacked in a container. The ToF camera, by determining the spatial position of the top layer, allows it to select the item that is not covered by any other and can be easily grasped by the robot arm. Another example is all palletizing and depalletizing applications. Creating a spatial map of objects significantly facilitates the process of configuring a system for automated product loading. ToF cameras are also excellent for checking the completeness of low-contrast loads, which are not so easily observed by a regular 2D camera.
There is also a high demand for ToF technology in the automotive industry. The rapidly growing trend for intelligent and autonomous cars necessitates the use of increasingly advanced devices for environmental control. ToF sensors are ideal for this, as by providing information about approaching objects, they allow for a rapid reaction of vehicle control and safety systems. ToF cameras can also play an identical role in all types of rovers and mobile robots, constructed both by hobbyists and commercial companies.
It’s also worth adding that the most widely used ToF camera in the world is the Microsoft Kinect 2.0. This motion sensor, created for the entertainment industry, is a full-fledged device based on time-of-flight technology.
