3D Camera – Time-of-Flight (ToF)

Spatial images

Spatial vision is extremely important to humans, and that is where the question arises. Why do most cameras show a flat image instead of a much more natural image in three dimensions? To answer this question, you need to analyze the history of how people tried to reproduce depth in images. The first method that has been mastered and is still used today is the method of stereography where, by observing one scene with two cameras, the effect of spatial depth was obtained. Digital image processing allowed to obtain a better method, thanks to combining two photos into one and such a manipulation of colors that in special glasses with one blue glass and the other red glass, each eye sees a different image. Currently, cinemas use the IMAX technology, thanks to the appropriate polarization of the image and the lenses in the glasses, it provides a smooth and three-dimensional image. All these technologies of specially crafted images with much greater requirements than images in two dimensions, this leads us to the conclusion that capturing images in 3D requires much more complicated processes than acquisition in two dimensions. More complex processes require more resources, more advanced technologies and interpretation of more data, which makes 3D acquisition cameras usually more expensive and more complicated to use. The 2D image also provides a more faithful representation of structures and colors. Does this make 3D cameras an unnecessary curiosity? Nothing could be more wrong. Today’s world of automation striving for automation creates a demand for spatial vision systems. Artificial intelligence-controlled cars or packaging machines are some of the examples of applications. And their number is constantly growing. For this reason, a number of technologies have been developed that allow cameras to acquire 3D image, which allow you to perfectly match the camera to the application.

3D image acquisition technologies

3D image acquisition technologies differ significantly from each other and represent completely different advantages. The aforementioned spatial images were obtained using the stereovision method. This method uses data from two camera images and uses an algorithm to combine the same point in space from two photos. Thanks to this and the knowledge of the relative position of the cameras, it is possible to recreate the depth of the image. This technology is almost identical to what takes place during our viewing process, which presents a number of advantages, such as the possibility of achieving high precision for short distances or the possibility of using two 2D cameras instead of a 3D camera. However, this method requires complex algorithms and a precise method of image retrieval. This makes stereovision expensive, complex, and difficult to use for real-time applications. A completely different approach to registration of a spatial image is the method of laser triangulation. The technology uses laser light which reflects off the observed object and goes to the camera. The angle at which the laser light is registered allows to calculate the distance from the camera to the object. This technology is ideal for applications where we want to know the exact distance of the observed objects from the camera, but for such a system to work properly it requires high-quality components. The system requires almost unchanged working conditions, the object cannot move too fast or be far away from the camera. Taking into account the disadvantages of previous systems, Basler created cameras based on ToF (Time of flight) technology. This technology, unlike the previous ones, does not require complete measuring systems, but one compact device. The only question is how does it work?

The principle of operation of ToF Cameras

The ToF method uses a modulated light source integrated with the camera, which is reflected from the observed object and registered by the sensor. This sounds very similar to the laser triangulation method, but here it is based on a completely different principle. The method is divided according to the mode of operation of the illuminator into continuous or pulsed. In the continuous ToF method, the phase shift between the light wave from the illuminator and the one read by the sensor is examined, which allows the distance to the object to be calculated. In the pulse method used in ToF Basler cameras, the shutter control unit synchronizes frame acquisition with light pulses, allowing only a part of the light sent by the illuminator to reach the sensor. To visualize it, you can use an analogy where each pixel is a country and the illuminator’s light is driving a truck going abroad. It takes some time for the truck to drive to the destination (here the object being observed) and back again. At that time, however, the state closes the borders and some of the trucks cannot return. Knowing the time after which the borders were closed after the trucks were shipped (assuming that they all traveled at the same speed), it can be determined whether the place to which the truck traveled was closer or further than the distance resulting from the product of the speed of the truck and the elapsed time divided by two. By changing the time after which the border closes, you can get more and more information about the distance of your destinations. Similarly, the shutter blocks the access of light reflected from an object that was at a distance greater than the product of the speed of light in the air and the time from the light pulse to the shutter closure divided by two. Then, after repeatedly changing the shutter speed, the signals that reached the sensor and the amount of light are summed up which reached it, you can determine how long it took the light to reach each pixel of the sensor and thus create a grid of points creating a spatial model of the observed object. It seems complicated, but it is the simplest and most accessible of all 3D image acquisition methods. A great proof of the effectiveness and ease of use of this technology is that the device using it has sold in millions of copies and is still used today. This device is the kinect camera for the Xbox 360, and now it is also starting to appear in smartphones.

Advantages and disadvantages of ToF cameras

ToF cameras definitely stand out from the competition in 3D acquisition with their convenience of use and affordability, however, it is not an ideal solution and has many disadvantages. By far the biggest disadvantage of using light waves returning to the sensor for imaging is that any other light present in the camera’s working space will negatively affect the accuracy of the measurement.

These cameras may also have problems modeling shiny or reflective objects. However, this disadvantage can also be an advantage. These cameras do not need additional illuminators and for some applications they will represent a significant cost reduction. They also have the greatest possibilities when it comes to imaging large spaces from all the above-mentioned technologies, they are also much faster than them. Therefore, they offer much more possibilities if they are to be used in real time.

Basler Blaze Camera

In response to market demand for affordable, accurate 3D imaging cameras, Basler developed Basler Blaze cameras using ToF technology. The camera is equipped with a VCSEL diode that allows the camera to work in daylight, an integrated Sony DepthSense ™ IMX556PLR-C sensor with CAPD (Current Assisted Photonic Demodulator) pixels and a Back-Illuminated CMOS matrix. This allows him to carry out accurate 3D measurements at a distance of up to 10 meters. The camera provides an image at 30 frames per second in VGA resolution, allowing for accurate visualization of the tested model. Blaze has an IP67 resistance class, and additionally, low weight and no moving parts make it ideal for mounting on a robot arm. The Gigabit Ethernet interface facilitates integration and enables trouble-free use in multi-camera systems without causing mutual interference. An additional advantage of this camera is that although the camera works in black and white, it can be easily integrated with the Basler ace 2D camera, which allows you to obtain a color model.

ToF cameras or 2D cameras

Is a 2D or 3D camera better in my application? The answer to this question largely depends on what measurements or inspections the camera will perform. 2D cameras have much better performance when it comes to color, texture, dimensions and shape of flat objects. When it comes to 3D cameras, they are great for analyzing the capacity, shape of the entire element or its position in 3 dimensions. So it will be perfect for logistic tasks such as palletizing or the precise work of a robotic arm. It also presents a decisive advantage over cameras working in two dimensions in medicine, where it allows you to precisely position the patient for examination or remotely monitor the movement of patients. If cost is of less importance, you can always use a two-camera system, such as Basler Blaze and Basler ace, which allow you to take advantage of both.