The following video presents What Is Time-of-Flight? – Vision Campus:
“A ToF camera delivers 3D image information and helps to solve tasks where 2D data isn’t enough. For inspection tasks in low-contrast situations or varying light conditions in particular, 3D technology offers an added dimension for capturing the properties needed for measurement. Factory automation, robotics and logistics are often used in applications where 3D image data can effectively supplement 2D data. In the latest Vision Campus video our expert Thies Moeller explains how this kind of camera works. Beyond this, he describes the benefits inherent to this camera, as well as influencing factors that can diminish the effectiveness of a ToF camera.
What is Time-of-Flight?
00:07 Introduction
01:15 Main components of Time-of-Flight camera
01:58 Advantages of Time-of-Flight cameras
02:11 Operating principle
02:21 Time-of-Flight cameras based on the principle of pulsed light sources
02:43 Time-of-Flight cameras using continuous waves
03:23 What does a ToF image look like?
03:46 Advantages and disadvantages of different 3D measuring techniques
04:01 Limitations: Scattered light, ambient light, multiple reflections, multiple cameras
04:58 Working range, depth accuracy
05:19 Applications (Logistics, Robotics. Medical, Autonomous Vehicles
05:55 Conclusion“
This is an update to the previous blog, Sony Acquires Softkinetic Systems S.A., a Provider of 3D Vision Technology, to Develop the Next Generation of Range Image Sensors & Solutions .
Sony today announced it has developed a back-illuminated time-of-flight (“ToF“) image sensor with a 10µm pixel pitch, the industry’s smallest.*1 This accomplishment was made possible combining the ToF image sensor technology of Softkinetic Systems S.A. (“Softkinetic”), a Sony subsidiary acquired in 2015, and Sony’s back-illuminated CMOS image sensor technology.
Sony: Image taken by CMOS image sensor
- The result is a more compact sensor with more accurate depth-sensing performance.
- This provides a depth map just as accurate as conventional methods*2 up to 1.5x further away. That is, the design delivers accuracy equivalent to conventional methods, even at 1.5x the distance.*2
Sony Comparison of obtained depth map at the same distance: New sensor (left) and conventional sensor (right)
![Sony Comparison of measurement distance when the depth accuracy is equivalent (Ambient light: 2k lux; light source power: 104 mW [even]; modulation frequency: 100 MHz)](https://i0.wp.com/www.photoxels.com/wp-content/uploads/2017/06/time-of-flight-img03-resize.jpg?resize=600%2C383&ssl=1)
Sony Comparison of measurement distance when the depth accuracy is equivalent (Ambient light: 2k lux; light source power: 104 mW [even]; modulation frequency: 100 MHz)
PRESS RELEASE
June 05, 2017
Sony Develops Back-Illuminated Time-of-Flight Image Sensor With the Industry’s Smallest Pixel Pitch*1
Delivers Depth Maps of Same Accuracy 1.5x Further Away Than Conventional Technology*2
Sony Corporation
Sony Semiconductor Solutions Corporation
Tokyo, Japan — Sony Corporation (“Sony”) today announced it has developed a back-illuminated time-of-flight (“ToF”) image sensor with a 10µm pixel pitch, the industry’s smallest.*1 This accomplishment was made possibly combining the ToF image sensor technology of Softkinetic Systems S.A. (“Softkinetic”), a Sony subsidiary acquired in 2015, and Sony’s back-illuminated CMOS image sensor technology. Research results on the sensor were made public at the “2017 Symposia on VLSI Technology and Circuits” on Monday, June 5, 2017 in Kyoto, Japan.
*1 Back-illuminated ToF image sensors, as of June 5, 2017, the date of this release.
*2 Compared to Sony’s front-illuminated ToF image sensor.
With ToF technology, the distance to an object is measured by the time it takes for light from a light source to reach the object and reflect back to the sensor. ToF image sensors detect distance information for every pixel, resulting in highly accurate depth maps. Accuracy can be enhanced even further by ensuring that the reflected light is efficiently received and that the processing required to measure distance is executed at a high frame rate. Additionally, power consumption can be reduced by efficiently collecting and processing the reflected light; the more efficient, the less light source power is needed.
At the heart of the ToF technology in Sony’s new product is Softkinetic’s proprietary Current Assisted Photonic Demodulator (“CAPD”), which employs a unique pixel architecture capable of high-speed modulation using drift current*3 in the pixel, allowing for detection of the light signal with an improved accuracy. This methodology raises the accuracy of distance measurement at the level of every pixel, thereby delivering accurate measurement and depth map acquisition even at significant distances.
Sony combined CAPD with its back-illuminated CMOS image sensor pixel technology to develop the new back-illuminated ToF image sensor, which features a 10µm pixel pitch, with the industry’s smallest.*1 The new sensor builds on the advantages of a back-illuminated pixel structure, which layers the circuitry beneath the photo detector. By optimizing the pixel construction and circuitry for CAPD, the new Sony product delivers improved light collection efficiency and enables high-speed distance measurement processing. The design delivers accuracy equivalent to conventional methods, even at 1.5x the distance.*2 The higher level of light collection efficiency also makes it possible to use a less-powerful light source, and this keeps power consumption down and allows the ToF camera module to be more compact.
In recent years, demand for more accurate depth sensing has been increasing, particularly in the fields of augmented reality (AR), virtual reality (VR), and other applications such as robotics and drones that require autonomous operations. This new sensor is part of the DepthSense® lineup, which makes features such as gesture and object recognition as well as obstacle detection possible. By delivering highly accurate depth-sensing performance in a compact package with low power consumption, the new sensor will help expand the range of applications for ToF image sensors.
*3 Internal pixel drift current: Unlike normal image sensors, this current is generated by the potential difference between two electrodes without forming a depletion layer.
Product Features
- CAPD pixel technology achieves high-efficiency signal output
- SoftKinetic’s CAPD method reads signals at differing phases utilizing light source flight time, multiple times, and then outputs the signal ratio to convert to the distance. Generally, in order to increase the depth map’s signal-to-noise ratio (SNR), it is necessary to place multiple readout circuits inside the pixel to read multiple signals at different phases accurately so that the delay of reflected light can be calculated precisely, as well as to improve the usage efficiency of the reflected light signal. Depth measurement accuracy also depends on the capacity to use a faster modulation frequency.
To satisfy all of these requirements, CAPD dynamically creates a potential gradient (slope for electron transfer) and performs high-speed transfer via internal pixel drift current.*3 This design ensures efficient, high-speed transfer of electrons converted from reflected light in the photo detector between two collectors, making it possible to obtain signals of different phases more accurately.
* The depth accuracy is governed by the relationship shown in the following equation.
σdepth: Variation of measured distance, Fmod: Modulation frequency, Cmod: Signal distribution contrast ratio
- Highly accurate back-illuminated ToF image sensor with a 10µm pixel pitch, the industry’s smallest*1
- ToF pixels generally contain multiple readout circuits inside the pixel, so the multiple transistors and circuitry on top of the photo detector (photodiode) in a front-illuminated CMOS image sensor interfere with reflected light from the object, which brings down distance measurement accuracy.
Sony applied back-illuminated CMOS image sensor pixel technology to improve the effective pixel aperture ratio. This design made the light collection efficiency equivalent to that of a 15µm pixel pitch front-illuminated sensor, but using a back-illuminated sensor with a 10µm pixel pitch. The result is a more compact sensor with more accurate depth-sensing performance. The higher light collection efficiency also made it possible to use a less-power light source, and this kept power consumption down and allowed the sensor module to be even more compact.
In addition, accompanying the adoption of the back-illuminated configuration, the pixel construction and in-pixel circuitry layout were optimized for a ToF sensor, making it possible to accelerate the detection of phase difference required for distance measurement. This made it possible to double the modulation frequency (100 MHz) compared to the conventional method*2 while maintaining the usage efficiency of reflected light, thereby obtaining a better depth map with improved accuracy*2 when measured at the same distance. This in turn provides a depth map just as accurate as conventional methods*2 up to 1.5x further away.
Key Specifications
| New sensor | Conventional sensor | |
|---|---|---|
| Light source wavelength | 850nm | 850nm |
| Pixel size | 10µm×10µm | 15µm×15µm |
| Modulation frequency | 100MHz | 50MHz |
| Modulation contrast | 0.85 | 0.6 |
| Integration time | 380µs | 380µs |
| Depth accuracy at 1 m | 5.9mm | 12.9mm |
* DepthSense is a registered trademark of Softkinetic Systems S.A.