Panasonic Develops Long-Range High-Resolution & High-Sensitivity TOF Image Sensor to Capture 3D Range Images of Objects Up to 250 m Even at Night with Poor Visibility: Applications Include Industrial/Monitoring/Automotive Sensing Cameras

The following video presents What Is Time-of-Flight? – Vision Campus:
“A ToF camera delivers 3D image information and helps to solve tasks where 2D data isn’t enough. For inspection tasks in low-contrast situations or varying light conditions in particular, 3D technology offers an added dimension for capturing the properties needed for measurement. Factory automation, robotics and logistics are often used in applications where 3D image data can effectively supplement 2D data. In the latest Vision Campus video our expert Thies Moeller explains how this kind of camera works. Beyond this, he describes the benefits inherent to this camera, as well as influencing factors that can diminish the effectiveness of a ToF camera.

What is Time-of-Flight?
00:07 Introduction
01:15 Main components of Time-of-Flight camera
01:58 Advantages of Time-of-Flight cameras
02:11 Operating principle
02:21 Time-of-Flight cameras based on the principle of pulsed light sources
02:43 Time-of-Flight cameras using continuous waves
03:23 What does a ToF image look like?
03:46 Advantages and disadvantages of different 3D measuring techniques
04:01 Limitations: Scattered light, ambient light, multiple reflections, multiple cameras
04:58 Working range, depth accuracy
05:19 Applications (Logistics, Robotics. Medical, Autonomous Vehicles
05:55 Conclusion“

PRESS RELEASE

Jun 19, 2018

Panasonic Develops Long-range TOF Image Sensor

This new image sensor is capable of detecting objects 250 m ahead at night with poor visibility and achieves high resolution by adopting APD with 250,000 pixels

Osaka, Japan – Panasonic Corporation announced today that it has developed a time-of-flight (TOF) image sensor [2] that uses avalanche photodiode (APD) [1] pixels and is capable of capturing range imaging of objects up to 250 m even at night with poor visibility. The sensor can be applied in a variety of fields including automotive range imaging and wide-area surveillance in the dark.

The newly developed TOF image sensor directly measures the time-of-flight of light between an object and the sensor at all the pixels and collectively obtains 3D range images for short to long distances. The use of APDs, which amplify input signals, as the photodetector and the incorporation of an in-pixel circuit that integrates weak input signals, and it enables the 3D range imaging of a place 250 m ahead. In addition, the new sensor has achieved the integration of the world’s best 250,000 pixels^*1 through the lamination of the electron multiplier and the electron storage as well as the area reduction of APD pixels. This has enabled long-range and high-resolution 3D range imaging, which was difficult previously.

*1: As a TOF image sensor equipped with multiplication pixels that has been presented at conferences, etc.

(As of Jun. 8, 2018, according to Panasonic data)

This development brings the following advantages.

High-resolution and high-sensitivity TOF image sensor that uses APD pixels
250,000 pixels: Approx. four times the conventional number of pixels^*1
Acquisition of long-distance 3D range images using TOF
1 – 250 m: Approx. twice that of conventional distances^*2

*2: During video imaging by a TOF image sensor equipped with multiplication pixels that has been presented at conferences, etc.

(As of Jun. 8, 2018, according to Panasonic data)

This development has been realized on the basis of the following technologies:

APD pixel technology
The area of APD pixels is significantly reduced while the multiplication performance is maintained through the lamination of the multiplier that amplifies photoelectrons and the electron storage that retains electrons.
Long-range measurement imaging technology
Each pixel is equipped with an integration circuit that counts the number of detected photons [3] that have arrived at the photodetector. Even a weak single-photon level reflection light is reliably captured to achieve high-density range images.

Conventional camera technologies such as stereo cameras have an issue of degraded recognition accuracy at night. On the other hand, LiDAR that employs infrared light can be used at night; however, its low resolution makes it hard to identify small objects, which can cause missed detection.

[Suitable applications]

Industrial/monitoring/automotive sensing cameras

[Notes]

This development result will be announced in the 2018 Symposia on VLSI Technology and Circuits, which will be held in Honolulu on June 20, 2018.

[Product features]

1. High-resolution and high-sensitivity TOF image sensor that uses APD pixels

Conventional image sensors convert a single photon that has entered their pixels to only a single electron and therefore have an issue of being susceptible to noise when the signal light is as weak as approximately a single-photon level. The newly developed sensor forms very fine pixels of 11.2 μm pitch by using the APD pixel technology that laminates the photoelectric conversion region, multiplication region, and signal storage as constituents, thereby achieving a high resolution of 250,000 pixels and high sensitivity signals amplified by 10,000 times.

Fig.1 Comparison diagrams of the photodetector structure of a conventional image sensor and the APD image sensor

Fig. 2 Comparison diagrams of imaging with and without signal multiplication in pixels

2. Acquisition of long-range 3D range images using TOF

Common TOF ranging calculates distances by measuring the time-of-flight of light emitted from a light source to hit an object, reflect, and return. Panasonic has developed a unique short-pulse TOF method that drives a high-speed 10-nanosecond shutter built in the sensor in sync with 10-nanosecond minimum short-pulse light. This newly developed technology enables the collective acquisition of a 3D range image by compositing multiple ranges from short to long.

Fig.3 Explanatory diagram of the short-pulse TOF method

Fig. 4 Explanatory diagram of image acquisition by the APD-TOF sensor for each range using short pulses

[Technology]

1. APD pixel technology

Panasonic has developed APD pixel technology that enables fine pixels while possessing a multiplication function by providing a multiplication region that multiplies one photon to 10,000 electrons and laminating this multiplication region to accommodate it in the same plane area with the photoelectric conversion and signal storage regions. This technology enables sharp and high-resolution imaging in dark or distant places.

2. Long-range measurement imaging technology

Panasonic has developed long-range measurement imaging technology that collectively obtains distance ranges from short to long as range images. The light reflected from a place 250 m away means that in terms of probability a signal of one photon barely arrives. The company has newly developed a weak light integration technology that reliably captures a weak signal of a single photon through the use of unique integration circuits built in all the APD pixels, which count the number of arrivals of photons. The long-range measurement imaging technology that combines the new short-pulse TOF method and weak light integration technology has achieved range images with both long-range measurement and high resolution at night, which was difficult for conventional stereo cameras and LiDAR. The new technology enables the acquisition of range images in which the position and shape of people and small objects, which are invisible to human eyes, can be discerned even at night from a long distance.

Fig. 5 Explanatory diagram of densification of weak light using the weak light integration technology

Fig. 6 3D range image obtained at night by the TOF image sensor

[Term descriptions]

[1] Avalanche photodiode (APD): An ordinary photodiode generates one electron from one photon. On the other hand, an avalanche photodiode (APD) applies a strong electric field to the one electron generated from a photon to cause the electron to strongly collide with other electrons in the material, thereby generating two electrons. This collision will be repeated with increasing scale with the initial collision as a trigger as if it were an avalanche, multiplying the electron by over 10,000 times in the end.
[2] Time-of-flight (TOF) image sensor: The time-of-flight (TOF) refers to the time-of-flight of light. Common TOF ranging consists of a light source and a light detector. The light emitted from the light source hits a subject and the reflected light arrives at the detector. During this period, the light is traveling at the speed of light. The distance to an object can be calculated by measuring the time-of-flight of light since the speed of light in the air is constant.
[3] Photon: A photon is the smallest unit of the energy of light, which cannot be broken down further. It is considered difficult for image sensors used in common cameras to detect a single-photon light signal because these sensors have noise with energy of about one photon.

About Panasonic

Panasonic Corporation is a worldwide leader in the development of diverse electronics technologies and solutions for customers in the consumer electronics, housing, automotive, and B2B businesses. Celebrating its 100^th anniversary in 2018, the company has expanded globally and now operates 591 subsidiaries and 88 associated companies worldwide, recording consolidated net sales of 7.982 trillion yen for the year ended March 31, 2018. Committed to pursuing new value through innovation across divisional lines, the company uses its technologies to create a better life and a better world for its customers. To learn more about Panasonic:
https://www.panasonic.com/global.