This is an update to the previous blog, Sony Develops Back-Illuminated Time-of-Flight Image Sensor With the Industry’s Smallest Pixel Pitch: To Make Gesture & Object Recognition and Obstacle Detection Possible for Augmented Reality, Virtual Reality & Other Applications Such as Robotics & Autonomous Drones.
SoftKinetic (a Belgian company in Brussels) becomes Sony Depthsensing Solutions.
The following video presents Car Infotainment Gesture Control – DepthSense® CARlib for Automotive:
“SoftKinetic brings advanced time-of-flight gesture control technology to automobiles: With DepthSense CARlib, there’s no need to push buttons or touch screens: from now on, interact with the infotainment system using natural hand gestures. Certified for the automotive market, and available now in premium vehicles, SoftKinetic’s 3D time-of-flight sensors and tracking library enable accurate hand and pose tracking.“
- SoftKinetic has been a leader in 3D vision and gesture recognition technologies: it has continuously delivered state-of-the-art solutions in the field of 3D sensing and processing through the development of CMOS 3D sensors, 3D camera reference designs, SDKs, algorithms, and applications for gesture recognition, object scanning, automotive control, and AR/VR (Augmented Reality/Virtual Reality).
- All of the above shall now be developed under Sony Depthsensing Solutions.
- Sony’s achievements included the integration of DepthSense® technology and gesture recognition software into premium vehicles and the integration of DepthSense® camera module and software designed by the Brussels-based company into Sony’s new Entertainment Robot “aibo”.
Sony Robot “aibo”
Sooner rather than later, machines, including cars, will be able to detect, recognise and react towards human beings in ways we used to only think possible in fiction. The Brussels-based team is committed to bringing not only consumer electronics products, but also robots and automobiles to the next level of evolution.
We trust that 3D depth-sensing technology will revolutionise the way humans and machines interact in the near future, and this conviction is only reinforced by the confidence Sony has put in Sony Depthsensing Solutions to carry out this (r)evolution.
Sony Ceo, Akihiro Hasegawa
The following video presents Sony Aibo robot dog returns with advanced AI (CNET News):
“Sony’s famous robo-pup Aibo is back with new tricks, including sensors that gather data from owners. But you can only fetch one now in Japan.“
Sony recently released a back-illuminated, time-of-flight (“ToF”) image sensor which is 1/2-type, VGA resolution and it delivers improved depth sensing performance
- The DepthSense® Time-of-Flight (ToF) pixel, called “CAPD” (Current Assisted Photonic Demodulator), allows better performance, smaller pixel sizes and higher integration than any other ToF technology on the market.
The following video presents SoftKinetic’s depth sense technology:
“Lucas Matney chats with Softinetic about their camera, sensor, and software that measure and apply depth sensing information.“
PRESS RELEASE
Sony Releases a Back-Illuminated Time-of-Flight Image Sensor
Offers VGA Resolution in a Compact Size and Enables Highly Accurate Range Finding Performance from Near to Far Distances
December 19, 2017
Sony Corporation
Sony Semiconductor Solutions Corporation
Tokyo, Japan — Sony Corporation (“Sony”) today announced the release of a back-illuminated, time-of-flight (“ToF”) image sensor which is 1/2-type, VGA resolution and it delivers improved depth sensing performance. Sony will start shipping sample units in April 2018. The new sensor is part of Sony’s DepthSense™ lineup, a group of depth-sensing image sensor products with range finding capability, and it is the first product in the lineup to adopt a back-illuminated ToF configuration.
Compact sensors that can provide accurate depth maps are used for various applications such as gesture recognition, object recognition, and obstacle detection, as well as for robotics and drones that require autonomous operations, or virtual reality, augmented reality and mixed reality systems, for which market expansion is anticipated. The small size of the new product was achieved thanks to development of a 10µm square pixel, with highly accurate distance measurement performance from close to far distances, which can be used in a wide range of applications in these fields.
Sony IMX456QL back-illuminated time-of-flight image sensor
| Model name | Sample shipment date | Mass-production shipment date (planned) |
Sample price (excluding tax) |
|---|---|---|---|
| IMX456QL back-illuminated ToF image sensor |
April 2018 | November 2018 | 3,000 JPY |
With ToF technology, the distance to an object is measured by the time it takes for light from a light source to reach the object and reflect back to the sensor. ToF image sensors detect distance information for every pixel, resulting in highly accurate depth maps. To further improve accuracy from near to far distances, reflected light must be efficiently received and the signal processing must be executed at a high frame rate.
The new sensor which adopts back-illuminated CMOS image sensor architecture allows for more accurate detection of the reflected light because of improved sensor sensitivity. While conventional ToF sensor has difficulty in measuring far distance of approximately 10 meters, the new product comes with a sensitivity raising mode, enabling distance measurement with a high rate of detection at these distances. It is also possible to capture high-precision depth maps in a VGA resolution at close distances of approximately 30 centimeters to 1 meter.
Additionally, because this sensor captures depth maps for each frame, it enables image capture at a higher frame rate than when using a laser to scan the object for distance measurement. This reduces distortion of moving subjects in depth maps.
Softkinetic Systems S.A., a Sony Semiconductor Solutions Corporation subsidiary involved in development of ToF image sensor technologies since its acquisition in 2015, was renamed to Sony Depthsensing Solutions Holding SA/NV as of December 18. This is due to the fact that the integration of technological development processes went swiftly and the commercialization of products under the Sony DepthSense™ brand is now on track with the launch of this product. Sony plans to establish this company as an R&D base specialized in depth sensing solutions, aiming to create ever stronger products.
Main Features
Sony IMX456QL back-illuminated time-of-flight image sensor delivers high resolution depth maps in high degree of accuracy
Delivers highly accurate distance detection from near to far distance using a back-illuminated TOF system
Sony IMX456QL back-illuminated time-of-flight image sensor delivers high resolution depth maps in high degree of accuracy
The new sensor delivers high resolution depth maps in high degree of accuracy from a close distance of approximately 30 centimeters to a long distance of approximately 10 meters with a single ToF camera module.
Suppresses distortion of moving subjects
Because this sensor captures depth maps for each frame, it delivers reduced distortion of moving subjects in depth maps, compared to using a laser to scan the object for distance measurement.
Software development kit provided to support camera development
A software development kit (hardware and software) will be provided with the sample shipment to make camera development easier.
Key Specifications
| Model name | IMX456QL |
|---|---|
| Number of effective pixels | 640 (H) X 480 (V), 0.307 megapixels |
| Image size | Diagonal 8.0mm (Type 1/2) |
| Recommended light source wavelength | 850nm/940nm |
| Pixel size | 10µm×10µm |
| Modulation frequency | 5～100MHz |
| Modulation Contrast | 0.8*1 |
| Depth sensing frame rate | 120fps |
| Distance precision at 1 m | 6mm*2 |
