AI Processors Feature Augmented Image Signal Processing for Self Driving Vehicles

July 15, 2022

Blog

Despite the advancements in AI processors, the image signal processing capacity of the system is not enough to balance the requirements of self-driving vehicles. The continuous improvement in ISP hardware technology can bridge this gap in the industry.

Ambarella recently launched its AI-based Image Signal Processor (AISP). Talking more about an ISP (Image Signal Processor), it is used to perform post-processing on the signal output for performing various feature extraction algorithms followed by classification. 

AI plays an important role in augmenting Image Signal Processing that is done by integrating the ISP hardware onto its System on Chip. It makes use of the different types of Neural networks to solve various image processing tasks, from simple binary classification to instance segmentation. 

Ambarella’s low-power systems on chip (SoCs) offer a wide variety of benefits, such as an efficient mechanism for high-quality video compression and advanced image processing with the help of robust deep neural networks. Hence, it helps intelligent cameras extract valuable data from high-resolution video streams. 

The fundamental approach of ISPs is quite suitable for security and automotive industries, where full-color vision is required in low-light areas. As traditional camera systems capture noisy videos in a low-light environment and dark shadows or blown-out highlights in high contrast situations, both of these cases are not ideal and result in the loss of unfavorable details for AI applications. AISP can provide color imaging at very low lux levels and with minimum noise as compared to traditional cameras. It also provides more natural color reproduction and reduced external illuminations by offering new levels of high dynamic range (HDR), which is 100X advanced compared to the traditional ISPs.

Image credits: Ambarella

Some key features of the AISP include:

• Supports mainstream sensors
• Offers seamless switching between different lighting conditions, including day, night, and HDR scenes
• Runs in parallel with other AI algorithms on the CVflow engine
• Enables advanced operational modes to fit application-specific use cases

AISP Application for Self Driving Cars

The AISP was initially demonstrated on the CV2 SoCs. The updates will be available across Ambarella’s entire CVflow SoC range of products at video resolutions up to 4K. In the most recent update, a CV3 AI domain controller SoC was launched, which would be suitable for implementing autonomous driving for vehicles with single and multi-camera ADAS, Driver Monitoring System for a precisely accurate ecosystem. The in-cabin solutions along with single and multi-channel electronic mirrors with blind-spot detection, would further enhance the functionalities.

With all automakers focusing on building autonomous cars, the demand for camera-based SoCs and computer vision technology has increased immensely. Hence, Ambarella could be fit for this job as along with being an AI silicon company, it "has deep technical knowledge in camera-based SoCs and enhanced computer vision capabilities."

The application perspective of this technology would be in a typical security camera trying to identify a license plate at night. On traditional cameras, the image captured would be noisy, and the text on the license plate would be unreadable, even to the best AI algorithm. And with black and white video, all the information about the car's color would be lost. By using Ambarella’s AISP technology, we can restore the colors at the source even before AI processing, and hence the license plate can be clearly read and the color of the car identified. This AISP technology in all reduces the total system costs of building high-quality cameras by using low-cost sensors and therefore is an excellent fit for self-driving vehicles.