Improving smartphone cameras with color sensor technology
November 10, 2016
The world of digital image capture has evolved dramatically in the past decade. Perhaps most striking and remarkable are the advances in smartphone ca...
The world of digital image capture has evolved dramatically in the past decade. Perhaps most striking and remarkable are the advances in smartphone camera technology. The impact of smartphone cameras has resulted in a significant reduction in standalone digital camera sales over the last several years, as consumers are increasingly satisfied with the features and image quality of their mobile phone cameras.
The quality and accuracy of image capture has become a key area of innovation and differentiation and because of that, OEMs are continually improving the capabilities of smartphone cameras. Ultimately, smartphone OEMs continue to strive to deliver a Digital Single-Lens Reflex (DLSR) camera experience on phones while avoiding a heavy camera body/lens and higher cost. Consumers enjoy the benefits of capturing pictures with improved simplicity, resolution, and more life-like color accuracy.
LG raises bar with smartphone color spectrum technology
A great example in the market today is LG, which has a long-standing reputation for delivering advanced camera technology in its smartphones and has received many awards from various photography and imaging organizations. Through close collaboration with LG, ams delivered color sensor technology enabling LG’s G5 flagship phone to deliver a color spectrum sensor (CSS) technology for its primary camera. The phone’s color sensor is located next to the flash LED on the backside of the phone (Figure 1).
[Figure 1 | Color sensor technology raised the bar for camera innovation and performance improvement.]
LG is one of the first smartphone OEMs to take advantage of a color sensor to enhance its camera capabilities. The new sensor enables measurement of the ambient light while also determining if the light source is artificial or natural. In addition, it provides the ability to intelligently distinguish if the light source is from the ambient environment or an object within the field of view. By understanding the exact lighting conditions with such precision, the phone can select the optimal white point.
How color sensing technology works
The critical capability of a color sensor is the precise measurement of correlated color temperature (CCT) as well as the infrared (IR) component of the ambient light. CCT is a metric that defines the color appearance of a light source by relating its color to a define reference. IR is the portion of the electromagnetic spectrum with wavelengths beyond visible light and in the 700 nm to 1 mm range. Light CCT ranges from cool colors (bluish white) to warm colors (yellowish and red).
With accurate CCT and IR measurements, the lighting source can be identified as natural or artificial (i.e. sunlight vs. LED, incandescent, and fluorescent) and used to set the optimal white point for the image capture. The spectral components of various light sources have wide variations in the visible (RGB) and invisible (IR) and therefore accurate sensing of both components is needed to distinguish each light source.
Take, for example, the spectral responsivity of the ams TCS3400 color sensor (Figure 2). The on-chip IR blocking filter minimizes the red, green, and blue (RGB) responses to IR light required for more precise color measurement.
[Figure 2 | Spectral responsivity of the ams TCS3400]
The response from the RGB channels can be used to determine the lighting environment CCT. In addition to RGB and IR sensing, the sensor also has a clear (C) channel, which provides a reference channel for isolation of the color measurement. The four RGBC channels each have a dedicated 16-bit data converter, allowing simultaneous measurements.
Why use discrete color sensors?
Using a discrete color sensor in conjunction with an image sensor allows for measurement of IR light. Typical smartphone image sensors block this portion of the light spectrum since it is not in the visible range and, more importantly, it can have adverse effects on image capture quality. Therefore, smartphone cameras image sensors typically implement an IR blocking filter to ensure that the camera does not sense IR. In some situations, IR light can pass through these blocking filters and result in unnatural color and other adverse effects on the image quality. The camera’s image processing algorithms can eliminate these IR-induced effects if they have a measure of the ratio of IR light to visible light in the ambient lighting.
Another challenge for cameras is the ability to distinguish between color reflected from an object and the color of the ambient light when capturing an image. The combined RGB and IR sensing capability of the color sensor allows a camera subsystem to automatically make this determination and subsequently set the optimal white point for the image capture. The accurate IR light measurements allow the best capture of the actual color of the object rather than a misinterpreted color due to the IR light in the surrounding environment. Selecting the optimal white point is critical for the most natural and balanced appearance of a picture.
Additional color sensor applications
Color sensing technology along with the ability to accurately measure the ambient light CCT and intensity enables other smartphone applications that enhance the user experience. Among these applications are automatic camera LED flash color control and automatic display white point setting.
Smartphone camera flash implementations may use multiple LEDs, white and non-white in color, to intelligently fire the LEDs with the right color, and amount of light, based on the CCT of the lighting environment. Automatic determination of the optimum flash intensity and color, based on the scenes lighting conditions, provides the ability to make both indoor and outdoor images more balanced.
Another feature enabled by color sensing is automatic adjustment of the display’s white point to match the lighting environment’s CCT. The display white point, sometimes referred to as reference white, is typically a fixed setting where the displayed ‘white color’ appears ‘white’ to our eyes. Vision science research has concluded that our viewing experience is best when the display white point is adjusted based on the CCT of the lighting environment. Therefore, a fixed setting is not ideal for smartphones that are used in a broad range of lighting conditions. The ability to dynamically adjust the display white point based on the lighting environment allows images to look more appealing and colors appear accurate for a better user experience.
ams AG
http://www.facebook.com/amsAnalog
