Introduction
As the automotive industry continues its path toward full automation, one area of focus has become the in-cabin monitoring systems, often referred to as driver and occupant monitoring systems (DMS/OMS). These systems use cameras and sensors to enhance the safety and comfort of drivers and passengers.
In-cabin systems are responsible for monitoring the vehicle's interior and the driver's response to the driving environment. So, they often need to work in sync with the exterior systems to ensure the driver makes proper adjustments as the conditions dictate. They can also further reduce human error accidents by focusing on the driver's alertness and making adjustments when necessary, e.g., slowing down the car due to drowsiness in the driver. In-cabin systems have seen so much positive improvement in the last few years that the European Union's General Safety Regulation (GSR) has begun mandating that all new cars from 2024 must implement DMS systems.
As these systems continue to become indispensable to road safety, manufacturers must have access to the proper calibration and characterization solutions during the development process to ensure the highest level of image quality performance and safety.
What do In-Cabin systems monitor?
In-cabin systems monitor drivers' and passengers' behavior and facial cues by capturing videos and images. They mostly use near-infrared (NIR) sensors combined with active illumination (e.g., LED or VCSEL) to ensure accuracy in very low-light conditions. NIR sensors are preferred as they do not need to rely on a visible light source that would be noticeable to the driver and passengers. Captured results are then fed into embedded software to analyze the car's interior.
Many of the aspects that in-cabin systems monitor fall into the area of machine vision, including:
- Driver facial expressions to assess distraction or unsafe emotional states
- Driver eye openness for drowsiness detection
- Driver gaze, e.g., looking down at a phone
- Driver's hand positions on the steering wheel
- Occupant presence and seatbelt usage
- Detection of children and proper child safety, e.g., child seats
- Size and posture of occupants to optimize airbag deployment
These in-cabin systems can make real-time assessments and adjustments (or recommendations) using machine vision learning algorithms when necessary.
When the car drives autonomously, these systems can also focus on human vision applications, such as webcam conferences or selfie-taking. A combination of RGB and IR camera systems could potentially be used to ensure proper functionality.
All told, these systems can significantly improve safety by detecting distraction and fatigue in the driver and optimizing safety features to enhance the safety of all occupants in the event of an accident. In addition, in-cabin cameras can improve user experience by personalizing commands and controlling various aspects of the driving experience.
Testing the image quality performance of in-cabin systems
As the reliance on in-cabin systems continues to grow, these systems must be rigorously tested during the development stages to ensure optimal performance and safety. Several image quality factors should be evaluated when testing their systems, including:
Resolution (SFR)
Camera resolution describes a camera's ability to reproduce a scene's fine details. One way to determine resolution performance is by measuring the spatial frequency response (SFR). These SFRs will produce a range of spatial frequencies on sale from 0% (a complete loss of scene information) to 100% (a perfect scene reproduction without data loss).
Resolution performance is crucial to determining the number of pixels per eye we have. It ensures enough detail is preserved, even in low-light settings, for determining eye movements and analyzing other driver and passenger facial cues. ISO 12233 outlines test methods that can be utilized for resolution measurements.
Distortion
Image distortion occurs when an image's straight lines appear to curve unnaturally. It is often the result of the lens's geometry. Distortion can be seen as a variation of the reproduction scale over the field. As a result, the same object is smaller in the image corner than in the image center. So, if the algorithm needs, e.g., 15 pixels per eye to work properly, this might work for the center but not for the corners.
ISO 17850 offers test methods for local geometric distortion measurements. In addition, our GEOCAL device provides geometric camera calibrations. It creates a grid of light spots originating from infinity for distortion measurements.
Dynamic range (OECF)
Dynamic range is the maximum contrast a digital camera system can capture from its maximum output level (saturation) down to an even signal-to-noise ratio. A camera system's dynamic range is determined by obtaining the OECF (opto-electronic conversion function), which describes how a camera converts varying light levels (optical input) into electronic signals (digital output).
Dynamic range performance is essential for in-cabin systems, as these systems must be able to perform in low-light or even almost no-light scenarios when analyzing, e.g., facial expressions. ISO 14524 outlines test methods for obtaining the OECF for dynamic range measurements.
Noise (SNR)
Noise describes the presence of artifacts in the image that do not originate from the original scene content. The noise level is determined using the signal-to-noise ratio, which plots the amount of noise compared to the desired signal. Using the SNR can help reduce the noise in low-light environments. However, noise mustn't be lowered to the point where the signal is no longer clear. Often, a little noise is necessary, especially in low-light environments.
In-cabin systems must ensure a balanced SNR to detect the correct contrasts and signals in a low-light cabin. ISO 15739 describes test methods for determining a camera system's noise levels.
Color Accuracy
Color accuracy describes the camera's ability to separate and reproduce colors in a scene. To measure color performance, a camera system must be calibrated based on a particular color space, such as sRGB. Correcting white balance is also a crucial aspect of color accuracy.
Color is primarily relevant for human vision applications, as machine vision relies on NIR camera systems and does not see color. RGB camera systems must, therefore, be properly color-calibrated to ensure color accuracy.
Timing Accuracy
Camera systems rely heavily on timing parameters to capture real-time scenes, particularly moving scenes. Frame rate, shutter speed, and autofocus are vital timing parameters that should be tested to ensure the system is not misinterpreting a scene due to movement or motion blur.
Even though in-cabin systems monitor people sitting, they are still moving around—especially important is monitoring the driver's head movements—and the camera system must be able to compensate for these movements and motion blurs.
NIR camera testing
Ensuring that in-cabin camera and sensor systems function in low-light or almost no-light scenarios requires using near-infrared (NIR) systems. Most systems use internal illumination up to 850 (visible) or 940 nm (not visible). We recommend testing these systems using illumination and measurement devices to measure and generate an illumination spectrum up to 940 nm.
LE7 VIS-IR uniform lightbox
The LE7 VIS-IR lightbox is powered by iQ-LED technology, allowing you to generate custom spectra during testing. The VIS-IR version has a spectral range of 380 – 1050 nm for testing in the visible and infrared ranges. This device is optimized for testing with transparent targets, such as the TE269 grayscale target for dynamic range and noise measurements. The TE292 camSPECS plate deals with color calibrations and measuring spectral sensitivities.
iQ-Flatlights VIS-IR
Another option for low-light measurements of in-cabin systems is the iQ-Flatlights VIS-IR*. These uniform lights are powered by iQ-LED technology. They are ideal for testing when using reflective test targets such as those for resolution measurements or those involving real human targets (e.g., mannequin heads).
*Please note that the iQ-Flatlights VIS-IR version is only available as a custom solution.
LED-Panel IR
We recommend the LED-Panel IR to test the timing parameters of NIR camera systems. This device can evaluate the performance of all the most important timing parameters, including autofocus, frame rate, shooting time lag, exposure time, and rolling shutter speed. The IR version is available with a peak wavelength of 850 nm (NIR) and a viewing angle of 80° or a peak wavelength of 940 nm (NIR) and a viewing angle of 90°.
Setting up an in-cabin test in a test lab
One of the main options for in-cabin testing is to use a real car with built-in test targets to evaluate camera performance. This type of test is effective but requires extensive mounting solutions to ensure targets are stationed in the correct areas of the car (for example, setting up 2-5 targets at the proper height where people are sitting). Depending on your lab setup and whether low-light and bright-light testing is possible, adequate illumination must also be considered.
For labs that can't use a real car or those with limited space, we recommend setting up your lab in a dark room with the proper illumination equipment (e.g., LE7 and/or iQ-Flatlights) and test targets. What is still important in this test setup is the height and distances of the targets (e.g., the passengers) to the device under test, which should match those from a real car.
Testing these systems in a test lab constantly evolves, and new solutions are being developed. Contact our lab for a consultation on what test setup and combination of test devices would work best for your requirements.
camPAS testing – iQ-Lab
Our iQ-Lab offers in-cabin testing services through our camPAS (Camera Performance for Automotive Systems) tests. We offer camPAS to clients who need independent and objective test results from a neutral third party to support their development decisions. Our lab team will work closely with you to design a test based on your requirements, create the proper test setup in our lab, and test your in-cabin camera systems.
Conclusion
In-cabin automotive camera systems are at the forefront of vehicle intelligence, playing a vital role in safety and user experience. Their success hinges on precise image quality and reliable performance under diverse real-world conditions.
By understanding the image quality factors and their related international standards crucial for in-cabin testing, developers can ensure that they use the proper test equipment and procedures to provide a higher user experience while meeting regulatory safety requirements.
Contact our sales team for a test lab consultation or other details regarding in-cabin test equipment.