What is flicker in image quality?
Light flicker is defined as a periodic change to the brightness or intensity of a light source. This change can be intentional, e.g., flashing light from an emergency vehicle, or unintentional, e.g., light fluctuations from digital road signs. As a result of its changing nature, flicker can be challenging to quantify in a test setting. Nevertheless, with the continuing advancement of cameras and sensors in specific industries, such as advanced driver assistance systems (ADAS), other automotive systems, and security, it is imperative to recreate and test flicker in a lab setting.
Modulate Mitigation Probability (MMP)
In traditional camera testing, camera systems are generally not tested for flickering light performance. Other cameras, such as automotive systems, must perform at the highest level in any illumination scenario to ensure safety standards are met. Unfortunately, there is currently no internationally recognized standard for flicker performance testing.
However, the IEEE-P2020 working group is in the process of creating the first international standard for ADAS. This group is made up of industry experts from a variety of fields. Many of our dedicated automotive camera testing engineers are active members. One of the KPIs they are developing is modulation mitigation probability (MMP). MMP will serve as the main KPI in the standard for testing the camera system's response to flickering light.
How does flicker occur?
Flicker often occurs from LED-based light sources using pulse width modulation (PWM) technology to control light intensity. Flicker is a temporal light modulation (TLM) of the light that hits a camera sensor. As that sensor has a discrete sampling of light intensity (exposure time), the combination of the light turning on and off quickly and the capturing process can lead to unwanted artifacts.
In general, the number of observable artifacts in an image or video due to flickering light sources depends on the frequency and duty cycle of the PWM used in the light source and the phase shift between the light source and the capture rate of the camera.
In the automotive industry, flickering light is broken down into two categories, reflectance flicker, and illuminant flicker.
Reflectance flicker refers to a flickering light source that illuminates the scene but is not within the camera's field of view (e.g., illumination in a parking garage). We evaluate reflectance flicker using the flicker modulation index (FMI).
$$FMI=100 \frac{x_{max}-x_{min}}{x_{max}+x_{min}}$$
xmax= The maximum signal measurement of the PWM light source captured by the image sensor for the entire captured video sequence
xmin= The minimum signal measurement of the PWM light source captured by the image sensor for the entire captured video sequence
Illuminant flicker refers to a flickering light source that illuminates the scene within the field of view of the camera (e.g., headlights of an approaching vehicle). We evaluate illuminant flicker using the same FMI equation seen above. Illuminant flicker, however, can often cause various HDR artifacts to occur in the scene. Thus, we use the flicker detection index (FDI) to evaluate the likelihood that the captured image will have enough contrast between the LED light and a light reference level for a given camera, duty cycle, etc.
$$FDI=Prob\left(\frac{x_{meas}-x_{ref\text{_}off}}{x_{ref\text{_}off}}\ge \tau \right)$$
Prob= empirical probability / xmeas= measured flickering signal level / xref-off= reference background ("off") light level / τ= flicker contrast threshold
While the FDI equation is a good indicator of flicker detection, it is not as accurate if the system must reproduce the light source signal level, typically required for automotive-grade cameras. Therefore, we utilize the modulation mitigation probability equation developed under the direction of the IEEE-P2020 working group. This equation will provide the likelihood that the captured image will accurately reproduce the light source signal level when compared with a reference light source.
$$MMP=Prob\left[ \overline{x_{ref\text{_on}}}\left( 1-\delta\right) \lt x_{meas} \lt \overline{x_{ref\text{_on}}}\left( 1+\delta \right)\right]$$
Prob= empirical probability / xmeas= measured flickering signal level / xref-off= reference expected light level / δ= acceptable threshold level
The importance of testing flicker
Light flicker can affect almost all industries that rely on cameras and sensors, but it's most evident in the automotive and security sectors. Flicker is common in these industries because the surrounding environments constantly change and produce a broad range of lighting situations. The camera and sensor systems in these industries need to mitigate the modulation of the light source (flicker) to adapt to the changing environment to ensure high performance and safety. For example, an ADAS system must always react correctly even with an inconsistent light flicker to avoid a dangerous driving situation.
Industry engineers understand the importance of flicker but find it difficult to test different flicker situations in the test lab reliably. As a result, our engineers at Image Engineering have developed various hardware and software solutions that can accurately reproduce light flicker in a test lab setting and then evaluate the camera's response to the flickering conditions.
Testing flicker in a test lab
Flicker has proven to be one of the more challenging camera system KPIs to recreate in a test lab setting. Cameras with short exposure times often experience stability problems due to inadequate testing devices. At IE, we have designed various illumination devices that can accurately simulate flickering scenarios for more reliable test results. Remember that a proper device must allow you to control the light source's intensity, frequency, duty cycle, and phase.
Once you have reliable test equipment, the next step is properly setting up your test scene. When testing reflectance flicker, we recommend setting up the light sources right outside the field of view of the device under test (DUT). For illuminant flicker, the light sources should be within the direct field of view of the DUT.
Tools for testing flicker
Testing flicker with the LG3 and TE269 test chart
The LG3 illumination box is the first lightbox on the market with flicker capabilities. Based on iQ-LED technology, the LG3 greatly expands the test areas for dynamic range measurements with the ability to illuminate a high contrast test target with over 150,000 lx. It is used with transparent test charts such as the TE269 test chart.
The TE269 test chart follows not only ISO 14524 and ISO 15739 but also IEC 62676-5, the first international standard regarding the measurement methods and descriptions of security and surveillance cameras. The chart is suited for high dynamic range measurements of 1,000,000:1 / 120 dB.
The flicker mode has a range of 10 to 500 Hz and variable duty cycle, meaning it can simulate varying flicker frequencies, e.g., those of traffic and streetlights. There is also a mode of 32 kHz/128 kHz and variable duty cycle.
The LG3 has proven to be a solution for accurately recreating different flicker situations in a test lab environment with these advanced capabilities.
Vega - a DC-powered light source
The Vega light source is a powerful solution for high precision measurements of the exceedingly short exposure times often seen in automotive-grade cameras. Vega uses thermally stabilized LEDs driven by DC (direct current) technology making it one of our most sophisticated and unique light sources.
Vega has exceptionally high stability that provides a previously unattainable consistency for measuring these types of cameras with very short exposure times. We actively stabilize the temperature so that the DC driver does not affect temperature regulation, and we can achieve temperature stability within half a degree. The temperature system works both ways (i.e., heating and cooling) and will remain consistent even when turning on/off the light output or changing the intensities.
Vega generates flicker from a wide frequency range. We have further developed the low-frequency functionality to include sine, triangular, and square waveforms, covering nearly all real-world scenarios.
Finally, Vega utilizes uniquely designed grayscale test charts optimized for system evaluations with very short exposure times.
Testing MMP with the DTS
The Dynamic Test Stand (DTS) is an ADAS testing solution for various KPIs (Key Performance Indicators), including MMP (flicker). The DTS can also test for the other KPIs outlined in the IEEE-P2020 standard, such as contrast transfer accuracy (CTA) and color separation probability (CSP).
The MMP mode of the DTS uses six contrast boxes based on Vega technology to check the camera system's response to modulated light across a dynamic range of 120 dB. Overall, the DTS has 216 unique luminances that can be tested. There are also four single LED lights for measuring the system's response to local flicker.
With a DTS, you can optimize the process of performing a regression test during the tuning mitigation functionality of a camera system.
The DTS also has built-in analysis software for analyzing the results once the cameras or sensor has undergone testing. The analysis will provide details on where the system must be improved. The MMP evaluation module is the only available analysis software for flicker on the market.
Flicker testing in the IQ-Lab
For over 25 years, our clients have been using the IQ-Lab for their image quality testing. When you send us your cameras, we will provide you with objective third-party results that you can trust. We always follow international standards when testing, and many of our test engineers participate in the working groups in updating and developing the standards.
When we test cameras in the IQ-Lab, we use the same test equipment that we offer to our clients. Automotive camera testing is no different; we usually use a combination of devices to test for flicker performance.
For example, in a camPAS test, we evaluate the flicker performance of automotive cameras and sensors using the Vega light source and grayscale test charts. This combination allows us to efficiently assess flicker performance in a test lab, thus significantly reducing the amount of test ride kilometers needed before the camera is brought to the market.
For questions and inquiries about our IQ-Lab test services, don't hesitate to contact , and we are happy to help you get started.
Conclusion
Flicker is a periodic change to the brightness or intensity of a light source and often has a negative impact on the performance of a camera and sensor system. In general, systems from the automotive (ADAS) and security industries frequently experience problems due to the inconsistencies in their environments.
When an ADAS camera, for example, encounters performance problems due to flicker, we will experience an unsafe driving situation. As a result, these systems must be thoroughly tested and evaluated in a test lab before making them available to the public.
Until recently, it wasn't easy to simulate flicker in a test lab setting until recently. However, our engineers have worked hard to develop multiple flicker performance testing solutions. Our test lab also uses each device we offer to the public for various flicker testing scenarios.
Are you searching for a flicker testing solution for your test lab? Before committing to a test system, we offer the opportunity to visit our headquarters, where your team can test your specific use case to ensure the test system is right for you. We can help you get started; contact our support team today for a consultation.