Standard Testing for IEEE-P2020

Test solutions for IEEE-2020 KPIs

IEE-2020 is the first internationally recognized standard that defines metrics and test methods to accurately evaluate the image quality performance of advanced driver assistance systems (ADAS).

We are very excited to announce that the IEEE-2020 standard was officially released to the public on March 21, 2025. Many of our engineers have actively contributed to this publication since its inception, and we look forward to finally sharing it with the industry. It is now available for download from the IEEE-2020 website.

IEEE-2020 is the first internationally recognized image quality performance standard for advanced driver assistance systems (ADAS). It defines metrics and test methods to evaluate these ADAS applications' performance accurately.

Metrics include:

Contrast Indicators
- Contrast Transfer Accuracy (CTA)
- Contrast Signal-to-Noise Ratio (CSNR)
Flicker (MMP)
Dynamic Range (CNR based)
Flare
Spatial Frequency Response (SFR)
Noise
Geometric Calibration

During the development of these metrics, the IEEE-2020 working group established definitions, test methods, and performance indicators to describe ADAS performance. Many of these metrics are rooted in traditional electronic imaging standards (e.g., ISO). Others, however, such as CTA and MMP, were established when it became clear that ADAS applications required new metrics to define performance. These new metrics are even more crucial when testing ADAS applications as these systems operate in rapidly changing environments and will be responsible for consumer safety.

For each metric, IEEE-2020 outlines test methods that can be used in a test lab to evaluate device performance. At Image Engineering, our R&D team closely followed the progress of the standard and developed testing equipment for each metric.

With IEEE-2020's official release, we look forward to assisting you in building an ADAS test lab. Check out our solutions for each metric below, or contact our sales team directly for more details.

Contrast indicator - Contrast Transfer Accuracy (CTA)

IEEE-2020 defines Contrast Transfer Accuracy (CTA) as how well the system under test can identify and separate different contrasts. CTA is a crucial KPI for ADAS applications when making adjustments to ensure consumer safety.

The graphic below shows a heat map that indicates where contrast transfer accuracy is accurate (yellow) and weak (purple).

We offer the VLS (Versatile Light System) solution for creating heat maps and testing CTA performance. This device allows you to create a dynamic test scene from the comfort of your test lab.

VLS solution for testing CTA performance

Contrast indicator - Contrast Signal-to-Noise Ratio (CSNR)

Contrast Signal-to-Noise Ratio (CSNR) refers to a camera's ability to separate contrasts by comparing the mean contrast to its standard deviation. As ADAS applications will encounter extensive contrasts, it's vital to understand how well the system can separate them and make proper adjustments.

The VLS (Versatile Light System) solution can be used to evaluate CSNR performance. This solution creates a dynamic test scene in the comfort of a test lab.

VLS solution for testing CSNR performance

Flicker - Modulated Light Mitigation Probability (MMP)

Modulated Light Mitigation Probability (MMP) or flicker is defined as how well the camera responds to flickering light situations, such as flickering broken road signs and oncoming traffic. The Vega light source generates flicker scenarios in a test lab using DC-driven technology. It is ideal for cameras with very short exposure times, such as those used in ADAS applications.

Using the Vega device to evaluate flicker performance

Dynamic Range

Dynamic range is an essential image quality factor for all electronic imaging industries. In IEEE-2020, the dynamic range is based on CNR (contrast-to-noise ratio) and is defined as the camera's ability to capture both bright and dark sections of the scene simultaneously. This KPI is crucial for ADAS applications that operate in vastly different lighting environments.

The image below shows a heat map that indicates where the dynamic range performance of a scene depending on the luminance.

We offer the VLS (Versatile Light System) and specifically designed grayscale targets for measuring dynamic range according to IEEE-2020.

VLS solution for testing dynamic range performance

Camera flare

Flare is unwanted light in an image that does not originate from the related scene object. It is one of the more difficult KPIs to test as it can be very random depending on the environment or the camera hardware.

Our R&D team is in the final stages of developing an advanced solution for testing flare. Contact our sales team for more information.

Spatial Frequency Response (SFR)

Spatial Frequency Response describes the resolution or details of a scene. This KPI is crucial for automotive applications, which are required to separate and identify specific objects in a scene. SFR is often more subjective in these settings as it is used to recreate images for the consumer (e.g., backup cameras).

IEEE-2020 outlines test methods that use slanted-edge test targets. We offer various slanted-edge targets, such as TE283C and other multipurpose test charts.

A slanted edge test chart for SFR measurements

Noise

Noise is defined as unwanted artifacts that appear in the digital output of the camera system. In automotive applications, too much noise can lead to false identifications or missed objects in a scene.

High Dynamic Range (HDR) scenes, such as those experienced by automotive systems, can lead to more noise. We use advanced OECF test targets, such as TE269 (shown above), to evaluate unwanted noise in a test scene.

Geometric Calibration

ADAS applications need to map the scene they are seeing to make proper readings and adjustments. As a result, these systems need to be geometrically calibrated to ensure accuracy.

Traditional geometric calibration methods use test targets for validation. However, this test method often requires multiple test targets in various sizes and relay lenses. This setup needs extensive lab space for proper calibration. Our R&D team developed an alternative approach using the GEOCAL. GEOCAL is a compact device that uses a beam-expanded laser and a grid of light spots originating from infinity instead of test targets.

Need help with testing according to IEEE-Standard 2020?

Find the right products and services for testing according to IEEE-Standard 2020.

Products & Solutions

We offer a variety of products and solutions for testing according to IEEE-Standard 2020. Find the best devices for your test lab.

See products

Test Services

Get professional camera testing with our image quality test lab, one of the world's largest independent camera test labs.

See Test Lab

International Standards

Test solutions for IEEE-2020 KPIs

Metrics include:

Contrast indicator - Contrast Transfer Accuracy (CTA)

Contrast indicator - Contrast Signal-to-Noise Ratio (CSNR)

Flicker - Modulated Light Mitigation Probability (MMP)

Dynamic Range

Camera flare

Spatial Frequency Response (SFR)

Noise

Geometric Calibration

Geometric Calibration

Need help with testing according to IEEE-Standard 2020?

Products & Solutions

Test Services

Further Readings

Image Quality Factor: Contrast Signal-to-Noise Ratio (CSNR)

Image Quality Factor: Flicker

Image Quality Factor: Contrast Transfer Accuracy (CTA)

Image Quality Test Lab: Testing Image Quality

Image Quality Test Lab: Automotive Testing

Geometric Calibration