White Paper: NIST FRVT 1:N Face Recognition Performance Assessment

**Find the 2022 NIST FRTE (formerly FRVT) Performance results here: Paravision’s Face Recognition Ranks as the Most Accurate in the World in Latest NIST FRTE (formerly FRVT) 1:N Report

Face Recognition Performance Assessment

NIST FRTE (formerly FRVT) 1:N Identification Report

Performance Assessment Background

On March 24, 2020, the National Institute of Standards and Technology (NIST) released a new report on the performance of leading face recognition algorithms. The report showed Paravision’s strong and continually improving performance across several metrics, earning top ranks in particular on NIST’s threshold-based matching and identification mode assessments.

NIST routinely evaluates face recognition algorithms on how well they can verify two pictures of the same person (1:1 Verification) and search through large collections of pictures to match an unknown face with an existing identity (1:N Identification). The NIST assessment summarized in this document evaluates algorithms on their 1:N performance. 

While the rankings published by NIST separate multiple algorithm versions from a given vendor, the tables in this report track company-level face recognition performance, which is measured by each company’s top rank (across all of their submitted versions) in any individual NIST metric. This document focuses on identification mode and threshold-based matching rather than investigation mode as the former metrics are more relevant to Paravision’s user base.

Face Recognition Performance Highlights

Paravision consistently performed in the top tier when benchmarked against a variety of domestic and international competitors, demonstrating leadership in multiple categories of testing. Some face recognition performance highlights include:

• #1 in the US in 8 of 9 threshold-based matching categories
• Top 3 worldwide in identification mode accuracy
• Top 3 worldwide in threshold-based matching accuracy
• #1 worldwide in profile (side view) matching accuracy

With these results, Paravision is the only company globally to be ranked in the top 5 for NIST FRTE (formerly FRVT) 1:1 and 1:N (Identification and Threshold-Based Matching) as of March 2020.

Face Recognition Performance Breakdown

Face recognition performance accuracy is judged on two broad metrics: Type I errors (false positives) and Type II errors (false negatives). False positives occur when a new face is incorrectly matched with a known face in the dataset; false negatives occur when a known face is not matched with its correct identity.

The NIST report’s Identification Mode and Threshold-Based Matching scores both assess combined low False Positive Identification Rates (FPIRs) and False Negative Identification Rates (FNIRs). 

Identification Mode is a subset of Threshold-Based Matching at FPIR=0.01. Strong performance on these tests shows an algorithm can process a high volume of face submissions accurately with minimal human review. Paravision scored in the top 5 worldwide in 8 of the 10 datasets below, while remaining the best performer in the US in all but one category.

Threshold-Based Matching Company Rankings

Identification Mode Company Rankings

Profile Matching Performance

Paravision performed especially well in NIST’s profile matching tests, where faces turned away from, or perpendicular to the camera could still be identified. Paravision scored #1 worldwide in the profile dataset in investigation and identification mode, and in two of the three threshold-based matching categories. 

While these capabilities are still in their nascent stages, the NIST report notes that “a few algorithms correctly match side-view photographs to galleries of frontal photos, with search accuracy approaching that of the best c. 2010 algorithms executing frontal-frontal search.

The capability to recognize a 90-degree change in viewpoint – pose invariance – has been a long-sought milestone in face recognition research.”

While Paravision scores highly, there remains room for improvement in matching profile-pose faces. At a highly constrained FPIR of 0.001, Paravision’s algorithm has a 39% FNIR – a result which may be too high to be usable across multiple applications despite being the #1 score worldwide. At FPIR = 0.01 and 0.1, Paravision delivers much more usable FNIR, at 18% and 11%, respectively. Paravision’s closest competitors have error rates nearly twice those.

Competitor Performance in Profile Threshold-Based Matching

Recognizing Aging Faces

As people get older, changes to the face’s skin, muscle, and bone structure shift their appearance, sometimes substantially. These changes tend to become more extreme as age progresses. NIST notes that “this is a gradual process affecting all human faces and, absent surgical intervention, is essentially irreversible over long time-scales. Aging increases false negative rates. In some applications aging effects are avoided by policy: faces are re-enrolled periodically. In other applications, this is not possible.” Effectively matching aging faces minimizes the number of re-enrollments needed, and makes applications that don’t support re-enrollment—like surveillance—operable over a longer time period. 

NIST measured the ability of algorithms to match search photos older than enrollment photos by 0 to 18 years, in two-year increments. Paravision achieved a top 3 rank worldwide, and is #1 in the US across all age categories, showing its algorithm’s lasting performance even with extended time gaps between search and enrollment photos.

Identification Mode Company Rankings by Number of Years Between Enrollment and Search Photos

Investigation Mode Company Rankings by Number of Years Between Enrollment and Search Photos

 

Performance Improvement over Time

Altogether, this NIST report shows how dramatically Paravision’s algorithms have improved in a short time. In the two years since Paravision first submitted algorithms to NIST, it has reduced its false-negative identification rates by up to 95%.

Paravision Face Recognition Performance – Threshold-Based Matching

NIST FRTE (formerly FRVT) 1:N March 2020 – False Negative Identification Rate (FNIR) – lower is better

Paravision Face Recognition Performance – Identification Mode

NIST FRTE (formerly FRVT) VT 1:N March 2020 – False Negative Identification Rate (FNIR) – lower is better

 

Over the past year and a half, Paravision has progressed dramatically, eclipsing both its competitors and its past scores. With NIST’s latest results, Paravision has shown its leadership in both the 1:1 and 1:N categories evaluations.

The company’s performance underscores its commitment to building computer vision that goes beyond human vision. Paravision will continue building industry-leading face recognition for its customers and will bring that same level of diligence to all future NIST evaluations.

**Find the 2022 results here: Paravision’s Face Recognition Ranks as the Most Accurate in the World in Latest NIST FRTE (formerly FRVT 1:N Report

 

For more information visit paravision.ai/benchmarks/ or book a meeting at paravision.ai/contact.

Paravision