Delivering Enterprise-Grade One-to-Many Face Recognition at Scale
Biometric identification systems have become crucial for both governments and enterprises that need to confirm individual identities and uncover fraudulent attempts to gain identity privileges. Many of these biometric systems are small: for example, your smartphone only needs to manage one user identity, or at most the identities of a few users. Other biometric systems manage tens of millions of identities or even more.
In the past, large-scale biometric identification systems were based on fingerprint biometrics, or sometimes on iris biometrics. For example, the Aadhaar system in India, the world’s largest biometric system, is primarily based on fingerprint and iris biometrics. Historically, fingerprint and iris biometrics were used in these systems because those modalities were significantly more accurate and suited to scale than face recognition.
However, in the past few years, face recognition systems evolved significantly and are now on a par with or more accurate than systems based on other biometric modalities, and are becoming more popular in government and enterprise applications.
This white paper explores some of the reasons why face recognition is now suitable for large-scale government and enterprise-grade applications, and how newer face recognition systems can accurately search databases with tens of millions of records, and can perform hundreds of matches per second. The paper discusses the following:
- How newer face recognition systems can scale to process searches on tens of millions of records.
- How newer face recognition systems can maintain biometric accuracy even at scale.
- The new use cases that face recognition systems can support.
- The necessary components for an enterprise-grade face recognition system.
Before delving into the capabilities of newer enterprise-grade face recognition systems, this paper will discuss critical terms used when discussing biometric recognition.
The first topic that must be addressed is the idea of identity—who a person is—and the ability to confirm that identity. A person’s identity may be confirmed by one or more of the following authentication factors:
- Something you know, such as a password.
- Something you have, such as a passport or driver’s license.
- Something you are, such as the features of your face.
- Something you do, such as using a finger to swipe particular patterns on a screen.
- Somewhere you are, or your geolocation.
Individuals can be uniquely identified by one of these factors, or by several factors in combination. Biometrics (“something you are”) such as face recognition are one of the tools used to uniquely identify an individual.
While biometrics may be captured in image form, they are not processed as images. Instead, biometric systems convert the original images to templates. Templates provide two benefits over the original images.
- Faster results due to smaller storage space and improved transmission time. In modern face recognition systems, a template is simply a vector that is on the order of 1 kB in size, vs. 100 kB for the original image.
- Improved privacy due to retention of only the data needed to identify individuals. While a compromised face image can be used by anyone to identify an individual, a compromised proprietary template is much more difficult to reverse engineer, and even then cannot be used to re-create the actual appearance of a person.
Two other key terms are verification and identification, which require vastly different biometric methods and systems.
Verification (one-to-one, i.e. 1:1) answers the question “Are you the person that you claim to be?” If a person claims to be Leonardo DiCaprio, and Mr. DiCaprio’s biometrics have been stored, then the person’s face, fingers, iris, or other biometrics can be compared to the stored biometrics for Leonardo DiCaprio to prove or disprove the asserted identity. Verification only requires comparison against a single biometric template, and is also known as a one-to-one or 1:1 search.
Identification (one-to-many, i.e. 1:N), on the other hand, answers the question “Who are you?” If I am suffering from amnesia, then my biometrics can be compared to the stored biometrics for missing persons. Hopefully, my biometrics will match those of a known missing person, which would allow my identification. Identification requires comparison against multiple biometric templates, and is also known as a one-to-many or 1:N search.
When examining how artificial intelligence is applied to biometrics, it is worthwhile to consider how AI defines the terms training, inference, algorithm, and model. As described later in this paper, model training iteratively analyzes and detects patterns in data, resulting in a model which is often colloquially referred to as an algorithm even if that is not technically correct. Inference takes the algorithm and analyzes new data to produce insights.
Finally, when examining independent evaluations of face recognition accuracy, there are four terms that need explanation.
- Used in one-to-one (verification) searches, the False Non-Match Rate (FNMR) is the proportion of comparisons that incorrectly result in a non-match when they should have resulted in a match.
- Used in one-to-one (verification) searches, the False Match Rate (FMR) is the proportion of comparisons that incorrectly result in a match when they should have resulted in a non-match.
- Used in one-to-many (identification) searches, the False Negative Identification Rate (FNIR) is the proportion of searches incorrectly resulting in a non-identification when they should have resulted in a positive identification.
- Used in one-to-many (identification) searches, the False Positive Identification Rate (FPIR) is the proportion of non-mated searches incorrectly producing one or more candidates when they should have resulted in none.
Typically, biometric systems are evaluated by establishing an acceptable false positive rate (FMR or FPIR for 1:1 or 1:N, respectively), and then mapping the threshold to a resulting false negative rate. FPIR is inherently more challenging than FMR because the possibility for false positives increases as the number of comparison points (i.e. database size) increases.
For instance, in the ongoing National Institute of Standards and Technology (NIST) Face Recognition Vendor Test (FRVT), the “Visa-Border” 1:1 test uses a 10-6 (1 in 1 million) FMR, which results in a 0.0027 FNMR for Paravison. For 1:N, NIST FRVT benchmarks FPIR at 0.003 on a database of 1.6M records, for which Paravision delivers an FNIR of 0.0038.
Scaling face recognition systems
Systems that perform identification are necessarily more complex than systems performing verification, since a biometric template has to be compared against a database of biometric templates. The system needs to evaluate the results of all of these comparisons, and usually ranks the results of these comparisons in the order of matching probability.
The first face recognition systems in the 1990s were capable of identification, but were incapable of performing identification on databases of tens of millions of individual records. The computer hardware and architectures of the day were incapable of supporting these large workloads. Even later systems (as of 2010) included algorithms with low accuracy, insufficient to create useful results.
Since that time, systems have improved in the following ways:
Artificial intelligence has dramatically improved face recognition accuracy. These improvements over the last decade, and even over the last three years, will be discussed in more detail later in this paper.
Computing architectures and hardware components have improved. Previous computers and on-premise computing models have given way to accelerated parallel processors, support of elastic, horizontal scaling to increase system capabilities, and virtualized component-based architectures, including Docker and Kubernetes. These improvements support managed distribution of the computing workload among devices.
Computing can take place beyond the premises. Systems are no longer limited to a single computer room. Where needed, cloud computing enables deployment of systems of unlimited size that are not limited to a single building or even a single geographic location. As computing needs increase, system managers can “spin up” additional server instances in near-real time.
All of these improvements have dramatically increased the scalability of face recognition systems. As described below, the improvements not only contribute to improved processing speed, but also to improved accuracy.
Providing accurate face recognition
One reason why early biometric systems relied on fingerprint and iris biometrics was because face recognition was not that accurate.
Even as late as 2010, face recognition systems were not as accurate as other biometric systems. In that year, the National Institute of Standards and Technology (NIST) reported (NIST Interagency Report 7709) that for the best-performing face recognition algorithm “the chance of identifying the unknown subject (at rank 1) in a database of 1.6 million criminal records is about 92%.”
NIST testing in 2018 provided dramatically more accurate results. NIST Interagency Report 8238 demonstrated that “the most accurate algorithms (found) matching entries, when present, in galleries containing 12 million individuals, with error rates below 0.2%.” These accuracy rates have continued to improve (in some cases with more than ten-fold reductions in errors since 2018) as NIST conducts its ongoing Facial Recognition Vendor Tests, and as organizations continue to submit newer algorithms.
Why did face recognition accuracy improve so dramatically over the past decade?
The introduction of artificial intelligence (AI) and machine learning (ML) to face recognition has resulted in dramatic accuracy improvements, as AI and ML experts have applied their knowledge to solving face recognition problems. As part of this, the experts have applied model training and inference to face recognition.
- The first component of AI, model training, requires an iterative process of analyzing a large amount of data, detecting patterns in that data, and generating an algorithm for that kind of pattern detection that creates a model. The model is checked for accuracy, and the process is repeated.
- The second component of AI, inference, takes the model (often referred to colloquially as an “algorithm”) generated by training and analyzes new data to produce insights. In some cases, these insights can form subsequent inputs to the iterative process described above.
As the AI and ML experts continuously apply model training and inference to face recognition, accuracy continues to improve. The graphic below documents how AI and ML have improved the accuracy of one vendor’s (Paravision’s) algorithm. It displays a dramatic improvement in the false negative identification rate (FNIR) over the course of just three years. These results are taken from the National Institute of Standards and Technology (NIST) Ongoing FRVT 1:N Identification test, results of which are continuously updated on the NIST website.
Applying face recognition to use cases
These improvements in face recognition accuracy are coupled with other benefits of face recognition.
- Ease of acquisition. Faces are by far the easiest biometrics to acquire. They can be acquired at a distance, when a person is moving, in either visible or infrared light, and (with proper camera placement) without requiring any special effort or interaction from the person.
- Ease of deployment. Most face recognition systems do not require any special capture equipment, and can work with standard still or video cameras supporting a variety of formats. Many entities already have the necessary cameras, and well-designed face recognition systems can integrate with these existing camera systems.
Face recognition supports use cases not easily supported by other biometrics.
- Enterprise physical access control. Fast, frictionless, touchless and secure facility access, from compact biometric readers that work with existing access control infrastructure. Face recognition enables highly secure biometric authentication from users who are in motion and at-a-distance.
- Identity Verification (IDV) and Know Your Customer (KYC). Digital identity verification has exploded in popularity as a way of enabling self-service and real-time onboarding for financial services, gig economy, and government programs that demand strong identity. Face recognition is uniquely appropriate here due to availability of high quality smartphone selfie cameras to capture and match with faces on identity documents.
- Travel and Borders. On every passport, state ID, and drivers license, the face is the ideal identity credential for travel and border. Meanwhile, in the airport or other border crossing, the ability to deliver fully touchless, intuitive, and fast biometric authentication makes face recognition the right tool for the job.
Six characteristics of a superior enterprise-grade face recognition system
Regardless of the particular applications, there are six key features that a superior enterprise-grade face recognition system requires.
- The face recognition system must deliver outstanding accuracy in challenging, real world environments and across demographics including age, gender, and race. The system vendor must provide transparency in performance with statistically significant, real-world benchmarks. NIST FRVT is singular in this regard. Other benchmarks, such as LFW, do not offer sufficient transparency and / diversity and challenge of data.
- The face recognition system vendor must be committed to ethical development and deployment of the technology, ensuring that the face recognition system has been developed and tested in the right way, and that it will not be deployed in such a manner that is contrary to privacy or other usage laws or standards.
- The face recognition system must support a variety of deployment models, including on-premises, cloud, mobile, edge, and hybrid solutions. Again, enterprises have taken advantage of the wide variety of deployment solutions available today. A face recognition system must operate in on-premises, cloud, mobile, edge and hybrid environments, and must support the unique security requirements of each environment for both enterprise and government users.
- The face recognition system must be based upon open systems standards. Almost all face recognition systems are integrated into other systems, such as camera networks and security systems. Therefore, the system must easily integrate with these other systems.
- The face recognition system must work with a variety of common image formats. Enterprises have sunk significant investments into existing systems, and do not want to replace these systems just to satisfy the needs of a new face recognition solution.
- The vendor of the face recognition system must be committed to continuous improvement of its solution. This includes a commitment to improving accuracy and performance, and a willingness to incorporate new hardware and software technologies into its solution as they are developed. Ideally, the vendor’s customers will realize continuous accuracy and speed benefits as the vendor improves its artificial intelligence, architecture, and other capabilities.
Deploying 1:N Face Recognition with Paravision
From Cloud to Edge, Paravision’s comprehensive face recognition product suite offers all of the tools necessary to develop and deploy mission-critical biometric identification and authentication solutions across a wide range of applications, including identity verification, physical access control, air travel and government services. Paravision’s toolset is modular and interoperable, enabling partners to develop the specific solution that meets their customers’ needs.
Paravision face recognition has been optimized for leading chipsets, from Intel CPU to NVIDIA GPUs to Ambarella SoCs, supports leading operating systems including Windows, Linux, and iOS, Android, and can be deployed at any level, from SDK to API-driven Docker containers.
Here, we will look at 3 different use cases where 1:N face recognition is useful, and how Paravision’s product suite can be integrated to meet the use case need.
- Enterprise access control
- Identity verification
- Border management
The application of AI and ML to face recognition, as well as other technological advances, have expanded the use of face recognition to a variety of use cases that cannot be accomplished with other biometric modalities.
Enterprises can implement these new capabilities to improve their business results, increase customer satisfaction, and ensure that the enterprises are operating properly.
Paravision can help enterprises realize these goals, regardless of the size of the enterprise database or the number of transactions the enterprise performs. Please contact Paravision for a more detailed analysis of your specific needs.