Biometric Face Comparison
Face comparison (formerly Face matching) is the process of determining whether two images of faces belong to the same person. This is used in identity verification scenarios to compare the face of ID document holder and the portrait on the document.
Another use case is second factor authentication for login, when the user presents his face to be verified with a previously stored face image of him.
Components providing face comparison
Face comparison is provided by mobile app libraries:
Note: On-device face comparison should be used only in special cases in offline mode when connectivity to DIS is not possible. Face comparison in DIS provides higher accuracy, leading to better UX and higher security.
The Digital Identity Service (DIS) provides the face comparison functions in two ways. Either as dedicated face API functions, or as part of the onboarding API.
| Onboarding API | Face API |
|---|---|
Face comparison is performed when both the customer’s selfie and document image is uploaded, and the inspect function is called. This matches the document portrait with the selfie. | Face comparison is performed when the face photo is provided and a reference photo is in the similarity function. |
| DOT Digital Identity Service - Customer Onboarding | DOT Digital Identity Service - Face Biometrics |
| Swagger: customers/{id}/inspect API call | Swagger: faces/{probe_face_id}/similarity API call |
Innovatrics face biometric algorithm ranks among the top in the NIST FRVT
Comparison steps
In order to compare two faces, the following steps are performed:
- Face detection - find the position of the face in the image
- Template extraction - compute representation of the face used for comparison
- Comparison - compare two face templates and output a similarity score
Face detection
The first step when performing face comparison is face detection. This is an important step, because there might be no face or multiple faces present in the picture. Once a face is detected, it can be used in the comparison process. There are various face detection modes available, fast mode provides lower latency, in accurate mode detection is more precise. Mobile devices only support fast mode, server components are configured by default to accurate mode.
Template extraction
Once the face has been detected, the face template can be generated. These templates can be cached on the application level to speed up the comparison. Once the reference image is uploaded to the server, the template can be generated and cached. When a user logs in and uses face as a second factor, face detection and extraction is only performed on the probe image from the user and the reference template is pulled from the cache. Only templates generated by the same mode and product version can be matched. During major product upgrades, templates must be regenerated as mentioned in the respective product changelog.
Comparison (formerly Matching)
Comparison calculates the similarity of the two templates, providing a comparison score. The higher the score, the more similar are the faces. This is usually a very fast operation.
Comparison threshold and comparison decision
The final comparison decision if the two face images belong to the same person is determined by the comparison score and the threshold. If the score is above the threshold this can be interpreted as a match, if the score is below the threshold it is a non-match.
Comparison accuracy
Comparison thresholds affects the decision mistakes. False match is when comparison decides that face images of two different persons are matching. False Match Rate FMR (formerly FAR) is the proportion of comparison trials that result in a false match on a given dataset with a given threshold. False non-match is when comparison decides that two face images of the same person do not match. False Non-match Rate FNMR (formerly FRR) is the proportion of comparison trials that result in a false non-match on the same dataset and with the same threshhold. The combination of these two FMR @ x% FNMR defines the accuracy of comparison.
Thresholds for comparison of ID document portrait vs selfie (Digital Identity Service)
The following characteristics have been measured on our ICAO face quality testing dataset using DOT Digital Identity Service (accurate extraction mode):
| FMR levels | FMR [%] | FNMR [%] | Score threshold |
|---|---|---|---|
| 1:1000 | 0.10 | 0.95 | 0.325 |
| 1:5000 | 0.02 | 1.5 | 0.39 |
| 1:10000 | 0.01 | 1.87 | 0.465 |
Example
If we require a FMR rate of 0.02 with Digital Identity Service, we have to set the threshold of the comparison score to 0.39. If we have a representative set of 10,000 mated face image pairs (slefie and of the same person), statistically 150 will in this case be incorrectly decided as a non-match. If we have 10,000 not mated face image pairs, statistically 2 will be wrongly decided as a match.
We recommend following comparison thresholds based on the use case:
- 0.325 Low quality input images or when expecting significant appearance change of individuals (person is younger on an ID card issued in the past)
- 0.35 Standard setting for comparing selfie with document portrait
- 0.39 Comparison aiming for high security
Thresholds for comparison of ID document portrait vs selfie (DOT Face library)
The following characteristics have been measured on our ICAO face quality testing dataset using the DOT Face mobile library for Android and iOS (fast extraction mode):
| FAR levels | FAR [%] | FRR [%] | Score threshold |
|---|---|---|---|
| 1:500 | 0.200 | 0.867 | 0.276 |
| 1:1000 | 0.100 | 1.205 | 0.297 |
| 1:5000 | 0.020 | 2.483 | 0.358 |
Setting the correct threshold depends on the security/convenience balance that is required for the specific use case. We recommend to use server side face comparison whenever possible as it has significantly higher accuracy, leading to better security and user experience. Please note that if images are of very low quality (e.g. selfie in a dark room, worn off document portrait) comparison score can be lower even for mated pairs and threshold must be adjusted accordingly.
During the initial configuration of the system, two thresholds can be set. If the score is below the bottom threshold, the result is automatically set to reject. If the score is above the top threshold it is automatically accepted. If the score is between the two thresholds, images go for review to a back office operator for a final decision. Over time, the range between the two thresholds can be minimized or even eliminated by setting a single decision threshold if not needed anymore.
NOTE
To add face comparison to your workflow, please consider the following:
- Image quality - if image quality is low, accuracy of the comparison decreases
- Age difference between the images - time difference between the capture of the two images is several years, the person’s appearance might have changed significantly.
Image vs template usage
When performing face comparison using images, face detection is always called internally and a template is generated. When using templates, face detection is skipped.
Using images
If you do not need the result of the face detection for other purposes, you can simply invoke face comparison with images. This is particularly useful when comparison is performed only once during the flow. An example would be a simple selfie vs identity document face comparison.
Using templates
If you need more data about the face, such as age estimation or passive liveness, the recommended approach is as follows:
- Invoke face detection with all the needed attributes and also with template extraction enabled
- Cache this template
- Use it for comparison
This approach can be used when we want to evaluate passive liveness and also perform face comparison. Calling verify with at least one template reduces processing time.
Templates can also be cached on the application level for use cases like login, where the same reference face is needed. Please note that templates are incompatible across major product upgrades, and must be regenerated by invoking the face detection on the source images again.