Face Auto Capture

Face auto capture is a process of capturing the face image suitable for biometric checks without the need of manually trigger the capture action. Image is captured automatically, when all requirements are satisfied.

Face auto capture is provided by the following components of the mobile app libraries.

Face Auto Capture

Image quality

Image quality is important aspect in face recognition. At the same time being too strict during the face capture can impact user experience. Therefore, the application should only require sufficient quality inputs for the specific use case in question. For example login should be quick, requiring only basic adjustments as opposed to passport quality image capture, where correct lighting and background uniformity are required. Quality can be controlled by various quality attributes but to simplify integration, pre-configured quality providers are available to cover most common use cases.

Quality Providers

There are three pre-defined quality providers, starting with the easiest for the user’s perspective.

  • Verification quality provider
  • Passive liveness quality provider
  • ICAO quality provider

Verification quality provider

Verification quality provider captures a face that is suitable for verification. This ensures face is in the center of the image and poses minimal constraints for the user.

Passive liveness quality provider

Passive liveness quality provider ensures that captured face has enough quality to fulfil passive liveness conditions and passive liveness can be evaluated. When you use this provider face capture and passive liveness can be performed as one step from the user’s perspective.

ICAO quality provider

ICAO quality provider captures a face that is similar to passport like photograph. In general, it is very difficult to produce passport-like photograph in uncontrolled environment. This quality provider is configured to produce maximum quality images with acceptable constraints for the user. It is advised to show best practices to the user before the capture, for example to be in a well-lit room, stand in front of plain background, avoid side light etc. To produce such photograph, user cooperation is required.

Passive liveness quality provider will cover most use cases as a large portion of them require verification with passive liveness evaluation without the need of passport-like quality photographs.

Face capture process

Face capture process is structured into steps that user must go through to capture image with sufficient quality. Each quality provider contains attributes that should be checked. User is instructed to comply with the requirements and the message is displayed until requirements are satisfied. There are two ranges that serve as hysteresis which enables the user to move a little out of the range when proceeding the following steps. However when user breaks the conditions significantly, they are reverted to the previous steps. When all requirements are satisfied, the process enters the “stay still” phase during which candidates are considered and photo with highest quality meeting all criteria is selected.

Recognition result

When face is detected various operations can be performed and more detailed info about the face can be obtained. This includes:

  • face features - important points and their coordinates [x,y] in the original image (e.g. nose tip, mouth corners)
  • ICAO attributes - photo attributes defined by ICAO (e.g. brightness, shadow)
  • face attributes - other photo attributes (e.g. passive liveness)
  • face template - compact representation of the face generated by a neural network. This template can be used for verification. Templates are not compatible across major product upgrades.
  • original image - image from the video stream on which all data is computed
  • ICAO cropped image - straightened image, cropped from the original that contains ICAO defined margin around the face.

Original image

All attributes are calculated from the original image.

Original image

ICAO cropped image

ICAO cropped image can be used for storing face images in a standard aspect and can be later used for template extraction or verification.

If you need to evaluate passive liveness on the server, cropped images cannot be used because more context around the face is needed for evaluation. For this purpose you should use the original image.

ICAO cropped image

Face features

Coordinates of following face features are available in the detection result:

  • rightEyeOuterCorner
  • rightEyeCentre
  • rightEyeInnerCorner
  • leftEyeInnerCorner
  • leftEyeCentre
  • leftEyeOuterCorner
  • noseTip
  • mouthCenter
  • mouthRightCorner
  • mouthLeftCorner
  • mouthUpperEdge
  • mouthLowerEdge
  • rightEyebrowOuterEnd
  • rightEyebrowInnerEnd
  • leftEyebrowInnerEnd
  • leftEyebrowOuterEnd
  • faceRightEdge
  • faceChinTip
  • faceLeftEdge

Subset of these points was rendered in the image below:

Face features in original image

ICAO attributes

ICAO specifies requirements for passport like photographs which include:

Scene attributes

  • sharpness
  • brightness
  • contrast
  • uniqueIntensityLevels
  • shadow
  • noseShadow
  • specularity
  • backgroundUniformity

Subject attributes

  • eyeGaze
  • eyeStatusRight
  • eyeStatusLeft
  • heavyFrame
  • mouthStatus
  • redEyeRight
  • redEyeLeft
  • roll
  • yaw
  • pitch
  • eyeDistance

Attribute describing overall confidence of the face detection:

  • faceConfidence

NOTE

If you need to detect a face on an already captured photograph, you can use the FaceDetector component. Subsequently, you can perform all face related operations. FaceDetector can detect multiple faces, if you need just one face, you can use eye distance to select the biggest face or if your workflow does not allow multiple faces, you can raise an error requiring a new photograph.