DOT Web Face Auto Capture

v3.3.0

Introduction

Web DOT Face Auto Capture is a web component using video stream from available phone or web camera to automatically capture an image of a user’s face with the required quality. The component renders the video stream and overlays it with a placeholder and instructions to guide the user to position their face correctly.

Supported Browsers

Web DOT Face Auto Capture was tested with:

  • Chrome on desktop (Windows, Mac and Linux) and mobile (Android, iPhone)

  • Firefox on desktop (Windows, Mac and Linux) and mobile (Android)

  • Edge on Windows

  • Safari on Mac and iPhone

  • WebView on Android

Known issues:

  • Components don’t work with WKWebView on iPhone

Privacy and security

This component can only be used in secure contexts due to MediaDevices API used for handling camera access. A secure context is, in short, a page loaded using HTTPS or the file:/// URL scheme, or a page loaded from localhost. Before accessing any camera, the component must always get the user’s permission. Browsers may offer a once-per-domain permission feature, but they must ask at least the first time, and the user has to specifically grant ongoing permission if they choose to do so. Browsers are required to display an indicator that shows that a camera or microphone is in use. More details can be found on MDN docs.

Basic Setup

Initialization

Web DOT Face Auto Capture can be installed via NPM

npm install @innovatrics/dot-face-auto-capture

or yarn

yarn add @innovatrics/dot-face-auto-capture

To manually integrate the Web DOT Face Auto Capture, download latest version from thehttps://github.com/innovatrics/dot-js-kit/releases[Github repository^]. Then add following line to dependencies in your package.json:

"dependencies": {
    "dot-face-auto-capture": "file:dot-face-auto-capture-[VERSION].tgz"
},

where [VERSION] is the Web DOT Face Auto Capture version integrated. This installs dot-face-auto-capture as an external module that can then be used (just like any other module in the code). For example, one could do import 'dot-face-auto-capture'; in the app.

Usage

Component has to be wrapped inside parent node with a defined height and width. The video stream will not render if the height is zero.
Properties cameraOptions needs to be passed into component after <x-dot-face-auto-capture/> tag is rendered.

import 'dot-face-auto-capture';

const FaceCamera = (props) => {
  useEffect(() => {
    const faceAutoCaptureHTMLElement = document.getElementById('x-dot-face-auto-capture');
    faceAutoCaptureHTMLElement.cameraOptions = props;
  });

  return <x-dot-face-auto-capture id="x-dot-face-auto-capture" />;
};

const Page = () => {

  const handleFacePhotoTaken = (image, resolution) => {
    // ...
  };

  return (
    <div className="container" style="height: 500px; width: 500px;">
      <FaceCamera
        imageType="png"
        cameraFacing="user"
        photoTakenCb={handleFacePhotoTaken}
      />
    </div>
  );
};

See also DOT Web Samples showing the usage of DOT Web Auto Capture components in different front-end technologies like React, Angular…​

TypeScript

Declaration file is bundled inside package. To use with TypeScript, import types from @innovatrics/auto-capture.

import { FaceCallback, FaceComponentData } from "@innovatrics/auto-capture";

Hosting of SAM wasm

The component needs to have access to the WebAssembly wasm binary file. It’s distributed in the package, and needs to be hosted by the website provider. By default, component try to fetch wasm file from <PROJECT_ORIGIN>/sam.wasm. This can be changed using the samWasmUrl property. If using Create React App, copy the`sam.wasm` file to public folder. In our example the final path is public/sam.wasm.

Face Auto Capture Component

  • (Optional) ['png'] string imageType – Format of the image returned after successful capture

    • 'jpeg'

    • 'png'

  • (Optional) ['user'] string cameraFacing – Defines which camera to acquire from the browser’s getUserMedia API

    • 'user' – The video source is facing toward the user; this is the selfie or front-facing camera on a smartphone

    • 'environment' – The video source is facing away from the user, thereby viewing their environment; this is the back camera on a smartphone

  • function photoTakenCb – Callback on successful image capture

  • function onError – Callback in the case that an error occurred (see Handling errors)

    • (e: Error) ⇒ void

  • (Optional) string samWasmUrl - URL link to the location where the wasm binary file is hosted

  • (Optional) object thresholds - Detection configuration

    • (Optional) [0.12] number faceConfidence - Detection confidence threshold

    • (Optional) [0.35] number minFaceSizeRatio - Minimum face size ratio relative to shorter side of the camera resolution

    • (Optional) [0.44] number maxFaceSizeRatio - Maximum face size ratio relative to shorter side of the camera resolution

    • (Optional) [0.07] number faceCenteringNarrowSide - Size of narrow side for center face validator

    • (Optional) [0.21] number faceCenteringWideSide - Size of wide side for center face validator

    • (Optional) [400] number sharpnessThreshold - Low sharpness threshold

    • (Optional) [400] number brightnessLowThreshold - Low brightness threshold

    • (Optional) [900] number brightnessHighThreshold - High brightness threshold

    • (Optional) [100] number hotspotsMediumThreshold - Medium hotspots threshold

    • (Optional) [800] number hotspotsHighThreshold - High hotspots threshold

  • (Optional) object uiCustomisation - UI customization of the component (see UI Customization).

Callback parameters

  • Blob image – Returned image on successful capture

  • object data

    • object cameraSettings - MediaTrackSettings object containing used webcam settings. The object in addition contains device name.

    • (Optional) object detection - Object contains all detection parameters and its values. Present if image was taken using auto capture (not manual capture).

    • object imageResolution - Width and height of the captured image.

Multi capture

Face auto capture component allows you to capture an unlimited number of faces without the need to reinitialize the webcam and detector. Component calls photoTakenCb callback on every captured face photo. When photoTakenCb is called, detection is paused. Camera stream and face detector stay initialized. Component is in waiting state. You should implement a custom UI for waiting state (e.g. overlay over our component). We provide a default UI, but is not customizable. To continue detection, dispatch custom event on face object.

face.dispatchEvent(
  new CustomEvent('face-auto-capture', {
    detail: { instruction: 'continue-detection' },
  }),
);

UI Customization

  • (Optional) object placeholder - Placeholder customization

    • (Optional) enum facePlaceholder - One of the predefined placeholders in component that can be selected:

      • 'circle-solid'

      • 'ellipse-solid'

      • 'man-solid'

      • 'woman-solid'

      • 'square-rounded-dash'

      • 'square-rounded-solid'

      • 'square-dash'

      • 'square-solid'

    • (Optional) string customSVG - Alternatively, in a future release, a string with custom svg will be able to be provided (see UI Customization examples)

  • (Optional) object instructions - Modification of default messages for localization or customization

    • (Optional) ['Stay still…'] string candidate_selection - Shown when all validations are passed, i.e. the image is suitable for capture

    • (Optional) ['Move back'] string face_too_close - Shown when the face is too close to the camera

    • (Optional) ['Move closer'] string face_too_far - Shown when the face is too far from the camera

    • (Optional) ['Center your face'] string face_centering - Shown when the face is not centered in the placeholder

    • (Optional) ['Center your face'] string face_not_present - Shown when no face is detected

    • (Optional) ['Turn face against light'] string sharpness_too_low - Shown when the face in the image is not sharp enough

    • (Optional) ['Turn face against light'] string brightness_too_low - Shown when the image is too dark

    • (Optional) ['Less light needed'] string brightness_too_high - Shown when the image is too bright.

    • (Optional) ['Turn face against light'] string hotspots_medium_present - Shown when the image has reflections

    • (Optional) ['Move away from bright light'] string hotspots_high_present - Shown when the face is too bright

  • (Optional) colors - Colors in Web DOT Face Auto Capture may be customized in integration

    • (Optional) ['white'] color placeholderColor - Color of the placeholder lines

    • (Optional) ['#00BFB2'] color placeholderColorSuccess - Color of the placeholder lines when all validations are passed

    • (Optional) ['white'] color instructionColor - Instruction background color

    • (Optional) ['#00BFB2'] color instructionColorSuccess - Instruction background color when all validations are passed

    • (Optional) ['black'] color instructionTextColor - Instruction text color

  • (Optional) loading - Modification of component loading state

    • (Optional) ['Loading. Please wait.'] string text - Shown text while loading component

UI Customization examples

<FaceCamera
  photoTakenCb={handleFacePhotoTaken}
  uiCustomisation = {{
    placeholder: {
      facePlaceholder: 'ellipse-solid',
    },
    instructions: {
      face_too_close: 'Your face is too close',
      face_too_far: 'Your face is too far',
    },
    colors: {
      placeholderColor: '#EEEEEE',
      instructionTextColor: '#080808',
    },
  }}
/>

Custom SVG placeholder images are not yet supported, but will be available in a future release

Handling errors

When an error occurs, we call onError callback with one parameter of type Error. We set name property to AutoCaptureError, and also message with more details. Component renders default UI for error state, but is not customizable, and integrator should implement own error handling. Component uses the MediaDevices API that provides access to connected media input devices like cameras and microphones. If the user denies permission to access or the webcam is not present, an error is thrown. We provide original error thrown by browser inside cause property of the error object. List of possible errors can be found on MDN docs.

Error example:

{
  name: "AutoCaptureError",
  message: "The webcam is already in use by another application",
  cause: DOMException: Could not start video source // Original error thrown by browser MediaDevices API
}

Appendix

Changelog

3.3.0 - 2022-06-15

Added
  • Add support for capturing multiple face photos

  • Add custom event face-auto-capture with continue-detection instruction to continue detection

Changed
  • When photoTakenCb is called, component switches into waiting state

  • Remove needs for using wrapper with defined height and width

3.2.0 - 2022-05-25

Added
  • Countly analytics tracking

  • Optional threshold sharpnessThreshold with default value 400 and Turn face against light instruction

  • Optional threshold brightnessLowThreshold with default value 400 and Turn face against light instruction

  • Optional threshold brightnessHighThreshold with default value 900 and Less light needed instruction

  • Optional threshold hotspotsMediumThreshold with default value 100 and Turn face against light instruction

  • Optional threshold hotspotsHighThreshold with default value 800 and Move away from bright light instruction

  • TypeScript support. Import types from @innovatrics/auto-capture

Changed
  • onError callback returns AutoCaptureError

  • onError callback is required

  • Show error screen when error occurs

Fixed
  • Stuck component when error occurs

3.1.2 - 2022-04-04

Changed
  • Remove force manual capture on instruction click

3.1.1 - 2022-03-15

Changed
  • Change optimalFaceSizeLimit and optimalFaceSizeParam to minFaceSizeRatio and maxFaceSizeRatio

  • Change faceCenterLimit to faceCenteringNarrowSide and faceCenteringWideSide

  • Set minFaceSizeRatio default value to 0.35 and maxFaceSizeRatio to 0.44

  • Set faceCenteringNarrowSide default value to 0.07 and faceCenteringWideSide to 0.21

3.1.0 - 2022-02-03

Fixed
  • Unify loading and instruction design

3.0.2 - 2022-01-28

Added
  • Add loading screen

  • Add loading.text property in uiCustomisation for configuring text on loading screen

Changed
  • Show new instruction only if 600ms elapsed since last instruction change

3.0.1 - 2022-01-25

Added
  • file sam.wasm

Fixed
  • safari version 15 on mac

Changed
  • Face detection accuracy improved.

  • Detection engine

Removed
  • blaze_models package

  • modelUrls option from cameraOptions

2.3.1 - 2022-01-10

Fixed
  • Fix bug causing integration error

2.3.0 - 2022-01-05

Added
  • add cameraSettings into photoTakenCb callBack function

  • Redesign of the UI of Face Auto Capture component.

Changed
  • photoTakenCb callBack function structure

2.2.2 - 2021-12-17

  • Incremental version upgrade

2.2.1 - 2021-12-09

Changed
  • Hide switch camera button if there is only 1 webcam available

2.2.0 - 2021-11-30

Added
  • thresholds.faceConfidence - Detection confidence threshold with default value 0.12

  • modelUrls - Object with path to the models needed for detection

Changed
  • integrated new neural network for face detection

  • dynamic import of TensorFlow.js libraries

2.1.0 - 2021-11-18

  • First released version