DOT Web Document Auto Capture

v2.3.1

Introduction

DOT Web Document Auto Capture is a web component using video stream from available phone or web camera to automatically capture an image of an ID document with the required quality. The component renders the video stream and overlays it with a placeholder and instructions to guide the user to position the document correctly.

Supported Browsers

DOT Web Document Auto Capture was tested with:

  • Chrome on desktop (Windows, Mac and Linux)

  • Firefox on desktop (Windows and Linux)

  • Chrome on Android and iPhone

  • Firefox on Android and iPhone

  • Safari on iPhone

The components does not work reliably with Safari on Mac.

Privacy and security

This component can only be used in secure contexts due to MediaDevices API used for handling camera access. A secure context is, in short, a page loaded using HTTPS or the file:/// URL scheme, or a page loaded from localhost. Before accessing any camera, component must always get user’s permission. Browsers may offer a once-per-domain permission feature, but they must ask at least the first time, and the user has to specifically grant ongoing permission if they choose to do so. Browsers are required to display an indicator that shows that a camera or microphone is in use. More details can be found on MDN docs.

Basic Setup

Initialization

To integrate the DOT Web Document Auto Capture, following line has to be added to dependecies in package.json:

"dependencies": {
    "dot-document-auto-capture": "file:dot-document-auto-capture-[VERSION].tgz",
}

where [VERSION] is the DOT Web Document Auto Capture version integrated. This installs dot-document-auto-capture as an external module that can be use then (just like any other module in the code) For example, one could do import 'dot-document-auto-capture'; in the app.

Usage

Component has to be wrapped inside parent node with a defined height and width. The video stream will not render, if the height is zero.
Properties cameraOptions needs to be passed into component after <x-dot-document-auto-capture/> tag was rendered.

import 'dot-document-auto-capture';

const DocumentCamera = (props) => {
  useEffect(() => {
    const documentAutoCaptureHTMLElement = document.getElementById('x-dot-document-auto-capture');
    documentAutoCaptureHTMLElement.cameraOptions = props;
  })

 return <x-dot-document-auto-capture id="x-dot-document-auto-capture" />;
};

const modelUrls = {
 modelJSON: 'blaze_model/docu_3cl_background/model.json',
 modelBin: 'blaze_model/docu_3cl_background/group1-shard1of1.bin',
};

const Page = () => {

  const handleDocumentPhotoTaken = (image, resolution) => {
    // ...
  };

  return (
    <div className="container" style="height: 500px; width: 500px;">
      <DocumentCamera
       imageType="png"
       cameraFacing="environment"
       photoTakenCb={handleDocumentPhotoTaken}
       modelUrls={modelUrls}
      />
    </div>
  );
};

Hosting of detection models and SAM wasm

Blaze models

The component needs to have access to a document detection model consisting of a binary and JSON. These are distributed in the package and need to be hosted by the website provider. If using Create React App copy model.json and group1-shard1of1.bin to public folder. In our example the final path is public/blaze_model/docu_3cl_background/group1-shard1of1.bin and public/blaze_model/docu_3cl_background/model.json.

Sam wasm

The component needs to have access to WebAssembly wasm binary file. It’s distributed in the package and need to be hosted by the website provider. By default, component try to fetch wasm file from <PROJECT_ORIGIN>/sam.wasm. This can by changed using samWasmUrl property. If using Create React App copy sam.wasm file to public folder. In our example the final path is public/sam.wasm.

Document Auto Capture Component

Properties

  • (Optional) ['png'] string imageType – Format of the image returned after successful capture

    • 'jpeg'

    • 'png'

  • (Optional) string cameraFacing – Defines which camera to acquire from browser’s getUserMedia API. Default camera facing for mobile phones is set to environment and for others platforms is set to user

    • 'user' – The video source is facing toward the user; this is the selfie or front-facing camera on a smartphone

    • 'environment' – The video source is facing away from the user, thereby viewing their environment; this is the back camera on a smartphone

  • function photoTakenCb – Callback on successful image capture

  • (Optional) function onError – Callback for case that an error occurred (see Handling errors)

    • (e: Error) ⇒ void

  • object modelUrls - Object with path to the models needed for detection

    • string modelJSON - URL link to the location where the model is hosted

    • string modelBin - URL link to the location where the model is hosted

  • (Optional) string samWasmUrl - URL link to the location where the wasm binary file is hosted

  • (Optional) object thresholds - Detection configuration

    • (Optional) [0.9] number confidenceThreshold - Detection confidence threshold

    • (Optional) [0.035] number placeholderErrorScoreThreshold - Maximum deviation for document position inside placeholder

    • (Optional) [400] number sharpnessThreshold - Low sharpness threshold

    • (Optional) [250] number brightnessLowThreshold - Low brightness threshold

    • (Optional) [900] number brightnessHighThreshold - High brightness threshold

    • (Optional) [100] number hotspotsScoreThreshold - Hotspots score threshold

  • (Optional) object uiCustomisation - UI customization of component (see UI Customization)

Callback parameters

  • Blob image – Returned image on successful capture

  • object data

    • object cameraSettings - MediaTrackSettings object containing used webcam settings. The object in addition contains device name.

    • (Optional) object detection - Object contains all detection parameters and its values. Present if image was taken using auto capture (not manual capture).

    • object imageResolution - Width and height of the captured image.

UI Customization

  • (Optional) object placeholder - Placeholder customization

    • (Optional) enum documentPlaceholder - One of the predefined placeholders in component that can be selected:

      • 'id-rectangle-corners-front'

      • 'id-rectangle-dash-front'

      • 'id-rectangle-dot-front'

      • 'id-rectangle-solid-front'

      • 'id-rounded-rectangle-photo-front'

      • 'id-rounded-rectangle-corners-front'

      • 'id-rounded-rectangle-dash-front'

      • 'id-rounded-rectangle-dot-front'

      • 'id-rounded-rectangle-solid-back'

      • 'id-rounded-rectangle-solid-front'

      • 'pass-rounded-rectangle-solid-back'

      • 'pass-rounded-rectangle-solid-back-blank'

    • (Optional) string customSVG - Alternatively, in a future release, a string with custom svg will be able to be provided (see UI Customization examples)

  • (Optional) object instructions - Modification of default messages for localization or customization

    • (Optional) ['Hold still…'] string candidate_selection - Shown when all validations are passed, i.e. image is suitable for capture

    • (Optional) ['Place document in rectangle'] string document_centering - Shown when the document is not centered inside the placeholder

    • (Optional) ['Move back'] string document_too_close - Shown when the document is too close to the camera

    • (Optional) ['Place document in rectangle'] string document_not_present - Shown when no document is detected

    • (Optional) ['Move closer'] string document_too_far - Shown when the document is too far from the camera.

    • (Optional) ['More light needed'] string sharpness_too_low - Shown when the document found in the image is not sharp enough

    • (Optional) ['More light needed'] string brightness_too_low - Shown when the image is too dark

    • (Optional) ['Less light needed'] string brightness_too_high - Shown when the image is too bright.

    • (Optional) ['Avoid reflections'] string hotspots_present - Shown when the document found in the image has reflections

  • (Optional) colors - Colors in DOT Web Document Auto Capture may be customized in integration

    • (Optional) ['white'] color placeholderColor - Color of the placeholder lines

    • (Optional) ['#00BFB2'] color placeholderColorSuccess - Color of the placeholder lines when all validations are passed

    • (Optional) ['white'] color instructionColor - Instruction background color

    • (Optional) ['#00BFB2'] color instructionColorSuccess - Instruction background color when all validations are passed

    • (Optional) ['black'] color instructionTextColor - Instruction text color

UI Customization examples

<DocumentCamera
  modelUrls={modelUrls}
  photoTakenCb={handleDocumentPhotoTaken}
  uiCustomisation = {{
    placeholder: {
      documentPlaceholder: 'id-rectangle-dash-front'
    },
    instructions: {
      document_too_close: 'Document is too close',
      document_too_far: 'Document is too far',
    },
    colors: {
      placeholderColor: '#EEEEEE',
      instructionTextColor: '#080808',
    },
  }}
/>

Custom SVG placeholder images are not yet supported, but will be available in a future release

Handling errors

Component uses the MediaDevices API that provides access to connected media input devices like cameras and microphones. If the user denies permission to access or the webcam is not present, an error is thrown and is handled by onError callback. List of possible errors can be found on MDN docs.

Appendix

Changelog

2.3.1 - 2022-01-10

Fixed
  • Fix bug causing integration error

2.3.0 - 2022-01-05

Added
  • add cameraSettings into photoTakenCb callBack function

Changed
  • photoTakenCb callBack function structure

  • Redesign of the UI of Document Auto Capture component.

2.2.2 - 2021-12-17

Added
  • samWasmUrl property for configure path to SAM wasm binary file

2.2.1 - 2021-12-09

Added
  • New SVG placeholder pass-rounded-rectangle-solid-back-blank

Changed
  • Hide switch camera button if there is only 1 webcam available

  • Default camera facing for mobile phones is set to environment and for others platforms is set to user

  • Computation of placeholderErrorScore for document placement inside placeholder

  • Default value for placeholderErrorScoreThreshold from 0.03 to 0.035

Fixed
  • On android phones choose correct default lens if more than one available

2.2.0 - 2021-11-30

Changed
  • dynamic import of TensorFlow.js libraries

2.1.0 - 2021-11-18

  • First released version