DOT Web Document Auto Capture

v3.4.2

Introduction

DOT Web Document Auto Capture is a web component that uses the video stream from an available phone or web camera to automatically capture an image of an ID document with the required quality. The component renders the video stream, and overlays it with a placeholder and instructions to guide the user to position the document correctly.

Supported Browsers

DOT Web Document Auto Capture was tested with:

Chrome on desktop (Windows, Mac and Linux) and mobile (Android, iPhone)
Firefox on desktop (Windows, Mac and Linux) and mobile (Android)
Edge on Windows
Safari on Mac and iPhone
WebView on Android

Known issues:

Components don’t work with WKWebView on iPhone

Privacy and security

This component can only be used in secure contexts due to MediaDevices API used for handling camera access. A secure context is, in short, a page loaded using HTTPS or the file:/// URL scheme, or a page loaded from localhost. Before accessing any camera, the component must always get the user’s permission. Browsers may offer a once-per-domain permission feature, but they must ask at least the first time, and the user has to specifically grant ongoing permission if they choose to do so. Browsers are required to display an indicator that shows that a camera or microphone is in use. More details can be found on MDN docs.

Basic Setup

Initialization

DOT Web Document Auto Capture can be installed via NPM

npm install @innovatrics/dot-document-auto-capture

or yarn

yarn add @innovatrics/dot-document-auto-capture

To manually integrate the DOT Web Document Auto Capture, download latest version from the Github repository. Then, add following line to dependecies in your package.json:

"dependencies": {
    "@innovatrics/dot-document-auto-capture": "file:dot-document-auto-capture-[VERSION].tgz",
}

where [VERSION] is the DOT Web Document Auto Capture version integrated. This installs dot-document-auto-capture as an external module that can be use then (just like any other module in the code) For example, one could do import '@innovatrics/dot-document-auto-capture'; in the app.

Usage

Document auto capture component is an web component which uses custom HTML <x-dot-document-auto-capture/> tag. Properties cameraOptions needs to be passed into component after <x-dot-document-auto-capture/> tag was rendered.

import '@innovatrics/dot-document-auto-capture';

const DocumentCamera = (props) => {
  useEffect(() => {
    const documentAutoCaptureHTMLElement = document.getElementById('x-dot-document-auto-capture');
    documentAutoCaptureHTMLElement.cameraOptions = props;
  })

 return <x-dot-document-auto-capture id="x-dot-document-auto-capture" />;
};

const Page = () => {

  const handleDocumentPhotoTaken = (image, resolution) => {
    // ...
  };

  // Save function reference to prevent unnecessary reload of component
  const handleError = useCallback(
    (error: Error) => {
      alert(error)
    },
    [],
  );

  return (
    <DocumentCamera
      imageType="png"
      cameraFacing="environment"
      photoTakenCb={handleDocumentPhotoTaken}
      onError={handleError}
    />
  );
};

See also DOT Web Samples showing the usage of DOT Web Auto Capture components in different front-end technologies like React, Angular…

TypeScript

Declaration file is bundled inside package. To use with TypeScript, import types from @innovatrics/dot-document-auto-capture.

import type { DocumentCallback, DocumentCameraProps } from '@innovatrics/dot-document-auto-capture';

Hosting of SAM wasm

The component needs to have access to the WebAssembly wasm binary file. It’s distributed in the package and needs to be hosted by the website provider. By default, the component will try to fetch the wasm file from <PROJECT_ORIGIN>/sam.wasm. This can by changed using samWasmUrl property. If using Create React App, copy sam.wasm file to public folder. In our example, the final path is public/sam.wasm.

Document Auto Capture Component

Properties

(Optional) ['png'] string imageType – Format of the image returned after successful capture
- 'jpeg'
- 'png'
(Optional) string cameraFacing – Defines which camera to acquire from browser’s getUserMedia API. Default camera facing for mobile phones is set to environment and for others platforms is set to user
- 'user' – The video source is facing toward the user; this is the selfie or front-facing camera on a smartphone
- 'environment' – The video source is facing away from the user, thereby viewing their environment; this is the back camera on a smartphone
function photoTakenCb – Callback on successful image capture
- (image, data) ⇒ void - (see Callback parameters)
function onError – Callback for the case that an error occurred (see Handling errors)
- (e: Error) ⇒ void
(Optional) string samWasmUrl - URL link to the location where the wasm binary file is hosted
(Optional) ['standard'] string validationMode – Sets the validation mode for document placement
- 'standard' – Document can be placed anywhere, but cannot leak out of photo
- 'strict' – Document must be placed inside placeholder
(Optional) [false] boolean detectionLayerVisible - If detection UI layer (tracking rectangle) is visible
(Optional) object thresholds - Detection configuration
- (Optional) [0.9] number confidenceThreshold - Detection confidence threshold
- (Optional) [0.035] number placeholderErrorScoreThreshold - Maximum deviation for document position inside placeholder
- (Optional) [400] number sharpnessThreshold - Low sharpness threshold
- (Optional) [250] number brightnessLowThreshold - Low brightness threshold
- (Optional) [900] number brightnessHighThreshold - High brightness threshold
- (Optional) [100] number hotspotsScoreThreshold - Hotspots score threshold
- (optional) [0.03] number outOfBoundsThreshold - Document out of bounds threshold
- (Optional) [0.43] number sizeSmallThreshold - Small size threshold
(Optional) object uiCustomisation - UI customization of component (see UI Customization)

Callback parameters

Blob image – Returned image on successful capture
object data
- object cameraSettings - MediaTrackSettings object containing used webcam settings. The object in addition contains device name.
- (Optional) object detection - Object contains all detection parameters and its values. Present if image was taken using auto capture (not manual capture).
- object imageResolution - Width and height of the captured image.

Multi capture

Document auto capture component allows you to capture an unlimited number of documents without the need to reinitialize the webcam and detector. This allows you to capture two sides of a document or multiple documents. Component calls photoTakenCb callback on every captured document photo. When photoTakenCb is called, detection is paused. Camera stream and document detector stay initialized. Component is in waiting state. You should implement a custom UI for waiting state (e.g. overlay over our component). We provide a default UI, but is not customizable. To continue detection, dispatch custom event on document object.

document.dispatchEvent(
  new CustomEvent('document-auto-capture', {
    detail: { instruction: 'continue-detection' },
  }),
);

UI Customization

(Optional) object placeholder - Placeholder customization
- (Optional) enum documentPlaceholder - One of the predefined placeholders in component that can be selected:
  - 'id-rectangle-corners-front'
  - 'id-rectangle-dash-front'
  - 'id-rectangle-dot-front'
  - 'id-rectangle-solid-front'
  - 'id-rounded-rectangle-photo-front'
  - 'id-rounded-rectangle-corners-front'
  - 'id-rounded-rectangle-dash-front'
  - 'id-rounded-rectangle-dot-front'
  - 'id-rounded-rectangle-solid-back'
  - 'id-rounded-rectangle-solid-front'
  - 'pass-rounded-rectangle-solid-back'
  - 'pass-rounded-rectangle-solid-back-blank'
- (Optional) string customSVG - Alternatively, in a future release, a string with custom svg will be able to be provided (see UI Customization examples)
(Optional) object instructions - Modification of default messages for localization or customization
- (Optional) ['Hold still…'] string candidate_selection - Shown when all validations are passed, i.e. image is suitable for capture
- (Optional) ['Place document in rectangle'] string document_centering - Shown when the document is not centered inside the placeholder
- (Optional) ['Move back'] string document_too_close - Shown when the document is too close to the camera
- (Optional) ['Place document in rectangle'] string document_not_present - Shown when no document is detected
- (Optional) ['Move closer'] string document_too_far - Shown when the document is too far from the camera.
- (Optional) ['More light needed'] string sharpness_too_low - Shown when the document found in the image is not sharp enough
- (Optional) ['More light needed'] string brightness_too_low - Shown when the image is too dark
- (Optional) ['Less light needed'] string brightness_too_high - Shown when the image is too bright.
- (Optional) ['Avoid reflections'] string hotspots_present - Shown when the document found in the image has reflections
(Optional) colors - Colors in DOT Web Document Auto Capture may be customized in integration
- (Optional) ['white'] color placeholderColor - Color of the placeholder lines
- (Optional) ['#00BFB2'] color placeholderColorSuccess - Color of the placeholder lines when all validations are passed
- (Optional) ['white'] color instructionColor - Instruction background color
- (Optional) ['#00BFB2'] color instructionColorSuccess - Instruction background color when all validations are passed
- (Optional) ['black'] color instructionTextColor - Instruction text color
(Optional) appStateInstructions - Modification of default messages for component state
- (Optional) loading - Component loading state
  - (Optional) ['Loading. Please wait.'] string text - Text shown while component is loading
  - (Optional) [true] boolean visible - Show/hide loading instruction while component is loading
- (Optional) waiting - Component waiting state
  - (Optional) ['Waiting for input'] string text - Text shown while component is waiting
  - (Optional) [true] boolean visible - Show/hide waiting instruction while component is waiting

UI Customization examples

<DocumentCamera
  photoTakenCb={handleDocumentPhotoTaken}
  uiCustomisation = {{
    placeholder: {
      documentPlaceholder: 'id-rectangle-dash-front'
    },
    instructions: {
      document_too_close: 'Document is too close',
      document_too_far: 'Document is too far',
    },
    colors: {
      placeholderColor: '#EEEEEE',
      instructionTextColor: '#080808',
    },
    appStateInstructions: {
      waiting: {
        visible: false
      }
    },
  }}
/>

Custom SVG placeholder images are not yet supported, but will be available in a future release

Handling errors

When an error occurs we call onError callback with one parameter of type Error. We set name property to AutoCaptureError and also message with more details. Component renders default UI for error state but is not customizable, and integrator should implement own error handling. Component uses the MediaDevices API that provides access to connected media input devices like cameras and microphones. If the user denies permission to access or the webcam is not present, an error is thrown. We provide original error thrown by browser inside cause property of the error object. List of possible errors can be found on MDN docs.

Error example:

{
  name: "AutoCaptureError",
  message: "The webcam is already in use by another application",
  cause: DOMException: Could not start video source // Original error thrown by browser MediaDevices API
}

Typescript declaration files export

Change

Dependencies update and upgrade

3.3.1 - 2022-07-06

Fixed

Zero height of component when error occurs

Add

add appStateInstructions property to uiCustomisation
add option to change appState instructions text and visibility

Change

interface for changing Loading appState instruction text

3.3.0 - 2022-06-15

Changed

Remove needs for using wrapper with defined height and width

3.2.0 - 2022-05-25

Added

Add support for capturing multiple document photos
Add custom event document-auto-capture with continue-detection instruction to continue detection
Add Countly analytics tracking
TypeScript support. Import types from @innovatrics/auto-capture

Changed

onError callback returns AutoCaptureError
onError callback is required
Show error screen when error occurs
When photoTakenCb is called, component switches into waiting state

Fixed

Stuck component when error occurs

3.1.2 - 2022-04-04

Changed

Remove force manual capture on instruction click

Add detectionLayerVisible with default value false

Fixed

Unify loading and instruction design

3.0.2 - 2022-01-31

Added

Add loading screen
Add loading.text property in uiCustomisation for configuring text on loading screen

Changed

Show new instruction only if 600ms elapsed since last instruction change
Changed default value for document_centering instruction

3.0.1 - 2022-01-25

Added

Introduced option to choose between normal and strict validation mode
Option to select between normal or strict validation mode.
OutOfBoundsValidator and OutOfBoundsThreshold
SizeSmallValidator and SizeSmallThreshold

Fixed

safari version 15 on mac

Changed

Document detection accuracy improved.
Detection area size
New `sam.wasm`file

Removed

blaze_models package
modelUrls option from cameraOptions

2.3.1 - 2022-01-10

Fixed

Fix bug causing integration error

2.3.0 - 2022-01-05

Added

add cameraSettings into photoTakenCb callBack function

Changed

photoTakenCb callBack function structure
Redesign of the UI of Document Auto Capture component.

2.2.2 - 2021-12-17

Added

samWasmUrl property for configure path to SAM wasm binary file

2.2.1 - 2021-12-09

Added

New SVG placeholder pass-rounded-rectangle-solid-back-blank

Changed

Hide switch camera button if there is only 1 webcam available
Default camera facing for mobile phones is set to environment and for others platforms is set to user
Computation of placeholderErrorScore for document placement inside placeholder
Default value for placeholderErrorScoreThreshold from 0.03 to 0.035

Fixed

On Android phones, choose correct default lens if more than one available

2.2.0 - 2021-11-30

Changed

dynamic import of TensorFlow.js libraries

2.1.0 - 2021-11-18

First released version