DOT Web Smile Liveness
v6.2.0
Notice
To determine the outcome of this component, it is essential to validate it by forwarding the data to DIS as a liveness record. If the smile liveness evaluation produces a negative result, the integrator should then provide the user with clear instructions on how to proceed.
Introduction
DOT Web Smile Liveness is set of Non-ui component and Ui component web components providing a semi-passive liveness capture. Components navigate user to capture two images. First one with neutral face expression and during the process of capturing second image user is asked to smile.
Components lays foundations for the multimodal capability of liveness detection based on the change in facial expression. The process can be performed within seconds and does not compromise the customer experience.
Non-ui component is a web component that renders the video stream from an available phone or web camera to automatically capture images of a user’s face with the required quality.
Ui component is a web component that renders an overlay over video stream. Overlay includes a placeholders, camera control buttons and instructions to guide the user to position their face correctly.
Both components should always be used together, unlike DOT Web Document Auto Capture or DOT Web Face Auto Capture, Smile Liveness Ui component can not be custom implemented.
Supported Browsers
DOT Web Smile Liveness was tested with:
Chrome on desktop (Windows, Mac and Linux) and mobile (Android, iPhone)
Firefox on desktop (Windows, Mac and Linux) and mobile (Android)****
Edge on Windows
Safari on Mac, iPad and iPhone
WebView on Android*
SafariVC on iPad and iPhone**
WKWebView on iPad and iPhone***
Known issues:
* Some older Android phones can experience issues with camera stream initialization using WebRTC. Our web components will work on most of them, but camera stream could be slow and laggy. This issue is caused by device and can be experienced in any website using WebRTC in Android WebView.
** Components are tested with SafariVC on devices iPhone 7 and newer with iOS 15 or iPadOS 15 or newer. Older devices or iOS versions are not officially supported and might experience issues.
*** In order to run DOT Web Smile Liveness in WKWebView on iPhone or iPad, allowsInlineMediaPlayback
and mediaTypesRequiringUserActionForPlayback
properties on WKWebViewConfiguration
must be set to true
.
**** Some older and/or low-end Android phones can experience issues with camera stream initialization using WebRTC when using Firefox browser. Our web components might display The webcam is already in use by another application
error. This issue is caused by the Firefox’s implementation of initialization of camera and can be experienced in any website using WebRTC in Firefox on Android. It is recommended to use other supported browsers in those cases.
Privacy and security
This component can only be used in secure contexts due to MediaDevices
API used for handling camera access. A secure context is, in short, a page loaded using HTTPS or the file:///
URL scheme, or a page loaded from localhost
. Before accessing any camera, the component must always get the user’s permission. Browsers may offer a once-per-domain permission feature, but they must ask at least the first time, and the user has to specifically grant ongoing permission if they choose to do so. Browsers are required to display an indicator that shows that a camera or microphone is in use. More details can be found on MDN docs.
Non-ui component
Requirements
Minimum required camera resolution for appropriate results is 720x720. Anything less is insufficient.
Initialization
DOT Web Smile Liveness can be installed via NPM, yarn or pnpm
npm install @innovatrics/dot-smile-liveness
To manually integrate the DOT Web Smile Liveness, download latest version from the Github repository. Then add following line to dependencies in your package.json:
"dependencies": {
"@innovatrics/dot-smile-liveness": "file:innovatrics-dot-smile-liveness-[VERSION].tgz"
},
where [VERSION]
is the DOT Web Smile Liveness version integrated. This installs dot-smile-liveness
as an external module that can then be used (just like any other module in the code). For example, one could do import '@innovatrics/dot-smile-liveness';
in the app.
Usage
Smile Liveness component is a web component which uses custom HTML <x-dot-smile-liveness />
tag.
Properties props
needs to be passed into component after <x-dot-smile-liveness />
tag was rendered.
import type {
SmileLivenessProps,
HTMLSmileLivenessElement,
OnCompleteData
} from "@innovatrics/dot-smile-liveness";
import { useEffect } from "react";
import "@innovatrics/dot-smile-liveness";
const SmileLiveness = (props: SmileLivenessProps) => {
useEffect(() => {
const smileLivenessHTMLElement = document.getElementById(
"x-dot-smile-liveness"
) as HTMLSmileLivenessElement | null;
if (smileLivenessHTMLElement) {
smileLivenessHTMLElement.props = props;
}
});
return <x-dot-smile-liveness id="x-dot-smile-liveness" />;
};
const Page = () => {
const handleOnComplete = (data: Array<OnCompleteData>, content: Uint8Array) => {
// ...
};
const handleError = (error: Error) => {
alert(error);
};
return (
<SmileLiveness
onComplete={handleOnComplete}
onError={handleError}
sessionToken="1111-222-333-4444"
/>
);
};
Alternatively, you can use useRef
and useLayoutEffect
to render a web component with its props.
import type {
SmileLivenessProps,
HTMLSmileLivenessElement,
} from "@innovatrics/dot-smile-liveness";
import { useLayoutEffect, useRef } from "react";
import "@innovatrics/dot-smile-liveness";
const SmileLiveness = (props: SmileLivenessProps) => {
const ref = useRef<HTMLSmileLivenessElement | null>(null);
useLayoutEffect(() => {
const element = ref.current;
if (element) {
element.props = props;
}
}, [props]);
return <x-dot-smile-liveness id="x-dot-smile-liveness" ref={ref} />;
};
iOS - Requesting DeviceMotion Permissions
To utilize the device motion sensor and provide instructions for device_tilted
instruction on iOS , it’s necessary to invoke the DeviceMotionEvent.requestPermission()
method and acquire user consent. This requirement is set by Apple and isn’t specific to the DOT Web Smile Liveness. The method returns a promise that yields a string – either "granted" or "denied". If the user has already granted permission, the promise will resolve instantly.
Unfortunately DeviceMotionEvent.requestPermission()
could only be called on a "user gesture" event, such as a button click. Requesting permission without a user gesture event isn’t feasible. Given that, DOT Web Smile Liveness doesn’t produce any suitable user gesture event, the method must be invoked from the parent component when a user gesture event takes place. Below is a TypeScript example illustrating the process:
type DeviceMotionEventWithPermissions = {
requestPermission: () => Promise<PermissionState>;
} & DeviceMotionEvent;
try {
const { requestPermission } = DeviceMotionEvent as unknown as DeviceMotionEventWithPermissions;
if (typeof requestPermission === 'function') {
const response = await requestPermission();
if (response === 'granted') {
// permissions granted
} else {
// permissions denied
}
}
} catch (error) {
// handle error in standard way
}
Without this permission, the device_tilted
instruction will not be available but the component will still work. Further information on this can be found in this article.
TypeScript
Declaration file is bundled inside package. To use with TypeScript, import types from @innovatrics/dot-smile-liveness
.
import type { SmileLivenessCallback, SmileLivenessProps } from "@innovatrics/dot-smile-liveness";
Hosting WASM files
During the initialization phase of the component, the component loads the required WASM binary files via HTTP fetch request. Please note that we do not provide any hosting for WASM files. That’s why it is essential that the customer provides the component with all WASM files from our distribution.
To do so, please copy wasm
directory from our distribution into your distribution every time you update to new version of the component. If you are using package manager, you can find wasm
directory in node_modules/@innovatrics/dot-smile-liveness/wasm
.
@innovatrics/
|__ dot-smile-liveness/
|__ wasm/ <-- copy this directory
|-- sam.wasm
|-- sam_simd.wasm
|-- dot_embedded_bg.wasm
...
|-- package.json
|-- README.md
...
By default, the component will try to fetch the desired wasm file from <PROJECT_ORIGIN>/wasm/<NAME_OF_DESIRED_FILE>.wasm
. This can be changed using wasmDirectoryPath
property. For example, if wasmDirectoryPath=/my-directory
property is provided, the component will try to fetch the desired wasm file from <PROJECT_ORIGIN>/my-directory/<NAME_OF_DESIRED_FILE>.wasm
. Please, do not provide absolute path to single wasm file, but only to the directory containing all wasm files. For more information, check out our samples on Github where you can find real examples of how to host wasm files in various technologies such as React, Angular, Vue, etc.
Troubleshooting
If you are having difficulties with hosting wasm files, check your browser console in dev tools. You should see an error message with more details about the problem. It’s also a good idea to look at the network tab in the developer tools and check that the wasm file has been loaded correctly. Please pay close attention to the actual response of the wasm load request, as an HTTP 200 status does not necessarily mean that the wasm file was loaded correctly. If you see application/wasm
in response type, then wasm file was loaded correctly. If you see e.g. text/html
in response type, then wasm file was not loaded correctly.
Licensing
In order to support wide adoption of the components, a free mode was developed in addition to the premium mode used by licensed customers.
Free mode
Components run in the free mode by default. This mode does not impose any feature limits, it just displays an watermark overlay over the component, to indicate not-licensed deployment. The overlay looks as on the image below:
Premium
To run components in premium mode, dot_embedded_bg.wasm
and valid license files are needed.
To initialize WASM binary file please follow a steps in Hosting WASM files.
To use your license file you need to copy it to project public directory as <PROJECT_ORIGIN>/iengine.lic
.
By default, the component will try to fetch the desired license file from <PROJECT_ORIGIN>/iengine.lic
or <PROJECT_ORIGIN>/license.lic
*. This can be changed using licensePath
property. For example if licensePath=/my-license.lic
property is provided, the component will try to fetch the license file from <PROJECT_ORIGIN>/my-license.lic
.
* Please note that fetching license from <PROJECT_ORIGIN>/license.lic
by default is deprecated and will be removed in future versions. Please use <PROJECT_ORIGIN>/iengine.lic
instead.
If one of the following steps fails, then component will run in [Freemium] mode:å
WASM binary file not found
license file not found
license is not valid
In order to obtain the license, please contact your Innovatrics’ representative.
Properties
function onComplete
- Callback on successful image capture(data: Array<OnCompleteData>, content: Uint8Array) ⇒ void
see Callback parameters
function onError
– Callback in the case that an error occurred (see Handling errors)(e: Error) ⇒ void
(Optional)
string sessionToken
– Unique identifier of the session(Optional)
string wasmDirectoryPath
- Path to the location, where the directory with all wasm binary files are hosted(Optional)
string licensePath
- Path to the license file(Optional)
HTMLElement styleTarget
- Provide an alternate DOM node to inject styles. DOM node has to exist inside DOM before auto-capture component is initialized. This is useful when rendering components in a shadow DOM. By default, the styles are injected into the<head>
element of the document. See [style-target] example for more details.(Optional)
object thresholds
- Detection configuration(Optional)
[0.16]
number minFaceSizeRatio
- Minimum face size ratio relative to shorter side of the camera resolution(Optional)
[0.20]
number maxFaceSizeRatio
- Maximum face size ratio relative to shorter side of the camera resolution
Callback parameters
array data
object data
Blob image
– Returned image on successful captureobject data
object detection
- Object contains all detection parameters and its values. Present if image was taken using auto capture (not manual capture).object imageResolution
- Width and height of the captured image.
Uint8Array content
- output for DOT Digital Identity Service
Handling errors
When an error occurs, we call onError
callback with one parameter of type Error
. We set name
property to AutoCaptureError
, and also message
with more details.
Component renders default UI for error state, but is not customizable, and integrator should implement own error handling.
Component uses the MediaDevices API that provides access to connected media input devices like cameras and microphones.
If the user denies permission to access or the webcam is not present, an error is thrown. We provide original error thrown by browser inside cause
property of the error object.
List of possible errors can be found on MDN docs.
Error example:
{
name: "AutoCaptureError",
message: "The webcam is already in use by another application",
cause: DOMException: Could not start video source // Original error thrown by browser MediaDevices API
}
Ui component
Requirements
Both components must be wrapped in parent div with position: relative
and overflow: hidden
.
<div style={{position: 'relative', overflow: 'hidden'}}>
<SmileLivenessUi showCameraButtons/>
<SmileLiveness
onComplete={handleOnComplete}
onError={handleError}
/>
</div>
Initialization
UI component can be installed via NPM, yarn or pnpm
npm install @innovatrics/dot-auto-capture-ui
To manually integrate UI component, download latest version from the Github repository. Add following line to dependencies in your package.json:
"dependencies": {
"@innovatrics/dot-auto-capture-ui": "file:innovatrics-dot-auto-capture-ui-[VERSION].tgz",
}
where [VERSION]
is the DOT Web Smile Liveness version integrated. This installs dot-auto-capture-ui
as an external module that can be used (just like any other module in the code). For example, one could do import '@innovatrics/dot-auto-capture-ui';
in the app.
Usage
Smile Liveness UI component is an web component which uses custom HTML <x-dot-smile-liveness-ui />
tag.
Properties props
needs to be passed into component after <x-dot-smile-liveness-ui />
tag was rendered.
import type {
SmileLivenessUiProps,
HTMLSmileLivenessUiElement,
} from "@innovatrics/dot-auto-capture-ui/smile-liveness";
import { useEffect } from "react";
import "@innovatrics/dot-auto-capture-ui/smile-liveness";
function SmileLivenessUi(props: SmileLivenessUiProps) {
useEffect(() => {
const uiElement = document.getElementById(
"x-dot-smile-liveness-ui",
) as HTMLSmileLivenessUiElement | null;
if (uiElement) {
uiElement.props = props;
}
});
return <x-dot-smile-liveness-ui id="x-dot-smile-liveness-ui" />;
}
Alternatively, you can use useRef
and useLayoutEffect
to render a web component with its props.
import type {
SmileLivenessUiProps,
HTMLSmileLivenessUiElement,
} from "@innovatrics/dot-auto-capture-ui/smile-liveness";
import { useLayoutEffect, useRef } from "react";
import "@innovatrics/dot-auto-capture-ui/smile-liveness";
function SmileLivenessUi(props: SmileLivenessUiProps) {
const ref = useRef<HTMLSmileLivenessUiElement | null>(null);
useLayoutEffect(() => {
const element = ref.current;
if (element) {
element.props = props;
}
}, [props]);
return <x-dot-smile-liveness-ui id="x-dot-smile-liveness-ui" ref={ref} />;
};
TypeScript
Declaration files are bundled inside package. Just import types from @innovatrics/dot-autocapture-ui/smile-liveness
.
Properties
(Optional)
object instructions
- Modification of default messages for localization or customization(Optional)
['Stay still…']
string candidate_selection
- Shown when all validations are passed, i.e. the image is suitable for capture(Optional)
['Move back']
string face_too_close
- Shown when the face is too close to the camera(Optional)
['Move closer']
string face_too_far
- Shown when the face is too far from the camera(Optional)
['Center your face']
string face_centering
- Shown when the face is not centered in the placeholder(Optional)
['Position your face into the circle']
string face_not_present
- Shown when no face is detected(Optional)
['Position your face into the circle']
string left_eye_not_present
- Shown when left eye is not detected(Optional)
['Position your face into the circle']
string right_eye_not_present
- Shown when right eye is not detected(Optional)
['Position your face into the circle']
string mouth_not_present
- Shown when mouth is not detected(Optional)
['Turn face against light']
string sharpness_too_low
- Shown when the face in the image is not sharp enough(Optional)
['Turn face against light']
string brightness_too_low
- Shown when the image is too dark(Optional)
['Less light needed']
string brightness_too_high
- Shown when the image is too bright(Optional)
['Smile :)']
string smile
- Shown after first image is captured and user is asked to smile(Optional)
['Keep neutral expression']
string keep_neutral_expression
- Shown when user is already smiling before the first image is captured and he is not asked to smile yet(Optional)
['Smile :)']
string smile_candidate_selection
- Shown when all validations are passed for second image, i.e. the image is suitable for capture(Optional)
['Hold your phone at eye level']
string device_pitched
- Shown when the device is tilted too much
(Optional)
string backdropColor
- To customize the color of the placeholder backdrop. E.g.:'rgba(10, 10, 10, 0.5)'
If you prefer not to use the backdrop, you can set the backdropColor to 'none'
.
(Optional)
theme
- Modification of theme properties.(Optional)
colors
- To customize colors of following properties(Optional)
['white']
color placeholderColor
- Color of the placeholder lines(Optional)
['#00BFB2']
color placeholderColorSuccess
- Color of the placeholder lines when all validations are passed(Optional)
['white']
color instructionColor
- Instruction background color(Optional)
['#00BFB2']
color instructionColorSuccess
- Instruction background color when all validations are passed(Optional)
['black']
color instructionTextColor
- Instruction text color
(Optional)
font
- To customize font with following properties(Optional)
['Montserrat']
string family
- Font family. Please note that the chosen font must be installed and available in the styles of the web components for it to display correctly.(Optional)
[14]
number minimumSize
- Minimum font size, from which the actual font size is calculated relative to the component width/height(Optional)
['normal']
string style
- Font style(Optional)
[600]
string|number weight
- Font weight
(Optional)
appStateInstructions
- Modification of default messages for component state(Optional)
loading
- Component loading state(Optional)
['Loading. Please wait.']
string text
- Text shown while component is loading(Optional)
[true]
boolean visible
- Show/hide loading instruction while component is loading
(Optional)
done
- Component finished capturing image(Optional)
['Waiting for input']
string text
- Text shown while component is waiting(Optional)
[true]
boolean visible
- Show/hide waiting instruction while component is waiting
(Optional)
[false]
boolean
showCameraButtons
- Show/hide camera buttons (switch camera and toggle mirror) on top of component(Optional)
HTMLElement styleTarget
- Provide an alternate DOM node to inject styles. DOM node has to exist inside DOM before auto-capture component is initialized. This is useful when rendering components in a shadow DOM. By default, the styles are injected into the<head>
element of the document. See [style-target] example for more details.
Example how to use UI properties:
<SmileLivenessUi
theme={{ colors: { placeholderColor: "green" }, font: { style: 'italic' } }}
backdropColor="rgba(10, 10, 10, 0.5)"
instructions={{ candidate_selection: 'Candidate selection' }}
appStateInstructions={{ loading: { text: 'Component is loading', visible: true } }}
showCameraButtons
/>
Example how to import fonts (using head in main html file):
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="" href="" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Title</title>
<!-- import font in the head -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Montserrat&display=swap" rel="stylesheet">
<!------->
</head>
</html>
Example of using components together
import type { OnCompleteData } from "@innovatrics/dot-smile-liveness";
import SmileLivenessUi from "./SmileLivenessUi";
import SmileLiveness from "./SmileLiveness";
const Smile = () => {
const handleOnComplete = (
data: Array<OnCompleteData>,
content: Uint8Array,
) => {
const [neutralFace, smileFace] = data;
console.log(neutralFace.data, neutralFace.image);
console.log(smileFace.data, smileFace.image);
};
const handleError = (error: Error) => {
alert(error);
};
return (
<div style={{ position: "relative", overflow: "hidden" }}>
<SmileLivenessUi showCameraButtons />
<SmileLiveness onComplete={handleOnComplete} onError={handleError} />
</div>
);
};
Example how to use styleTarget
prop with Ui component when using Shadow DOM:
This approach also works for Non-ui component.
import type {
SmileLivenessUiProps,
HTMLSmileLivenessUiElement,
} from "@innovatrics/dot-auto-capture-ui/smile-liveness";
import { useEffect } from "react";
import "@innovatrics/dot-auto-capture-ui/smile-liveness";
function SmileLivenessUi(props: SmileLivenessUiProps) {
useEffect(() => {
const uiElement = document.getElementById(
"x-dot-smile-liveness-ui",
) as HTMLSmileLivenessUiElement | null;
const styleNode = document.createElement('div');
uiElement?.shadowRoot?.append(styleNode);
if (uiElement) {
uiElement.props = {
...props,
styleTarget: styleNode,
};
}
});
return <x-dot-smile-liveness-ui id="x-dot-smile-liveness-ui" />;
}
HTML of example above will look like this:
<x-dot-smile-liveness-ui id="x-dot-smile-liveness-ui">
#shadow-root (open)
<div>
<style data-styled="active" data-styled-version="6.1.0">
/* CSS styles will be injected here*/
</style>
</div>
</x-dot-smile-liveness-ui>
Code Samples
See also DOT Web Samples showing the usage of DOT Web Auto Capture components in different front-end technologies like React, Angular…
Appendix
Changelog
6.2.0 - 2024-11-06
Added
iengine.lic
as another default license file name. Now the component will automatically load eitheriengine.lic
orlicense.lic
Changed
Smile detection improvements
license.lic
default license file name flagged as deprecated
Fixed
Custom element prop types
6.1.8 - 2024-10-01
Fixed
Wrong image in
onPhotoTaken
callback
6.1.7 - 2024-10-01
Fixed
Error while switching between cameras on Android devices
6.1.6 - 2024-09-11
Fixed
Theme customization prop
6.1.5 - 2024-09-02
Changed
Improved security measures and enhanced protection against vulnerabilities
6.1.4 - 2024-08-21
Fixed
camera initialization on iOS, when multiple components are used on the same page
6.1.3 - 2024-08-21
Fixed
unnecessary camera permission request when photo was taken
6.1.2 - 2024-08-07
Fixed
customization of "smile" and "keep_neutral_expression" instructions
Added
ref
property to custom HTMLElement type
6.1.1 - 2024-08-01
Incremental version upgrade
6.1.0 - 2024-07-09
Added
font size, family, weight and style customization
Introduced detection of facial expression, autocapture of face in the smiling phase.
Changed
Update
SAM
to version 1.39.3
6.0.0 - 2024-05-27
Added
License validation
dot_embedded_bg.wasm
- WASM binary file for license validationlicensePath
- path to license fileTiers - freemium/premium mode
Freemium overlay component Non-UI components
Changed
Update
SAM
to version 1.38.1Use smile detection by
FaceAutoCapture
component
Removed
Removed
SmileCustomEvent.ANIMATION_END
5.2.8 - 2024-04-10
Changed
Performance and documentation optimization
Added
Added
minFaceSizeRatio
andmaxFaceSizeRatio
properties to control the size of the face detection area.
5.2.7 - 2024-03-01
Changed
Performance and documentation optimization
5.2.6 - 2024-02-20
Changed
Fix typo in
package.json
5.2.5 - 2024-02-20
Changed
Remove
type:module
frompackage.json
file in order to fix issue with Angular 16
5.2.4 - 2024-02-14
Fixed
Component not loading in certain conditions on Android in Firefox
5.2.3 - 2023-12-14
Added
styleTarget
property for specifying an alternate DOM node to inject styles. This is useful when rendering components in a shadow DOM. If not specified, the styles are injected into the<head>
element of the document.
5.2.2 - 2023-11-27
Removed
Use of
eval()
function in third-party library
5.2.1 - 2023-10-18
Changed
Update
SAM
to version 1.35.3Improved detection speed
Minimum camera resolution validation changed from validating both sides to validating just the shorter side. Now, minimal size of the shorter side is 720px.
5.2.0 - 2023-10-03
Changed
Update
SAM
to version 1.35.0Improved detection speed
Added
wasmDirectoryPath
- path to directory with Web assembly files.Support for multiple Web assembly files. The component will automatically load the
sam.wasm
orsam_simd.wasm
files from the specified folder, based on the device support of SIMD instructions. Web assembly files have to be placed inwasm
folder.
Removed
samWasmUrl
- property was removed. UsewasmDirectoryPath
property instead.
5.1.0 - 2023-09-28
First released version