logo logo

DOT iOS Document library

v2.3.0

Introduction

DOT iOS Document provides components for digital document capture using the latest Innovatrics Small Area Matcher (SAM) image processing library. It wraps the core functionality of SAM to higher level module which is easy to integrate into an iOS application. Components having UI are available as UIViewController and can be embedded into application’s existing UI or presented using standard methods (ex. show(), push(), present()).

List of Non-UI Components

DOCUMENT DETECTOR

Stateless component for performing document detection on an image and perspective correction on the detected image region.

IMAGE PARAMETERS ANALYZER

Stateless component for computing image parameters such as brightness, sharpness and hotspots score.

DOCUMENT AUTO CAPTURE CONTROLLER

Component that performs detection on images and provides hints to guide custom UI components. Contains logic to auto capture documents that conform to detection validation rules.

List of UI Components

DOCUMENT CAPTURE PLACEHOLDER VIEW CONTROLLER

Visual component for capturing good quality documents suitable for verification. Intended to be embedded into existing UI screens. Contains placeholder to visually guide the user.

DOCUMENT CAPTURE FREE VIEW CONTROLLER

Visual component for capturing any document. Intended to be embedded into existing UI screens.

Requirements

  • Xcode 11.4+

  • iOS 10.0+

  • Swift or Objective-C

  • CocoaPods

Distribution

DOT iOS Document is distributed as a XCFramework - DotDocument.xcframework using Cocoapods with its dependencies stored in our public github repository. It can be easily integrated into XCode with custom definition of podspecs. First step is to insert following line of code on top of you Podfile.

Podfile
source 'https://github.com/innovatrics/innovatrics-podspecs'

Then DOT iOS Document dependency must be specified in Podfile. Dependencies of DOT iOS Document will be downloaded alongside it.

Podfile
source 'https://github.com/innovatrics/innovatrics-podspecs'

use_frameworks!

target 'YOUR_TARGET' do

pod 'dot-document'

end

In case of CocoaPods problem with pod install, try to clone the private pod repository manually.

pod repo remove innovatrics
pod repo add innovatrics https://github.com/innovatrics/innovatrics-podspecs

Sample Project

Usage and configuration of DOT iOS Document are demonstrated in DOT iOS Kit Sample demo project. To run the sample, you must install pods and then open DOTSample.xcworkspace.

Permissions

Applications that make use of camera require following permission in Info.plist:

Info.plist
<key>NSCameraUsageDescription</key>
	<string>Your usage description</string>

Localization

String resources can be overridden in your application and alternative strings for supported languages can be provided following these two steps:

  1. Add your own Localizable.strings file to your project using standard iOS localization mechanism. To change a specific text override corresponding key in this Localizable.strings file.

  2. Set the localization bundle to the bundle of your application (preferably during the application launch in your AppDelegate).

Use this setup if you want to use standard iOS localization mechanism, which means your App uses system defined locale.

import DotDocument

DotDocumentLocalization.bundle = .main

Custom localization

You can override standard iOS localization mechanism by providing your own translation dictionary and setting the DotDocumentLocalization.useLocalizationDictionary flag to true. Use this setup if you do not want to use standard iOS localization mechanism, which means your App ignores system defined locale and uses its own custom locale.

import DotDocument

guard let localizableUrl = Bundle.main.url(forResource: "Localizable", withExtension: "strings", subdirectory: nil, localization: "de"),
      let dictionary = NSDictionary(contentsOf: localizableUrl) as? [String: String]
else { return }

DotDocumentLocalization.useLocalizationDictionary = true
DotDocumentLocalization.localizationDictionary = dictionary
Localizable.strings
"document_autocapture.instruction.brightness_low" = "More light needed";
"document_autocapture.instruction.brightness_high" = "Less light needed";
"document_autocapture.instruction.sharpness_low" = "More light needed";
"document_autocapture.instruction.document_small" = "Move closer";
"document_autocapture.instruction.document_large" = "Move back";
"document_autocapture.instruction.document_not_detected" = "Scan document";
"document_autocapture.instruction.does_not_fit_placeholder" = "Center document";
"document_autocapture.instruction.hotspots_score_high" = "Avoid reflections";
"document_autocapture.instruction.hold_still" = "Hold still...";

Logging

DOT iOS Document supports logging using a global Logger instance. You can set the log level as follows:

import DotDocument

// Set debug log level
Logger.logLevel = .debug

Non-UI Components

Document Detector

/// Detect document in image and return `DetectionResult` that describes the detected document corners and attributes.
@objc public static func detect(image: Image) throws -> DetectionResult

/// Warp detected region in the input `Image` and produce a new `Image` with perspective corrected region.
@objc public static func warpPerspective(image: Image, widthHeightRatio: Double, corners: [Double]) throws -> Image

The DocumentDetector class provides two static methods that can be used to implement custom document detection applications. To detect documents in Image use the detect method which will generate DetectionResult response. This result type contains basic information from the detection algorithm such as the detection confidence, width to height ratio and position of the document corners in the image.

import UIKit
import DotDocument

let image = UIImage(named: "document_image.jpg")!.cgImage!
let detectionImage = Image(cgImage: image)
let detectionResult = try! DocumentDetector.detect(image: detectionImage)
print(detectionResult)
(confidence: 0.904, widthHeightRatio: 1.5864, corners: [0.13738317757009347, 0.3475823405746321, 0.8943925233644859, 0.3433777154870357, 0.9037383177570093, 0.7224947442186405, 0.12990654205607477, 0.7203924316748423])

Another method provided by the DocumentDetector is warpPerspective, this method will create a new Image that will contain a perspective corrected cutout of the document based on the detection corners and width to height ratio.

let warp = try! DocumentDetector.warpPerspective(image: image, widthHeightRatio: detectionResult.widthHeightRatio, corners: detectionResult.corners)

Image Parameters Analyzer

/// Analyze an image and return `ImageParameters`.
@objc public static func analyze(image: Image) throws -> ImageParameters

To compute parameters of an image you can use ImageParametersAnalyzer and its method analyze. The result type contains sharpness, brightness and hotspots score of the image.

import UIKit
import DotDocument

let image = UIImage(named: "document_image.jpg")!.cgImage!
let image = Image(cgImage: image)
let imageParams = try! ImageParametersAnalyzer.analyze(image: image)
print(imageParams)
(sharpness: 0.322, brightness: 0.667, hotspotsScore: 0.0)

Document Auto Capture Controller

/// Delegate to pass detection and capture events to.
@objc weak public var delegate: DocumentAutoCaptureControllerDelegate?

/// Document detection evaluator.
@objc public var evaluator: DocumentAutoCaptureFrameParametersEvaluator?

/// Active auto capture type.
@objc public let autoCaptureType: DocumentAutoCaptureType

/// Restart auto capture process.
@objc public func restart()

/// Schedule document detection on the provided image.
@objc public func detect(imageBatch: ImageBatch)

Validation is a process that in essence checks the DocumentAutoCaptureFrameParameters contents and accepts or rejects a document. Depending on application the validation rules will vary. One of the simplest approaches to validation is to check the detection confidence value by defining a detection threshold. Detected documents that have confidence above the threshold are accepted as valid documents.

For this purpose we provide DocumentAutoCaptureController class that manages detection and validation process in separate queue asynchronously and informs a delegate of the detection result. To validate the document, DocumentAutoCaptureController requires DocumentAutoCaptureFrameParametersEvaluator with appropriate array of validators. This evaulator combines results of all validations into single result - array of hints.

When initializing DocumentAutoCaptureController class, you may choose which DocumentAutoCaptureType will be used. If you choose DocumentAutoCaptureType.simple, primaryImage of ImageBatch will be used for document detection and image parameters evaluation. If you choose DocumentAutoCaptureType.withSecondaryImage, then primaryImage of ImageBatch will be used for document detection and secondaryImage of ImageBatch will be used for evaluation of image parameters.

// Delegate to catch asynchronous detection and capture events
class CaptureDelegate: DocumentAutoCaptureControllerDelegate {
    func onDetection(image: Image, frameParameters: DocumentAutoCaptureFrameParameters, documentAutoCaptureHints: [DocumentAutoCaptureHintWrapper]) {
        print("Document detection - \(frameParameters)")
    }
    func onCapture(image: Image, frameParameters: DocumentAutoCaptureFrameParameters) {
        print("Document is valid")
    }
}

// Detection controller to offload detection into a queue and validate the results
let captureController = DocumentAutoCaptureController()
// Associate delegate with the controller
captureController.delegate = CaptureDelegate()
// Add a validator that simply checks the detection confidence
captureController.evaluator = DocumentAutoCaptureFrameParametersEvaluator(validators: [DocumentNotDetectedValidator(confidenceThreshold: 0.2)])

// Add new detection to the queue
captureController.detect(imageBatch: SimpleImageBatch(image: documentImage))

Delegate action onDetection is triggered after each detection and provides the delegate with additional information in a form of an array of DetectionHint enum values, this value is .documentNotDetected in case of the used DocumentNotDetectedValidator.

When the hints array is not empty the validator has determined the document to be not valid and the onCapture event will never be triggered.

Detection hints are especially useful for guiding the user experience when detecting images from video frames. This is used extensively in the provided UIViewController implementation DocumentCapturePlaceholderViewController.

Available Validators

Validators implement the DocumentAutoCaptureFrameParametersValidator protocol and contain simple checks of detected document parameters. We provide the following list of simple validators that check single aspect of the DocumentAutoCaptureFrameParameters.

Table 1. List of available basic validators
ValidatorConditionHints

DocumentLargeValidator

detected document width / image width < documentWidthToImageWidthRatioThreshold

.documentLarge

DocumentSmallValidator

detected document width / image width > documentWidthToImageWidthRatioThreshold

.documentSmall

BrightnessHighValidator

ImageParameters.brightness < threshold

.brightnessHigh

BrightnessLowValidator

ImageParameters.brightness > threshold

.brightnessLow

SharpnessLowValidator

ImageParameters.sharpness > threshold

.sharpnessLow

DocumentNotDetectedValidator

DetectionResult.confidence > confidenceThreshold

.documentNotDetected

DocumentDoesNotFitPlaceholderValidator

detected document corners to placeholder corners distance < detectedToPlaceholderCornersDistanceThreshold

.documentDoesNotFitPlaceholder

HotspotsScoreHighValidator

ImageParameters.hotspotsScore < threshold

.hotspotsScoreHigh

Hold Still Phase

It is beneficial for the capture quality to instruct user to hold still while multiple high quality detections are collected. This allows DocumentAutoCaptureController to select the detection with the best attributes as the capture candidate. When the first valid document is detected the DocumentAutoCaptureController will enter hold still phase and will generate .holdStill hint. After specified delay the hint will disappear and the collection phase will end with a capture event, returning the best quality detection that ocurred.

UI Components

Document Capture Placeholder View Controller

/// Delegate to receive detection and capture events asynchronously.
@objc weak public var delegate: DocumentCaptureViewControllerDelegate?

/// Source of detection images.
@objc public var documentSource: DocumentSourceProtocol?

/// Document auto capture controller.
@objc public var captureController: DocumentAutoCaptureController?

/// Begin document capture.
@objc public func start()

/// Restart document capture process.
@objc public func restart()

/// Force capture of next detection
@objc public func capture()

To quickly integrate document detection into any application a customizable DocumentCapturePlaceholderViewController UI component is provided. This UI component is already pre-configured to capture documents from various sources such as phone camera, static images or video. To start document capture process start() method has to be called. Document capture process will automatically stop when it successfully finishes or when the UI component will disappear. You can call restart() to restart the capture process from the beginning.

import UIKit
import DotDocument

class CaptureDelegate: DocumentCaptureViewControllerDelegate {
    func documentDetected(_ controller: DocumentCaptureViewController, image: Image, frameParameters: DocumentAutoCaptureFrameParameters, documentAutoCaptureHints: [DocumentAutoCaptureHintWrapper]) {
        print("Document detection - \(frameParameters)")
    }
    func documentCaptured(_ controller: DocumentCaptureViewController, image: Image, frameParameters: DocumentAutoCaptureFrameParameters) {
	print("Document captured - \(frameParameters)")
    }
}
let controller = DocumentCapturePlaceholderViewController.create()
controller.delegate = CaptureDelegate()

navigationController?.pushViewController(controller, animated: true)

Document Auto Capture View Controller Configuration

To configure document source and validator thresholds, you can use DocumentAutoCaptureViewControllerConfiguration. It has following attributes:

/// Document source.
@objc public let documentSource: DocumentSourceProtocol

/// Document detection confidence threshold in range [0, 1.0].
@objc public let confidenceThreshold: Double

/// Minimal accepted sharpness threshold in range [0, 1.0].
@objc public let sharpnessLowThreshold: Double

/// Minimal accepted brightness threshold in range [0, 1.0].
@objc public let brightnessLowThreshold: Double

/// Maximal accepted brightness threshold in range [0, 1.0].
@objc public let brightnessHighThreshold: Double

/// Maximal accepted hotspots score threshold in range [0, 1.0].
@objc public let hotspotsScoreHighThreshold: Double
let configuration = DocumentAutoCaptureViewControllerConfiguration(
			documentSource: CameraDocumentSource(),
			confidenceThreshold: 0.2,
			sharpnessLowThreshold: 0.85,
			brightnessLowThreshold: 0.25,
			brightnessHighThreshold: 0.9,
			hotspotsScoreHighThreshold: 0.008)

let controller = DocumentCapturePlaceholderViewController.create(configuration: configuration)

Document Capture View Controller Style

To customize UI components, you can use DocumentCaptureViewControllerStyle. It has following attributes:

/// Hint label font.
@objc public let hintFont: UIFont

/// Hint label text color.
@objc public let hintTextColor: UIColor

/// Hint label text color during capture.
@objc public let hintCapturingTextColor: UIColor

/// Hint background color.
@objc public let hintBackgroundColor: UIColor

/// Hint background color during capture.
@objc public let hintCapturingBackgroundColor: UIColor

/// Placeholder color.
@objc public let placeholderColor: UIColor

/// Placeholder color during capture.
@objc public let placeholderCapturingColor: UIColor
let style = DocumentCaptureViewControllerStyle(hintTextColor: .red,
                                               hintCapturingTextColor: .green,
                                               hintBackgroundColor: .gray,
                                               hintCapturingBackgroundColor: .gray)

let controller = DocumentCapturePlaceholderViewController.create(style: style)

Document Image Sources

Document detection relies on the document source instance to handle any hardware configuration or image pre-processing and transformation. Document sources provide images or video frames which you can use for the document detection.

During development or for testing purposes you can switch do different DocumentSourceProtocol implementation as CameraDocumentSource is only available on iOS device and does not work in simulator. The following basic document sources are implemented:

  • ImageDocumentSource - Document detection from static image, periodically. You can switch the image at anytime to implement various testing scenarios.

  • VideoDocumentSource - Document detection in video. Especially useful for tests and debugging in simulator.

  • CameraDocumentSource - Document detection from camera feed. This is the primary source of images for document detection but it is not supported in the simulator.

For more complex document sources you may use your own DocumentSourceProtocol implementation.

Document Capture Free View Controller

/// Delegate to receive detection and capture events asynchronously.
@objc weak public var delegate: DocumentCaptureViewControllerDelegate?

/// Source of detection images.
@objc public var documentSource: DocumentSourceProtocol?

/// Document auto capture controller.
@objc public var captureController: DocumentAutoCaptureController?

/// Begin document capture.
@objc public func start()

/// Restart document capture process.
@objc public func restart()

/// Force capture of next detection
@objc public func capture()

To quickly integrate document detection into any application a customizable DocumentCaptureFreeViewController UI component is provided. This UI component is already pre-configured to capture documents from various sources such as phone camera, static images or video.

import UIKit
import DotDocument

class CaptureDelegate: DocumentCaptureViewControllerDelegate {
    func documentDetected(_ controller: DocumentCaptureViewController, image: Image, frameParameters: DocumentAutoCaptureFrameParameters, documentAutoCaptureHints: [DocumentAutoCaptureHintWrapper]) {
        print("Document detection - \(frameParameters)")
    }
    func documentCaptured(_ controller: DocumentCaptureViewController, image: Image, frameParameters: DocumentAutoCaptureFrameParameters) {
	print("Document captured - \(frameParameters)")
    }
}
let controller = DocumentCaptureFreeViewController.create()
controller.delegate = CaptureDelegate()

navigationController?.pushViewController(controller, animated: true)

type: redirect redirect: https://developers.innovatrics.com/digital-onboarding/docs/latest-version-matrix/ robots: noindex ---

Changelog

Unreleased

2.3.0 - 2021-05-14

Added

  • DocumentAutoCaptureViewControllerConfiguration to enable additional configuration of UI components

  • DocumentAutoCaptureViewControllerConfiguration property documentSource

  • DocumentAutoCaptureViewControllerConfiguration property confidenceThreshold

  • DocumentAutoCaptureViewControllerConfiguration property sharpnessLowThreshold

  • DocumentAutoCaptureViewControllerConfiguration property brightnessLowThreshold

  • DocumentAutoCaptureViewControllerConfiguration property brightnessHighThreshold

  • DocumentAutoCaptureViewControllerConfiguration property hotspotsScoreHighThreshold

  • DocumentDoesNotFitPlaceholderValidator

Changed

  • DocumentCapturePlaceholderViewController.create()

  • DocumentCaptureFreeViewController.create()

  • removed BorderMarginValidator

  • removed DocumentCenteredValidator

  • removed DocumentRotationValidator

  • removed SharpnessHighValidator

  • removed DocumentAutoCaptureHint.sharpnessHigh, .widthToHeightLow, .widthToHeightHigh, .documentNotCentered

  • BrightnessHighValidator.maxBrightness to .threshold

  • BrightnessLowValidator.minBrightness to .threshold

  • DocumentLargeValidator.maxSize to .documentWidthToImageWidthRatioThreshold

  • DocumentSmallValidator.minSize to .documentWidthToImageWidthRatioThreshold

  • DocumentNotDetectedValidator.minConfidence to .confidenceThreshold

  • SharpnessLowValidator.minSharpness to .threshold

  • localization keys

2.2.2 - 2021-05-03

Fixed

  • allow multiline hint label

2.2.1 - 2021-03-31

Fixed

  • performance issue in document autocapture process

Changed

  • renamed SingleImageBatch to SimpleImageBatch

  • PlaceholderImageBatch.init to init(image: Image, detectionFrame: CGRect, imageParametersFrame: CGRect)

2.2.0 - 2021-03-17

Added

  • DotDocumentLocalization.localizationDictionary and .useLocalizationDictionary to enable overriding of standard iOS localization mechanism

Changed

  • renamed Localization class to DotDocumentLocalization

2.1.0 - 2021-01-27

Changed

  • add DocumentCaptureViewController.start() to start capture process explicitly

  • capture process of DocumentCaptureViewController will no longer start implicitly

2.0.0 - 2021-01-13

Added

  • Localization class, to support localization in more complex projects

  • HotspotsScoreHighValidator

  • ImageParametersAnalyzer

  • ImageParameters

  • protocol ImageBatch

  • SingleImageBatch

  • PlaceholderImageBatch

  • DocumentAutoCaptureFrameParameters

  • DocumentAutoCaptureFrameParametersEvaluator

  • DocumentAutoCaptureType.simple and DocumentAutoCaptureType.secondaryImageParameters

  • DocumentSourceDelegate.sourceNotAuthorized() to allow handling of camera permission

  • DocumentCaptureViewControllerDelegate.documentSourceNotAuthorized() to allow handling of camera permission

Changed

  • renamed framework and module to DotDocument

  • changed localization keys

  • protocol DetectionValidatorProtocol renamed to DocumentAutoCaptureFrameParametersValidator

  • DocumentCaptureController renamed to DocumentAutoCaptureController

  • DocumentCaptureControllerDelegate renamed to DocumentAutoCaptureControllerDelegate

  • removed DocumentAutoCaptureController.detectionValidator added .evaluator of type DocumentAutoCaptureFrameParametersEvaluator instead

  • DocumentAutoCaptureController now detects from ImageBatch instead of Image, to allow more complex auto capture workflows

  • removed DocumentCaptureViewController.requestHighResolutionImage(), added .highResolutionCapture instead

  • removed DocumentCaptureViewController.startDocumentCapture(), .stopDocumentCapture(), added .restart() instead

  • removed DocumentCaptureController.startDetection(), .stopDetection() added DocumentAutoCaptureController.restart() instead

  • DocumentAutoCaptureFrameParametersValidator now requires DocumentAutoCaptureFrameParameters instead of DetectionResult

  • DocumentAutoCaptureControllerDelegate and DocumentCaptureViewControllerDelegate now provides DocumentAutoCaptureFrameParameters instead of DetectionResult

  • ImageParameters.brightness, .sharpness, .hotspotsScore and DetectionResult.confidence is now normalized to [0, 1.0]

  • confidence, brightness, sharpness, hotspotsScore validators take normalized input values

  • removed SequenceValidator use DocumentAutoCaptureFrameParametersEvaluator instead

  • removed SteadyValidator, hold still phase is always present in auto capture process and is handled by DocumentAutoCaptureController

1.2.2 - 2020-12-17

Added

  • support for iOS Simulator arm64 architecture

1.2.1 - 2020-11-04

Fixed

  • for CameraDocumentSource override NSObject.init() with convenience CameraDocumentSource.init() and with CameraDocumentSourcePreset.fullHD as default parameter.

1.2.0 - 2020-11-04

Added

  • CameraDocumentSourcePreset enum

Changed

  • CameraDocumentSource.init() has CameraDocumentSourcePreset parameter.

1.1.5 - 2020-10-23

Fixed

  • crop high resolution image from DocumentCapturePlaceholderViewController

  • clear preview layer when CameraDocumentSource.stopSession() is called

Changed

  • CameraDocumentSource.orientation type to AVCaptureVideoOrientation

Added

  • documentCaptureDidLayoutSubviews to DocumentCaptureViewControllerDelegate

1.1.4 - 2020-10-02

Fixed

  • orientation of high resolution image returned from DocumentCaptureViewController

1.1.3 - 2020-09-17

Changed

  • updated SAM to 2.0.0

1.1.2 - 2020-09-16

Fixed

  • SAM architecture selection when building for release

1.1.1 - 2020-08-28

Fixed

  • camera orientation wrong initial value when presenting DocumentCaptureViewController

1.1.0 - 2020-08-25

Changed

  • detect document from cropped image

  • DocumentCaptureController has startDetection(), stopDetection()

  • DocumentSourceProtocol has startSession(), stopSession()

  • DocumentCaptureViewController has startDocumentCapture(), stopDocumentCapture()

1.0.2 - 2020-08-04

Changed

  • use Operations in DocumentCaptureController

1.0.1 - 2020-08-04

Changed

  • removed type constraint from AnnotationLayerProtocol

1.0.0 - 2020-08-03

Fixed

  • fixed memory access issue when converting images to Image

Changed

  • CameraDocumentSource learned orientation support

  • rename DocumentCaptureSimpleViewController to DocumentCapturePlaceholderViewController

  • reworked validation process in DocumentCapturePlaceholderViewController

  • DocumentCaptureViewControllerDelegate is now shared between view controllers

  • removed DocumentDoesNotFitPlaceholderValidator

  • DocumentSmallValidator and DocumentLargeValidator now calculate using area instead of width

  • DocumentCaptureController.detectionWidth is now public

  • Image conversion to and from vImage_Buffer is now public

  • added Image transformation support

  • added transformation support to Camera and Video document source

Added

  • Added DocumentCaptureViewControllerStyle

  • Added DocumentCaptureFreeViewController

  • Landscape support in UI components

  • DocumentRotationValidator

  • DocumentCenterValidator

  • Logging support

0.2.0 - 2020-07-17

Changed

  • minimal supported iOS version to 10.0

0.1.1 - 2020-07-16

Changed

  • renamed module to DOTDocument

0.1.0 - 2020-07-13

  • First release

← DOT Digital Identity Service
DOT iOS Document library →