Data & Privacy

Data, including the type of data processed, storage location, retention period, encryption methods, data transfer, and isolation practices, are key considerations when handling privacy.

Types of Data#

The system processes and stores various types of data, each serving a specific purpose and requiring appropriate handling. The data types are categorized as follows:

Face Biometry#

Mechanism#

When Face Detection is turned on (default setting), the face detection neural network analyzes incoming frames for faces.

Each segment passing quality threshold is sent to face extraction neural network that creates face template. A face template is a mathematical vector representation of a face. The original image or physical appearance cannot be reversibly reconstructed from this data. Primary purpose of face template is matching between each other for similarity - vector comparison.

The face template itself is anonymous - it does not contain any name, ID, or personal details.

Each extracted template is compared to all enrolled templates in a process called matching.

Considerations#

Detected pedestrians or objects may include faces of people passing by that haven’t been processed or matched. Also, the full image of a detection might contain faces or details that weren’t specifically targeted. For instance, when focusing on detecting cars or trucks, a pedestrian’s visible face or a truck driver’s face might still appear in the background or in a cropped part of the image.

The SmartFace offers robust and flexible configuration of the privacy settings.

Data Storage Locations#

There are two data storage systems:

• Relational databases, either PostgreSQL or Microsoft SQL
• BLOB storage, either Min.IO or AWS S3

Data Retention Period#

The retention period can be configured from one to an unlimited number of days. Data is continuously stored, with a cleanup job running daily that safely deletes all data older than the defined age.

BLOB data storage for images can be turned off entirely, so no pictures of detected faces, pedestrians or objects is stored at all.

Data Encryption#

Data encryption includes two parts: encrypting data at rest (where it is stored) and encrypting data in transit.

Encryption at Rest#

Full disk encryption is recommended. Additionally, each data storage service supports further encryption:

• PostgreSQL encryption
• Microsoft SQL encryption
• MinIO encryption

Encryption in Transit#

All internal SmartFace communication can be configured to run encrypted via TLS. Running TLS requires custom TLS certificates, so by default, this is disabled.

Data Transfer#

The SmartFace solution (SmartFace Station, SmartFace Platform, SmartFace Embedded Stream Processor) only downloads binaries, executables, and Docker images over the internet. If needed, these components can be downloaded to another machine and transferred into an isolated environment via local storage (e.g., USB).

In default configuration SmartFace does not send any data to the internet. Each SmartFace component communicates with others via a (local) network; no internet connection is required.

Data Isolation#

Multiple deployment strategies are available, ranging from public internet-facing deployment to completely isolated, secure local networks without internet access.

Single Machine Deployment#

SmartFace is deployed on a single machine with a USB camera directly attached or IP cameras on a secured, isolated LAN without internet access. No data leaves the SmartFace machine.

Single Machine on Local Network#

SmartFace is deployed on a secured, isolated LAN without internet access. One or many IP cameras or Edge devices are connected in the LAN. No data leaves the LAN.

SmartFace in a Datacenter#

SmartFace is deployed in a datacenter, with one or many Edge devices sending data over the internet via SSL-secured connections. Data is securely stored in the datacenter.

SmartFace on Multiple Locations#

Multiple SmartFace instances are deployed at different locations connected over the internet. Data is securely stored at each location and transmitted over the internet via TLS secured connections.

Privacy Settings#

The SmartFace system offers robust and flexible privacy settings to minimize the storage of sensitive data while maintaining the functionality required for real-time biometric identification and analytics. By configuring various options for watchlist management, data retention, and live video source storage, organizations can tailor the system to meet specific privacy and compliance requirements.

The next section provides an overview of the available settings and configurations, helping users strike the right balance between operational efficiency and data privacy.

High-Level Privacy Configuration Possibilities

• Anonymized Watchlist Management
  Omit or anonymize sensitive fields like names, labels, and IDs during registration, and disable image storage if not required.
• Data Retention Control
  Automatically remove detection, image, and match data older than a specified retention period through database cleanup.
• Global Data Storage Restriction
  Completely disable data storage from live video sources while retaining real-time event processing capabilities.
• Per-Source Data and Detection Configuration
  Customize data storage and detector enablement settings for each RTSP camera or EdgeStream.
• Save Strategies
  Limit database storage based on detection modality (e.g., store only matched faces or disable storage entirely).
• Image Data Control
  Disable storage of full-frame and cropped image data for privacy-sensitive operations.
• EdgeStream-Specific Privacy Settings
  Prevent the transmission of unidentified faces and selectively disable features like image crop or video preview. These settings provide granular control over how biometric and video data is collected, stored, and used, ensuring compliance with privacy standards and regulations.

Anonymized Watchlist Management#

A record representing an individual person that can be recognized by the system is called a “Watchlist Member.”

The Watchlist Member entity contains several fields that may include sensitive information, such as display name, full name, ID, labels, registration image, and crop image. None of these fields are required for the identification process.

Fields that can be omitted during the registration process:

• Display Name: This can be left empty, or an anonymized value can be used.
• Full Name: This can also be left empty, or an anonymized value can be used.
• Labels: Labels could be omitted.
• ID: An anonymized ID from an external system can be used, or it can be left empty during registration. In the latter case, a random GUID is generated by the system.

Image data is required during the registration process to extract the biometric template. However, storing the full frame and crop image in the SmartFace database can be disabled by setting the following environment variable for the API service: NoSqlDataStorageDisabled=false

This setting applies to the following services: API, Base, VideoDataCollector, VideoDataAggregator, Camera, and EdgeStreamProcessor.

Data Retention Control#

All data collected during normal operation of processing live video sources (e.g. EdgeStreams and RTSP cameras) typically includes detected faces, palms, and other objects. These data can be removed on a daily basis using the Database Cleanup Process.

There are three types of data related to a single event:

1. Detection: Includes time, position, analytic data, and biometric templates.
2. Image Data: Includes full frame and crop images, if image storage is not disabled by setting NoSqlDataStorageDisabled=false.
3. Match Result: Contains details about the identified Watchlist Member. Match results are created only if the matching score exceeds the threshold defined in the relevant watchlist.

The Database Cleanup Process removes all detections and related image data older than a defined time range (default: 14 days). Additionally, it can be configured to remove match results related to these detections.

Global Data Storage Restriction#

At the global level, it is possible to disable the transmission of all data from video sources to the service responsible for storing data in the database. This effectively stops database usage altogether. However, real-time notifications such as faceProcessed, objectProcessed, identificationEvent, matchResult, and noMatchResult remain unaffected, ensuring that real-time event-based business logic continues to function.

To disable data transmission globally, send the following request:

PUT /api/v1/Setup/DataStorage/Video

Payload example:

{
  "storageMode": "None"
}

Per-Source Data and Detection Configuration#

Storing data collected during normal operation from Live Video Sources can be restricted using various settings directly in the SmartFace Station for both RTSP Cameras and Edge Streams.

Detection of specific object types or modalities can be disabled per live video source.

For RTSP cameras, disable detectors via resourceId on the camera entity:

PUT /api/v1/Cameras

Payload example:

{
...
  "faceDetectorResourceId": "none",
  "pedestrianDetectorResourceId": "none",
  "objectDetectorResourceId": "none",
...
}

For EdgeStreams, disable detectors via boolean properties on the EdgeStream entity:

PUT /api/v1/EdgeStreams

Payload example:

{
...
  "settings": {
    "visualCodeDetection": {
      "enable": false,
      ...
    },
    "faceDetection": {
      "enable": false,
      ...
    },
    ...
  },
...
}

Save Strategies#

Data storage in the database can be limited using save strategies, which are applicable to each detection modality. In the context of privacy, the none and matchedOnly strategies are particularly relevant.

• MatchedOnly: Stores only faces matched against a Watchlist Member.
• None: Stores nothing unless the linked object is configured to save.

More about Save strategies could be found here: Data Retention and Backup

You can set the saving strategies within the SmartFace Station (for both RTSP Cameras and Edge Cameras) or using the APIs.

Using the API for RTSP Cameras:

PUT /api/v1/Cameras

Payload example:

{
...
  "faceSaveStrategy": "MatchedOnly",
  "pedestrianSaveStrategy": "none",
  "objectSaveStrategy": "none",
...
}

Using the API for Edge Cameras:

PUT /api/v1/EdgeStreams

PUT /api/v1/EdgeStreams

{
...
  "faceSaveStrategy": "MatchedOnly",
  "objectSaveStrategy": "none",
...
}

Supported values:
FaceSaveStrategy: Balanced, None, All, MatchedOnly
PedestrianSaveStrategy: Balanced, None, All
ObjectSaveStrategy: Balanced, None, All, MatchedOnly

Image Data Control#

The storage of full-frame image data can be disabled per camera or EdgeStream by setting saveFrameImageData to false.

For RTSP cameras, you can also disable video preview:

PUT /api/v1/Cameras

Payload example:

{
...
  "saveFrameImageData": false,
  "mpeG1PreviewEnabled": false,
...
}

For edgeStreams there is also possibility to turn off sending crop image to server on EdgeStream entity via:

PUT /api/v1/EdgeStreams

Payload example:

{
...
  "saveFrameImageData": false,
  "settings": {
    "faceDetection": {
      "crop": {
        "enable": false,
        ...
      }
    },
...
}

EdgeStream-Specific Privacy Settings#

For EdgeStreams, it is possible to prevent the transmission of unidentified faces to the server, ensuring the system only processes known individuals. Configure this on the EdgeStream entity:

PUT /api/v1/EdgeStreams

Payload example:

{
...
  "messaging": {
    "identifiedOnly": true
    ...
    },
...
}