logo logo

Watchlist Management

SmartFace Embedded Stream Processor supports the identification of the detected face against the watchlist stored directly on the camera or edge device.

To register and manage watchlist members and their faces you can use MQTT topics or REST API:

SFEStream Processor Watchlist Management via MQTT

This part of documentation describes the MQTT topics used for watchlist members/users management in SFE Stream Processor.

Using MQTT for communication with the camera requires the MQTT broker and MQTT Client to run as part of your solution.

SmartFace Platform is already integrated with SFE Stream Processor using MQTT.

Root topic

All topics are prefixed with the root :settings.connection.topic and :settings.connection.client_id which are configured in the settings file.

Response Topics

SFE Stream Processor uses MQTT v3 which does not support response topics as MQTT v5 does. Instead, responses are always sent to the same topic that was used as a request with the /response suffix added. Additionally, in some topics, it is possible to extend request topics with the :request_id to differentiate between specific requests when dealing with multiple requests at the same time.

Message Payloads

The MQTT message payload format is configurable in the settings file. Supported formats are JSON, YAML and Protobuf.

Result types are used to report the result of an operation Result<Message, ErrorMessage>). The result type is either Message or ErrorMessage. The Message type is specific to the operation while the ErrorMessage type is common and contains an error message string. Any of the result types can be used as a response payload unless otherwise specified.

There are three message types used in the MQTT topics:

  • Retained<Message> - Messages that are retained by the broker and sent to new subscribers
  • Periodic<Message> - Messages that are sent periodically
  • Response<Message> - Messages that are sent as a response to a request

Health Topic

:topic/:client_id/health
  => Retained(HealthReport)

The health topic is used to send the health report of the stream processor to the MQTT broker. The health report is updated when the health status changes.

  • Online - pushed immediately after connecting to an MQTT broker
  • Offline - pushed after the stream processor is restarted (e.g. settings file has changed) or correctly shutdown (SIGTERM/SIGINT)
  • OfflineDisconnected - pushed as a last will message after losing connection to the broker or in case of any other unexpected failure

See also the health proto message format

FrameData Topic

:topic/:client_id/frame_data
  => Periodic(FrameData)

FrameData topic is currently the primary output topic of the stream processor. It is used to periodically send the frame data to the MQTT broker. Depending on the configuration, the frame data can contain the following information:

  • Face detection bounding boxes
  • Face landmarks
  • Face crops
  • Face templates
  • Face identification results
  • Active tracking ids
  • New tracking ids
  • Lost tracking ids
  • Full frame data

See also the frame_data proto message format

User Topics

Stream Processor manages a user database that can be used to store face templates and user information. The user database is stored in the stream processor storage directory.

Add new face

Add a new face for a specific user UUID using topic ../user/:user_id/face/add. The response is an updated user record or an error. The payload can be either an image or a template. Images are processed by the stream processor and a template is extracted from the image. The template is then added to the user database. The template extraction is done using the same settings as the stream processor. Valid image formats are JPEG and PNG. :user_id is expected to be in UUID v4 format, for example: 4af8b586-1361-4ff2-9a58-034aa9a03c0f

:topic/:client_id/user/:user_id/face/add/[:request_id]
  <= Image or Template bytes
  => Response(Result<Record, Error>)

Example usage with MQTT CLI Client:

# To create new user with face or add face to existing user with image
mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user/4af8b586-1361-4ff2-9a58-034aa9a03c0f/face/add -m:file face_image.jpg

# To create new user with face or add face to existing user with template
mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user/4af8b586-1361-4ff2-9a58-034aa9a03c0f/face/add -m:file face_template.icf

Add metadata for specific user

Use topic ../user/:user_id/metadata/:key/:value to associate any metadata with a specific user. Metadata is defined as key:value pair.

:topic/:client_id/user/:user_id/metadata/:key/:value/[:request_id]
  <= Any message
  => Response(Result<(), Error>)

Example usage with MQTT CLI Client:

mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user/4af8b586-1361-4ff2-9a58-034aa9a03c0f/metadata/company/innovatrics -m:empty

List all users

The topic ../user is used to query the entire user database. The response is a list of all users (including templates and metadata) stored in the database or an error.

:topic/:client_id/user
  <= Any message
  => Response(Result<Records, Error>)

Example usage with MQTT CLI Client:

mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user -m:empty

Get specific user

The topic ../user/:user_id topic is used to query a specific user (including metadata) stored in the user database. The response is the user information or an error.

:topic/:client_id/user/:user_id
  <= Any message
  => Response(Result<Record, Error>)

Example usage with MQTT CLI Client:

mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user/4af8b586-1361-4ff2-9a58-034aa9a03c0f -m:empty

Delete user

Delete a user from the user database using topic ../user/:user_id/delete. The response is an empty message or an error.

:topic/:client_id/user/:user_id/delete/[:request_id]
  <= Any message
  => Response(Result<(), Error>)

Example usage with MQTT CLI Client:

mqtt pub -h 10.11.96.36 -t edge-stream/rsc101-usb-2/user/4af8b586-1361-4ff2-9a58-034aa9a03c0f/delete -m:empty

See also the user proto message format

DB Topics

Update the user database according to the update request that contains a list of users to be added, updated or deleted. The update response will contain a DSID string that can be used to identify the database state.

:topic/:client_id/db/update/[:request_id]
  <= UpdateRequest
  => Response(Result<UpdateResponse, Error>)

Get the status of the user database. The response will contain the last DSID string that was used to update the database state.

:topic/:client_id/db/status/[:request_id] 
  <= Any message
  => Response(Result<StatusResponse, Error>)

See also the db proto messages format