Watchlist Management

SmartFace Embedded Stream Processor supports the identification of the detected face against the watchlist stored directly on the camera or edge device.

To register and manage watchlist members and their faces you can use MQTT topics or REST API:

SFEStream Processor Watchlist Management via MQTT

This part of documentation describes the MQTT topics used for watchlist members/users management in SFE Stream Processor.

Using MQTT for communication with the camera requires the MQTT broker and MQTT Client to run as part of your solution.

SmartFace Platform is already integrated with SFE Stream Processor using MQTT.

Root topic

All topics are prefixed with the root :settings.connection.topic and :settings.connection.client_id which are configured in the settings file.

Response Topics

SFE Stream Processor uses MQTT v3 which does not support response topics as MQTT v5 does. Instead, responses are always sent to the same topic that was used as a request with the /response suffix added. Additionally, in some topics, it is possible to extend request topics with the :request_id to differentiate between specific requests when dealing with multiple requests at the same time.

Message Payloads

The MQTT message payload format is configurable in the settings file. Supported formats are JSON, YAML and Protobuf.

Result types are used to report the result of an operation Result<Message, ErrorMessage>). The result type is either Message or ErrorMessage. The Message type is specific to the operation while the ErrorMessage type is common and contains an error message string. Any of the result types can be used as a response payload unless otherwise specified.

There are three message types used in the MQTT topics:

  • Retained<Message> - Messages that are retained by the broker and sent to new subscribers
  • Periodic<Message> - Messages that are sent periodically
  • Response<Message> - Messages that are sent as a response to a request

Health Topic

:topic/:client_id/health
  => Retained(HealthReport)

The health topic is used to send the health report of the stream processor to the MQTT broker. The health report is updated when the health status changes.

  • Online - pushed immediately after connecting to an MQTT broker
  • Offline - pushed after the stream processor is restarted (e.g. settings file has changed) or correctly shutdown (SIGTERM/SIGINT)
  • OfflineDisconnected - pushed as a last will message after losing connection to the broker or in case of any other unexpected failure

See also the health proto message format

FrameData Topic

:topic/:client_id/frame_data
  => Periodic(FrameData)

FrameData topic is currently the primary output topic of the stream processor. It is used to periodically send the frame data to the MQTT broker. Depending on the configuration, the frame data can contain the following information:

  • Face detection bounding boxes
  • Face landmarks
  • Face crops
  • Face templates
  • Face identification results
  • Active tracking ids
  • New tracking ids
  • Lost tracking ids
  • Full frame data

See also the frame_data proto message format

User Topics

:topic/:client_id/user
  <= Any message
  => Response(Result<Records, Error>)

Stream Processor manages a user database that can be used to store face templates and user information. The user database is stored in the stream processor storage directory. The use topic is used to query the entire user database. The response is a list of all users in the database or an error.

:topic/:client_id/user/:user_id
  <= Any message
  => Response(Result<Record, Error>)

The user topic is used to query a specific user in the user database. The response is the user information or an error.

:topic/:client_id/user/:user_id/register[/:request_id]
  <= Any message
  => Response(Result<Record, Error>)

Register a new user in the user database. The response is the user information or an error.

:topic/:client_id/user/:user_id/delete/[:request_id]
  <= Any message
  => Response(Result<(), Error>)

Delete a user from the user database. The response is an empty message or an error.

:topic/:client_id/user/:user_id/face/add/[:request_id]
  <= Image or Template bytes
  => Response(Result<Record, Error>)

Add a new face template to a user in the user database. The response is an updated user record or an error. The payload can be either an image or a template.

Images are processed by the stream processor and a template is extracted from the image. The template is then added to the user database. The template extraction is done using the same settings as the stream processor. Valid image formats are JPEG and PNG.

See also the user proto message format

DB Topics

:topic/:client_id/db/update/[:request_id]
  <= UpdateRequest
  => Response(Result<UpdateResponse, Error>)

Update the user database according to the update request that contains a list of users to be added, updated or deleted. The update response will contain a DSID string that can be used to identify the database state.

:topic/:client_id/db/status/[:request_id] 
  <= Any message
  => Response(Result<StatusResponse, Error>)

Get the status of the user database. The response will contain the last DSID string that was used to update the database state.

See also the db proto messages format

SFEStream Processor Watchlist Management via REST API

This page describes the REST API calls for the management of the Watchlists stored directly on the camera or edge device.

Using REST API for Watchlist management requires SFE REST Client running on your edge platform.

POST /api/v1/WatchlistMembers/{id}/AddFace

Register a new watchlist member or add a face to an existing watchlist member:

  • If the watchlist member with the specified ID does not exist, a new watchlist member is registered.
  • If the watchlist member with the specified ID already exists, a face is added.

Path parameters:

  • id - The ID of the watchlist member to add a new face

Request body:

  • data - Binary data containing an image in JPEG format or a valid face template

Response:

  • The response contains the face template encoded in base64 if the face was added successfully.
  • Otherwise, it returns an error.

Example:

curl POST http://10.11.80.38:4242/api/v1/WatchlistMembers/lipo/AddFace -T C:\Users\mlipovsky\Downloads\lipo.jpg

GET /api/v1/WatchlistMembers

Get all watchlist members.

Response:

  • The response contains all watchlist members with their face templates encoded in base64.
  • Otherwise, it returns an error.

Example:

curl GET http://10.11.80.38:4242/api/v1/WatchlistMembers

GET /api/v1/WatchlistMembers/{id}

Get a watchlist member with a specified ID.

Path parameters:

  • id - The ID of the watchlist member to get

Output:

  • If a watchlist member with the specified ID exists the response contains this watchlist member ID with his face templates encoded in base64.
  • Otherwise, it returns an error.

Example:

curl GET http://10.11.80.38:4242/api/v1/WatchlistMembers/lipo

DELETE /api/v1/WatchlistMembers/{id}

Delete a watchlist member with the specified ID.

Path parameters:

  • id - The ID of a watchlist member to delete

Output:

  • If a watchlist member with the specified ID exists the response contains this watchlist member ID with his face templates encoded in base64.
  • Otherwise, it returns an error.

Example:

curl DELETE http://10.11.80.38:4242/api/v1/WatchlistMembers/lipo