Stream Processor
The Embedded Stream Processor is the face-recognition pipeline that runs directly on a smart camera or an AI box. Instead of streaming video to the server, the device processes its own camera input and publishes only the results — face metadata, crops and templates — as FrameData messages over MQTT (Message Queuing Telemetry Transport). Face Matcher receives these messages as an edge stream and treats them like any other camera: matching against watchlists, events, notifications and history all work the same way. See On-edge processing for when to choose this over server-side RTSP.
What runs on the device
The pipeline reads frames from the camera sensor (or from a GStreamer source on AI boxes), then runs these stages on the device's neural-network accelerator. Every stage after tracking can be switched on or off in the device's settings.yaml, so you decide how much work happens on the edge and how much stays on the server.
| Stage | What it does | Setting |
|---|---|---|
| Face detection | Finds faces in each frame, returns bounding boxes and a confidence score | face_detection |
| Tracking | Assigns a tracking ID to each face across frames so one person produces one tracklet | tracking |
| Face landmarks | Locates 23 facial keypoints used for alignment, head pose and mask detection | always on |
| Template extraction | Produces a compact face template for matching | face_extraction |
| Face identification | Matches the template against a watchlist stored on the device | face_identification |
| Passive liveness | Scores whether the face is a real person or a presented photo, screen or mask (Distant or Nearby mode) | face_liveness_passive |
What it sends
The device publishes a FrameData message to the MQTT broker whenever a new face appears and at least every messaging.interval milliseconds. Depending on configuration, a message carries bounding boxes, landmarks, face crops, face templates, identification candidates, liveness scores and the active, new and lost tracking IDs; full frames can be included for debugging. Messages are serialized as Protobuf by default, or as JSON or YAML. A retained health message reports whether the device is online. The full topic layout and message schema are on the MQTT API page.
How it relates to Face Matcher
Face Matcher's broker is RabbitMQ with the MQTT plugin enabled (host port 1883). Two engine services handle edge devices:
edge-stream-processorsubscribes toedge-stream/<client_id>/frame_datafor every registered edge stream and feeds the results into the same processing chain as server-side cameras — watchlist matching, save strategies, notifications and event history. If the device sends no templates, the server extracts them from the crops.edge-streams-state-synchronizerkeeps the watchlists you select in Station synchronized to the device's local database, so identification can run on the edge and the device reports a match, not just a face.
You register each device in Station as an edge stream with a client ID, and Station then pushes processing settings and the device license to it and shows its health. See Connect to Face Matcher.
Two integration paths
| Path | Broker | Who consumes FrameData | Watchlists | Use when |
|---|---|---|---|---|
| With Face Matcher (recommended) | Face Matcher's RabbitMQ, port 1883 | The edge-stream-processor service | Synchronized from Face Matcher | You want events, history, notifications and Station management |
| Standalone | Your own MQTT broker (or the mqtt_broker binary shipped with AI-box packages) | Your application, subscribed to the device topics | Managed through the device's own user and db topics | You are building a local application and do not need the server |
Both paths use the same device software and the same settings.yaml; only the connection section differs.
Where to go next
- Supported devices — tested cameras and AI boxes.
- Install on a device — per device family, including the hardware ID and license.
- Settings reference — every key in
settings.yaml. - GStreamer input — video sources on AI boxes.
The on-device algorithms come from the Innovatrics Embedded Toolkit, which is a separate product; this section documents only what you need to run it as an edge stream. For issues with the device software, contact sfembedded-integration@innovatrics.com.