SmartFace Platform

SmartFace Platform v5_4.30.1#

Release date: 26-February-2025

Important Information
To use new palm related features, a correct license needs to be used. If in doubt, please contact your sales representative

Changed#

• Toolkit was updated to 1.13.3. to fix speed regression introduced in previous version when license was verified for each identify call
• Fix possible race condition when multiple edge stream were created simultaneously
• Add more 409 possible response code descriptions to Palm search related API endpoints

SmartFace Platform v5_4.30.0#

Release date: 02-February-2025

Important Information
To use new palm related features, a correct license needs to be used. If in doubt, please contact your sales representative

Added#

• SFCam service now supports palm processing
• EdgeStreamProcessor service now supports processing of palm data
• API configuration for palm processing was added via POST/PUT Camera endpoints via
  • PalmDetectorResourceId was added. Supported values are cpu or none
  • PalmTemplateGeneratorResourceId was added. Supported value is cpu
  • PalmDetectorConfig object on PUT/POST Camera API endpoints
• Added support for AssumedRole authentication in the migrate faces command, allowing S3 access
• Added support for palm synchronization from leader to follower

Changed#

• API endpoints of Camera and EdgeStream now accept configuration properties for PalmDetection, namely
  • PalmDetection_Enable
  • PalmDetection_MaxDetections
  • PalmDetection_BoundingBoxExtension
  • PalmDetection_Tracking_InputThreshold
  • PalmDetection_Tracking_OutputThreshold
  • PalmDetection_Tracking_Stability
  • PalmDetection_Tracking_MaxFramesLost
• IdentificationEventNotificationDTO was extended with PalmModalityInfo which will be set when detected palm was identified/not identified
• Edge stream POST/PUT endpoints now support setting of PalmDetection related settings
• GRPC NX Integration may not work correctly when GRPC connection is broken due to server processing error. This is known issue and will be addressed later on
• Updated API descriptions of allowed save strategies
• Improved validation messages with better descriptions for all services
• migrate-faces admin cli command parameter alias was changed from --s3-sk to -s3-sk. This was typo. Full version of parameter --s3-secret-key stay unchanged.

Fixed#

• Fixed handling of the deprecated but accepted ObjectSaveStrategy value Best for EdgeStream endpoints
• Fixed a bug where deleting a member with palms left the frame and its blobs in the database
• Fixed off by one validation for the /contract/smartface/watchlist_member_count license value

SmartFace Platform v5_4.29.0#

Release date: 13-December-2024

Important Information
To use new palm related features, a correct license needs to be used. If in doubt, please contact your sales representative

New palm modality added, including enrolling palms for Watchlist Members, searching by palm images and palm templates via REST API, including in LFIS. Added new endpoints, new threshold for palms for Watchlists.

Added an option for the SFE Stream Processor to disable sending of notifications of unmatched people. This means that people who are not recognized by the Edge camera do not generate notifications - their data will never leave the Edge camera.

Several optimizations, including RabbitMQ updated in Windows deployment, removing Edge streams from the platform removes them from the MQTT to avoid retaining previous configuration and overriding the settings.

Technical Changelog#

Added#

• Palm modality for LFIS and REST API
• Option to disable SFE Stream Processor notifications for unmatched persons

Changed#

• Removing Edge streams from platform applied to MQTT Broker
• RabbitMQ optimalized in Windows deployment

Fixed#

• Incorrectly exposed face property for an individual in the GraphQL.

SmartFace Platform v5_4.28.2#

Release date: 09-December-2024

Important Information
A bug was fixed where under some circumstances the watchlist matcher did not properly update the watchlist database. This is related to the watchlist synchronization, therefore it is recommended to update if you are using watchlist synchronization and the rebuild the synchronization stream re-running the create-wl-stream-generation.sh

The services starting process was updated to prevent cold start waiting. Now all the services get ready during their start, they do not wait for the fist detection for full initialization.

Fixed a bug where when re-registering a watchlist member with parameter keepAutoLearnPhotos: true, assigned autolearn faces are deleted, but the fields NotMaskedAveragedTemplate and MaskedAveragedTemplate are not changed.

Technical Changelog#

Changed#

• Updated the services starting to prevent cold start waiting

Fixed#

• Possible inconsistency of the watchlist matchers database
• Averaged autolearn templates are not processed correctly when re-registering watchlist members

SmartFace Platform v5_4.28#

Release date: 08-October-2024

Important Information
⚠️ For the version v5_4.28 of the SmartFace Platform we have changed the Save strategies. Added new strategies: None and MatchedOnly and updated of strategy Balanced.

⚠️ With the version v5_4.28 the new Face Validation is enabled for Watchlist Member Enrollment. Only faces matching the requirements can be enrolled. The requirements can be configured.

Updated save strategies
New strategy None was added. It saves face/pedestrian only when it is linked to a pedestrian/face that is about to be saved based on its saving strategy.

Another new strategy MatchedOnly was added. It saves face/object only when matched to a watchlist member and match score is sufficiently higher than previously saved face/object with the same matched member.

The Balanced was updated. It saves face/object also in all cases that MatchedOnly strategy would.

Save strategies are now available for objects using the SFE Stream Processor.

Identification based on Time-based One Time Password encoded in QR code (QR TOTP)
New identification type / modality was added using the QR codes. New REST API endpoints were added: an API endpoint for creating TOTP secrets for watchlist members and a conditionally enabled endpoint displaying currently valid QR code for watchlist member with enabled TOTP.

Added support for “matching” parsed QR codes from the SmartFace Embedded Stream Processor against watchlist member database, evaluation and notifications ( ObjectProcessed, IdentificationEvent ).

Additional features for the LFIS
Added Pedestrian and Object detection. Biometric templates are available via the REST API and they can be queried using the GraphQL API. Pedestrian and Object detection endpoint in API (and LFIS setup)

Image validation and filtering for registration We have added new filtering Face Validation for the images used for registration, to ensure high quality of the registration image.

Technical Changelog#

Added#

• Added new save strategy “None”
• Added new save strategy “MatchedOnly”
• Added new QR Modality using Time-based One Time Password encoded in QR code (QR TOTP)
• Added Pedestrian and Object detection for LFIS
• Added ability to download Biometric Templates for Watchlist member and to query them
• Added Face Validation with conditions for Watchlist Member registration
• Added IdentificationEvent notification

Changed#

• Updated save strategy “Balanced”
• Updated checking of compatibility, now we are checking compatibility of the SmartFace Server to ensure the AVX2 instructions are supported
• Updated SFAdmin to redirect logs of the SFAdmin into console

Fixed#

• Fixed case sensitive search when using the PostgreSQL
• Fixed a bus in the Rapid Video Processing where the service gets stuck

SmartFace Platform v5_4.27#

Release date: 31-July-2024

Important Information
⚠️ For the version v5_4.27 of the SmartFace Platform we have changed the Save strategies. New Balanced strategy became the default saving strategy for Faces and Pedestrians, the First Face, Best Face and First Face and Best Face became deprecated.

New Saving Strategies
Today there are All, Balanced, None and Matched Only Saving Strategies. The Balanced is the new default. None keeps Faces linked to Pedestrians and Pedestrians linked to Faces.

Updated Liveness Notifications
The liveness checks are performed and now the information about the results is part of the notifications.

Speed improvement for linked Faces and Pedestrinas
The Database querying was optimized thus improving overall speed of the linked Faces and Pedestrians.

Technical Changelog#

Added#

• New Balanced Saving Strategy

Changed#

• None Saving Strategy was changed to allow keeping Faces and Pedestrians consistent for the purpose of linking. Now all saving strategies allow linking between Pedestrians and Faces.
• EdgeStreamStateSynchronizer throws an error if the WL Stream is not available.

Fixed#

• Fixed bug related to propagating the Client ID
• Video processing has few bugs fixed
• Fixed Odata propagating Edge license
• Several Minor fixes

SmartFace Platform v5_4.26#

Release date: 10-June-2024

Important Upgrade Recommendations
⚠️ We highly recommend upgrading to SmartFace Platform version 4.26 from versions 4.24 and 4.25. This upgrade addresses a bug in our matcher, ensuring improved stability and performance.

Docker Engine Requirements
ℹ️ The minimum required version of Docker Engine has been updated to 20.10.10. To ensure optimal compatibility and performance, we strongly advise using the latest versions of Docker Engine, Docker CLI and Docker Compose along with this update.

GPU Requirements
ℹ️ If you have GPU processing enabled, please note the minimal version for NVIDIA drivers has been updated to >= 520.61, minimal version of CUDA has been updated to version 11.8.

Liveness Thresholds
ℹ️ With the updated Liveness, the Liveness Thresholds have been updated. Please read more here

Improved and more accurate liveness detection We have integrated IFace 5.14. With this improvement, the default condition string for nearby and distant liveness has been changed to FACE_CONFIDENCE: [1000; 10000]. For more technical information click on “Read more” below.

Body parts detection is no longer supported This is due to the discontinuation of support for the OpenVINO toolkit, which powered our Body Parts Detector.

Technical Changelog#

Added#

• Added ClientId field to MatchNotification and NoMatchNotification notifications

Changed#

• Upgraded IFace to version 5.14

  • Distant liveness algorithm can now be configured via appsettings in RpcLiveness service, section Liveness__DistantAlgorithm, supported values are fast (default) and accurate
  • The FAR/FRR ratios and their relation to the Liveness Score has changed. Please read more here

• Optimized matching with SFE Toolkit library when identifying templates in large watchlists

• Matcher service pre-fetches an additional request from message broker

• SF platform no longer builds and releases Nvidia Jetson-specific docker images. As a replacement there are images that can be run on any arm64 device but without the support of any GPU acceleration. For the GPU support we can use the SmartFace Stream Processor using the same machine.

  • All images released in SF platform are now multiplatform and docker engine will pull the relevant one based on the platform of the host machine

• Changed default condition string for nearby and distant liveness to FACE_CONFIDENCE: [1000; 10000]

• Grouping now filters out faces with low template quality

• RpcBodyPartsDetector service has been removed, API compatibility is maintained, related ResourceIds are marked deprecated, if still in use, they will fall back to sfe_object_cpu_remote detection for pedestrian detection

  • If body_parts_cpu_remote is selected as liveness check resource id, no liveness check will be performed and Not Performed result will be returned
  • OpenVino installation has been removed from prerequisity installer

• changed default values for OnnxRuntimeConfiguration__SolverInterThreads and OnnxRuntimeConfiguration__SolverIntraThreads to 1 so SFE Toolkit and IFace default behavior is consistent

• removed IFaceConfig from Config table, replaced with appsettings section IFaceConfiguration where ThreadNum and ThreadManagementMode can be configured

• Individuals in GraphQL API now queries directly from the Individual table, instead of the IndividualsWithBestFace view

Fixed#

• Fixed an issue where watchlist matcher database may have become corrupt
• Matcher no longer loads autolearn faces from database. Only regular faces and averaged templates are used
• Update of LiveGrouping GroupingStep configuration property now correctly recalculates next update iteration of LiveGrouping according to new value

SmartFace Platform v5_4.25.2#

Release date: 12-April-2024

Added#

• Centralized configuration for smart cameras

  • This enhancement allows users to configure the SFE Stream processor using both the SmartFace Platform and SmartFace Station, with added functionality to upload licenses for your smart cameras.
  • Centralized configuration of the SFE Stream processor is achieved via the existing API endpoint /api/v1/EdgeStreams and in Edge stream configuration detail in SmartFace Station.
  • For more technical information click on “Read more” below.

• Multi-site Watchlists Synchronization

  • Advanced feature based on Leader-Folloer architecture has been added. It enables the automatic distribution of watchlists from a central Leader site to multiple Follower sites deployed across different locations.
  • For more technical information click on “Read more” below.

• Upgrade from v5_4.24 to v5_4.25

Fixed#

• SmartFace Platform v5_4.25.1

  • Fixed handling of edge case of Camera service face tracking process

• SmartFace Platform v5_4.25.2

  • Fixed Windows installer issue occuring when re-installing SmartFace on computer with DB synchronization follower already installed
  • Removed Log properties of EdgeStream settings API

Technical Changelog#

Added#

• New SFDbSynchronizationLeader and SFDbSynchronizationFollower services which allow Watchlist database synchronization from one leader SmartFace instance to multiple follower SmartFace instances
• PUT, POST, DELETE methods on Watchlists and WatchlistMembers can be set to readonly via FeatureManagement.ReadOnlyWatchlists config setting for SmartFace.Api service, also configurable via FeatureManagement__ReadOnlyWatchlists environment variable.
• Windows platform installler now includes additional page for setup of SFDbSynchronizationFollower service. When checkbox is selected it will use provided configuration fields and test whether they are correct by making GRPC call to provided Leader address. If everything is correct then :
  • SFDbSynchronizationFollower service will be registered and started afterwards
  • ReadOnlyWatchlists feature flag will be set for SFApi service
• Added configuration API for SmartFace Embedded Stream Processors as an extension of pre-existing EdgeStream endpoint /api/v1/EdgeStreams, configuration is forwarded to the MQTT from API
  • Removed OperationMode__MatchingDataStrategy and OperationMode__LivenessDataStrategy from appsetings and env.all, these values are now dependent on the configuration of the Embedded stream processor
• GraphQL API subscription to listen for SmartFace Embedded Stream Processor health report
• Added new endpoint LiveGrouping/Streams mirroring functionality of LiveGrouping/Cameras, but with support for both cameras and EdgeStreams
• Possibility to disable LiveGrouping processing with new property Enabled on LiveGrouping endpoint (The property defaults to true, so LiveGroupings are enabled by default)
• Added tests for multitenancy to sf-cloud-func-tests image for sf-helm cloud tests
• Added linear interpolation for bounding boxes in preview to smooth out the tracking preview
• Added optional parameter --dependencies-availability-timeout to sfadmin to specify the timeout for the attempt to connect to RabbitMq and database. If not specified, retry will be not attempted, behaviour remains unchanged.
• Added metrics for SFBase DbCleanup Worker

Changed#

• Default value of EdgeStream property SaveFrameImageData was changed from true to false
• Added property Note that was missing into api/v1/WatchlistMembers/Register body payload
• Optimized camera preview performance when performance is cpu bound
• Optimized RPC call handling when dataflow input queue is full
• Optimized memory usage of Camera process in case of non-functioning template extraction
• run-migration admin cli command now requires --tenant-id option. For non multitenancy deployments use value default
• Added support for setting and testing authentication via sf-cloud-func-tests image for sf-helm cloud tests
• Serialization method of watchlist synchronization messages was changed to protobuf
• Improved logging in SFBase DbCleanup Worker
• HTML documentation for REST and LFIS APIs contain customized title and meta description for better SEO

Fixed#

• Fixed edge cases of DbCleanup, Watchlist Autolearn and Session cleanup jobs when scheduled to run around 23:59:59
• Fixed SetupDbCleanup endpoint not accepting 00:00:00 as a valid cleanupStart value, also added 23:59:59 as upper bound of cleanupStart to validation
• DbCleanupWorker and Tracklet Recovery are now synchronized so they will not run in parallel
• Camera worker service is now correctly restarted after live grouping change
• Fixed issue when empty tracklet was created in database which may occur when match RPC request was canceled
• Fixed issue when some tracklets may be never finished when using remote face detection
• Fixed edge case of EdgeStream synchronizer when synchronizing against an empty database
• Fixed hanging reference during deinitialization of GStreamer pipeline
• Fixed calculation of face area change
• Restored previous behavior of matcher logging count of all watchlist members being loaded on startup
• Fixed a potential issue that could occur when recalculating the averaged template of a member after a autolearn face was deleted via the api/v1/WatchlistMembers/{id}/RemoveFace API endpoint.
• Added an annotation for possible 408 response status code for api/v1/WatchlistMembers/{id}/RemoveFace endpoint (in case the RPC call for averaging templates fails).
• Fixed missing OpeAPI Description annotation for 200 OK response of /api/v1/Images/{id} endpoint.
• Grouping endpoints now correctly validate grouping description maximum length of 500 characters

SmartFace Platform v5_4.24#

Release date: 04-January-2024

Changed#

• Fast matching
  • New improved matching is now approximately 2x faster, ensuring quicker and more efficient results.

Technical Changelog#

Added#

• Replaced IDKit with SFE Toolkit in RpcWatchlistMatcher.
• Fixed issue when video record related notifications were sent before database transaction was completed.
• Added possibility to turn off saving of crop images by already existing configuration option NoSqlDataStorageDisabled which is now applicable in Camera service and EdgeStreamProcessor service.
• RabbitMQ version was updated in provided dependencies. Check RabbitMQ 3.12 Release Notes for further instructions on upgrading.
• Fixed default values used in ObjectDetectorConfig in POST /api/v1/Cameras endpoint if they were not specified in request.
• Releasing/distributing new Cloud Functional tests image used in SmartFace helm chart
• Windows installer has new page where you can specify whether watchlist stream population should be done by installer.

SmartFace Platform v5_4.23#

Release date: 13-December-2023

Added#

• The selective watchlists synchronisation to Edge streams

  • Now, you can effortlessly select specific watchlists and synchronize them to chosen edge streams via REST API endpoint /api/v1/Edgestreams/{id}/WatchlistSynchronization/.

• Connection between face and body

  • SmartFace is capable of linking detected faces to the corresponding body. The facial and pedestrian data are stored in the database only if the “All” save strategy is chosen for both face and pedestrian detection.

Changed#

• Tracking improvements in SmartFace Platform were implemented.

Technical Changelog#

Added#

– Added option to synchronize selected watchlists to edge streams. This option can be configured via EdgeStream API endpoint on path {id}/WatchlistSynchronization. Supported values are None, All and Selected. When Selected is used, then you can specify which watchlists should be synchronized.

• Added index on StreamId column in tables VisualObjects and Frames.

• Added possibility to specify MaxOldnessDetectionFrames for IFace tracking, which represents number of redection frames after which suspended tracklet will be definitely marked as lost. Default value is 2, which is minimal allowed IFace value. The value can be set as an environment variable TrackingConfiguration__MaxOldnessDetectionFrames in the docker environment, or by using the camera’s SmartFace.Camera.appSettings.json in the Windows environmnent such as in the example below:

  "TrackingConfiguration": {
  "MaxOldnessDetectionFrames": 2
},

• SmartFace will now try to evaluate whether detected faces and pedestrians from same frame represents same “person” by linking them together by their Ids (Face will contain Id of Pedestrian and vice versa). It is enabled by default in the SmartFace Platform configuration. However this linking behavior is only supported when All save strategy is used for both Face and Pedestrians. If you want to disable this behavior altogether you can set false value to ObjectLinking__Enabled docker environment variable in camera services or in the Windows environment in camera’s SmartFace.Camera.appSettings.json as per example below:

"ObjectLinking": {
  "Enabled": false
},

Changed#

• SmartFace binaries package was removed.

• Performance of loading RpcFaceMatcher service has been improved

• Fixed possible EdgeStreamsStateSynchronizer bug when synchronization for EdgeStream could get stuck when exception was thrown during db update.

• GraphQLApi service now declares CORS policy that allows any origin as other API and OData API services.

SmartFace Platform v5_4.22#

Release date: 11-October-2023

Added#

• The SmartFace Platform introduced new functionality - Watchlist synchronization to Edge streams. This is avalable for SmartFace Embedded Stream Processor version 2.5 and higher.

  • Now, it is possible to synchronize watchlist members registered in SmartFace with Edge streams and perform matching on edge devices.
  • To facilitate this functionality, a new REST API endpoint, ‘/api/v1/Edgestreams/{id}/WatchlistSynchronization’, has been added.
  • Please note watchlist synchronization with the Edge streams is disabled by default. To enable it, please take a look at steps in the documentation

• Database optimization has been successfully implemented.

• New IFace 5.7 was integrated with improved liveness check.

Technical Changelog#

Added#

• Added new service EdgeStreamsStateSynchronizer which seamlessly synchronizes watchlist members to edge streams.

• Added new endpoint into EdgeStream API endpoints on path {id}/WatchlistSynchronization where you can control whether watchlist synchronization should be enabled or not. By default synchronization is disabled. Supported values are None and All.

• Added new command populate-wl-update-log-stream to admin-cli which will initialize and populates data for already existing watchlist members in database. This command should be used when synchronization to Edge stream cameras is desired.

• Added new configuration options SynchronizationConfig__UpdateLogReadBatchCount and SynchronizationConfig__BatchTriggerMs to tweak batching configuration of EdgeStreamsStateSynchronizer

• Added new configuration options OperationMode__MatchingDataStrategy and OperationMode__LivenessDataStrategy into EdgeStreamProcessor which affects how platform is treating matching and liveness data coming from Edge stream camera. By default platform will use ServerOnly strategy for both matching/liveness data coming from Edge stream. Possible configuration values are:

  • EdgeStreamOnly - Platform will not call any matching or liveness services based on detected faces from Edge stream, but it will only use identifications and liveness results from Edge stream. However configuration of SpoofDetectorResourceIds defined in EdgeStream are still honored. This means that only configured liveness types are used for liveness evaluation.
  • ServerOnly- Platform will ignore identifications and liveness results from Edge stream and will do matching and liveness calls to internal services.

• Added new Rpc service into RpcWatchlistMatcher which provides watchlist member metadata for requested ExternalId of member

• AuthenticationType configuration option for S3 connection supporting authentications other than instance profile in AWS

  • InstanceProfile represents previously implemented behavior
  • AssumedRole can be used with AWS IRSA
  • AccessKeyAndSecret can now also be used in conjunction with bucket endpoint

• UseBucketRegion configuration option for S3 connection making the the configuration more explicit

  • previously this behavior implicitly was controlled by supplying value to BucketRegion property
  • when false then Endpoint configuration property is used
  • when true then BucketRegion configuration property is used

Changed#

• Coordinates and confidences on all VisualObject types have been changed from double to float in postgre and MSSQL databases.
• Template data in MSSQL databases in VisualObjects.Template, WatchlistMembers.NotMaskedAveragedWildTemplate and WatchlistMembers.MaskedAveragedWildTemplate have been changed from varbinary(max) to varbinary(600).
• Type of FullName field of WatchlistMember table has been limited to maximum of 200 characters and API validation rules limiting maximum length of FullName and DisplayName of watchlists have been added.
• IFace upgraded to version 5.7.0
• Default condition string for distant liveness changed.

Fixed#

• Fixed GraphQL serialization issue for FirstFaceBestFace face save strategy on Camera, EdgeStream and VideoRecord aggregate roots.
• Fixed issue when multiple object types could be mixed in one tracklet
• Fixed validation of FaceSaveStrategy field in VideoRecord Create and Update requests.
• Fixed bug when some frame blob images of VideoRecords were incorrectly removed.
• template, templateVersion, serializedId properties on GraphQL Face type were removed
• Optimized cleanup procedures for large databases