Pedestrian attributes extraction

In addition to pedestrian detection, the SmartFace Platform provides advanced functionality to extract valuable attributes of detected pedestrians. This chapter will guide you through the process of leveraging the SmartFace Platform’s capabilities to extract and store pedestrian attributes.

System configuration#

By configuring the system and utilizing services and REST APIs, you can access valuable information about detected pedestrians and integrate it into your workflow.

Service for pedestrian attributes extraction#

The SmartFace Platform utilizes the RpcPedestrianExtractor service for extracting pedestrian attributes. This service supports various resources such as: cpu, gpu, any routing.

The SmartFace Platform sends requests with a pedestrian crop image, the service returns the extracted attributes based on the service’s configuration. All attributes obtained through the service are stored in the database and included in the PedestrianInsertedNotificationDTO for further use and integration.

REST API#

To configure the pedestrian attributes extraction, you can utilize the REST API provided by the SmartFace Platform, using endpoint PUT /api/v1/Cameras on your REST API port 8098. By accessing the pedestrianExtractorResourceId parameter within the Camera entity, you have control over enabling or disabling attribute extraction and specifying the desired resource to be utilized. Valid options for pedestrianExtractorResourceId include cpu_remote, gpu_remote, any_remote. These values determine the resource allocation for the attribute extraction process.

Pedestrian attributes and interpretation#

We recognize two types of attributes:

• confidence attributes - raw attributes directly returned by the underlying attribute extractor. The confidence values associated with these attributes are mapped to a range of 0 to 10 000.

• interpreted attributes - derived from confidence attributes through the evaluation of a confidence threshold or by combining multiple attributes.

Below you will find a list of supported pedestrian attributes.

Confidence attributes#

Interpreted attributes#

Threshold Interpretation formula#

There are 3 possible outcomes of threshold evaluation for interpreted attributes

• If confidence value is above upper threshold → TRUE

• If confidence value is below threshold → FALSE

• If confidence value is between low and upper threshold → Indecisive (attrribute will never be included in response)

Pedestrian attributes configuration#

SmartFace Platform allows you to configure which attributes should be returned by extractor and also thresholds for interpretation in RpcPedestrianExtractor.

Windows installation#

Default configuration is as follows and can be found in appsettings.json for the Windows installation.

"Extraction": {
        "Attributes": {
          "AppendInterpretedAttributesUnderLowerThreshold": false,
          "AppendConfidenceAttributes": false,
          "AppendInterpretedAttributes": true,

          "InterpretationThresholds": {
            "CommonThresholdUpper": 8000,
            "CommonThresholdLower": 4000,
            "GlassesThresholdUpper": 8000,
            "GlassesThresholdLower": 4000,
            "HoldThresholdUpper": 7000,
            "HoldThresholdLower": 3000
          }
        }
      }

Linux installation#

For the Docker setup, please add the environmental variables for the pedestrian-extractor service in the docker-compose.yml file:

- Extraction__Attributes__AppendInterpretedAttributesUnderLowerThreshold=false
- Extraction__Attributes__AppendConfidenceAttributes=false
- Extraction__Attributes__AppendInterpretedAttributes=true
- Extraction__Attributes__InterpretationThresholds__CommonThresholdUpper=8000
- Extraction__Attributes__InterpretationThresholds__CommonThresholdLower=4000
- Extraction__Attributes__InterpretationThresholds__GlassesThresholdUpper=8000
- Extraction__Attributes__InterpretationThresholds__GlassesThresholdLower=4000
- Extraction__Attributes__InterpretationThresholds__HoldThresholdUpper=7000
- Extraction__Attributes__InterpretationThresholds__HoldThresholdLower=3000

The resulting service would look like this:

pedestrian-extractor:
    image: ${REGISTRY}sf-pedestrian-extractor:${SF_VERSION}
    container_name: SFPedestrianExtractorCpu
    restart: unless-stopped
    environment:
      - RabbitMQ__Hostname
      - RabbitMQ__Username
      - RabbitMQ__Password
      - RabbitMQ__Port
      - AppSettings__Log_RollingFile_Enabled=false
      - AppSettings__USE_JAEGER_APP_SETTINGS
      - JAEGER_AGENT_HOST
      - Gpu__GpuEnabled=true
      - Gpu__GpuNeuralRuntime=Tensor
      - Extraction__Attributes__AppendInterpretedAttributesUnderLowerThreshold=false
      - Extraction__Attributes__AppendConfidenceAttributes=false
      - Extraction__Attributes__AppendInterpretedAttributes=true
      - Extraction__Attributes__InterpretationThresholds__CommonThresholdUpper=8000
      - Extraction__Attributes__InterpretationThresholds__CommonThresholdLower=4000
      - Extraction__Attributes__InterpretationThresholds__GlassesThresholdUpper=8000
      - Extraction__Attributes__InterpretationThresholds__GlassesThresholdLower=4000
      - Extraction__Attributes__InterpretationThresholds__HoldThresholdUpper=7000
      - Extraction__Attributes__InterpretationThresholds__HoldThresholdLower=3000
    volumes:
      - "./iengine.lic:/etc/innovatrics/iengine.lic"
      - "/var/tmp/innovatrics/tensor-rt:/var/tmp/innovatrics/tensor-rt"
     runtime: nvidia

Attributes filtering#

You can control how many attributes are returned by setting Append configuration properties.

Thresholds#