ModalitiesGallery Class Reference

Gallery of references. More...

#include <ModalitiesGallery.h>

Collaboration diagram for ModalitiesGallery:

Classes
class	ModalitiesCandidates
	Contains candidates per each modality. More...
class	Candidate
	Found candidate. More...

Public Member Functions
void	Add (std::shared_ptr< ModalitiesGalleryReference > reference)
	Adds ModalitiesGalleryReference into gallery.
void	Remove (size_t index)
	Removes the ModalitiesGalleryReference stored at the specified index from the gallery.
std::shared_ptr< ModalitiesCandidates >	IdentifyModalities (const std::shared_ptr< ModalitiesTemplate > &probe, size_t maxCandidates) const
	Search for given probe.
std::vector< Candidate >	Identify (const std::shared_ptr< ModalitiesTemplate > &probe, size_t maxCandidates) const
	Search for given probe.

Static Public Member Functions
static std::shared_ptr< ModalitiesGallery >	Create ()
	Create gallery of references.
static std::shared_ptr< ModalitiesGallery >	Create (ModalitiesGalleryType type)
	Create gallery of references.
static std::shared_ptr< ModalitiesGallery >	Create (const ModalitiesGalleryConfig &config)
	Create gallery of references.
static std::vector< Candidate >	Consolidate (const std::shared_ptr< ModalitiesCandidates > &identifiedModalityCandidates, size_t maxCandidates)
	Merges previously identified modality candidates into list of overall best candidates.
static std::vector< Candidate >	Consolidate (const std::shared_ptr< ModalitiesCandidates > &identifiedModalityCandidates, size_t maxCandidates, const ModalitiesMatcher &matcher)
	Merges previously identified modality candidates into list of overall best candidates.

Detailed Description

Gallery of references.

Note

The number of references in any ModalitiesGallery is limited by license. You can have more created galleries in your code. The number limits overall references in all galleries.

The ModalitiesGallery stores ModalitiesGalleryReference objects for later identification. Adding data to this gallery in higher-level language bindings is less efficient than in IndexedModalitiesGallery. This inefficiency arises because it creates objects derived from C++ interfaces while also maintaining references to the original high-level language objects.

These high-level language references are necessary because objects returned from the C++ implementation lose their original type information. To address this, the gallery remaps types based on the ID from ModalitiesGalleryReference. However, this approach consumes more memory, as it stores both the identification related data and a reference to the user object.

Despite the increased memory usage, this gallery is more convenient to use as IndexedModalitiesGallery. Since the entire user object is stored within the gallery, users do not need to manually map objects to their identifiers in their code or reload ModalitiesTemplate for consolidation. However, users should be mindful of their object size. The object itself is not required for IdentifyModalities, only the Index and ModalitiesTemplate provided by ModalitiesGalleryReference are used. The relevant identification data is copied from ModalitiesTemplate.

Once a ModalitiesGalleryReference is added to the gallery, its associated ModalitiesTemplate is no longer relevant until consolidation. If memory usage is a critical concern, users can implement ModalitiesGalleryReference::GetModalitiesTemplates() in such a way that the function loads the template on each call instead of storing the object in memory or use IndexedModalitiesGallery.

Each ModalitiesGalleryReference contains a ModalitiesTemplate used for identification. The ModalitiesGallery::Identify() function returns this reference to the SDK user, who can then downcast it to their custom type to retrieve additional information.

Possible approaches for adding custom types to the gallery include:

• Inheritance: Extend ModalitiesGalleryReference to include custom data.
• Composition: Store a reference to a custom object within ModalitiesGalleryReference.

Inheritance

You can inherit our ModalitiesGalleryReference in your types.

Composition

You can embed your type into adapter of ModalitiesGalleryReference.

The identification (ModalitiesGallery::Identify()) has two phases

• searching in gallery for best candidates per modality (details in ModalitiesGallery::IdentifyModalities()).
• merging lists of candidates per modality into final candidates list (details in ModalitiesGallery::Consolidate()).

The enrollment SDK provides following types of gallery

• Print identification based on ICSTemplate
  • c++

    // Recreate your own data types from data stored during enrollment
    auto johnTemplate =
      ModalitiesTemplate::Create(BinaryFile::ReadAll("output/john.tmpl"));
    auto john = std::make_shared<UserType>(0, "john", johnTemplate);
    auto markTemplate =
      ModalitiesTemplate::Create(BinaryFile::ReadAll("output/mark.tmpl"));
    auto mark = std::make_shared<UserType>(1, "mark", markTemplate);
     
    // Create a gallery and add users.
    auto gallery = ModalitiesGallery::Create();
    gallery->Add(john);
    gallery->Add(mark);
     
    // Identify a probe in the gallery.
    constexpr size_t MaxCandidates = 2; // size of result set of candidates
    auto result = gallery->Identify(johnTemplate, MaxCandidates);
     
    // Output identification results.
    for (const auto& c : result)
    {
        auto user = std::dynamic_pointer_cast<UserType>(c.GetCandidate());
        std::cout << "   Name is " << user->name << " score is ["
                  << c.GetScore().GetPrintScore() << ", "
                  << c.GetScore().GetFaceScore() << ", "
                  << c.GetScore().GetIrisScore() << "]" << std::endl;
    }

  • java

    // Recreate your own data types from data stored during enrollment.
    ModalitiesTemplate johnTemplate = ModalitiesTemplate.Create(
            BinaryFile.ReadAll("output/john.tmpl"));
    UserType john = new UserType(0, "john", johnTemplate);
    ModalitiesTemplate markTemplate = ModalitiesTemplate.Create(
            BinaryFile.ReadAll("output/mark.tmpl"));
    UserType mark = new UserType(1, "mark", markTemplate);
     
    // Create a gallery and add users.
    ModalitiesGallery gallery = ModalitiesGallery.Create();
    gallery.Add(john);
    gallery.Add(mark);
     
    // Identify a probe in the gallery.
    long maxCandidates = 2; // size of result set of candidates
    ArrayList<ModalitiesGallery.Candidate> result = gallery.Identify(
            johnTemplate, maxCandidates);
     
    // Output identification results.
    for (ModalitiesGallery.Candidate c : result) {
        UserType user = (UserType) c.GetCandidate();
        System.out.println("   Name is " + user.name + " score is ["
                           + c.GetScore().GetPrintScore() + ", "
                           + c.GetScore().GetFaceScore() + ", "
                           + c.GetScore().GetIrisScore() + "]");
    }

  • csharp

    // Recreate your own data types from data stored during enrollment
    var johnTemplate =
        ModalitiesTemplate.Create(BinaryFile.ReadAll("output/john.tmpl"));
    var john = new UserType(0, "john", johnTemplate);
    var markTemplate =
        ModalitiesTemplate.Create(BinaryFile.ReadAll("output/mark.tmpl"));
    var mark = new UserType(1, "mark", markTemplate);
     
    // Create a gallery and add users.
    var gallery = ModalitiesGallery.Create();
    gallery.Add(john);
    gallery.Add(mark);
     
    // Identify a probe in the gallery.
    var maxCandidates = 2U; // size of result set of candidates
    var result = gallery.Identify(johnTemplate, maxCandidates);
     
    // Output identification results.
    foreach (ModalitiesGallery.Candidate c in result)
    {
        var user = c.GetCandidate() as UserType;
        System.Console.WriteLine("   Name is " + user.name + " score is [" +
                                 c.GetScore().GetPrintScore() + ", " +
                                 c.GetScore().GetFaceScore() + ", " +
                                 c.GetScore().GetIrisScore() + "]");
    }

Member Function Documentation

Add()

void Add ( std::shared_ptr< ModalitiesGalleryReference > reference )

inline

Adds ModalitiesGalleryReference into gallery.

The reference contains ModalitiesTemplate used for identification.

Parameters

reference

ModalitiesGalleryReference to add

Exceptions

EnrollmentException	when it is not possible to add reference
NullArgumentException	when reference is null
NullArgumentException	when reference template is null
EnrollmentInvalidArgumentException	when at least one probe print has position different as PrintPosition::LEFT_INDEX, PrintPosition::LEFT_LITTLE, PrintPosition::LEFT_MIDDLE, PrintPosition::LEFT_THUMB, PrintPosition::LEFT_RING, PrintPosition::RIGHT_INDEX, PrintPosition::RIGHT_LITTLE, PrintPosition::RIGHT_MIDDLE, PrintPosition::RIGHT_THUMB, PrintPosition::RIGHT_RING or no position.
NullArgumentException	when some print in probe prints is null.
NullArgumentException	when some face in probe faces is null.
NullArgumentException	when some iris in probe irises is null.

Consolidate() [1/2]

std::vector< Candidate > Consolidate ( const std::shared_ptr< ModalitiesCandidates > & identifiedModalityCandidates, size_t maxCandidates )

inlinestatic

Merges previously identified modality candidates into list of overall best candidates.

It uses ModalitiesVerifyMatcher with default ModalitiesVerifyMatcher::Config for merging identified candidates in gallery.

It creates list of unique candidates and verifies each of them with probe stored in ModalitiesCandidates. It uses 1 to 1 Modalities Verification. Once modalities scores are calculated it sorts candidates according to maximal score of all modalities scores.

Parameters

identifiedModalityCandidates	modality candidates identified by IdentifyModalities().
maxCandidates	maximum candidates to return

Returns

merged candidates

References Consolidate(), and GlobalMultiModalityExecutor::Get().

Referenced by Consolidate(), and Identify().

Here is the call graph for this function:

Here is the caller graph for this function:

Consolidate() [2/2]

std::vector< Candidate > Consolidate ( const std::shared_ptr< ModalitiesCandidates > & identifiedModalityCandidates, size_t maxCandidates, const ModalitiesMatcher & matcher )

inlinestatic

Merges previously identified modality candidates into list of overall best candidates.

It create list of unique candidates and verify each of them with probe stored in ModalitiesCandidates. It uses 1 to 1 Modalities Verification. Once modalities scores are calculated it sorts candidates according to maximal score of all modalities scores.

Parameters

identifiedModalityCandidates	modality candidates identified by IdentifyModalities().
maxCandidates	maximum candidates to return
matcher	ModalitiesMatcher to make multi modal verification

Returns

merged candidates

Create() [1/3]

std::shared_ptr< ModalitiesGallery > Create ( )

inlinestatic

Create gallery of references.

It will create gallery with default configuration.

Returns

created gallery

Exceptions

EnrollmentException

when it is not possible to create gallery

References IndexedModalitiesGallery::Create().

Here is the call graph for this function:

Create() [2/3]

std::shared_ptr< ModalitiesGallery > Create ( const ModalitiesGalleryConfig & config )

inlinestatic

Create gallery of references.

Parameters

config

Config of gallery

Returns

created gallery

Exceptions

EnrollmentException

when it is not possible to create gallery

References IndexedModalitiesGallery::Create(), and ModalitiesGalleryConfig::ToIndexedModalitiesGalleryConfig().

Here is the call graph for this function:

Create() [3/3]

std::shared_ptr< ModalitiesGallery > Create ( ModalitiesGalleryType type )

inlinestatic

Create gallery of references.

Parameters

type	ModalitiesGalleryType of gallery

Returns

created gallery

Exceptions

EnrollmentException

when it is not possible to create gallery

References IndexedModalitiesGallery::Create(), and ModalitiesGalleryConfig::SetGalleryType().

Here is the call graph for this function:

Identify()

std::vector< Candidate > Identify ( const std::shared_ptr< ModalitiesTemplate > & probe, size_t maxCandidates ) const

inlinenodiscard

Search for given probe.

It uses ModalitiesVerifyMatcher with default ModalitiesVerifyMatcher::Config for merging identified candidates in gallery.

The identification has two phases

• searching in gallery for best candidates per modality (details in IdentifyModalities()).
• merging lists of candidates per modality into final candidates list (details in Consolidate()).

Parameters

probe	probe that is searched in gallery
maxCandidates	maximum number of returned identified candidates

Returns

identified candidates

Exceptions

EnrollmentException	when it is not possible to identify probe
NullArgumentException	when probe is null
NullArgumentException	when some print in probe prints is null.
NullArgumentException	when some face in probe faces is null.
NullArgumentException	when some iris in probe irises is null.

References Consolidate(), and IdentifyModalities().

Here is the call graph for this function:

IdentifyModalities()

std::shared_ptr< ModalitiesCandidates > IdentifyModalities ( const std::shared_ptr< ModalitiesTemplate > & probe, size_t maxCandidates ) const

inlinenodiscard

Search for given probe.

Parameters

probe	probe that is searched in gallery
maxCandidates	maximum number of returned identified candidates

Returns

identified candidates per modalities without score. To get scores and unique candidates list use function Consolidate().

Exceptions

EnrollmentException	when it is not possible to identify probe
NullArgumentException	when probe is null
NullArgumentException	when some print in probe prints is null.
NullArgumentException	when some face in probe faces is null.
NullArgumentException	when some iris in probe irises is null.

Referenced by Identify().

Here is the caller graph for this function:

Remove()

void Remove ( size_t index )

inline

Removes the ModalitiesGalleryReference stored at the specified index from the gallery.

The provided index MUST match the one returned by ModalitiesGalleryReference::GetIndex() when the reference was originally added to the gallery.

Parameters

index

The reference index to remove.

Exceptions

EnrollmentException

If the provided index does not exist in the gallery.

---

The documentation for this class was generated from the following file:

• ModalitiesGallery.h