-
The image is passed to the model that has an encoder-decoder that performs detection and ReID
- Detection:
- Heat Map: The model generates a heatmap where each peak represents an object center. This allows FairMOT to locate the central point of each detected object.
- Box size: the model also predicts the width and height of each object, providing the bounding box dimensions.
- Centre offset: Since down sampling occurs during encoding, the exact object center can shift. A center offset is computed to refine and re-align the detected object center accurately in the input image.
- Re-ID Embeddings: The Re-ID branch generates high-dimensional feature embeddings for each detected object, uniquely encoding its appearance. This embedding acts as a signature to help differentiate between individual objects across frames
-
In the Association phase, the model passes the bounding boxes and Re-ID feature extractions for the current frame. It uses a two stage matching strategy between the Previous frame result and the current frame result
Stage 1:
- Kalman Filter Prediction: FairMOT uses a Kalman filter to predict the positions of objects in the current frame based on past trajectories. These predictions are color-coded as cyan in the visualization.
- Cosine Distance Calculation: For each object detected in the previous frame, the model computes the cosine distance between its Re-ID embedding and the embeddings of detected objects in the current frame. This helps to measure the similarity of objects based on appearance.
- Mahalanobis Distance Calculation: The model also computes the Mahalanobis distance between the Kalman filter’s predicted positions and the actual detected positions in the current frame. This measure helps estimate how closely objects' predicted and actual positions align.
- Fusing Distance Scores: The cosine and Mahalanobis distances are fused to generate a final similarity score between pairs of objects across frames. This fusion helps maintain robustness in tracking, accounting for both appearance and positional information.
- Hungarian Algorithm Matching: The Hungarian algorithm matches the bounding boxes of objects from the previous frame to those in the current frame, based on the computed similarity scores. If the similarity score exceeds a certain threshold, the model assigns the same ID to matched detections, establishing continuity in tracking.
Stage 2:
If there are objects in the current frame that couldn’t be matched with previous frame objects, FairMOT initiates a second stage to handle these cases:
- Overlap-Based Comparison: This stage compares the spatial overlap between bounding boxes of unmatched detections and tracklets from previous frames.
- Threshold-Based Assignment: If the overlap exceeds a specified threshold, the object retains the former ID, assuming it's the same object reappearing. Otherwise, FairMOT assigns a new ID to the object, treating it as a new detection.
-
Kalman filter: an algorithm that provides estimates of the current state of a system, typically for applications in tracking and navigation. Helps maintain object identities across frames by predicting where an object is likely to appear next.
-
Cosine distance: measure of similarity between two non-zero vectors in an n-dimensional space, based on the cosine of the angle between them. cosine distance helps compare feature embeddings of detected objects across frames. A smaller cosine distance (close to 0) means the features are similar, indicating that the object is likely the same
-
Mahalanobis distance: measure of the distance between a point and a distribution, taking into account the correlations between variables. It’s commonly used to measure how far a point is from the mean of a distribution. Mahalanobis distance is used to compare the predicted location of an object to the actual observed location. This helps in associating detected objects across frames by determining if a detection aligns with a predicted tracklet position
-
Hungarian Algorithm: It finds the optimal one-to-one matching between two sets, minimizing or maximizing a given cost or distance.
