Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions documentation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Unit Tests Documentation

This directory contains the unit tests for the ML-CV-Target-Tracking project, focusing on isolated testing of specific modules and components.

The tests are written using the `pytest` framework and utilize `unittest.mock` for isolating components from hardware interfaces and external dependencies.

## Structure

### 1. Object Tracker Worker (`test_object_tracker_worker.py`)
Tests the core data flow of the hardware tracking thread.
- **`test_worker_read_loop_data_flow`**: Validates the `object_tracker_read_loop` function. Ensures that data correctly shifts from the hardware queue, goes through the tracklets parsing logic, and is successfully placed into the generic software output queue. It uses mock devices and exceptions (`StopLoopException`) to test iteration control safely.

### 2. Object Tracker (`test_object_tracker.py`)
Tests the parsing and translation of raw hardware tracking data into the system's generalized `TrackedObject` format.
- **`test_parse_tracklets_valid_data`**: Verifies that successful hardware tracklets correctly translate millimeter coordinates into meters and denormalize bounding box Regions of Interest (ROI) into correct pixel dimensions.
- **`test_parse_tracklets_ignores_removed`**: Ensures that detections flagged with a `REMOVED` status by the hardware are filtered out of the active tracklets payload.
- **`test_parse_tracklets_unknown_label_fallback`**: Determines that bounding labels falling outside the defined label map boundaries default reliably to the stringified integer index.

### 3. Software Tracker (`test_software_tracker.py`)
Validates the fallback custom software-based tracking algorithm, which stitches independent frames of raw detections into continuous, ID-persistent tracked objects.
- **`test_tracker_initialization`**: Verifies the tracker starts correctly with a clean slate and resets ID counters.
- **`test_new_track_creation`**: Evaluates that a brand-new, unseen detection triggers a brand-new track with a `NEW` tracking status.
- **`test_track_persistence_and_smoothing`**: Tests exponential smoothing logic (`alpha` parameter), verifying that bounding boxes and coordinates calculate partial adjustments seamlessly between contiguous frames.
- **`test_track_lost_and_removed`**: Simulates temporal tracking loss, checking that missed detections transition tracks to `LOST` status, and eventually drop out of memory entirely upon exceeding the `max_lost_frames` duration tolerance.
- **`test_id_contention_highest_iou_wins`**: In scenarios where multiple detections overlap an ongoing track, this guarantees that the highest Intersection over Union (IoU) claims the existing ID.
- **`test_sub_threshold_iou_spawns_new_track`**: Validates the `iou_threshold` strictness parameters; weak overlaps must not mistakenly hijack existing identities, but must spawn entirely new tracks instead.

## Running the Unit Tests

You can execute the unit tests from the workspace root by using `pytest`:

```bash
# Run all unit tests
pytest tests/unit/

# Run tests with verbose output
pytest -vv tests/unit/
```
34 changes: 34 additions & 0 deletions documentation/test_software_tracker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# SoftwareTracker Unit Tests & Bug Report

## Overview
This document details the unit testing suite created for the `SoftwareTracker` (`modules/object_tracker/software_tracker.py`) and outlines a critical logic bug discovered during edge-case testing.

## Test Suite Implementation (`test_software_tracker.py`)
A new unit test file was created to verify the state transitions of the software tracker. The tests validate the following expected behaviors:
* **Initialization:** Verifies the tracker starts with an empty state and an ID counter at 1.
* **Track Creation:** Ensures a new, unmatched detection correctly spawns a `TrackedObject` with `TrackingStatus.NEW`.
* **Track Persistence & Smoothing:** Confirms that a matching detection in a subsequent frame upgrades the status to `TrackingStatus.TRACKED` and correctly applies the Exponential Moving Average (EMA) to smooth the spatial coordinates and bounding box.
* **Track Expiration:** Validates that tracks without matching detections transition to `TrackingStatus.LOST` and are completely purged from memory after `max_lost_frames` is exceeded.

## The Bug: ID Contention (The "Crossing Paths" Problem)
While the tracker handles isolated, sequential detections perfectly, it fails when processing ambiguous detections (e.g., when two objects cross paths).

A test case (`test_id_contention_highest_iou_wins`) was written to simulate two distinct detections in the current frame overlapping with a single tracked object from the previous frame.

**Expected Behavior:** The detection with the highest Intersection over Union (IoU) should claim the existing track ID. The second detection (the "loser") should be forced to spawn a new track ID.
**Actual Behavior:** The tracker assigned *both* detections to the same ID sequentially. The test resulted in `len(results) == 1` instead of `2`.

## Root Cause Analysis
The bug stems from how detections are matched to existing tracks in `software_tracker.py`.

Currently, the `update()` method iterates through each detection and calls `_find_best_match()` to find the highest IoU track. However, there is no mechanism to "lock" a track once it has been claimed.
1. The tracker processes `Detection A`, finds it overlaps with `Track 1`, and updates `Track 1`'s state.
2. The tracker immediately processes `Detection B`, sees it *also* overlaps with `Track 1`, and updates `Track 1` again, entirely overwriting the data from `Detection A`.

Because the loop evaluates detections independently rather than globally, multiple detections can hijack the same track ID in a single frame, causing the tracker to permanently lose tracking data for the overwritten objects.

## Proposed Fix
The `_find_best_match` logic needs to be replaced with a **Global Greedy Matching** algorithm.
1. Compute the IoU for all possible detection-track pairs.
2. Sort these pairs by highest IoU.
3. Assign matches greedily, ensuring that once a track or a detection is claimed in a frame, it is removed from the pool of available options.
93 changes: 93 additions & 0 deletions documentation/tests_documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Tests Documentation

This document provides a comprehensive overview of all test files and individual test cases within the `tests/` and `modules/common/tests/` directories, describing their purpose and functionality.

---

## 1. Object Tracking & Control Unit Tests (`tests/unit/`)

### [test_object_tracker.py](../../tests/unit/test_object_tracker.py)
Tests the parsing of raw hardware tracklets into `TrackedObject` formats.
- **`test_parse_tracklets_valid_data`**: Verifies that raw hardware tracklets are correctly parsed into `TrackedObject`s.
- **`test_parse_tracklets_ignores_removed`**: Verifies that tracklets flagged as `REMOVED` by the hardware are safely skipped.
- **`test_parse_tracklets_unknown_label_fallback`**: Ensures that, if the hardware returns a label index outside the predefined map, it gracefully falls back to the stringified index representation.

### [test_object_tracker_worker.py](../../tests/unit/test_object_tracker_worker.py)
- **`test_worker_read_loop_data_flow`**: Verifies that the worker loop correctly accesses and moves data from the hardware input queue downstream to the output queue.

### [test_sitl_connection.py](../../tests/unit/test_sitl_connection.py)
Tests Software-In-The-Loop (SITL) operational connectivity.
- **`test_connection`**: Basic connection verification to ensure the automated system can communicate correctly to the virtual SITL environment.

### [test_software_tracker.py](../../tests/unit/test_software_tracker.py)
Evaluates logic to bridge, sustain, or expire target tracks.
- **`test_tracker_initialization`**: Verifies the software tracker initializes properly with an empty state.
- **`test_new_track_creation`**: Validates a new valid detection spawns a new tracking instance with status `NEW`.
- **`test_track_persistence_and_smoothing`**: Checks that a heavily overlapping detection in a sequential frame adequately updates the pre-existing track ID.
- **`test_track_lost_and_removed`**: Ensures a track appropriately becomes `LOST` if missed, and eventually is removed if lost for too long.
- **`test_id_contention_highest_iou_wins`**: Given two overlapping detections against a known track, enforces that the detection providing the highest Intersection over Union (IoU) rightfully claims the ID.
- **`test_sub_threshold_iou_spawns_new_track`**: Validates a detection with an IoU falling lower than the baseline threshold does not falsely match, effectively defaulting to spawning a new distinct track.

### [test_velocity_control.py](../../tests/unit/test_velocity_control.py)
Script to evaluate velocity command formulation and output against targets.
- **Integration Function**: Includes primary methodologies (`send_velocity_command` and `calculate_velocity_commands`) utilizing simple proportional P-controllers. Converts target errors to velocities and leverages `pymavlink` via local NED frames to actuate tracking directives safely.

---

## 2. Common Modules Unit Tests (`modules/common/tests/unit/`)

### Data Encoding & Decoding
* **[test_message_encoding_decoding.py](../../modules/common/tests/unit/data_encoding/test_message_encoding_decoding.py)**
- **`test_encoding_decoding`**: Function to verify encode and decode workflows for system messaging.
* **[test_metadata_encoding_decoding.py](../../modules/common/tests/unit/data_encoding/test_metadata_encoding_decoding.py)**
- **`test_encoding_metadata`**: Function to check if image/payload metadata operates symmetrically via encode/decode workflows.
* **[test_image_encode_decode.py](../../modules/common/tests/unit/test_image_encode_decode.py)**
- **`test_image_encode_decode`**: Primary testing sequence compressing and decompressing images. Accommodates the understanding that JPEG encoding holds lossy properties.

### GPS, KML, and Conversion Systems
* **[test_kml_conversion.py](../../modules/common/tests/unit/test_kml_conversion.py)**
Tests processes saving locational metadata to KML configurations.
- **`test_with_save_path`**: Asserts files save properly assuming correct validation path.
- **`test_nonexistent_save_path`**: Simulates and analyzes saving files under uncreated directories.
- **`test_no_locations`**: Validates script execution on Empty geographical lists.
- **`test_named_locations` / `test_locations`**: Sub-checks for named lists vs pure coordinate entries.
* **[test_local_global_conversion.py](../../modules/common/tests/unit/test_local_global_conversion.py)**
Tests internal invocation methods for `DroneOdometry`, location mappings, and coordinates.
- Test suites covering `test_position_global_from_position_local`, `test_drone_odometry_local_from_global`, `test_position_local_from_position_global`, etc. assure coordinate conversions are properly dispatched.

### Telemetry Network Endpoints (`network/`)
* **[test_send_image.py](../../modules/common/tests/unit/network/test_send_image.py)**: Integration test combining image encoding sequences, sending over internal network topologies (TCP/UDP).
* **[test_tcp.py](../../modules/common/tests/unit/network/test_tcp.py)** / **[test_udp.py](../../modules/common/tests/unit/network/test_udp.py)**: Verify message payloads sending to dummy echo servers accurately over expected sockets.

### Hardware & Operations Output
* **[test_logger.py](../../modules/common/tests/unit/test_logger.py)**
Standardizes debugging processes across the software system.
- Asserts logging functions work universally across varying output targets (`test_log_to_file`, `test_debug_log_info_to_stdout`, etc.).
- Evaluates capturing video/image frames inside logs (`test_log_with_frame_info`).
* **[test_qr.py](../../modules/common/tests/unit/test_qr.py)**
Tests fundamental QR pattern scanner implementations matching historical metrics (e.g. `test_2023_task1_route`, `test_2023_task2_routes`).
* **[test_read_yaml.py](../../modules/common/tests/unit/test_read_yaml.py)**
- Tests whether the global `read_yaml` configurator accesses configuration items or safely flags `test_open_config_file_not_found`.
* **[test_hitl_inject_position.py](../../modules/common/tests/unit/test_hitl_inject_position.py)**
- Tests injecting random coordinate streams effectively into the Hardware-In-The-Loop (HITL) setup loop printed functionally to console mechanisms.

---

## 3. Common Modules Integration Tests (`modules/common/tests/integration/`)
These files generally serve as standalone test scripts verifying hardware endpoints or robust integration paths.

### Cameras & Vision Testing
* **[test_camera_arducamir.py](../../modules/common/tests/integration/test_camera_arducamir.py)**: Physically tests an ArducamIR configuration.
* **[test_camera_opencv.py](../../modules/common/tests/integration/test_camera_opencv.py)**: Verifies the default generic OpenCV optical camera outputs.
* **[test_camera_picamera2.py](../../modules/common/tests/integration/test_camera_picamera2.py)**: Checks native Raspberry Pi `picamera2` environment pipelines.
* **[test_camera_qr_example.py](../../modules/common/tests/integration/test_camera_qr_example.py)**: Evaluates a dynamic end-to-end QR code reading using localized vision setups.

### Emulator & Hardware in The Loop (HITL)
* **[test_camera_emulator.py](../../modules/common/tests/integration/camera_emulator/test_camera_emulator.py)** & **[test_camera_read.py](../../modules/common/tests/integration/camera_emulator/test_camera_read.py)**: Streamlines tests evaluating virtual Windows OBS stream capabilities injecting feed streams acting as mock image sensors.
* **[test_hitl_json_parser.py](../../modules/common/tests/integration/hitl/test_hitl_json_parser.py)**: Parses pre-recorded coordinate maps continuously demonstrating the position emulator capabilities against real targets.
* **[test_threading.py](../../modules/common/tests/integration/hitl/test_threading.py)**: Assesses threading concurrency overlaps validating safe runtime threading across mock GPS updates and local virtual cameras.

### Flight Controllers
* **[test_flight_controller.py](../../modules/common/tests/integration/test_flight_controller.py)**: Standard connectivity test fetching device outputs straight to diagnostic outputs.
* **[test_flight_controller_mission_ended.py](../../modules/common/tests/integration/test_flight_controller_mission_ended.py)**: Evaluates autopilot waypointing metrics testing proper flight termination signals when a drone achieves its end destination.
* **[test_send_messages.py](../../modules/common/tests/integration/test_send_messages.py)**: Checks end-to-end capabilities transmitting arbitrary command messages actively to the established drone endpoint via the FlightController class.
19 changes: 19 additions & 0 deletions modules/object_tracker/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
"""
ObjectTracker module for persistent tracking of detected objects across frames.

Two modes available:
1. On-device (DepthAI): Use configure_tracker_node() + parse_tracklets()
2. Software (host): Use SoftwareTracker.update(detections)
"""

from .tracked_object import TrackedObject, TrackingStatus
from .detection import Detection

# On-device DepthAI tracker
from .object_tracker import configure_tracker_node, parse_tracklets

# Software tracker (accepts Detection objects)
from .software_tracker import SoftwareTracker

# Workers
from .object_tracker_worker import object_tracker_run, object_tracker_read_loop
26 changes: 26 additions & 0 deletions modules/object_tracker/detection.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""
Detection input class - interface contract with detection team.

This matches the Detection class from the SpatialDetectionNetwork team.
"""

from dataclasses import dataclass


@dataclass
class Detection:
"""
Standardized detection result from SpatialDetectionNetwork.

This is the input format we receive from the detection team.
"""

label: str
confidence: float
x: float # spatial X (meters, camera frame)
y: float # spatial Y (meters, camera frame)
z: float # spatial Z / depth (meters, camera frame)
xmin: float # bbox left (pixels or normalized)
ymin: float # bbox top (pixels or normalized)
xmax: float # bbox right (pixels or normalized)
ymax: float # bbox bottom (pixels or normalized)
Loading