# Test Specification — Tier1Detector

## Acceptance Criteria Traceability

| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-01 | Tier 1 latency ≤100ms per frame on Jetson Orin Nano Super | PT-01, PT-02 | Covered |
| AC-04 | New YOLO classes (black entrances, branch piles, footpaths, roads, trees, tree blocks) P≥80%, R≥80% | IT-01, AT-01 | Covered |
| AC-05 | New classes must not degrade detection performance of existing classes | IT-02, AT-02 | Covered |
| AC-26 | Total RAM ≤6GB (Tier1 engine portion) | PT-03 | Covered |

---

## Integration Tests

### IT-01: Detect Returns Valid Detections for Known Classes

**Summary**: Verify detect() produces correctly structured Detection objects with expected class labels on a reference image.

**Traces to**: AC-04

**Input data**:
- Reference frame (1920x1080) containing annotated footpath_winter, branch_pile, dark_entrance
- Pre-exported TRT FP16 engine with all target classes

**Expected result**:
- list[Detection] returned, len > 0
- Each Detection has: centerX/centerY in [0,1], width/height in [0,1], classNum ≥ 0, label in class_names, confidence in [0,1]
- At least one detection matches each annotated object (IoU > 0.5)
- Segmentation masks present for seg-capable classes (non-None numpy arrays with correct HxW shape)

**Max execution time**: 200ms (includes preprocessing + inference)

**Dependencies**: TRT engine file, class names config

---

### IT-02: Existing Classes Not Degraded After Adding New Classes

**Summary**: Verify baseline classes maintain their mAP after YOLOE set_classes includes new target classes.

**Traces to**: AC-05

**Input data**:
- Validation set of 50 frames with existing-class annotations (vehicles, people, etc.)
- Baseline mAP recorded before adding new classes
- Same engine with new classes added via set_classes

**Expected result**:
- mAP50 for existing classes ≥ baseline - 2% (tolerance for minor variance)
- No existing class drops below P=75% or R=75%

**Max execution time**: 30s (batch of 50 frames)

**Dependencies**: TRT engine, validation dataset, baseline mAP record

---

### IT-03: Detect Handles Empty Frame Gracefully

**Summary**: Verify detect() on a blank/black frame returns an empty detection list without errors.

**Traces to**: AC-04

**Input data**:
- All-black numpy array (1920, 1080, 3)

**Expected result**:
- Returns empty list (no detections)
- No exception raised

**Max execution time**: 100ms

**Dependencies**: TRT engine

---

### IT-04: Load Raises EngineLoadError for Invalid Engine Path

**Summary**: Verify load() raises EngineLoadError when engine file is missing or corrupted.

**Traces to**: AC-04

**Input data**:
- engine_path pointing to non-existent file
- engine_path pointing to a 0-byte file

**Expected result**:
- EngineLoadError raised in both cases
- is_ready() returns false

**Max execution time**: 1s

**Dependencies**: None

---

### IT-05: NMS-Free vs NMS Detection Output Consistency

**Summary**: Verify both YOLOE-26 (NMS-free) and YOLOE-11 (may need NMS) produce non-overlapping detections.

**Traces to**: AC-04

**Input data**:
- Reference frame with multiple closely spaced objects
- Two TRT engines: one NMS-free (YOLOE-26), one requiring NMS (YOLOE-11)

**Expected result**:
- Both engines produce detections with minimal overlap (IoU between any two detections of same class < 0.5)
- Detection count within ±20% of each other

**Max execution time**: 500ms

**Dependencies**: Both TRT engines

---

## Performance Tests

### PT-01: Single Frame Inference Latency

**Summary**: Measure end-to-end detect() latency on Jetson Orin Nano Super.

**Traces to**: AC-01

**Load scenario**:
- Single frame at a time (sequential)
- 100 frames of varying content
- Duration: ~10s
- Ramp-up: none

**Expected results**:

| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Latency (p50) | ≤50ms | >100ms |
| Latency (p95) | ≤80ms | >100ms |
| Latency (p99) | ≤100ms | >100ms |

**Resource limits**:
- GPU memory: ≤2.5GB
- CPU: ≤30% (preprocessing only)

---

### PT-02: Sustained Throughput at Target FPS

**Summary**: Verify Tier1 can sustain 30 FPS inference without frame drops over a 60-second window.

**Traces to**: AC-01

**Load scenario**:
- 30 frames/second, continuous
- Duration: 60 seconds
- Ramp-up: immediate

**Expected results**:

| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Sustained FPS | ≥30 | <25 |
| Frame drop rate | 0% | >2% |
| Max latency spike | ≤150ms | >200ms |

**Resource limits**:
- GPU memory: ≤2.5GB (stable, no growth)
- GPU utilization: ≤90%

---

### PT-03: GPU Memory Consumption

**Summary**: Verify TRT engine stays within allocated GPU memory budget.

**Traces to**: AC-26

**Load scenario**:
- Load engine, run 100 frames, measure memory before/after
- Duration: 30 seconds
- Ramp-up: engine load

**Expected results**:

| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| GPU memory at load | ≤2.0GB | >2.5GB |
| GPU memory after 100 frames | ≤2.0GB (no growth) | >2.5GB |
| Memory leak rate | 0 MB/min | >10 MB/min |

**Resource limits**:
- GPU memory: ≤2.5GB hard cap

---

## Security Tests

### ST-01: Engine File Integrity Validation

**Summary**: Verify the engine loader detects a tampered or corrupted engine file.

**Traces to**: AC-04

**Attack vector**: Corrupted model file (supply chain or disk corruption)

**Test procedure**:
1. Flip random bytes in a valid TRT engine file
2. Call load() with the corrupted file

**Expected behavior**: EngineLoadError raised; system does not execute arbitrary code from the corrupted file.

**Pass criteria**: Exception raised, is_ready() returns false.

**Fail criteria**: Engine loads successfully with corrupted file, or system crashes/hangs.

---

## Acceptance Tests

### AT-01: New Class Detection on Validation Set

**Summary**: Verify P≥80% and R≥80% on new target classes using the validation dataset.

**Traces to**: AC-04

**Preconditions**:
- Validation set of ≥200 annotated frames with footpaths, branch piles, dark entrances, roads, trees, tree blocks
- TRT FP16 engine exported with all classes

**Steps**:

| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Run detect() on all validation frames | Detections produced for each frame |
| 2 | Match detections to ground truth (IoU > 0.5) | TP/FP/FN counts per class |
| 3 | Compute precision and recall per class | P ≥ 80%, R ≥ 80% for each new class |

---

### AT-02: Baseline Class Regression Test

**Summary**: Verify existing YOLO classes maintain detection quality after adding new classes.

**Traces to**: AC-05

**Preconditions**:
- Baseline mAP50 recorded on existing validation set before new classes added
- Same validation set available

**Steps**:

| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Run detect() on baseline validation set | Detections for existing classes |
| 2 | Compute mAP50 for existing classes | mAP50 ≥ baseline - 2% |
| 3 | Check per-class P and R | No class drops below P=75% or R=75% |

---

## Test Data Management

**Required test data**:

| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| validation_new_classes | 200+ frames with annotations for all 6 new classes | Annotated field imagery | ~2 GB |
| validation_baseline | 50+ frames with existing class annotations | Existing test suite | ~500 MB |
| blank_frames | All-black and all-white frames for edge cases | Generated | ~10 MB |
| corrupted_engines | TRT engine files with flipped bytes | Generated from valid engine | ~100 MB |

**Setup procedure**:
1. Copy TRT engine to test model directory
2. Load class names from config
3. Call load(engine_path, class_names)

**Teardown procedure**:
1. Unload engine (if applicable)
2. Clear GPU memory

**Data isolation strategy**: Each test uses its own engine load instance. No shared state between tests.
