ai-training/_docs/02_document/tests/blackbox-tests.md

Blackbox Test Scenarios

BT-AUG: Augmentation Pipeline

BT-AUG-01: Single image produces 8 outputs

• Input: 1 image + 1 valid label from fixture dataset
• Action: Run Augmentator.augment_inner() on the image
• Expected: Returns list of exactly 8 ImageLabel objects
• Traces: AC: 8× augmentation ratio

BT-AUG-02: Augmented filenames follow naming convention

• Input: Image with stem "test_image"
• Action: Run augment_inner()
• Expected: Output filenames: test_image.jpg, test_image_1.jpg through test_image_7.jpg; matching .txt labels
• Traces: AC: Augmentation output format

BT-AUG-03: All output bounding boxes in valid range

• Input: 1 image + label with multiple bboxes
• Action: Run augment_inner()
• Expected: Every bbox coordinate in every output label is in [0.0, 1.0]
• Traces: AC: Bounding boxes clipped to [0, 1]

BT-AUG-04: Bounding box correction clips edge bboxes

• Input: Label with bbox near edge: 0 0.99 0.5 0.2 0.1
• Action: Run correct_bboxes()
• Expected: Width reduced so bbox fits within [margin, 1-margin]; no coordinate exceeds bounds
• Traces: AC: Bounding boxes clipped to [0, 1]

BT-AUG-05: Tiny bounding boxes removed after correction

• Input: Label with tiny bbox that becomes < 0.01 after clipping
• Action: Run correct_bboxes()
• Expected: Bbox removed from output (area < correct_min_bbox_size)
• Traces: AC: Bounding boxes with area < 0.01% discarded

BT-AUG-06: Empty label produces 8 outputs with empty labels

• Input: 1 image + empty label file
• Action: Run augment_inner()
• Expected: 8 ImageLabel objects returned; all have empty labels lists
• Traces: AC: Augmentation handles empty annotations

BT-AUG-07: Full augmentation pipeline (filesystem integration)

• Input: 5 images + labels copied to data/ directory in tmp_path
• Action: Run augment_annotations() with patched paths
• Expected: 40 images (5 × 8) in processed images dir; 40 matching labels in processed labels dir
• Traces: AC: 8× augmentation, filesystem output

BT-AUG-08: Augmentation skips already-processed images

• Input: 5 images in data/; 3 already present in processed/ dir
• Action: Run augment_annotations()
• Expected: Only 2 new images processed (16 new outputs); existing 3 untouched
• Traces: AC: Augmentation processes only unprocessed images

---

BT-DSF: Dataset Formation

BT-DSF-01: 70/20/10 split ratio

• Input: 100 images + labels in processed/ dir
• Action: Run form_dataset() with patched paths
• Expected: train: 70 images+labels, valid: 20, test: 10
• Traces: AC: Dataset split 70/20/10

BT-DSF-02: Split directories structure

• Input: 100 images + labels
• Action: Run form_dataset()
• Expected: Created: train/images/, train/labels/, valid/images/, valid/labels/, test/images/, test/labels/
• Traces: AC: YOLO dataset directory structure

BT-DSF-03: Total files preserved across splits

• Input: 100 valid images + labels
• Action: Run form_dataset()
• Expected: count(train) + count(valid) + count(test) == 100 (no data loss)
• Traces: AC: Dataset integrity

BT-DSF-04: Corrupted labels moved to corrupted directory

• Input: 95 valid + 5 corrupted labels (coords > 1.0)
• Action: Run form_dataset() with patched paths
• Expected: 5 images+labels in data-corrupted/; 95 across train/valid/test splits
• Traces: AC: Corrupted labels filtered

---

BT-LBL: Label Validation

BT-LBL-01: Valid label accepted

• Input: Label file: 0 0.5 0.5 0.1 0.1
• Action: Call check_label(path)
• Expected: Returns True
• Traces: AC: Valid YOLO label format

BT-LBL-02: Label with x > 1.0 rejected

• Input: Label file: 0 1.5 0.5 0.1 0.1
• Action: Call check_label(path)
• Expected: Returns False
• Traces: AC: Corrupted labels detected

BT-LBL-03: Label with height > 1.0 rejected

• Input: Label file: 0 0.5 0.5 0.1 1.2
• Action: Call check_label(path)
• Expected: Returns False
• Traces: AC: Corrupted labels detected

BT-LBL-04: Missing label file rejected

• Input: Non-existent file path
• Action: Call check_label(path)
• Expected: Returns False
• Traces: AC: Missing labels handled

BT-LBL-05: Multi-line label with one corrupted line

• Input: Label file: 0 0.5 0.5 0.1 0.1\n3 0.5 0.5 0.1 1.5
• Action: Call check_label(path)
• Expected: Returns False (any corrupted line fails the whole file)
• Traces: AC: Corrupted labels detected

---

BT-ENC: Encryption

BT-ENC-01: Encrypt-decrypt roundtrip (arbitrary data)

• Input: 1024 random bytes, key "test-key"
• Action: decrypt_to(encrypt_to(data, key), key)
• Expected: Output equals input bytes exactly
• Traces: AC: AES-256-CBC encryption

BT-ENC-02: Encrypt-decrypt roundtrip (ONNX model)

• Input: azaion.onnx bytes, model encryption key
• Action: decrypt_to(encrypt_to(model_bytes, key), key)
• Expected: Output equals input bytes exactly
• Traces: AC: Model encryption

BT-ENC-03: Empty input roundtrip

• Input: b"", key "test-key"
• Action: decrypt_to(encrypt_to(b"", key), key)
• Expected: Output equals b""
• Traces: AC: Edge case handling

BT-ENC-04: Single byte roundtrip

• Input: b"\x00", key "test-key"
• Action: decrypt_to(encrypt_to(b"\x00", key), key)
• Expected: Output equals b"\x00"
• Traces: AC: Edge case handling

BT-ENC-05: Different keys produce different ciphertext

• Input: Same 1024 bytes, keys "key-a" and "key-b"
• Action: encrypt_to(data, "key-a") vs encrypt_to(data, "key-b")
• Expected: Ciphertexts differ
• Traces: AC: Key-dependent encryption

BT-ENC-06: Wrong key fails decryption

• Input: Encrypted with "key-a", decrypt with "key-b"
• Action: decrypt_to(encrypted, "key-b")
• Expected: Output does NOT equal original input
• Traces: AC: Key-dependent encryption

---

BT-SPL: Model Split Storage

BT-SPL-01: Split respects size constraint

• Input: 10000 encrypted bytes
• Action: Split into small + big per SMALL_SIZE_KB = 3 logic
• Expected: small ≤ max(3072 bytes, 20% of total); big = remainder
• Traces: AC: Model split ≤3KB or 20%

BT-SPL-02: Reassembly produces original

• Input: 10000 encrypted bytes → split → reassemble
• Action: small + big
• Expected: Equals original encrypted bytes
• Traces: AC: Split model integrity

---

BT-CLS: Annotation Class Loading

BT-CLS-01: Load 17 base classes

• Input: classes.json
• Action: AnnotationClass.read_json()
• Expected: Dict with 17 unique class entries (base IDs)
• Traces: AC: 17 base classes

BT-CLS-02: Weather mode expansion

• Input: classes.json
• Action: AnnotationClass.read_json()
• Expected: Same class at offset 0 (Norm), 20 (Wint), 40 (Night); e.g., ID 0, 20, 40 all represent ArmorVehicle
• Traces: AC: 3 weather modes

BT-CLS-03: YAML generation produces 80 class names

• Input: classes.json + dataset path
• Action: create_yaml() with patched paths
• Expected: data.yaml contains nc: 80, 17 named classes + 63 Class-N placeholders
• Traces: AC: 80 total class slots

---

BT-HSH: Hardware Hash

BT-HSH-01: Deterministic output

• Input: "test-hardware-info"
• Action: Security.get_hw_hash() called twice
• Expected: Both calls return identical string
• Traces: AC: Hardware fingerprinting determinism

BT-HSH-02: Different inputs produce different hashes

• Input: "hw-a" and "hw-b"
• Action: Security.get_hw_hash() on each
• Expected: Results differ
• Traces: AC: Hardware-bound uniqueness

BT-HSH-03: Output is valid base64

• Input: "test-hardware-info"
• Action: Security.get_hw_hash()
• Expected: Matches regex ^[A-Za-z0-9+/]+=*$
• Traces: AC: Hash format

---

BT-INF: ONNX Inference

BT-INF-01: Model loads successfully

• Input: azaion.onnx bytes
• Action: OnnxEngine(model_bytes)
• Expected: No exception; engine object created with valid input_shape and batch_size
• Traces: AC: ONNX inference capability

BT-INF-02: Inference returns output

• Input: ONNX engine + 1 preprocessed image
• Action: engine.run(input_blob)
• Expected: Returns list of numpy arrays; first array has shape [batch, N, 6+]
• Traces: AC: ONNX inference produces results

BT-INF-03: Postprocessing returns valid detections

• Input: ONNX engine output from real image
• Action: Inference.postprocess()
• Expected: Returns list of Annotation objects; each Detection has x,y,w,h ∈ [0,1], cls ∈ [0,79], confidence ∈ [0,1]
• Traces: AC: Detection format validity

---

BT-NMS: Overlap Removal

BT-NMS-01: Overlapping detections — keep higher confidence

• Input: 2 Detection objects at same position, confidence 0.9 and 0.5, IoU > 0.3
• Action: remove_overlapping_detections()
• Expected: 1 detection returned (confidence 0.9)
• Traces: AC: NMS IoU threshold 0.3

BT-NMS-02: Non-overlapping detections — keep both

• Input: 2 Detection objects at distant positions, IoU < 0.3
• Action: remove_overlapping_detections()
• Expected: 2 detections returned
• Traces: AC: NMS preserves non-overlapping

BT-NMS-03: Chain overlap resolution

• Input: 3 Detection objects: A overlaps B (IoU > 0.3), B overlaps C (IoU > 0.3), A doesn't overlap C
• Action: remove_overlapping_detections()
• Expected: ≤ 2 detections; highest confidence per overlapping pair kept
• Traces: AC: NMS handles chains

---

BT-AQM: Annotation Queue Message Parsing

BT-AQM-01: Parse Created annotation message

• Input: Msgpack bytes matching AnnotationMessage schema (status=Created, role=Validator)
• Action: Decode and construct AnnotationMessage
• Expected: All fields populated: name, detections, image bytes, status == "Created", role == "Validator"
• Traces: AC: Annotation message parsing

BT-AQM-02: Parse Validated bulk message

• Input: Msgpack bytes with status=Validated, list of names
• Action: Decode and construct AnnotationBulkMessage
• Expected: Status == "Validated", names list matches input
• Traces: AC: Bulk validation parsing

BT-AQM-03: Parse Deleted bulk message

• Input: Msgpack bytes with status=Deleted, list of names
• Action: Decode and construct AnnotationBulkMessage
• Expected: Status == "Deleted", names list matches input
• Traces: AC: Bulk deletion parsing

BT-AQM-04: Malformed message raises exception

• Input: Invalid msgpack bytes
• Action: Attempt to decode
• Expected: Exception raised
• Traces: AC: Error handling for malformed messages