mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 21:11:13 +00:00
[AZ-345] [AZ-346] [AZ-347] [AZ-349] Archive batch 57 task specs
Move completed task specs from _docs/02_tasks/todo/ to _docs/02_tasks/done/ now that the four tickets are In Testing. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -1,209 +0,0 @@
|
||||
# C3 DISK+LightGlue Primary Matcher
|
||||
|
||||
**Task**: AZ-345_c3_disk_lightglue
|
||||
**Name**: C3 DISK+LightGlue Primary Matcher
|
||||
**Description**: Implement `DiskLightGlueMatcher`, the production-default `CrossDomainMatcher` (per D-C3-1 = (a)). For each top-N=3 candidate in a `RerankResult`: extract DISK keypoints + descriptors from the nav-camera frame and the candidate tile via the C7 `InferenceRuntime` (TensorRT 10.3 FP16 primary, ONNX-Runtime fallback); match keypoints via the shared `LightGlueRuntime` helper (AZ-278); filter inliers + compute median reprojection residual via the shared `RansacFilter` helper (AZ-282); record the result in a `CandidateMatchSet`. Sort surviving candidates descending by inlier count (tie-break: lower median residual ranked higher); return the best as `MatchResult.best_candidate_idx`. Implements the drop-and-continue contract (Invariant 4) for per-candidate `MatcherBackboneError`. Updates the constructor-injected `RollingHealthWindow` after each frame. Composition-root wired via the AZ-344 factory.
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-344 (Protocol + factory + DTOs + errors + RollingHealthWindow), AZ-263_initial_structure, AZ-269_config_loader, AZ-278_lightglue_runtime (shared LightGlue helper), AZ-282_ransac_filter (shared RANSAC helper), AZ-298_c7_tensorrt_runtime (DISK forward via TRT), AZ-299_c7_onnxrt_fallback (DISK forward via ONNX-RT fallback), AZ-303_c6_storage_interfaces (`tile_pixels_handle` from `RerankResult`; tile pixel decode), AZ-281_engine_filename_schema (DISK engine self-describing filename), AZ-321_c10_engine_compiler (DISK + LightGlue engine compile path), AZ-266_log_module, AZ-272_fdr_record_schema
|
||||
**Component**: c3_matcher (epic AZ-257 / E-C3)
|
||||
**Tracker**: AZ-345
|
||||
**Epic**: AZ-257 (E-C3)
|
||||
|
||||
### Document Dependencies
|
||||
|
||||
- `_docs/02_document/contracts/c3_matcher/cross_domain_matcher_protocol.md` — Protocol contract (every invariant satisfied; drop-and-continue is INV-4).
|
||||
- `_docs/02_document/components/04_c3_matcher/description.md` — § 1 D-C3-1 = (a) production-default; § 5 error handling; § 7 shared helper serial access; § 9 logging.
|
||||
- `_docs/02_document/module-layout.md` — `c3_matcher` Per-Component Mapping (`disk_lightglue.py` Internal); `BUILD_MATCHER_DISK_LIGHTGLUE` row (ON for airborne / research / replay-cli).
|
||||
- `_docs/02_document/contracts/shared_helpers/lightglue_runtime.md` — single-pair / multi-pair API.
|
||||
- `_docs/02_document/contracts/shared_helpers/ransac_filter.md` — RANSAC + median residual API.
|
||||
- `_docs/02_document/contracts/c2_5_rerank/rerank_strategy_protocol.md` — `RerankResult` consumed at input boundary.
|
||||
- `_docs/02_document/contracts/c7_inference/inference_runtime_protocol.md` — DISK forward via `InferenceRuntime`.
|
||||
- `_docs/02_document/components/04_c3_matcher/tests.md` — C3-IT-01 (best-candidate inlier count p5 ≥ 80); C3-IT-02 (deterministic best_candidate_idx); C3-IT-03 (cross-domain MRE p95 < 2.5 px); C3-IT-04 (tilt ±20° + 350m outliers); C3-IT-05 (`InsufficientInliersError` propagation); C3-PT-01 (latency p95 ≤ 180 ms; per-candidate ≤ 60 ms; GPU mem ≤ 800 MB).
|
||||
|
||||
## Problem
|
||||
|
||||
Without this task: `compose_root` cannot wire when `config.matcher.strategy = "disk_lightglue"` (the default value); F3 / F6 cannot run; AC-1.1 (best-candidate inlier count p5 ≥ 80) has no producer; AC-2.2 (cross-domain MRE p95 < 2.5 px) is unmeasurable; AC-NEW-7 cache-poisoning safety budget loses its primary detection signal (low-inlier frames in MatcherHealth). The DISK+LightGlue choice is locked per Mode B Fact #110 / D-C3-1; without this task the locked decision is unrealised.
|
||||
|
||||
## Outcome
|
||||
|
||||
- `src/gps_denied_onboard/components/c3_matcher/disk_lightglue.py` defining:
|
||||
- `DiskLightGlueMatcher` class implementing the `CrossDomainMatcher` Protocol (AZ-344).
|
||||
- Constructor: `__init__(self, runtime: InferenceRuntime, lightglue_runtime: LightGlueRuntime, ransac_filter: RansacFilter, fdr_client: FdrClient, health_window: RollingHealthWindow, config: MatcherConfig)`. The strategy holds the DISK engine ID (returned by `runtime.load_engine`) plus references to the constructor-injected `LightGlueRuntime` + `RansacFilter`.
|
||||
- `match(frame, rerank_result, calibration)`:
|
||||
1. Decode + preprocess the nav-camera frame ONCE (resize / normalise per DISK's input contract).
|
||||
2. Run DISK forward on the query frame → `(query_keypoints, query_descriptors)`.
|
||||
3. `survivors: list[CandidateMatchSet] = []`, `dropped = 0`.
|
||||
4. For each `RerankCandidate` in `rerank_result.candidates`:
|
||||
a. Decode + preprocess the candidate tile (from `tile_pixels_handle`).
|
||||
b. Try DISK forward on the tile → `(tile_keypoints, tile_descriptors)`. On failure: wrap as `MatcherBackboneError`; emit ERROR log + FDR record `kind="matcher.backbone_error"` with `tile_id` + `phase="disk_forward"`; `dropped += 1`; continue.
|
||||
c. Try `lightglue_runtime.match_pair(query_keypoints, query_descriptors, tile_keypoints, tile_descriptors)` → `correspondences` (raw matches before RANSAC). On failure: wrap as `MatcherBackboneError`; phase="lightglue_match"; drop; continue.
|
||||
d. `ransac_result = ransac_filter.filter(correspondences, threshold_px=config.ransac_threshold_px)` → `RansacResult(inlier_correspondences, ransac_outlier_count, per_candidate_residual_px)`. The helper handles RANSAC + median residual computation.
|
||||
e. If `ransac_result.inlier_correspondences.shape[0] == 0`: emit DEBUG log `kind="c3.matcher.zero_inliers"`; `dropped += 1`; continue.
|
||||
f. Append `CandidateMatchSet(tile_id=candidate.tile_id, inlier_count=ransac_result.inlier_correspondences.shape[0], inlier_correspondences=ransac_result.inlier_correspondences, ransac_outlier_count=ransac_result.ransac_outlier_count, per_candidate_residual_px=ransac_result.per_candidate_residual_px)` to `survivors`.
|
||||
5. Determine `survivor_max_inliers = max(s.inlier_count for s in survivors)` (or 0 if empty).
|
||||
6. If `len(survivors) == 0` OR `survivor_max_inliers < config.min_inliers_threshold`: emit ERROR log `kind="c3.matcher.insufficient_inliers"` + FDR record `kind="matcher.insufficient_inliers"`; `health_window.update(now, best_inlier_count=0, had_backbone_error=(dropped > 0))`; raise `InsufficientInliersError`.
|
||||
7. Sort `survivors` descending by `inlier_count`; ties broken by `per_candidate_residual_px` ascending. The first survivor is the best.
|
||||
8. `best = survivors[0]`. If `best.per_candidate_residual_px > config.residual_warn_threshold_px`: emit WARN log `kind="c3.matcher.residual_above_threshold"` (will trigger AdHoP at C3.5).
|
||||
9. `health_window.update(now, best_inlier_count=best.inlier_count, had_backbone_error=(dropped > 0))`.
|
||||
10. Emit FDR record `kind="matcher.frame_done"` with `{frame_id, candidates_input, candidates_dropped, best_inlier_count, best_residual_px, best_tile_id}`.
|
||||
11. Return `MatchResult(frame_id=rerank_result.frame_id, per_candidate=survivors, best_candidate_idx=0, reprojection_residual_px=best.per_candidate_residual_px, matched_at=monotonic_ns(), matcher_label="disk_lightglue", candidates_input=len(rerank_result.candidates), candidates_dropped=dropped)`.
|
||||
- `health_snapshot()`: returns `self._health_window.snapshot()`.
|
||||
- Module-level `create(config, lightglue_runtime, ransac_filter, inference_runtime, health_window) -> CrossDomainMatcher`:
|
||||
1. `disk_weights_path = config.matcher.disk_weights_path` (TRT engine produced by AZ-321).
|
||||
2. Load DISK engine via `inference_runtime.load_engine(disk_weights_path)`.
|
||||
3. Construct `DiskLightGlueMatcher(...)`.
|
||||
- Composition-root wiring path for `config.matcher.strategy == "disk_lightglue"`.
|
||||
- Logging per description.md § 9: INFO ready; WARN residual-above-threshold; ERROR insufficient-inliers + backbone-error; DEBUG per-frame inlier+residual list (gated).
|
||||
- FDR records: `matcher.frame_done` (always per frame), `matcher.backbone_error` (per error), `matcher.insufficient_inliers` (per all-failed event).
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
- `DiskLightGlueMatcher` class implementing `CrossDomainMatcher` exactly per the AZ-344 contract.
|
||||
- DISK forward via C7 `InferenceRuntime` (TRT primary; ONNX-RT fallback chain owned by C7 — this task consumes the unified interface).
|
||||
- LightGlue matching via shared helper.
|
||||
- RANSAC + median residual via shared `RansacFilter` helper.
|
||||
- Drop-and-continue per-candidate error handling (Invariant 4).
|
||||
- Below-threshold all-failed → `InsufficientInliersError`.
|
||||
- Deterministic best-candidate selection (Invariant 3).
|
||||
- `RollingHealthWindow.update` after each frame.
|
||||
- Composition-root wiring path.
|
||||
- Logging + FDR record emission per description.md § 9.
|
||||
- Unit tests covering Invariants 1–9, drop-and-continue, below-threshold, deterministic ordering, `tile_pixels_handle` reference semantics, composition-root wiring path.
|
||||
- `BUILD_MATCHER_DISK_LIGHTGLUE` flag wiring (ON in airborne / research / replay-cli; OFF in operator-tooling).
|
||||
|
||||
### Excluded
|
||||
- The Protocol + DTOs + errors + factory + `RollingHealthWindow` — owned by AZ-344.
|
||||
- The `LightGlueRuntime` helper — already AZ-278.
|
||||
- The `RansacFilter` helper — already AZ-282.
|
||||
- The C7 `InferenceRuntime` — owned by AZ-297..AZ-300.
|
||||
- DISK engine compile (.onnx → .trt) — owned by AZ-321; this task consumes the produced engine.
|
||||
- ALIKED+LightGlue (AZ-346) and XFeat (AZ-347).
|
||||
- Component-internal acceptance tests beyond Invariants 1–9 + drop-and-continue smoke: C3-IT-01 (recall floor), C3-IT-03 (cross-domain MRE), C3-IT-04 (tilt outliers), C3-PT-01 (latency NFR), are deferred to Step 9 / E-BBT.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: Protocol conformance**
|
||||
`isinstance(DiskLightGlueMatcher(...), CrossDomainMatcher)` returns `True`.
|
||||
|
||||
**AC-2: Best-candidate selection — argmax(inlier_count) + tie-break**
|
||||
Given a `RerankResult` with N=3 candidates whose computed inlier counts are [120, 80, 120] and median residuals [1.4, 1.0, 1.1]
|
||||
When `match(...)` is called
|
||||
Then `best_candidate_idx == 0` (the candidate with `inlier_count=120` AND `residual=1.1` (lower than the other 120-inlier candidate's 1.4)); `per_candidate[0].inlier_count == 120 AND per_candidate_residual_px == 1.1`; `per_candidate[1].inlier_count == 120 AND per_candidate_residual_px == 1.4`; `per_candidate[2].inlier_count == 80`.
|
||||
|
||||
**AC-3: Drop-and-continue on per-candidate `MatcherBackboneError`**
|
||||
Given an `InferenceRuntime` test double that raises `RuntimeError` on the 2nd candidate's DISK forward and succeeds on others
|
||||
When `match(...)` is called
|
||||
Then `len(per_candidate) == 2`; `candidates_dropped == 1`; ONE ERROR log `kind="c3.matcher.backbone_error"` is emitted with `tile_id` + `phase="disk_forward"`; ONE FDR record `kind="matcher.backbone_error"` is emitted; success path continues.
|
||||
|
||||
**AC-4: Drop-and-continue on per-candidate LightGlue failure**
|
||||
Given a `LightGlueRuntime` test double that raises on the 1st candidate's match call
|
||||
When `match(...)` is called
|
||||
Then the candidate is dropped with `phase="lightglue_match"`; ERROR log + FDR record emitted; remaining candidates processed.
|
||||
|
||||
**AC-5: Below-threshold → `InsufficientInliersError`**
|
||||
Given `config.matcher.min_inliers_threshold = 60` AND every candidate's RANSAC inlier count is < 60
|
||||
When `match(...)` is called
|
||||
Then `InsufficientInliersError` is raised; ONE ERROR log `kind="c3.matcher.insufficient_inliers"` + ONE FDR record `kind="matcher.insufficient_inliers"` are emitted; `health_window.update(now, best_inlier_count=0, had_backbone_error=False)` is invoked.
|
||||
|
||||
**AC-6: All-failed → `InsufficientInliersError`**
|
||||
Given every candidate's DISK forward raises
|
||||
When `match(...)` is called
|
||||
Then `InsufficientInliersError` is raised; per-candidate ERROR logs + final ERROR log emitted; `health_window.update(now, best_inlier_count=0, had_backbone_error=True)` is invoked.
|
||||
|
||||
**AC-7: WARN log on residual above threshold**
|
||||
Given the best candidate's `per_candidate_residual_px = 4.2` AND `config.matcher.residual_warn_threshold_px = 2.5`
|
||||
When `match(...)` returns
|
||||
Then ONE WARN log `kind="c3.matcher.residual_above_threshold"` with `{residual_px: 4.2, threshold_px: 2.5}` is emitted.
|
||||
|
||||
**AC-8: `health_window.update` invoked after every `match` (success or failure)**
|
||||
Given any `match(...)` call (success, partial drop, all-failed)
|
||||
When the call completes (returns normally OR raises `InsufficientInliersError`)
|
||||
Then `health_window.update(...)` is invoked exactly ONCE for that frame; `best_inlier_count` matches the actual best inlier count (0 on all-failed); `had_backbone_error == True` if any candidate dropped due to backbone failure.
|
||||
|
||||
**AC-9: `inlier_correspondences` shape contract**
|
||||
Given a successful `match(...)`
|
||||
When inspecting any `CandidateMatchSet`
|
||||
Then `inlier_correspondences.shape == (inlier_count, 4)`; `dtype == float32`.
|
||||
|
||||
**AC-10: Deterministic — same inputs → bit-identical MatchResult**
|
||||
Given fixed inputs and deterministic test doubles
|
||||
When `match(...)` is called 3 times
|
||||
Then all three returns have identical `per_candidate` content (same inlier_counts, same residuals, same best_candidate_idx).
|
||||
|
||||
**AC-11: Composition-root wiring**
|
||||
Given `config.matcher.strategy = "disk_lightglue"` AND a constructed shared `LightGlueRuntime` AND `RansacFilter` AND `InferenceRuntime`
|
||||
When `compose_root(config)` runs
|
||||
Then a `DiskLightGlueMatcher` instance is wired; ONE INFO log `kind="c3.matcher.ready"` with `{strategy: "disk_lightglue", min_inliers_threshold, residual_warn_threshold_px}` is emitted; the strategy's `_lightglue_runtime` is identity-equal to the runtime root's shared helper.
|
||||
|
||||
**AC-12: FDR `matcher.frame_done` per frame**
|
||||
Given a successful `match(...)` returning best candidate with inlier_count=120 and residual=1.1, dropped=1
|
||||
When the call completes
|
||||
Then ONE FDR record `kind="matcher.frame_done"` is emitted with structured fields `{frame_id, candidates_input: 3, candidates_dropped: 1, best_inlier_count: 120, best_residual_px: 1.1, best_tile_id: <tuple>}`.
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance** (deferred validation to C3-PT-01):
|
||||
- `match` p95 ≤ 180 ms (3 candidates × ~60 ms DISK forward + LightGlue match + RANSAC).
|
||||
- Per-candidate p95 ≤ 60 ms.
|
||||
- GPU memory ≤ 800 MB combined (DISK engine + LightGlue engine resident).
|
||||
|
||||
**Compatibility**
|
||||
- DISK engine file format owned by C10 + C7; this task consumes via `config.matcher.disk_weights_path`.
|
||||
- Upstream DISK research code drop pinned per Plan-phase; weight changes require C10 rebuild + C3-IT-03 re-run.
|
||||
|
||||
**Reliability**
|
||||
- Drop-and-continue per candidate (Invariant 4).
|
||||
- Single-thread by contract (INV-1).
|
||||
- `InsufficientInliersError` triggers C5 VIO-only fallback (AC-3.5); does NOT crash.
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| AC Ref | What to Test | Required Outcome |
|
||||
|--------|--------------|------------------|
|
||||
| AC-1 | Protocol conformance | `isinstance` returns `True` |
|
||||
| AC-2 | Best-candidate + tie-break | Lower residual wins among tied inliers |
|
||||
| AC-3 | DISK forward fails on 2nd | 2 survivors; ERROR log + FDR record |
|
||||
| AC-4 | LightGlue fails on 1st | 2 survivors; phase="lightglue_match" |
|
||||
| AC-5 | All below threshold | `InsufficientInliersError`; health update |
|
||||
| AC-6 | All forwards fail | `InsufficientInliersError`; per-candidate logs |
|
||||
| AC-7 | Residual > warn threshold | WARN log emitted |
|
||||
| AC-8 | Health update invoked once per `match` | One update per call regardless of outcome |
|
||||
| AC-9 | Correspondences shape | (I, 4) float32; I == inlier_count |
|
||||
| AC-10 | Determinism | 3 calls return identical content |
|
||||
| AC-11 | `compose_root` wiring | Wired; INFO log; helper identity-shared |
|
||||
| AC-12 | FDR `frame_done` emission | Correct structured fields |
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Drop-and-continue is mandatory** — Invariant 4; per-candidate exceptions never propagate.
|
||||
- **Median residual, not mean** — Invariant 8; computed inside `RansacFilter`.
|
||||
- **Constructor injection only** — no `import gps_denied_onboard.config` inside the strategy module.
|
||||
- **`LightGlueRuntime` and `RansacFilter` are constructor-injected** — never instantiated here.
|
||||
- **DISK engine load at `create` time, NOT at first frame** — engine-output assertion fires at startup.
|
||||
- **Tile pixel decode is per-call** — but the underlying `tile_pixels_handle` is page-cache-backed (not copied into the strategy).
|
||||
- **`RollingHealthWindow.update` is called EXACTLY once per `match`** — including the all-failed path.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: DISK upstream code drop ships an unsupported ONNX op for TRT 10.3**
|
||||
- *Mitigation*: engine compile is C10's responsibility (AZ-321). If C10 cannot build the engine, this task is blocked upstream — surface via tracker dependency mechanism.
|
||||
|
||||
**Risk 2: `LightGlueRuntime.match_pair` API not yet defined**
|
||||
- *Mitigation*: AZ-278 defines the helper API; this task consumes whatever AZ-278 ships. If only single-pair is provided, this task wraps single-pair calls in a per-candidate loop (already structured that way). Surface to AZ-278 implementer at decompose-step-4.
|
||||
|
||||
**Risk 3: Tile pixel decode is non-trivial cost on hot path**
|
||||
- *Mitigation*: tile pixels arrive as page-cache-backed handles from C6; decode (JPEG → ndarray) happens once per candidate. If profiling shows this is a bottleneck, a future optimization pre-decodes adjacent tiles in C6's mmap layer.
|
||||
|
||||
**Risk 4: Deterministic best-candidate tie-break depends on stable sort**
|
||||
- *Mitigation*: Python's `list.sort()` is stable; the implementation uses `sorted(survivors, key=lambda s: (-s.inlier_count, s.per_candidate_residual_px))` which is deterministic. Test AC-2 asserts the exact ordering on a tie scenario.
|
||||
|
||||
**Risk 5: `RollingHealthWindow` drift between matcher implementations**
|
||||
- *Mitigation*: ONE `RollingHealthWindow` class owned by AZ-344; constructor-injected into every concrete matcher. AZ-345/AZ-346/AZ-347 use the same instance type via the same constructor injection.
|
||||
|
||||
## Runtime Completeness
|
||||
|
||||
- **Named capability**: `DiskLightGlueMatcher` — production-default `CrossDomainMatcher` for cross-domain feature matching (architecture / E-C3 / `solution.md` / D-C3-1 / AC-1.1 + AC-2.2 + AC-3.1).
|
||||
- **Production code that must exist**: real `DiskLightGlueMatcher` calling real C7 `InferenceRuntime` with real TRT-compiled DISK engine; real shared `LightGlueRuntime` calls; real shared `RansacFilter` for inlier filtering + median residual; real `RollingHealthWindow.update` after each frame; real composition-root wiring.
|
||||
- **Allowed external stubs**: `FakeInferenceRuntime`, `FakeLightGlueRuntime`, `FakeRansacFilter`, `FakeFdrClient`, synthetic frame fixtures for unit tests.
|
||||
- **Unacceptable substitutes**: a Python+NumPy implementation of DISK forward (would not satisfy C3-PT-01 latency); a different RANSAC implementation per matcher (would defeat AZ-282 helper); skipping `RollingHealthWindow.update` on the all-failed path (would lose the health signal C5 needs); calling `LightGlueRuntime` in batch mode without per-candidate inlier breakdown; using the mean residual instead of the median (would violate INV-8).
|
||||
@@ -1,120 +0,0 @@
|
||||
# C3 ALIKED+LightGlue Secondary Matcher
|
||||
|
||||
**Task**: AZ-346_c3_aliked_lightglue
|
||||
**Name**: C3 ALIKED+LightGlue Secondary Matcher
|
||||
**Description**: Implement `AlikedLightGlueMatcher`, the secondary `CrossDomainMatcher`. Same architecture as `DiskLightGlueMatcher` (AZ-345) — DISK is replaced by ALIKED for the per-frame keypoint+descriptor extraction step; LightGlue + RANSAC stages are unchanged. Selectable via `config.matcher.strategy = "aliked_lightglue"`. ALIKED is the candidate alternative if D-C3-1 IT-12 verdict shifts away from DISK; until then it ships as the secondary path linked into airborne / research binaries (per ADR-002, both backbones can be linked; only one is selected at runtime).
|
||||
**Complexity**: 3 points
|
||||
**Dependencies**: AZ-344 (Protocol + factory + DTOs + errors + RollingHealthWindow), AZ-263_initial_structure, AZ-269_config_loader, AZ-278_lightglue_runtime, AZ-282_ransac_filter, AZ-298_c7_tensorrt_runtime, AZ-299_c7_onnxrt_fallback, AZ-303_c6_storage_interfaces, AZ-281_engine_filename_schema (ALIKED engine self-describing filename), AZ-321_c10_engine_compiler (ALIKED engine compile path), AZ-266_log_module, AZ-272_fdr_record_schema
|
||||
**Component**: c3_matcher (epic AZ-257 / E-C3)
|
||||
**Tracker**: AZ-346
|
||||
**Epic**: AZ-257 (E-C3)
|
||||
|
||||
### Document Dependencies
|
||||
|
||||
- `_docs/02_document/contracts/c3_matcher/cross_domain_matcher_protocol.md` — Protocol contract (every invariant satisfied; mirrors AZ-345's contract behavior).
|
||||
- `_docs/02_document/components/04_c3_matcher/description.md` — § 1 ALIKED secondary; § 5 same error handling; § 9 logging.
|
||||
- `_docs/02_document/module-layout.md` — `c3_matcher` Per-Component Mapping (`aliked_lightglue.py` Internal); `BUILD_MATCHER_ALIKED_LIGHTGLUE` row.
|
||||
- `_docs/02_document/contracts/shared_helpers/lightglue_runtime.md`.
|
||||
- `_docs/02_document/contracts/shared_helpers/ransac_filter.md`.
|
||||
- `_docs/02_document/contracts/c2_5_rerank/rerank_strategy_protocol.md`.
|
||||
- `_docs/02_document/contracts/c7_inference/inference_runtime_protocol.md`.
|
||||
|
||||
## Problem
|
||||
|
||||
Without this task: D-C3-1 IT-12 evaluation has no comparison point against DISK; if a future cycle's IT-12 verdict shifts the production-default to ALIKED, the airborne binary cannot be re-configured without a new task; the ADR-002 build-time exclusion machinery is under-tested (only one matcher would exist). ALIKED is also the documented fallback if DISK's licensing or upstream maintenance changes mid-cycle.
|
||||
|
||||
## Outcome
|
||||
|
||||
- `src/gps_denied_onboard/components/c3_matcher/aliked_lightglue.py` defining:
|
||||
- `AlikedLightGlueMatcher` class implementing the `CrossDomainMatcher` Protocol.
|
||||
- Constructor identical shape to `DiskLightGlueMatcher` (AZ-345); the only differences are: ALIKED engine loaded instead of DISK, `matcher_label = "aliked_lightglue"`, ALIKED-specific preprocessor (resize / normalise per the upstream ALIKED contract).
|
||||
- `match` method: identical control flow to AZ-345's `match` — drop-and-continue, RANSAC + median residual, deterministic best-candidate selection, `RollingHealthWindow.update`, FDR `matcher.frame_done`. The ONLY difference is the keypoint+descriptor extraction step calls the ALIKED engine instead of DISK.
|
||||
- `health_snapshot()` delegates to the constructor-injected `RollingHealthWindow`.
|
||||
- Module-level `create(config, lightglue_runtime, ransac_filter, inference_runtime, health_window) -> CrossDomainMatcher`:
|
||||
1. `aliked_weights_path = config.matcher.aliked_weights_path` (TRT engine produced by AZ-321).
|
||||
2. Load ALIKED engine via `inference_runtime.load_engine(...)`.
|
||||
3. Construct `AlikedLightGlueMatcher(...)`.
|
||||
- Composition-root wiring path for `config.matcher.strategy == "aliked_lightglue"`.
|
||||
- `BUILD_MATCHER_ALIKED_LIGHTGLUE` flag wiring (per ADR-002): ON in airborne + research binaries; OFF in operator-tooling.
|
||||
- ALIKED-specific preprocessor lives next to the strategy in the same module (NOT in `helpers/` — preprocessing parameters are weights-coupled per the same rule applied in AZ-337 / AZ-345).
|
||||
- All logging + FDR records identical structure to AZ-345 with `matcher_label = "aliked_lightglue"`.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
- `AlikedLightGlueMatcher` implementation per the `CrossDomainMatcher` Protocol.
|
||||
- ALIKED forward via C7 `InferenceRuntime`.
|
||||
- LightGlue matching via shared helper.
|
||||
- RANSAC + median residual via `RansacFilter`.
|
||||
- Same drop-and-continue + below-threshold + best-candidate selection semantics as AZ-345.
|
||||
- Same `RollingHealthWindow.update` invocation pattern.
|
||||
- Composition-root wiring path.
|
||||
- ALIKED-specific preprocessor inline.
|
||||
- Unit tests covering Invariants 1–9 + drop-and-continue + below-threshold + deterministic ordering, parametrised so they share fixtures with AZ-345's tests where possible.
|
||||
- `BUILD_MATCHER_ALIKED_LIGHTGLUE` flag wiring.
|
||||
|
||||
### Excluded
|
||||
- The Protocol + DTOs + errors + factory + `RollingHealthWindow` — owned by AZ-344.
|
||||
- `LightGlueRuntime` (AZ-278) and `RansacFilter` (AZ-282) helpers.
|
||||
- C7 runtime stack (AZ-297..AZ-300).
|
||||
- ALIKED engine compile (AZ-321).
|
||||
- Component-internal acceptance tests beyond Protocol + invariants smoke: deferred to Step 9 / E-BBT.
|
||||
- DISK matcher (AZ-345) and XFeat matcher (AZ-347).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1 through AC-12**: identical contract to AZ-345 AC-1..AC-12 with `matcher_label = "aliked_lightglue"` and ALIKED-specific tile preprocessing. The Protocol invariants are the same; the implementation is the same modulo backbone. Tests parametrise across both backbones so any divergence is caught.
|
||||
|
||||
**AC-special-1: ALIKED engine output schema is asserted at `create` time**
|
||||
Given a TRT engine whose ALIKED output dimensionality differs from the upstream-published value (e.g., descriptor_dim != expected)
|
||||
When `AlikedLightGlueMatcher.create(...)` is called
|
||||
Then `ConfigurationError` is raised with the offending shape; the strategy is NOT instantiated.
|
||||
|
||||
**AC-special-2: Strategy selection — `config.matcher.strategy == "aliked_lightglue"`**
|
||||
Given the runtime composition with `config.matcher.strategy = "aliked_lightglue"` AND `BUILD_MATCHER_ALIKED_LIGHTGLUE = ON`
|
||||
When `compose_root(config)` runs
|
||||
Then an `AlikedLightGlueMatcher` is instantiated; ONE INFO log `kind="c3.matcher.ready"` with `{strategy: "aliked_lightglue", ...}` is emitted; `_lightglue_runtime` identity-equal to the runtime root's shared helper.
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance** (deferred validation to C3-PT-01):
|
||||
- Same envelope as AZ-345: `match` p95 ≤ 180 ms; per-candidate ≤ 60 ms; GPU mem ≤ 800 MB.
|
||||
|
||||
**Compatibility**
|
||||
- ALIKED engine file format owned by C10 + C7; consumed via `config.matcher.aliked_weights_path`.
|
||||
|
||||
**Reliability**
|
||||
- Same as AZ-345: drop-and-continue, single-thread by contract, `InsufficientInliersError` triggers VIO-only fallback.
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| AC Ref | What to Test | Required Outcome |
|
||||
|--------|--------------|------------------|
|
||||
| AC-1..AC-12 | Identical to AZ-345 AC-1..AC-12 with ALIKED label | Same outcomes; `matcher_label = "aliked_lightglue"` |
|
||||
| AC-special-1 | ALIKED engine output shape mismatch | `ConfigurationError` at create time |
|
||||
| AC-special-2 | `compose_root(config="aliked_lightglue")` | Wired; INFO log emitted; helper identity-shared |
|
||||
| Parametrised drop-and-continue | Run AZ-345's drop-and-continue tests against ALIKED matcher fixture | Same drop-and-continue semantics |
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Same constraints as AZ-345** — drop-and-continue mandatory, median residual, constructor injection, helpers constructor-injected, ALIKED engine load at `create` time, `RollingHealthWindow.update` called exactly once per `match`.
|
||||
- **ALIKED-specific preprocessing parameters are hard-coded** — weights-coupled (same rule as DISK and UltraVPR); making them config-knobs would let an operator silently break the AC-1.1 inlier floor.
|
||||
- **Both DISK and ALIKED engines may be linked into the same binary** — ADR-002 allows multiple backbones at link time; only `config.matcher.strategy` selects which is instantiated. NOT mutually exclusive at build time (operator-tooling excludes both via `BUILD_MATCHER_*` flags OFF).
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: ALIKED upstream code drop preprocessing differs from DISK in non-obvious ways**
|
||||
- *Mitigation*: ALIKED preprocessor lives next to the strategy with hard-coded parameters; tests assert the preprocessor matches the upstream-published values; engine compile (AZ-321) consumes the same parameters.
|
||||
|
||||
**Risk 2: ALIKED's keypoint count distribution differs from DISK** (e.g., ALIKED returns more or fewer keypoints by default)
|
||||
- *Mitigation*: LightGlue and RANSAC are agnostic to keypoint count distribution; the median residual + inlier count metrics are normalised. C3-IT-01 (deferred) measures this empirically.
|
||||
|
||||
**Risk 3: Switching from DISK to ALIKED at runtime requires a corpus rebuild**
|
||||
- *Mitigation*: NO. C2's descriptor index (built by C10) is for VPR retrieval, not for cross-domain matching. C3 operates per-frame on raw tile pixels; switching matcher backbones does not require corpus rebuild. Documented in description.md § 8 (independent paths).
|
||||
|
||||
## Runtime Completeness
|
||||
|
||||
- **Named capability**: `AlikedLightGlueMatcher` — secondary `CrossDomainMatcher` (architecture / E-C3 / `solution.md` / AC-1.1 partition).
|
||||
- **Production code that must exist**: real `AlikedLightGlueMatcher` calling real C7 `InferenceRuntime` with real TRT-compiled ALIKED engine; same shared `LightGlueRuntime` + `RansacFilter` + `RollingHealthWindow` invocation pattern as AZ-345.
|
||||
- **Allowed external stubs**: same as AZ-345 — `FakeInferenceRuntime`, `FakeLightGlueRuntime`, `FakeRansacFilter`, `FakeFdrClient`.
|
||||
- **Unacceptable substitutes**: same as AZ-345 — Python+NumPy ALIKED forward; per-strategy RANSAC; skipping `RollingHealthWindow.update` on all-failed path; using mean residual instead of median.
|
||||
@@ -1,140 +0,0 @@
|
||||
# C3 XFeat Alternate Lightweight Matcher
|
||||
|
||||
**Task**: AZ-347_c3_xfeat
|
||||
**Name**: C3 XFeat Alternate Lightweight Matcher
|
||||
**Description**: Implement `XFeatMatcher`, the lightweight alternate `CrossDomainMatcher`. XFeat combines feature extraction AND matching in a single forward pass (no separate LightGlue stage); selectable via `config.matcher.strategy = "xfeat"`. Target use case: low-power / thermal-throttled scenarios where DISK+LightGlue's combined cost (~180 ms p95) exceeds the C4 hybrid's degraded budget. Drop-and-continue + below-threshold + best-candidate selection contracts inherited from the Protocol unchanged. RANSAC + median residual still computed via the shared `RansacFilter`.
|
||||
**Complexity**: 3 points
|
||||
**Dependencies**: AZ-344 (Protocol + factory + DTOs + errors + RollingHealthWindow), AZ-263_initial_structure, AZ-269_config_loader, AZ-282_ransac_filter, AZ-298_c7_tensorrt_runtime, AZ-299_c7_onnxrt_fallback, AZ-303_c6_storage_interfaces, AZ-281_engine_filename_schema (XFeat engine self-describing filename), AZ-321_c10_engine_compiler (XFeat engine compile path), AZ-266_log_module, AZ-272_fdr_record_schema
|
||||
**Component**: c3_matcher (epic AZ-257 / E-C3)
|
||||
**Tracker**: AZ-347
|
||||
**Epic**: AZ-257 (E-C3)
|
||||
|
||||
### Document Dependencies
|
||||
|
||||
- `_docs/02_document/contracts/c3_matcher/cross_domain_matcher_protocol.md` — Protocol contract.
|
||||
- `_docs/02_document/components/04_c3_matcher/description.md` — § 1 XFeat alternate (lightweight); § 5 error handling; § 9 logging.
|
||||
- `_docs/02_document/module-layout.md` — `BUILD_MATCHER_XFEAT` row.
|
||||
- `_docs/02_document/contracts/shared_helpers/ransac_filter.md` — RANSAC filtering API.
|
||||
- `_docs/02_document/contracts/c2_5_rerank/rerank_strategy_protocol.md`.
|
||||
- `_docs/02_document/contracts/c7_inference/inference_runtime_protocol.md`.
|
||||
|
||||
## Problem
|
||||
|
||||
Without this task: there is no lightweight matcher option for thermal-throttled scenarios; if the C4 hybrid switches to Jacobian (per ADR-006 / D-CROSS-LATENCY-1) but C3's per-frame budget still allows the heavy DISK+LightGlue path, the system has no mechanism to reduce C3's cost too. XFeat is also the documented mandatory simple-baseline alternative for IT-12 comparative study (AC-2.1a engine rule applied at the matcher level, with NetVLAD acting at the VPR level).
|
||||
|
||||
## Outcome
|
||||
|
||||
- `src/gps_denied_onboard/components/c3_matcher/xfeat.py` defining:
|
||||
- `XFeatMatcher` class implementing the `CrossDomainMatcher` Protocol.
|
||||
- Constructor: `__init__(self, runtime: InferenceRuntime, ransac_filter: RansacFilter, fdr_client: FdrClient, health_window: RollingHealthWindow, config: MatcherConfig)`. Note: NO `lightglue_runtime` argument — XFeat does not use LightGlue.
|
||||
- `match(frame, rerank_result, calibration)`:
|
||||
1. Decode + preprocess the nav-camera frame ONCE.
|
||||
2. For each `RerankCandidate` in `rerank_result.candidates`:
|
||||
a. Decode + preprocess the candidate tile.
|
||||
b. Run XFeat forward (single pass: outputs combined `correspondences` directly — XFeat fuses extraction + matching).
|
||||
c. On failure: drop-and-continue (`MatcherBackboneError`, `phase="xfeat_forward"`).
|
||||
d. RANSAC + median residual via `ransac_filter.filter(correspondences, threshold_px=...)` — same helper as DISK+LightGlue.
|
||||
e. Append `CandidateMatchSet` if survivors > 0.
|
||||
3. Below-threshold / all-failed → `InsufficientInliersError` (same semantics as AZ-345).
|
||||
4. Sort survivors descending by `inlier_count`; ties broken by `per_candidate_residual_px` ascending.
|
||||
5. WARN on residual above threshold; INFO on ready; FDR `matcher.frame_done` per frame.
|
||||
6. `RollingHealthWindow.update` after each frame (success or failure).
|
||||
7. `matcher_label = "xfeat"`.
|
||||
- Module-level `create(config, lightglue_runtime, ransac_filter, inference_runtime, health_window) -> CrossDomainMatcher`:
|
||||
1. `lightglue_runtime` is accepted in the signature for factory uniformity but NOT stored / used.
|
||||
2. `xfeat_weights_path = config.matcher.xfeat_weights_path` (TRT engine produced by AZ-321).
|
||||
3. Load XFeat engine via `inference_runtime.load_engine(...)`.
|
||||
4. Construct `XFeatMatcher(...)`.
|
||||
- Composition-root wiring path for `config.matcher.strategy == "xfeat"`.
|
||||
- `BUILD_MATCHER_XFEAT` flag wiring (ON in research; ON in airborne if config selects it; OFF in operator-tooling).
|
||||
- All logging + FDR records identical structure to AZ-345 with `matcher_label = "xfeat"`.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
- `XFeatMatcher` implementation per the `CrossDomainMatcher` Protocol.
|
||||
- XFeat forward via C7 `InferenceRuntime`.
|
||||
- RANSAC + median residual via shared `RansacFilter` (NO LightGlue).
|
||||
- Same drop-and-continue + below-threshold + best-candidate selection as AZ-345.
|
||||
- Same `RollingHealthWindow.update` invocation pattern.
|
||||
- Composition-root wiring path.
|
||||
- XFeat-specific preprocessor inline.
|
||||
- Unit tests covering Invariants 1–9 + drop-and-continue + below-threshold + deterministic ordering. Parametrised across XFeat-specific test fixtures (lightweight model output is different shape from DISK).
|
||||
- `BUILD_MATCHER_XFEAT` flag wiring.
|
||||
|
||||
### Excluded
|
||||
- The Protocol + DTOs + errors + factory + `RollingHealthWindow` — owned by AZ-344.
|
||||
- `RansacFilter` (AZ-282).
|
||||
- `LightGlueRuntime` (AZ-278) — XFeat does NOT consume this helper; the factory's signature includes it for uniformity but XFeat's `create` ignores the parameter.
|
||||
- C7 runtime stack (AZ-297..AZ-300).
|
||||
- XFeat engine compile (AZ-321).
|
||||
- Component-internal acceptance tests beyond Protocol + invariants smoke.
|
||||
- DISK matcher (AZ-345) and ALIKED matcher (AZ-346).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1 through AC-10**: identical contract to AZ-345 AC-1..AC-10 (Protocol conformance, best-candidate selection, drop-and-continue, below-threshold, residual WARN, health update, correspondences shape, determinism). `matcher_label = "xfeat"`.
|
||||
|
||||
**AC-11: Composition-root wiring**
|
||||
Given `config.matcher.strategy = "xfeat"` AND `BUILD_MATCHER_XFEAT = ON`
|
||||
When `compose_root(config)` runs
|
||||
Then an `XFeatMatcher` instance is wired; ONE INFO log `kind="c3.matcher.ready"` with `{strategy: "xfeat", ...}` is emitted. The strategy does NOT hold a reference to `LightGlueRuntime` (verifiable via `not hasattr(strategy, "_lightglue_runtime")` OR `strategy._lightglue_runtime is None`).
|
||||
|
||||
**AC-12: FDR `matcher.frame_done` per frame**
|
||||
Same shape as AZ-345 AC-12 with `matcher_label = "xfeat"`.
|
||||
|
||||
**AC-special-1: XFeat single-pass forward — no LightGlue call**
|
||||
Given a `match(...)` call where the `LightGlueRuntime` test double is provided to the factory
|
||||
When the call completes
|
||||
Then `lightglue_runtime.match_*` is NEVER invoked (verified by mock assertion `lightglue_runtime.match_pair.assert_not_called()`).
|
||||
|
||||
**AC-special-2: XFeat lower latency than DISK+LightGlue (informational, not gated)**
|
||||
Given identical hardware and identical inputs
|
||||
When `match(...)` is microbenchmarked × 100 frames
|
||||
Then XFeat's per-call p95 is < AZ-345's per-call p95 (informational metric; if XFeat is NOT faster, that's a backbone misconfiguration, not a contract violation. Documented in the test report; does NOT block this AC).
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance** (deferred to C3-PT-01):
|
||||
- `match` p95 ≤ 100 ms (informational target; XFeat is the lightweight option). NOT a hard gate; the hard gate is C3-PT-01's overall envelope.
|
||||
- GPU memory ≤ 300 MB (XFeat single engine; smaller than DISK+LightGlue).
|
||||
|
||||
**Compatibility**
|
||||
- XFeat engine file format owned by C10 + C7.
|
||||
|
||||
**Reliability**
|
||||
- Same as AZ-345.
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| AC Ref | What to Test | Required Outcome |
|
||||
|--------|--------------|------------------|
|
||||
| AC-1..AC-10 | Identical to AZ-345 AC-1..AC-10 with `matcher_label = "xfeat"` | Same outcomes |
|
||||
| AC-11 | `compose_root(config="xfeat")` | Wired; INFO log; no LightGlue dependency |
|
||||
| AC-12 | FDR `frame_done` emission | Correct fields; `matcher_label = "xfeat"` |
|
||||
| AC-special-1 | LightGlue NOT invoked | `lightglue_runtime.match_pair.assert_not_called()` |
|
||||
| AC-special-2 | Latency comparison | (Informational; not gated) |
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Same constraints as AZ-345** — drop-and-continue mandatory, median residual, constructor injection, helpers constructor-injected, engine load at `create` time, `RollingHealthWindow.update` called exactly once per `match`.
|
||||
- **`LightGlueRuntime` is NOT consumed** — the factory's `create` signature accepts it for uniformity (so AZ-344's factory can call all three matchers' `create` with the same args) but XFeatMatcher does NOT store or use it. Test AC-special-1 enforces this.
|
||||
- **XFeat-specific preprocessing parameters are hard-coded** (weights-coupled, same rule as DISK and ALIKED).
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: XFeat output schema differs from DISK+LightGlue output (correspondences format)**
|
||||
- *Mitigation*: XFeat outputs `correspondences` ndarray of shape `(M, 4)` with columns `(px_query, py_query, px_tile, py_tile)` — same as the post-LightGlue output of DISK+LightGlue. The shared `RansacFilter` consumes this format identically. If XFeat's upstream output differs, this task adapts inside the strategy.
|
||||
|
||||
**Risk 2: XFeat's RANSAC inlier counts may be systematically lower** (lighter-weight model produces noisier matches)
|
||||
- *Mitigation*: AC-2.1a engine rule applies (XFeat is the simple baseline at the matcher level); the ≥ 80 inlier count floor (AC-1.1) may not hold for XFeat. C3-IT-01 measures this; if XFeat fails AC-1.1 on Derkachi, it remains as the "engine rule" comparison baseline NOT the production-default — same engine-rule semantics as NetVLAD at C2.
|
||||
|
||||
**Risk 3: Linking three backbones into one binary inflates GPU memory headroom**
|
||||
- *Mitigation*: per ADR-002 / D-C7-13, only the SELECTED backbone's engine is loaded at `create` time. Linking does NOT load engines; loading happens lazily in each backbone's `create`. The factory only invokes ONE `create` per binary lifetime.
|
||||
|
||||
## Runtime Completeness
|
||||
|
||||
- **Named capability**: `XFeatMatcher` — alternate lightweight `CrossDomainMatcher` (architecture / E-C3 / `solution.md` / AC-2.1a engine rule at matcher level).
|
||||
- **Production code that must exist**: real `XFeatMatcher` calling real C7 `InferenceRuntime` with real TRT-compiled XFeat engine; real shared `RansacFilter` for inlier filtering + median residual; real `RollingHealthWindow.update` after each frame; real composition-root wiring.
|
||||
- **Allowed external stubs**: `FakeInferenceRuntime`, `FakeRansacFilter`, `FakeFdrClient`, `FakeLightGlueRuntime` (passed but unused).
|
||||
- **Unacceptable substitutes**: a Python+NumPy XFeat forward (would not satisfy the lightweight-target latency); using a different RANSAC implementation; storing/calling `LightGlueRuntime` (would defeat XFeat's single-pass design).
|
||||
@@ -1,209 +0,0 @@
|
||||
# C3.5 AdHoPRefiner — TRT engine + perspective preconditioning + conditional gate
|
||||
|
||||
**Task**: AZ-349_c3_5_adhop_refiner
|
||||
**Name**: C3.5 `AdHoPRefiner` — production-default conditional refiner
|
||||
**Description**: Implement `AdHoPRefiner`, the production-default `ConditionalRefiner` strategy. The strategy applies the conditional gate (`mr.reprojection_residual_px <= residual_threshold_px` → passthrough; otherwise → invoke), runs the OrthoLoC AdHoP TRT engine forward via the C7 `InferenceRuntime` to perform perspective preconditioning, recomputes inlier correspondences via the shared `RansacFilter` (AZ-282) on the preconditioned features, and writes the refined correspondences + new median residual back into a fresh `MatchResult` with `refinement_label = "adhop"`. On `RefinerBackboneError` (TRT exception, OOM, NaN, shape mismatch), the strategy CATCHES the exception inside `refine_if_needed`, logs ERROR + emits FDR record, and returns the input `MatchResult` unchanged with `refinement_label = "passthrough"` AND `was_invoked()` = True (Invariant 4 — passthrough fall-through). The strategy ALSO maintains a 60 s rolling invocation-rate counter for the WARN log when the rate exceeds `config.refiner.invocation_rate_warn_threshold` (per description.md § 9). Composition-root wired via the `build_refiner_strategy` factory (TBD AZ-? from the C3.5 Protocol task) when `config.refiner.strategy = "adhop"`.
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-348 (Protocol + factory + DTOs + errors + `PassthroughRefiner`), AZ-263_initial_structure, AZ-269_config_loader, AZ-282_ransac_filter (shared RANSAC helper), AZ-298_c7_tensorrt_runtime (AdHoP forward via TRT), AZ-299_c7_onnxrt_fallback (AdHoP forward via ONNX-RT fallback), AZ-281_engine_filename_schema (AdHoP engine self-describing filename), AZ-321_c10_engine_compiler (AdHoP engine compile path), AZ-266_log_module, AZ-272_fdr_record_schema
|
||||
**Component**: c3_5_adhop (epic AZ-258 / E-C3.5)
|
||||
**Tracker**: AZ-349
|
||||
**Epic**: AZ-258 (E-C3.5)
|
||||
|
||||
### Document Dependencies
|
||||
|
||||
- `_docs/02_document/contracts/c3_5_adhop/conditional_refiner_protocol.md` — the public contract this task implements; producer/consumer split assigns this task as the AdHoPRefiner consumer.
|
||||
- `_docs/02_document/components/05_c3_5_adhop/description.md` — § 1 architectural pattern; § 2 `ConditionalRefiner` interface + DTO enrichments (`refinement_label`, `refinement_added_latency_ms`); § 5 error handling (passthrough fall-through on `RefinerBackboneError`); § 7 caveats (threshold tuning); § 9 logging (per-frame DEBUG, rolling-rate WARN at 0.25, ERROR on backbone failure).
|
||||
- `_docs/02_document/components/05_c3_5_adhop/tests.md` — C3.5-IT-01 (residual reduction ≥ 90% of invocations); C3.5-IT-02 (passthrough fall-through bit-identical); C3.5-IT-03 (invocation rate < 0.30 on Derkachi normal); C3.5-PT-01 (latency budget).
|
||||
- `_docs/02_document/contracts/shared_helpers/ransac_filter.md` — RANSAC filtering API.
|
||||
- `_docs/02_document/contracts/c7_inference/inference_runtime_protocol.md` — `InferenceRuntime` API.
|
||||
- `_docs/02_document/architecture.md` — ADR-001, R10 (latency under thermal throttle).
|
||||
|
||||
## Problem
|
||||
|
||||
Without this task, C3.5 has no real refinement path; the system would always run the `PassthroughRefiner`, defeating the AC-2.2 hard-frame portion (cross-domain MRE < 2.5 px after refinement). The conditional gate also keeps AC-4.1's E2E latency budget intact on the steady-state path (passthrough costs ~0.5 ms while the AdHoP invocation costs ~30–90 ms; running it every frame would blow the F3 budget). Without proper passthrough fall-through on `RefinerBackboneError`, a single TRT failure would cascade into a frame skip → C5 visual_propagated → false health degradation → false F6 satellite re-localisation trigger.
|
||||
|
||||
## Outcome
|
||||
|
||||
- `src/gps_denied_onboard/components/c3_5_adhop/adhop_refiner.py` defining:
|
||||
- `AdHoPRefiner` class implementing the `ConditionalRefiner` Protocol.
|
||||
- Constructor: `__init__(self, runtime: InferenceRuntime, ransac_filter: RansacFilter, fdr_client: FdrClient, config: RefinerConfig, adhop_engine_handle)`. The `adhop_engine_handle` is the loaded TRT engine returned by `runtime.load_engine(...)` at `create` time.
|
||||
- `_was_invoked: bool` (private flag; reset to False at every call).
|
||||
- `_invocation_window`: a rolling-60s counter of (timestamp_ns, was_invoked) tuples for the rate WARN log. Bounded to ~180 entries at 3 Hz; pruned by lazy expiry.
|
||||
- `refine_if_needed(frame, mr, residual_threshold_px) -> MatchResult`:
|
||||
1. `start_ns = time.monotonic_ns()`.
|
||||
2. Defensive: `if residual_threshold_px <= 0: raise ValueError(...)`.
|
||||
3. **Gate**: `if mr.reprojection_residual_px <= residual_threshold_px:`
|
||||
- `self._was_invoked = False`.
|
||||
- Append `(now, False)` to `_invocation_window`; prune.
|
||||
- Return `mr` unchanged (object reference; refinement_label stays at default `"passthrough"`).
|
||||
4. **Invoked path**: `self._was_invoked = True`. Append `(now, True)` to `_invocation_window`; prune.
|
||||
5. Compute current invocation rate over the 60s window. If rate > `config.refiner.invocation_rate_warn_threshold` (default 0.25), emit ONE WARN log per minute (rate-limited): `kind="c3_5.refiner.invocation_rate_high"` with `{rate, target_threshold, frame_id}`.
|
||||
6. **Try**:
|
||||
a. Decode + preprocess the nav-camera frame ONCE.
|
||||
b. For the BEST candidate only (`mr.per_candidate[mr.best_candidate_idx]`): decode + preprocess the candidate tile pixels via the existing `tile_pixels_handle`.
|
||||
c. Run AdHoP TRT engine forward (single forward; output: perspective-preconditioned correspondences). Implementation detail: AdHoP takes the original correspondences AS INPUT and produces refined correspondences with the perspective preconditioning applied; this is method-agnostic per OrthoLoC.
|
||||
d. RANSAC + median residual via `self._ransac_filter.filter(refined_correspondences, threshold_px=...)` — same helper as C3.
|
||||
e. Build a NEW `CandidateMatchSet` for the best candidate with the refined `inlier_correspondences`, refined `inlier_count`, refined `per_candidate_residual_px`. Other candidates in `per_candidate` left unchanged.
|
||||
f. Build a NEW `MatchResult` via `dataclasses.replace(mr, per_candidate=[...new best, *unchanged others], reprojection_residual_px=new_best_residual, refinement_label="adhop", refinement_added_latency_ms=elapsed_ms)`.
|
||||
g. INFO log `kind="c3_5.refiner.frame_done"` (DEBUG-level per description.md § 9; promote to INFO only on the SUCCESS-after-many-failures recovery transition; details in implementation).
|
||||
h. FDR `refiner.frame_done` record with `{frame_id, was_invoked: true, refinement_label: "adhop", refinement_added_latency_ms, pre_residual_px, post_residual_px, inlier_count_before, inlier_count_after}`.
|
||||
i. Return the new `MatchResult`.
|
||||
7. **Except `RefinerBackboneError`** (or any TRT-runtime failure that the `InferenceRuntime` raises and a guard layer maps to `RefinerBackboneError` per ADR-001 error contract):
|
||||
- ERROR log `kind="c3_5.refiner.backbone_error"` with `{frame_id, exc_type, phase}`.
|
||||
- FDR `refiner.frame_done` record with `{frame_id, was_invoked: true, refinement_label: "passthrough", refinement_added_latency_ms: elapsed_ms_so_far, error: true}`.
|
||||
- Return the input `mr` unchanged (refinement_label stays at default `"passthrough"`).
|
||||
- **Critical**: the exception is NEVER re-raised out of `refine_if_needed` (Invariant 4). Other exception types (e.g., `MemoryError`) ARE re-raised because the runtime contract is that only the documented `RefinerBackboneError` class is convertible to passthrough.
|
||||
- `was_invoked() -> bool`: return `self._was_invoked`.
|
||||
- Module-level `create(config, ransac_filter, inference_runtime) -> ConditionalRefiner`:
|
||||
1. `adhop_weights_path = config.refiner.adhop_weights_path` (TRT engine produced by AZ-321).
|
||||
2. Load AdHoP engine via `inference_runtime.load_engine(adhop_weights_path)` — happens ONCE at startup.
|
||||
3. Construct `AdHoPRefiner(runtime=inference_runtime, ransac_filter=ransac_filter, fdr_client=..., config=config.refiner, adhop_engine_handle=adhop_engine_handle)`.
|
||||
- Composition-root wiring path: when `config.refiner.strategy == "adhop"` AND the AdHoP engine compile artifact is present, the AZ-? factory invokes `adhop_refiner.create(...)`.
|
||||
- All FDR + log records have `refinement_label` set per the post-refinement outcome.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
- `AdHoPRefiner` implementation per the `ConditionalRefiner` Protocol.
|
||||
- Conditional gate (`<=` semantics, inclusive, deterministic).
|
||||
- AdHoP TRT engine forward via C7 `InferenceRuntime`.
|
||||
- RANSAC + median residual recomputation via the shared `RansacFilter`.
|
||||
- Passthrough fall-through on `RefinerBackboneError` (Invariant 4).
|
||||
- Bit-identical correspondence preservation when the gate decides passthrough (Invariant 5).
|
||||
- 60 s rolling invocation-rate counter + WARN log emission (rate-limited to ONE warning per minute).
|
||||
- Per-frame FDR record emission with full provenance fields.
|
||||
- Composition-root wiring path.
|
||||
- Unit tests covering Invariants 1–9 + gate semantics + passthrough fall-through + invocation-rate accounting.
|
||||
|
||||
### Excluded
|
||||
- The Protocol + DTO extension + errors + factory + `PassthroughRefiner` — owned by AZ-? (Protocol task).
|
||||
- The `RansacFilter` helper — already AZ-282.
|
||||
- The C7 `InferenceRuntime` — owned by AZ-297..AZ-300.
|
||||
- The AdHoP TRT engine compile path — owned by AZ-321.
|
||||
- C3.5-IT-01..03 + C3.5-PT-01 component-internal acceptance tests — deferred to E-BBT (AZ-262). Unit tests in this task cover the per-method invariant smoke.
|
||||
- The `OrthoLoC` upstream code drop — vendored separately (Plan-phase pin); this task consumes the runtime-loadable AdHoP engine only.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: Protocol conformance**
|
||||
`AdHoPRefiner` instance passes `isinstance(refiner, ConditionalRefiner)`.
|
||||
|
||||
**AC-2: Gate inclusive semantics (Invariant 3)**
|
||||
Given `mr.reprojection_residual_px == residual_threshold_px` (equality)
|
||||
When `refine_if_needed` is called
|
||||
Then the strategy returns `mr` unchanged AND `was_invoked()` is False.
|
||||
And: `mr.reprojection_residual_px = residual_threshold_px + 1e-6` triggers the invoked path AND `was_invoked()` is True.
|
||||
|
||||
**AC-3: Successful AdHoP refinement produces enriched `MatchResult`**
|
||||
Given a `MatchResult` with `reprojection_residual_px = 5.0`, `residual_threshold_px = 2.5`, and a stub AdHoP engine that produces refined correspondences with median residual `1.2`
|
||||
When `refine_if_needed` is called
|
||||
Then the output `MatchResult` has:
|
||||
- `refinement_label == "adhop"`
|
||||
- `reprojection_residual_px ≈ 1.2`
|
||||
- `refinement_added_latency_ms > 0`
|
||||
- The best candidate's `inlier_correspondences` reflects the refined coordinates (NOT byte-identical to input).
|
||||
- `was_invoked()` returns True.
|
||||
|
||||
**AC-4: Passthrough fall-through on `RefinerBackboneError` (Invariant 4)**
|
||||
Given a stub AdHoP engine that raises `RefinerBackboneError` on forward
|
||||
When `refine_if_needed` is called with a residual above threshold
|
||||
Then:
|
||||
- The output `MatchResult` IS the input `mr` (object reference).
|
||||
- `refinement_label == "passthrough"` (default value preserved).
|
||||
- `was_invoked()` returns True (the attempt counted).
|
||||
- ERROR log `kind="c3_5.refiner.backbone_error"` emitted ONCE.
|
||||
- FDR `refiner.frame_done` record emitted with `error: true`.
|
||||
- The exception is NEVER re-raised (test asserts no exception escapes).
|
||||
|
||||
**AC-5: Other exception types DO re-raise**
|
||||
Given a stub AdHoP engine that raises `MemoryError` (NOT `RefinerBackboneError`)
|
||||
When `refine_if_needed` is called
|
||||
Then `MemoryError` propagates out (not converted to passthrough). Documents the closed-set semantics of Invariant 4.
|
||||
|
||||
**AC-6: Bit-identical correspondences on gate-decided passthrough (Invariant 5)**
|
||||
Given `mr.reprojection_residual_px = 1.0` AND `residual_threshold_px = 2.5` (gate → passthrough)
|
||||
When `refine_if_needed` is called
|
||||
Then for every candidate `i`, `out.per_candidate[i].inlier_correspondences IS mr.per_candidate[i].inlier_correspondences` (same object reference). Output's `refinement_label` stays at default `"passthrough"`.
|
||||
|
||||
**AC-7: `_invocation_window` accuracy**
|
||||
Given a sequence of 30 frames at 3 Hz with 10 invoked + 20 gate-passthroughs
|
||||
When `refine_if_needed` has been called for each
|
||||
Then the strategy's internal rate calculation reports `10/30 == 0.333` over the 10s window.
|
||||
|
||||
**AC-8: Invocation-rate WARN is rate-limited**
|
||||
Given the invocation rate exceeds `invocation_rate_warn_threshold = 0.25` for 60 consecutive seconds
|
||||
When the strategy is exercised over that period
|
||||
Then ONE (and only ONE) WARN log per 60 s window is emitted (rate-limited).
|
||||
|
||||
**AC-9: `was_invoked()` semantics matches Invariant 8**
|
||||
- Gate-decided passthrough → False.
|
||||
- AdHoP-success → True.
|
||||
- AdHoP-fall-through (backbone error) → True.
|
||||
|
||||
**AC-10: Composition-root wiring**
|
||||
Given `config.refiner.strategy = "adhop"` AND the AdHoP engine artifact path is valid
|
||||
When `compose_root(config)` runs
|
||||
Then an `AdHoPRefiner` instance is wired; ONE INFO log `kind="c3_5.refiner.ready"` with `{strategy: "adhop", residual_threshold_px}` is emitted; the strategy holds reference to the SAME `RansacFilter` instance as C3 + C4 (identity-shared).
|
||||
|
||||
**AC-11: FDR `refiner.frame_done` shape**
|
||||
Every `refine_if_needed` call (regardless of gate decision) emits exactly ONE FDR record with the documented field set.
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- `refine_if_needed` p95 (gate-passthrough) ≤ 1 ms (per C3.5-PT-01 with margin).
|
||||
- `refine_if_needed` p95 (AdHoP-invoked) ≤ 90 ms target / 150 ms hard limit (per C3.5-PT-01).
|
||||
- `_invocation_window` update p99 ≤ 5 µs (deque-based or ring-buffer; pruning is amortised O(1)).
|
||||
|
||||
**Compatibility**
|
||||
- AdHoP engine file format owned by C10 + C7.
|
||||
|
||||
**Reliability**
|
||||
- Single-thread by contract (Invariant 1); no internal locking.
|
||||
- TRT errors NEVER cascade out of `refine_if_needed` (Invariant 4).
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| AC Ref | What to Test | Required Outcome |
|
||||
|--------|--------------|------------------|
|
||||
| AC-1 | Protocol conformance | Passes `isinstance` |
|
||||
| AC-2 | Gate `<=` inclusive | equality → passthrough; +ε → invoked |
|
||||
| AC-3 | AdHoP success path | `refinement_label="adhop"`; refined residual; latency > 0 |
|
||||
| AC-4 | Backbone error → passthrough fall-through | Same-reference output; ERROR log; no escape |
|
||||
| AC-5 | Other exceptions re-raise | `MemoryError` propagates |
|
||||
| AC-6 | Gate passthrough byte-identical | Same object reference |
|
||||
| AC-7 | Invocation rate accuracy | `10/30 == 0.333` |
|
||||
| AC-8 | WARN log rate-limited | One per 60 s |
|
||||
| AC-9 | `was_invoked()` semantics | Three cases match |
|
||||
| AC-10 | Composition wiring | Ready log; identity-shared `RansacFilter` |
|
||||
| AC-11 | FDR record shape | Exactly one per call; documented fields |
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Single-threaded** by contract; the `_invocation_window` is non-thread-safe by design.
|
||||
- **AdHoP backbone errors NEVER propagate out** of `refine_if_needed` (Invariant 4). Other exception types DO propagate (AC-5 closed-set semantics).
|
||||
- **Gate uses `<=` not `<`** — equality at the threshold means passthrough (deterministic, documented).
|
||||
- **Engine load is ONCE at `create` time** — never lazy on first frame; ensures the F1 takeoff cold-start cost is bounded and deterministic.
|
||||
- **`RansacFilter` is constructor-injected, identity-shared** with C3 + C4 — composition root constructs ONE instance.
|
||||
- **WARN log is rate-limited to ONE per 60 s** — avoids log flooding when the threshold is mis-tuned.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: AdHoP TRT engine OOM on Jetson under thermal-throttle conditions**
|
||||
- *Mitigation*: passthrough fall-through (Invariant 4) on `RefinerBackboneError`; downstream pose estimator handles the same `MatchResult` shape regardless of refinement outcome. The repeated-OOM scenario is detectable via the `_invocation_window` "almost-always-error" pattern → operator-tooling pre-flight raises the residual threshold per R10.
|
||||
|
||||
**Risk 2: AdHoP refinement produces a result with HIGHER residual than the input** (degenerate frames)
|
||||
- *Mitigation*: per the contract, the strategy returns whatever AdHoP produces; downstream pose estimator gates on the new residual. C3.5-IT-01 measures the improvement rate (≥ 90%); the remaining ≤10% are accepted (documented).
|
||||
|
||||
**Risk 3: `_invocation_window` deque grows unbounded if `refine_if_needed` is called from the wrong thread / without pruning**
|
||||
- *Mitigation*: lazy prune at every append; bounded by ~180 entries at 3 Hz × 60 s. Memory upper bound is ~6 KB. Single-thread invariant covers the racing concern.
|
||||
|
||||
**Risk 4: AdHoP weights / engine file path missing at startup**
|
||||
- *Mitigation*: `create(...)` raises `RefinerConfigError` (caught at composition root) before the strategy is wired in. F1 takeoff abort follows the existing error handling pattern.
|
||||
|
||||
## Runtime Completeness
|
||||
|
||||
- **Named capability**: `AdHoPRefiner` — production-default `ConditionalRefiner` (architecture / E-C3.5 / `solution.md` / R10).
|
||||
- **Production code that must exist**: real `AdHoPRefiner` calling real C7 `InferenceRuntime` with real TRT-compiled AdHoP engine; real shared `RansacFilter` for inlier filtering + median residual; real conditional gate at the documented `<=` semantics; real passthrough fall-through on `RefinerBackboneError`; real 60 s rolling invocation-rate counter + rate-limited WARN log; real per-frame FDR record emission; real composition-root wiring.
|
||||
- **Allowed external stubs**: `FakeInferenceRuntime`, `FakeRansacFilter`, `FakeFdrClient` for tests.
|
||||
- **Unacceptable substitutes**: a Python+NumPy AdHoP forward (would not satisfy the latency budget); using a different RANSAC implementation; allowing `RefinerBackboneError` to propagate out (Invariant 4 violation); deferring the `_invocation_window` to a future task (C3.5-IT-03 fails without it); using `<` instead of `<=` for the gate (would create a deterministic-replay divergence on equality-ish frames).
|
||||
Reference in New Issue
Block a user