azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-22 16:01:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

[autodev] Step 12 cycle-1 sync: tests/resilience+traceability

Browse Source 
Backfill the uncommitted Step 12 (Test-Spec Sync) output for the
resilience-tests and traceability-matrix surfaces; these were
produced by the test-spec skill in cycle-update mode but never
landed as a git commit before the flow moved to Step 13.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-19 16:49:26 +03:00
parent 1ca9a59b0b
commit bb9c408597
2 changed files with 41 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_document/tests/resilience-tests.md
+37
View File
@@ -106,3 +106,40 @@
- Recovery gate is honored — early recovery attempts (FC GPS healthy for <10 s) MUST NOT promote spoofed GPS back into the estimator.
**Duration**: ~10 min total for three windows.
---
### NFT-RES-05: Composition-root bootstrap contract (replay-mode minimal config, operator-error contract, Tier-2 log boundary)
**Summary**: Validates the airborne composition-root bootstrap contract introduced by AZ-591 / AZ-618 / AZ-687. Three sub-cases pin the cross-cycle plumbing that ALL Tier-1 and Tier-2 replay tests depend on:
(a) replay-mode `build_pre_constructed(config)` succeeds without a `c6_tile_cache` config block (AZ-687 AC-687-1);
(b) on a misconfigured airborne bootstrap, the SUT exits non-zero with stderr carrying the `airborne_bootstrap:` prefix, the consuming component slug, the missing infrastructure key or `BUILD_*` flag, and one actionable sentence (AZ-618 operator-facing-error-contract NFR + AC-4 non-success branch);
(c) every successful Tier-2 replay run logs both `replay.compose_root.ready` and `replay.input.frame_emitted` to stdout (AZ-618 AC-5, AZ-687 AC-687-3).
**Traces to**: AC-NEW-1 (precondition for cold-start TTFF), AC-4.1 (precondition for latency budget) — protects the assembly path every product-AC test relies on.
**Preconditions**:
- Tier-1 acceptable for sub-cases (a) and (b); sub-case (c) is Tier-2-only.
- For (a): replay CLI invocation with a synthesized `Config` that has `mode == "replay"` and `components` that does NOT include a `c6_tile_cache` block. Public surface: `scripts/run_replay.sh` (or equivalent CLI entrypoint) pointed at `_docs/00_problem/input_data/flight_derkachi/`.
- For (b): minimal live-mode `Config` selecting a component strategy whose required `BUILD_*` flag is set to OFF in the environment (e.g. `c2_vpr.strategy="net_vlad"` with `BUILD_PYTORCH_FP16_RUNTIME=OFF`). No SUT internals are imported; the test reads stderr.
- For (c): standard Tier-2 Jetson e2e replay invocation per `tests/e2e/replay/test_derkachi_1min.py` (AC-1, AC-2, AC-5, AC-6).
**Fault injection** (sub-case b only): set the gating `BUILD_*` env var to `OFF` for a strategy that the config selects, then start the SUT.
**Steps**:
| Sub-case | Step | Consumer Action | Expected Behavior |
|----------|------|-----------------|-------------------|
| (a) replay minimal | 1 | Start SUT in replay mode against `derkachi-fixture`, with no `c6_tile_cache` block in the synthesized config | SUT process reaches the replay coordinator without exiting; stdout contains `replay.compose_root.ready` within 60 s |
| (a) replay minimal | 2 | Observe at least one outbound frame emission | stdout contains at least one `replay.input.frame_emitted` log line within the same run |
| (b) misconfig | 1 | Start SUT in live mode with a config that selects a strategy whose `BUILD_*` flag is OFF | SUT exits with `EXIT_GENERIC_FAILURE` (`1`) |
| (b) misconfig | 2 | Read SUT stderr | stderr contains `airborne_bootstrap:` prefix AND the consuming component slug (e.g. `c2_vpr`) AND either the missing infrastructure key (e.g. `c7_inference`) or the gating `BUILD_*` flag name AND one actionable sentence (regex: `set \`BUILD_[A-Z0-9_]+\`` OR `ensure \`[a-z0-9_]+\.[a-z0-9_]+\` is`) |
| (c) Tier-2 log boundary | 1 | Run `scripts/run-tests-jetson.sh tests/e2e/replay/test_derkachi_1min.py` (or its compose-bypass equivalent) | Each of AC-1 / AC-2 / AC-5 / AC-6 produces stdout that crosses both `replay.compose_root.ready` AND `replay.input.frame_emitted` log boundaries |
**Pass criteria**:
- (a) Process does not exit during bootstrap; both boundary log lines appear within 60 s of process start.
- (b) Exit code is `1`; stderr matches the four-field contract (prefix + component slug + missing key/flag + actionable sentence).
- (c) Every AC sub-test in the Tier-2 invocation has both log lines in its captured stdout; batch report `Tier-2 evidence:` field references the terminal log path per `_docs/02_document/tests/tier2-jetson-testing.md`.
**Duration**: ~3 min (Tier-1 sub-cases a+b combined); sub-case (c) is observed inside the existing Tier-2 e2e replay run, no separate duration.
**Rationale**: This scenario formalizes the cross-cycle bootstrap invariant. The Tier-1 contract path bypasses `compose_root`'s registry-driven assembly via `replay_components_factory`, so without this test the failure modes AZ-687 caught (replay-mode `KeyError`) and AZ-618 caught (empty `pre_constructed`) regress silently — both were missed by the Tier-1 suite that was 3343/0 green at the moment of the Jetson rerun. See `_docs/02_document/tests/tier2-jetson-testing.md` § Rationale.