Oleksandr Bezdieniezhnykh
6e4a575221
Batch 88 — adds four resource-limit blackbox scenarios + pure-logic helpers + unit tests: - NFT-LIM-01 Jetson memory (AC-NEW-13): tier2_only; Plan A/B budgets; AC-4 OOM-event scan; 30 s warm-up window; VmRSS + tegrastats streams. - NFT-LIM-02 FDR size (AC-7.3): 30 min → 8 h linear extrapolation against 50 GiB; ±60 s replay-window slack for AC-1. - NFT-LIM-03+05 storage (AC-7.4 + AC-NEW-12 + RESTRICT-STORAGE): aggregate ≤ 100 GiB across tile-cache + tile-cache-write + fdr-output; thumbnail-log < 1 GiB strict 8 h-extrapolated. - NFT-LIM-04 thermal (AC-NEW-5 PARTIAL): tier2_only; CPU/SoC p99 ≤ T_throttle − 5 °C; throttle-event scan; PARTIAL annotation written to traceability-status.json. Thresholds fixture lives at e2e/fixtures/jetson/thermal-thresholds.json (moved from the task spec's suggested tests/fixtures/ path so the file stays inside the blackbox_tests Owns: e2e/** envelope). All four helpers are public-boundary-only (no src/gps_denied_onboard imports). Scenarios skip cleanly in the Tier-1 docker harness pending AZ-595 (SITL replay builder) for the four shared fixture inputs and AZ-444 (Tier-2 Jetson runner) for the tier2_only scenarios. Code review: PASS_WITH_WARNINGS (0/0/2/1). Both Mediums are carried-over write_csv_evidence + _resolve_fixture_path duplication, deferred to AZ-446 (batch 89). Low is the self-resolved AZ-443 fixture ownership drift documented in the review. Tests: 1223 e2e/_unit_tests passing (+1 vs. batch 87 from the new directory-layout entry); 24 resource_limit scenarios collect and skip cleanly under runner/pytest.ini. Co-authored-by: Cursor <cursoragent@cursor.com>
Blackbox Test Harness (e2e/)
This directory is the public-boundary test harness for gps-denied-onboard. It is owned by the blackbox_tests cross-cutting entry in _docs/02_document/module-layout.md and implements task AZ-406 (Test Infrastructure Bootstrap) plus its downstream test-task siblings (AZ-407..AZ-446).
The harness runs in two execution tiers (environment.md § Two-tier execution profile):
- Tier-1 — workstation Docker.
cd e2e/docker && docker compose -f docker-compose.test.yml up --build --abort-on-container-exit e2e-runner - Tier-2 — Jetson Orin Nano Super hardware loop.
./e2e/jetson/run-tier2.sh --fc-adapter <ardupilot|inav> --vio-strategy <okvis2|klt_ransac>
Both tiers emit the same CSV report format (one row per test) per environment.md § Reporting.
Layout
e2e/
├── docker/ Tier-1 entrypoint (docker-compose.test.yml + Tier-2 bridge override + secrets mount)
├── jetson/ Tier-2 entrypoint (run-tier2.sh + systemd unit + tegrastats/jtop parsers)
├── runner/ e2e-runner image (Dockerfile, conftest, pytest plugins, helpers, requirements)
├── fixtures/ Fixture builders (tile-cache, age-injector, injectors/, mock-suite-sat, secrets, security)
├── tests/ Pytest target — `positive/`, `negative/`, `performance/`, `resilience/`, `security/`, `resource_limit/`
└── _unit_tests/ Out-of-container unit tests for the harness internals (run as part of the project test suite)
Public-Boundary Discipline (hard rule)
The e2e-runner image MUST NOT import any module from the SUT source tree (src/gps_denied_onboard/**). The only legal interaction surfaces are:
- MAVLink (ArduPilot SITL — UDP 14550)
- MSP2 (iNav SITL — TCP 5760)
- HTTP/JSON (mock-suite-sat-service — port 8080)
- Filesystem read of the FDR archive after a run (
fdr-outputvolume)
This rule is enforced by:
- The runner
Dockerfilebuilding from a base image that does NOT install the SUT package. - Layout discipline: no
import gps_denied_onboard.*in any file undere2e/. - Compose
e2e-net.internal: true— no external network egress (RESTRICT-SAT-1, NFT-SEC-02).
See _docs/02_document/tests/environment.md for the full per-service spec.
RUN_ID and report paths
Each invocation must set RUN_ID (defaults to local-${USER}-${EPOCH} in development; CI sets it from the workflow run id). Reports land at:
e2e-results/run-${RUN_ID}/report.csve2e-results/run-${RUN_ID}/evidence/(per-run.tlog, FDR archives, screenshots, profiler traces, tegrastats CSV, jtop CSV)
The e2e-results/ directory is gitignored.
How to add a new blackbox scenario
- Decompose the scenario into a task spec under
_docs/02_tasks/todo/. - Implement the test under the appropriate
e2e/tests/<category>/folder. - The conftest's session-scoped
(fc_adapter, vio_strategy)parameterization automatically applies — opt out with@pytest.mark.parametrizeoverrides. - Trace the scenario to the AC/RESTRICT IDs it exercises via the
traces_topytest marker — the CSV reporter emits this verbatim.
How to add a new fixture builder
Fixture builders live under e2e/fixtures/ and may be standalone Python modules (for runtime injectors) or Dockerized helpers (for tile-cache / mock-suite-sat). Each builder must:
- Be reproducible — given the same input, produce bit-identical output.
- Document its output volume / path in
_docs/02_document/tests/test-data.md. - Have a corresponding unit test under
e2e/_unit_tests/fixtures/.
Out-of-container unit tests
The harness's internal Python — CSV reporter, helpers, parsers, mock app, conftest skip rules — is unit-tested under e2e/_unit_tests/. These tests do NOT require Docker, SITL, or any external service and run as part of the project's main pytest invocation (testpaths extension in pyproject.toml).