mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 08:41:12 +00:00
[AZ-700] gps-denied-render-map: HTML map of estimated vs truth tracks
New operator-side console-script renders a self-contained HTML map (folium / Leaflet) comparing the estimator's JSONL track against the tlog ground-truth track. Pinned visual style: red truth + blue estimated polylines, start/end markers per track, 100 m + 50 m scale circles, optional AZ-699 accuracy-summary banner, and an --offline-tiles mode (with optional local tile-URL template) for Jetsons without internet. folium is gated behind a new [operator-tools] optional-dep so the airborne binary's cold-start NFR is unaffected (C12 binary doesn't import the new module). 14 new unit tests pin polyline count, marker count, scale-circle radii, summary embedding, offline-tile behaviour, and full CLI smoke. Zero mypy --strict errors. Refines the 2026-05-20 Jetson-only test policy: unit tests may run locally, e2e/perf/resilience/security stay Jetson-only. Documented in _docs/02_document/tests/environment.md (Where each tier runs) and .cursor/rules/testing.mdc (Test environment for this project). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -1,17 +1,40 @@
|
||||
# Test Environment
|
||||
|
||||
> **Active policy — 2026-05-20**: **all tests run on Jetson only.** The Jetson
|
||||
> Orin Nano Super (or a Jetson-equivalent arm64 agent) is the single canonical
|
||||
> test environment for every tier of testing — unit, integration, blackbox /
|
||||
> e2e, performance, resilience, security, resource-limit. Workstation x86
|
||||
> Docker (the historical "Tier-1" path) is **deprecated** and is not a
|
||||
> supported test environment going forward; the Tier-1 sections below are
|
||||
> retained as historical reference / traceability only. CI test pipelines
|
||||
> target the colocated arm64 Jetson Woodpecker agent (see
|
||||
> `_docs/04_deploy/ci_cd_pipeline.md`); local-development test runs SHOULD
|
||||
> use `scripts/run-tests-jetson.sh` against the configured `jetson-e2e` SSH
|
||||
> alias rather than `scripts/run-tests.sh`. This decision supersedes the
|
||||
> 2026-05-09 "both" decision recorded in the § Test Execution section.
|
||||
> **Active policy — 2026-05-20 (refined)**: the canonical CI / release-gate
|
||||
> test environment is the Jetson Orin Nano Super (or a Jetson-equivalent
|
||||
> arm64 agent). **Unit tests** (`pytest tests/unit/`) MAY be run on a local
|
||||
> developer workstation for fast iteration — they are hardware-agnostic by
|
||||
> construction, the suite is fully synthetic, and Jetson SSH round-trips add
|
||||
> latency without adding signal. **Blackbox / e2e / performance / resilience
|
||||
> / security / resource-limit tests** (`tests/e2e/`, `e2e/tests/`,
|
||||
> `tests/perf/`, etc.) MUST run on the Jetson — never on a local workstation
|
||||
> — because their pass criteria are tied to Jetson wall-clock latency,
|
||||
> thermal envelope, and the real-camera + real-FC SITL loop. Workstation x86
|
||||
> Docker (the historical "Tier-1" path) is **deprecated** as a supported
|
||||
> e2e environment; the Tier-1 sections below are retained as historical
|
||||
> reference / traceability only. CI e2e pipelines target the colocated
|
||||
> arm64 Jetson Woodpecker agent (see `_docs/04_deploy/ci_cd_pipeline.md`);
|
||||
> local-development e2e runs SHOULD use `scripts/run-tests-jetson.sh`
|
||||
> against the configured `jetson-e2e` SSH alias rather than
|
||||
> `scripts/run-tests.sh`. This refinement supersedes the 2026-05-20 "all
|
||||
> tiers on Jetson" wording and the 2026-05-09 "both" decision recorded in
|
||||
> the § Test Execution section.
|
||||
|
||||
## Where each tier runs (active policy)
|
||||
|
||||
| Tier | Local workstation | Jetson (canonical) | When local is the only option |
|
||||
|------|--------------------|--------------------|-------------------------------|
|
||||
| Unit (`tests/unit/`) | ✅ allowed and encouraged for dev iteration | ✅ also run as part of the Jetson CI lane | always |
|
||||
| Blackbox / e2e (`tests/e2e/`, `e2e/tests/`) | ❌ forbidden — placeholder fixtures + missing hardware = false-negative runs | ✅ required for any merge / release decision | never — if Jetson is unreachable, the e2e verdict is "not run" rather than a local result |
|
||||
| Performance / resilience / security / resource-limit | ❌ forbidden | ✅ required | never |
|
||||
| Thermal chamber (AC-NEW-5) | ❌ forbidden | ✅ chamber Jetson only | never |
|
||||
|
||||
Practical consequences:
|
||||
|
||||
- A PR may merge on green local unit tests + green Jetson e2e tests.
|
||||
- A PR MAY NOT merge on green local unit tests alone — the Jetson e2e lane is the binding signal.
|
||||
- When the Jetson agent is offline, the e2e verdict is "pending Jetson" — record the gap (e.g. via `_docs/_process_leftovers/`) rather than substituting a local run.
|
||||
- Tests in `tests/e2e/` that gate on `RUN_REPLAY_E2E` or `@pytest.mark.tier2` will SKIP locally; this is correct behaviour, not a failure to investigate.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -263,11 +286,21 @@ The captured-fixture builder framework (`e2e/fixtures/sitl_replay_builder/`) reg
|
||||
|
||||
## Test Execution
|
||||
|
||||
**Decision (2026-05-20)** — **Jetson only.** Supersedes the 2026-05-09 "both" decision below. All tests (unit, integration, blackbox / e2e, performance, resilience, security, resource-limit) run on the Jetson Orin Nano Super (or a Jetson-equivalent arm64 agent). The workstation x86 Docker path is deprecated. Rationale captured in `_docs/LESSONS.md` (2026-05-20 entry): repeated workstation-vs-Jetson environment divergences (Dockerfile build order, missing `libgl1`, gtsam wheel availability, venv symlink resolution, lazy-import side-effect registration) were producing false-negative test runs and consuming engineering time without ever exercising the production-equivalent hardware path.
|
||||
**Decision (2026-05-20, refined later that day)** — **Jetson is the binding e2e environment; unit tests may run locally.** This refines the earlier "Jetson only for everything" wording. Rationale captured in `_docs/LESSONS.md` (2026-05-20 entries):
|
||||
|
||||
- The original "Jetson-only across all tiers" decision came from repeated workstation-vs-Jetson environment divergences in the e2e / build path (Dockerfile build order, missing `libgl1`, gtsam wheel availability, venv symlink resolution, lazy-import side-effect registration). Those divergences are real and continue to justify Jetson as the binding e2e environment.
|
||||
- Forcing the unit-test suite over an SSH-orchestrated Jetson loop added 30–90 s per iteration without producing any signal the local interpreter doesn't already produce. The unit suite is fully synthetic — no camera, no SITL, no Jetson-specific runtime — so a local PASS is equivalent to a Jetson PASS for that tier.
|
||||
|
||||
**Operational entry points**:
|
||||
- Local-development: `scripts/run-tests-jetson.sh` against the configured `jetson-e2e` SSH alias (see `_docs/03_implementation/jetson_harness_setup.md` for one-time setup).
|
||||
- CI: `.woodpecker/01-test.yml` on the colocated arm64 Jetson agent (see `_docs/04_deploy/ci_cd_pipeline.md`).
|
||||
|
||||
| Tier | Entry point | Where it runs |
|
||||
|------|-------------|---------------|
|
||||
| Unit (`tests/unit/`) | `pytest tests/unit/ -q` directly, or `scripts/run-tests.sh` | local workstation (Python 3.10+ venv) |
|
||||
| Blackbox / e2e (`tests/e2e/`, `e2e/tests/`) | `scripts/run-tests-jetson.sh` (local dev) / `.woodpecker/01-test.yml` (CI) | colocated arm64 Jetson Woodpecker agent — see `_docs/04_deploy/ci_cd_pipeline.md` |
|
||||
| Performance / resilience / security / resource-limit | same as e2e | Jetson only |
|
||||
| AC-NEW-5 thermal chamber | quarterly + pre-release | `self-hosted-jetson-orin-chamber` |
|
||||
|
||||
A green local unit-test run is necessary-but-not-sufficient for merge; the Jetson e2e lane is the binding signal.
|
||||
|
||||
The remainder of this section preserves the original 2026-05-09 decision context for traceability.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user