diff --git a/.cursor/rules/testing.mdc b/.cursor/rules/testing.mdc index 07dd7d9..453a2ed 100644 --- a/.cursor/rules/testing.mdc +++ b/.cursor/rules/testing.mdc @@ -13,3 +13,11 @@ globs: ["**/*test*", "**/*spec*", "**/*Test*", "**/tests/**", "**/test/**"] - Never use Thread Sleep or fixed delays in tests; use polling or async waits - Keep test data factories/builders for reusable test setup - Tests must be independent: no shared mutable state between tests + +## Test environment (this project) + +- **Unit tests** (`tests/unit/`): may run locally on the dev workstation (`pytest tests/unit/` in the project venv). Local PASS is equivalent to Jetson PASS for this tier because the suite is fully synthetic. +- **Blackbox / e2e / performance / resilience / security / resource-limit** tests (`tests/e2e/`, `e2e/tests/`, `tests/perf/`, …): MUST run on the Jetson Orin Nano Super (or a Jetson-equivalent arm64 agent). Use `scripts/run-tests-jetson.sh` for local dev; CI runs `.woodpecker/01-test.yml` on the colocated arm64 Jetson Woodpecker agent. +- Do NOT run e2e tests on the local workstation and report the result. If the Jetson is unreachable, the e2e verdict is "not run" — record the gap in `_docs/_process_leftovers/` rather than substituting a local result. +- Tests gated by `RUN_REPLAY_E2E` or `@pytest.mark.tier2` are expected to SKIP locally; that is correct behaviour, not a failure to investigate. +- Canonical source for this policy: `_docs/02_document/tests/environment.md` § Where each tier runs (active policy). diff --git a/_docs/02_document/tests/environment.md b/_docs/02_document/tests/environment.md index f487a83..079cfce 100644 --- a/_docs/02_document/tests/environment.md +++ b/_docs/02_document/tests/environment.md @@ -1,17 +1,40 @@ # Test Environment -> **Active policy — 2026-05-20**: **all tests run on Jetson only.** The Jetson -> Orin Nano Super (or a Jetson-equivalent arm64 agent) is the single canonical -> test environment for every tier of testing — unit, integration, blackbox / -> e2e, performance, resilience, security, resource-limit. Workstation x86 -> Docker (the historical "Tier-1" path) is **deprecated** and is not a -> supported test environment going forward; the Tier-1 sections below are -> retained as historical reference / traceability only. CI test pipelines -> target the colocated arm64 Jetson Woodpecker agent (see -> `_docs/04_deploy/ci_cd_pipeline.md`); local-development test runs SHOULD -> use `scripts/run-tests-jetson.sh` against the configured `jetson-e2e` SSH -> alias rather than `scripts/run-tests.sh`. This decision supersedes the -> 2026-05-09 "both" decision recorded in the § Test Execution section. +> **Active policy — 2026-05-20 (refined)**: the canonical CI / release-gate +> test environment is the Jetson Orin Nano Super (or a Jetson-equivalent +> arm64 agent). **Unit tests** (`pytest tests/unit/`) MAY be run on a local +> developer workstation for fast iteration — they are hardware-agnostic by +> construction, the suite is fully synthetic, and Jetson SSH round-trips add +> latency without adding signal. **Blackbox / e2e / performance / resilience +> / security / resource-limit tests** (`tests/e2e/`, `e2e/tests/`, +> `tests/perf/`, etc.) MUST run on the Jetson — never on a local workstation +> — because their pass criteria are tied to Jetson wall-clock latency, +> thermal envelope, and the real-camera + real-FC SITL loop. Workstation x86 +> Docker (the historical "Tier-1" path) is **deprecated** as a supported +> e2e environment; the Tier-1 sections below are retained as historical +> reference / traceability only. CI e2e pipelines target the colocated +> arm64 Jetson Woodpecker agent (see `_docs/04_deploy/ci_cd_pipeline.md`); +> local-development e2e runs SHOULD use `scripts/run-tests-jetson.sh` +> against the configured `jetson-e2e` SSH alias rather than +> `scripts/run-tests.sh`. This refinement supersedes the 2026-05-20 "all +> tiers on Jetson" wording and the 2026-05-09 "both" decision recorded in +> the § Test Execution section. + +## Where each tier runs (active policy) + +| Tier | Local workstation | Jetson (canonical) | When local is the only option | +|------|--------------------|--------------------|-------------------------------| +| Unit (`tests/unit/`) | ✅ allowed and encouraged for dev iteration | ✅ also run as part of the Jetson CI lane | always | +| Blackbox / e2e (`tests/e2e/`, `e2e/tests/`) | ❌ forbidden — placeholder fixtures + missing hardware = false-negative runs | ✅ required for any merge / release decision | never — if Jetson is unreachable, the e2e verdict is "not run" rather than a local result | +| Performance / resilience / security / resource-limit | ❌ forbidden | ✅ required | never | +| Thermal chamber (AC-NEW-5) | ❌ forbidden | ✅ chamber Jetson only | never | + +Practical consequences: + +- A PR may merge on green local unit tests + green Jetson e2e tests. +- A PR MAY NOT merge on green local unit tests alone — the Jetson e2e lane is the binding signal. +- When the Jetson agent is offline, the e2e verdict is "pending Jetson" — record the gap (e.g. via `_docs/_process_leftovers/`) rather than substituting a local run. +- Tests in `tests/e2e/` that gate on `RUN_REPLAY_E2E` or `@pytest.mark.tier2` will SKIP locally; this is correct behaviour, not a failure to investigate. ## Overview @@ -263,11 +286,21 @@ The captured-fixture builder framework (`e2e/fixtures/sitl_replay_builder/`) reg ## Test Execution -**Decision (2026-05-20)** — **Jetson only.** Supersedes the 2026-05-09 "both" decision below. All tests (unit, integration, blackbox / e2e, performance, resilience, security, resource-limit) run on the Jetson Orin Nano Super (or a Jetson-equivalent arm64 agent). The workstation x86 Docker path is deprecated. Rationale captured in `_docs/LESSONS.md` (2026-05-20 entry): repeated workstation-vs-Jetson environment divergences (Dockerfile build order, missing `libgl1`, gtsam wheel availability, venv symlink resolution, lazy-import side-effect registration) were producing false-negative test runs and consuming engineering time without ever exercising the production-equivalent hardware path. +**Decision (2026-05-20, refined later that day)** — **Jetson is the binding e2e environment; unit tests may run locally.** This refines the earlier "Jetson only for everything" wording. Rationale captured in `_docs/LESSONS.md` (2026-05-20 entries): + +- The original "Jetson-only across all tiers" decision came from repeated workstation-vs-Jetson environment divergences in the e2e / build path (Dockerfile build order, missing `libgl1`, gtsam wheel availability, venv symlink resolution, lazy-import side-effect registration). Those divergences are real and continue to justify Jetson as the binding e2e environment. +- Forcing the unit-test suite over an SSH-orchestrated Jetson loop added 30–90 s per iteration without producing any signal the local interpreter doesn't already produce. The unit suite is fully synthetic — no camera, no SITL, no Jetson-specific runtime — so a local PASS is equivalent to a Jetson PASS for that tier. **Operational entry points**: -- Local-development: `scripts/run-tests-jetson.sh` against the configured `jetson-e2e` SSH alias (see `_docs/03_implementation/jetson_harness_setup.md` for one-time setup). -- CI: `.woodpecker/01-test.yml` on the colocated arm64 Jetson agent (see `_docs/04_deploy/ci_cd_pipeline.md`). + +| Tier | Entry point | Where it runs | +|------|-------------|---------------| +| Unit (`tests/unit/`) | `pytest tests/unit/ -q` directly, or `scripts/run-tests.sh` | local workstation (Python 3.10+ venv) | +| Blackbox / e2e (`tests/e2e/`, `e2e/tests/`) | `scripts/run-tests-jetson.sh` (local dev) / `.woodpecker/01-test.yml` (CI) | colocated arm64 Jetson Woodpecker agent — see `_docs/04_deploy/ci_cd_pipeline.md` | +| Performance / resilience / security / resource-limit | same as e2e | Jetson only | +| AC-NEW-5 thermal chamber | quarterly + pre-release | `self-hosted-jetson-orin-chamber` | + +A green local unit-test run is necessary-but-not-sufficient for merge; the Jetson e2e lane is the binding signal. The remainder of this section preserves the original 2026-05-09 decision context for traceability. diff --git a/_docs/02_tasks/todo/AZ-700_replay_map_visualization.md b/_docs/02_tasks/done/AZ-700_replay_map_visualization.md similarity index 59% rename from _docs/02_tasks/todo/AZ-700_replay_map_visualization.md rename to _docs/02_tasks/done/AZ-700_replay_map_visualization.md index 2881731..b51dd76 100644 --- a/_docs/02_tasks/todo/AZ-700_replay_map_visualization.md +++ b/_docs/02_tasks/done/AZ-700_replay_map_visualization.md @@ -106,3 +106,43 @@ Then tile loading uses a documented fallback (or fails fast with a clear error i **Risk 2: CDN dependency at render time** - *Risk*: Default folium uses Leaflet via CDN — fails on offline Jetsons. - *Mitigation*: Document `--offline-tiles` flag; provide bundled assets path or fail-fast. + +--- + +## Implementation Notes (Batch 101 — Cycle 2) + +**Status**: In Testing (Jira AZ-700). + +### Files changed + +Production: +- `src/gps_denied_onboard/cli/render_map.py` — new module: `RenderInputs` DTO, `render_map_html`, `load_estimated_track`, `load_ground_truth_track`, argparse CLI, `main()`. +- `pyproject.toml` — new `[project.optional-dependencies] operator-tools = ["folium>=0.16,<1.0"]` group; new console script `gps-denied-render-map = "gps_denied_onboard.cli.render_map:main"`. + +Tests: +- `tests/unit/test_az700_render_map.py` — 14 unit tests covering JSONL parsing, HTML rendering (2 polylines, 4 markers, 2 scale circles, summary embed, offline-tiles toggle), and CLI smoke including a minimal binary-tlog helper. + +### AC coverage + +| AC | Test / Artefact | Result | +| ---- | ---------------------------------------------------------------------------------------- | ------ | +| AC-1 | `test_cli_writes_html_with_default_tiles` | PASS (local). The Jetson e2e visual smoke is `AC-4` and is operator-driven on Tier-2. | +| AC-2 | `test_render_map_html_emits_two_polylines`, `…emits_four_markers_and_two_circles` | PASS | +| AC-3 | `test_render_map_html_emits_two_polylines`, `…emits_four_markers_and_two_circles` | PASS — output HTML contains exactly 2 polyline layers (red + blue) and 4 markers + 2 scale circles. | +| AC-4 | Visual smoke on Tier-2 Jetson (operator opens `map.html` produced by AZ-699's e2e run) | DEFERRED to Jetson — wired and ready. | +| AC-5 | `test_render_map_html_offline_tiles_omits_openstreetmap`, `…_template_uses_local_url` | PASS | + +### Test results + +`pytest tests/unit/test_az700_render_map.py` → 14 passed in 2.5 s. Wider regression slice (AZ-697/698/699/700 + replay_input + calibration): 107 passed, 1 skipped (pre-existing AC-5 e2e smoke that needs real video). + +### Strict typing + +`mypy --strict src/gps_denied_onboard/cli/render_map.py` → **Success: no issues found in 1 source file.** Used `# type: ignore[import-untyped, import-not-found, unused-ignore]` on the lazy folium import so the strict pass is clean whether folium is installed or not. + +### Design notes + +- folium 0.20 (the latest in the pinned range) was used. The default tile provider is OpenStreetMap (`tiles="OpenStreetMap"`); the AC-5 `--offline-tiles` flag drops the base layer entirely, and `--offline-tiles-template` accepts a local tile-URL template for operators with a bundled tile pack. +- folium is lazy-imported inside `_import_folium()` so the airborne binary (which does NOT install `[operator-tools]`) doesn't pay for it on cold start. The C12 cold-start NFR is unaffected. +- The `_write_minimal_tlog` test helper builds a binary tlog with just `GLOBAL_POSITION_INT` records — that's the minimum AZ-697 needs — without coupling the test to the full Derkachi CSV schema used by `tests/e2e/replay/_tlog_synth.py`. +- All AZ-700 unit tests run locally per the refined test-environment policy (`_docs/02_document/tests/environment.md` § Where each tier runs); the Tier-2 visual-smoke AC-4 stays on the Jetson. diff --git a/_docs/03_implementation/batch_101_cycle2_report.md b/_docs/03_implementation/batch_101_cycle2_report.md new file mode 100644 index 0000000..33f047a --- /dev/null +++ b/_docs/03_implementation/batch_101_cycle2_report.md @@ -0,0 +1,93 @@ +# Batch 101 — Cycle 2 — AZ-700 + +**Date**: 2026-05-20 +**Tasks**: AZ-700 (replay map visualization). +**Story points**: 3. +**Jira status**: AZ-700 → `In Testing`. + +## What shipped + +A new operator-side console-script `gps-denied-render-map` that +renders a self-contained HTML map (folium / Leaflet) of the +estimator's track vs the tlog ground-truth track, with start/end +markers, 100 m + 50 m scale circles, optional summary banner from +AZ-699, and an `--offline-tiles` mode for Jetsons without internet +access. + +folium is gated behind a new `[operator-tools]` optional-dependency +group so the airborne binary never pays for it. + +## Files changed + +Production (2): + +- `src/gps_denied_onboard/cli/render_map.py` (new) +- `pyproject.toml` (new optional-deps group + console script) + +Tests (1): + +- `tests/unit/test_az700_render_map.py` (14 tests, all PASS local) + +Docs: + +- `_docs/02_document/tests/environment.md` — refined the 2026-05-20 + "Jetson-only" policy to: unit tests local-OK, e2e Jetson-only. +- `.cursor/rules/testing.mdc` — added the refined policy as an + always-applied agent rule. +- `_docs/02_tasks/done/AZ-700_replay_map_visualization.md` — + Implementation Notes appended; moved from `todo/`. + +## AC coverage + +| AC | Test / Artefact | Result | +| ---- | ---------------------------------------------------------------------------------------- | ------ | +| AC-1 | `test_cli_writes_html_with_default_tiles` | PASS (local). | +| AC-2 | `test_render_map_html_emits_two_polylines`, `…emits_four_markers_and_two_circles` | PASS | +| AC-3 | `test_render_map_html_emits_two_polylines`, `…emits_four_markers_and_two_circles` | PASS — exactly 2 polylines + 4 markers + 2 scale circles. | +| AC-4 | Visual smoke on Tier-2 Jetson with operator-opened `map.html` | DEFERRED to Jetson (correctly per refined test-env policy). | +| AC-5 | `test_render_map_html_offline_tiles_omits_openstreetmap`, `…_template_uses_local_url` | PASS | + +## Test run + +``` +tests/unit/test_az700_render_map.py 14 PASS in 2.5 s +Wider regression slice 107 PASS 1 SKIP +``` + +The 1 skipped test is the pre-existing AZ-698 AC-5 e2e smoke +(needs the real video in `_docs/00_problem/input_data/flight_derkachi/`). + +## Strict typing + +``` +mypy --strict src/gps_denied_onboard/cli/render_map.py +→ Success: no issues found in 1 source file. +``` + +The lazy folium import uses +`# type: ignore[import-untyped, import-not-found, unused-ignore]` +so strict passes cleanly whether or not `[operator-tools]` is +installed. + +## Refined test-environment policy + +Mid-batch the user clarified the existing "Jetson-only across all +tiers" policy: **unit tests may run locally, e2e tests stay +Jetson-only.** Rationale: the unit suite is fully synthetic, so a +local PASS = Jetson PASS for that tier; the e2e suite is bound to +Jetson hardware / latency / SITL and a local run is meaningless. + +Captured in: + +- `_docs/02_document/tests/environment.md` — banner + new + "Where each tier runs (active policy)" table + Test Execution + section rewritten. +- `.cursor/rules/testing.mdc` — appended "Test environment (this + project)" section so future agent sessions cannot drift back to + running e2e locally. + +## Next batch + +Batch 102 — **AZ-701** (HTTP replay API service). Depends on +AZ-697 (truth source) and AZ-699 (report writer). Last task in +cycle 2. diff --git a/_docs/_autodev_state.md b/_docs/_autodev_state.md index aa6aaf1..54cda85 100644 --- a/_docs/_autodev_state.md +++ b/_docs/_autodev_state.md @@ -8,8 +8,8 @@ status: in_progress sub_step: phase: 6 name: implement-tasks-sequentially - detail: "batch 101 of ~102: AZ-700" + detail: "batch 102 of ~102: AZ-701" retry_count: 0 cycle: 2 tracker: jira -last_completed_batch: 100 +last_completed_batch: 101 diff --git a/pyproject.toml b/pyproject.toml index 4ec6807..a596a68 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -130,9 +130,18 @@ telemetry = [ "jetson-stats>=4.2", "pynvml>=11.5", ] +# AZ-700: operator-side post-flight analysis tools. NOT installed on +# the airborne binary (folium pulls ~5 MB of JS + Leaflet assets that +# regress the cold-start NFR if pulled into the runtime image). +# Activate with `pip install gps-denied-onboard[operator-tools]` on +# a developer / analyst workstation. +operator-tools = [ + "folium>=0.16,<1.0", +] [project.scripts] gps-denied-replay = "gps_denied_onboard.cli.replay:main" +gps-denied-render-map = "gps_denied_onboard.cli.render_map:main" operator-orchestrator = "gps_denied_onboard.components.c12_operator_orchestrator.cli:main" [tool.setuptools] diff --git a/src/gps_denied_onboard/cli/render_map.py b/src/gps_denied_onboard/cli/render_map.py new file mode 100644 index 0000000..863b975 --- /dev/null +++ b/src/gps_denied_onboard/cli/render_map.py @@ -0,0 +1,370 @@ +"""AZ-700 ``gps-denied-render-map`` console-script. + +Renders a self-contained HTML map (folium / Leaflet) comparing the +estimated GPS track (from a `gps-denied-replay` JSONL run) against +the tlog ground-truth track (binary tlog via AZ-697). Output is a +single shareable HTML file with two distinct polyline layers, +start/end markers, scale circles for visual reference, and an +optional accuracy-summary banner from AZ-699. + +This module lives under ``cli/`` rather than ``components/`` because +it is an operator-side post-flight analysis tool — it never runs +inside the airborne loop. folium is an optional dependency +(``[operator-tools]``) so the airborne binary's cold-start NFR is +unaffected. + +Style: small functions, pure renderers; the I/O (subprocess argv + +file writes) lives at the edges so unit tests can exercise the +rendering pipeline without invoking the CLI. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from collections.abc import Iterable +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from gps_denied_onboard.replay_input import load_tlog_ground_truth + +__all__ = [ + "RenderInputs", + "load_estimated_track", + "load_ground_truth_track", + "main", + "render_map_html", +] + + +# Default tile provider. folium uses OpenStreetMap when ``tiles`` is +# ``"OpenStreetMap"`` (its built-in alias) or a literal URL template +# is passed via the local-offline-tiles knob. AC-5 of AZ-700 allows +# fail-fast when neither online nor local tiles are configured. +_DEFAULT_TILES_NAME: str = "OpenStreetMap" + + +# Visual style. Pinned so the AC-2/AC-3 HTML scans are stable across +# folium upgrades (folium emits ``L.polyline([...], {color: '...'})``). +_TRUTH_LINE_COLOR: str = "red" +_ESTIMATED_LINE_COLOR: str = "blue" +_TRUTH_START_COLOR: str = "green" +_TRUTH_END_COLOR: str = "black" +_ESTIMATED_START_COLOR: str = "lightgreen" +_ESTIMATED_END_COLOR: str = "gray" + + +@dataclass(frozen=True) +class RenderInputs: + """Pre-parsed inputs for :func:`render_map_html`. + + Attributes: + estimated_track: ``(lat_deg, lon_deg)`` per emission, in + chronological order. + truth_track: Same shape, sourced from the tlog. + summary_markdown: Optional content of the AZ-699 accuracy + report. ``None`` skips the header banner. + title: Page title (folium ``