mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 22:11:12 +00:00
Oleksandr Bezdieniezhnykh
6599d828d2
Three blackbox-harness tasks landed together — all depend only on
AZ-406 and unblock the FT-* / NFT-* scenario tasks scheduled for
batches 69+.
AZ-407 — Static fixture builders (3pt):
* tile-cache-builder/{builder.py, Dockerfile, build.sh} produces a
deterministic tile-cache-fixture Docker volume from
_docs/00_problem/input_data/. Reproducibility primitives: sorted
iteration, frozen PIL JPEG settings, FAISS HNSW32 built single-
threaded with seeded stub descriptors.
* age-injector/{age_injector.py, inject.sh} clones the volume and
shifts capture_date by N×30.44 days; tile JPEG bytes preserved
bit-identical. Emits synth-age-7mo + synth-age-13mo volumes.
* cold-boot/cold_boot_fixture.json: frozen FC pose snapshot at
Derkachi sector centre, schema v1.
* secrets/mavlink-test-passkey.txt: 64-hex with required
`# TEST ONLY` header line per AC-5. Passkey-equality test now
compares the secret line after stripping the header.
* security/cve-2025-53644.jpg: synthetic 158-byte malformed JPEG
(truncated SOS marker). OpenCV 4.11.x rejects gracefully with
imdecode → None. AZ-439 will sharpen for ASan instrumentation.
* Top-level Makefile with `make fixtures` / `make fixtures-*` /
`make e2e-tier1*` / `make unit-tests` targets.
AZ-444 — Tier-2 Jetson harness wrapper (5pt):
* run-tier2.sh rewritten as orchestrator. Detects local
(aarch64 + TIER2_HOST=localhost) vs remote (ssh into TIER2_HOST).
New flags: -k/--selector, --build-kind production|asan,
--reflash (gated behind TIER2_REFLASH_ACK=1 two-key gate),
--dry-run.
* tier2-on-jetson.sh (new) — on-device delegate. Verifies
gps-denied-onboard{,-asan}.service health; restarts with 5s
tolerance; spawns tegrastats + jtop parallel samplers; tails
ASan unit's journal in asan mode; drives docker compose with
TIER=tier2-jetson; forwards SELECTOR to pytest -k.
* docker/run-tier1.sh (new) — selector-parity sibling.
* AC-1 (selector parity) and AC-6 (reflash gating) unit-tested via
--dry-run output assertions. AC-2/AC-3/AC-4/AC-5 are hardware-
loop ACs verified by the Tier-2 runtime smoke (no Jetson in the
unit-test layer).
AZ-445 — CSV reporter + evidence bundler refinements (2pt):
* reporting/nfr_recorder.py (new) — pytest plugin. Provides the
`nfr_recorder` fixture with record_metric(name, value, ac_id)
and partial(ac_id, reason). At session end emits:
- per-nfr/<scenario_id>.json (AC-1)
- traceability-status.json with every AC ID parsed from
traceability-matrix.md, classified Covered/PARTIAL/NOT
COVERED with source scenario IDs (AC-2)
- regression-baseline.json with all numeric metrics (AC-3)
* csv_reporter.py extended — `_outcome_to_result` consults the
aggregator; rows flip PASS → PARTIAL when an AC was marked
PARTIAL by nfr_recorder (AC-4). Graceful fallback when
aggregator isn't registered (unit-test contexts).
* conftest.py registers nfr_recorder in pytest_plugins.
* New --traceability-matrix CLI flag seeds the NOT COVERED rows.
Build / config:
* pyproject.toml dev extras: added Pillow>=10.4,<13.0 for the
tile-cache-builder unit test (broad enough to keep torchvision's
Pillow 12 pin happy; the production builder runs inside its own
Docker image with its own pin).
* Updated test_directory_layout.py to cover 10 new files + replaced
the byte-equal passkey assertion with the header-stripping
variant.
Test results:
* 157 focused tests pass (was 97 in batch 67; +60 new across this
batch). No regressions.
Module-layout / spec drift:
* AZ-407 spec text says `tests/fixtures/...`; module-layout
blackbox_tests entry (commit d7a17a8) authoritatively places the
harness under `e2e/`. Implementation followed the layout entry.
* AZ-444 spec mentions `e2e/tier2/run-tier2.sh`; AZ-406 placed it
at `e2e/jetson/run-tier2.sh`. Kept at `e2e/jetson/` for
consistency.
* Cold-boot README ownership: corrected from AZ-419 to AZ-407 per
AZ-419's own Dependencies field.
Specs archived to _docs/02_tasks/done/. Jira tickets transitioned to
In Testing on commit.
Co-authored-by: Cursor <cursoragent@cursor.com>
100 lines
2.9 KiB
Bash
Executable File
100 lines
2.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Tier-1 (workstation Docker) entrypoint. Selector-parity sibling of
|
|
# `e2e/jetson/run-tier2.sh`.
|
|
#
|
|
# Usage:
|
|
# ./run-tier1.sh \
|
|
# --fc-adapter <ardupilot|inav> \
|
|
# --vio-strategy <okvis2|klt_ransac|vins_mono> \
|
|
# [-k <pytest selector>] \
|
|
# [--build-kind <production|asan>] \
|
|
# [--enable-chamber] \
|
|
# [--dry-run]
|
|
#
|
|
# AZ-444 AC-1: this script + run-tier2.sh accept the same `-k <selector>`
|
|
# flag and emit the same pytest invocation modulo the TIER env var.
|
|
|
|
set -euo pipefail
|
|
|
|
FC_ADAPTER=""
|
|
VIO_STRATEGY=""
|
|
SELECTOR=""
|
|
BUILD_KIND="production"
|
|
ENABLE_CHAMBER=0
|
|
DRY_RUN=0
|
|
|
|
usage() {
|
|
grep -E '^# ' "$0" | sed 's/^# //' >&2
|
|
exit 1
|
|
}
|
|
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--fc-adapter) FC_ADAPTER="$2"; shift 2 ;;
|
|
--vio-strategy) VIO_STRATEGY="$2"; shift 2 ;;
|
|
-k|--selector) SELECTOR="$2"; shift 2 ;;
|
|
--build-kind) BUILD_KIND="$2"; shift 2 ;;
|
|
--enable-chamber) ENABLE_CHAMBER=1; shift ;;
|
|
--dry-run) DRY_RUN=1; shift ;;
|
|
-h|--help) usage ;;
|
|
*) echo "Unknown arg: $1" >&2; usage ;;
|
|
esac
|
|
done
|
|
|
|
if [[ -z "$FC_ADAPTER" || -z "$VIO_STRATEGY" ]]; then
|
|
echo "ERROR: --fc-adapter and --vio-strategy are required" >&2
|
|
usage
|
|
fi
|
|
|
|
case "$FC_ADAPTER" in
|
|
ardupilot|inav) ;;
|
|
*) echo "ERROR: --fc-adapter must be ardupilot or inav (got: $FC_ADAPTER)" >&2; exit 2 ;;
|
|
esac
|
|
|
|
case "$VIO_STRATEGY" in
|
|
okvis2|klt_ransac|vins_mono) ;;
|
|
*) echo "ERROR: --vio-strategy must be okvis2 | klt_ransac | vins_mono (got: $VIO_STRATEGY)" >&2; exit 2 ;;
|
|
esac
|
|
|
|
case "$BUILD_KIND" in
|
|
production|asan) ;;
|
|
*) echo "ERROR: --build-kind must be production or asan (got: $BUILD_KIND)" >&2; exit 2 ;;
|
|
esac
|
|
|
|
: "${RUN_ID:=tier1-$(date -u +%Y%m%dT%H%M%SZ)-${FC_ADAPTER}-${VIO_STRATEGY}}"
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
|
|
|
PYTEST_ARGS=("/test-suite")
|
|
PYTEST_ARGS+=("--csv=/e2e-results/run-${RUN_ID}/report.csv")
|
|
PYTEST_ARGS+=("--csv-columns=test_id,test_name,traces_to,fc_adapter,vio_strategy,tier,started_at_utc,execution_time_ms,result,error_message,evidence_paths")
|
|
PYTEST_ARGS+=("--evidence-out=/e2e-results/run-${RUN_ID}/evidence")
|
|
PYTEST_ARGS+=("--build-kind=${BUILD_KIND}")
|
|
[[ "${ENABLE_CHAMBER}" -eq 1 ]] && PYTEST_ARGS+=("--enable-chamber")
|
|
[[ -n "${SELECTOR}" ]] && PYTEST_ARGS+=("-k" "${SELECTOR}")
|
|
|
|
COMPOSE_CMD=(
|
|
docker compose
|
|
-f "${SCRIPT_DIR}/docker-compose.test.yml"
|
|
run --rm
|
|
-e TIER=tier1-workstation
|
|
-e BUILD_KIND="${BUILD_KIND}"
|
|
e2e-runner
|
|
pytest "${PYTEST_ARGS[@]}"
|
|
)
|
|
|
|
if [[ "${DRY_RUN}" -eq 1 ]]; then
|
|
echo "[tier1] --dry-run:"
|
|
echo "[tier1] RUN_ID=${RUN_ID}"
|
|
echo "[tier1] ${COMPOSE_CMD[*]}"
|
|
exit 0
|
|
fi
|
|
|
|
RUN_ID="${RUN_ID}" \
|
|
FC_ADAPTER="${FC_ADAPTER}" \
|
|
VIO_STRATEGY="${VIO_STRATEGY}" \
|
|
TIER="tier1-workstation" \
|
|
"${COMPOSE_CMD[@]}"
|
|
|
|
echo "[tier1] Suite complete. RUN_ID=${RUN_ID}"
|