mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 22:31:13 +00:00
811b04e605
Adapt C11 HttpTileDownloader to the AZ-505 v1.0.0 tile-inventory
contract (POST /api/satellite/tiles/inventory + GET /tiles/{z}/{x}/{y})
and wire the Jetson e2e harness against the real parent-suite
satellite-provider service. Closes Phase 1 of 5 for AZ-777; STOP
gate before Phase 2 (Derkachi catalog seed).
C11 changes:
- _LIST_PATH / _GET_PATH replaced with _INVENTORY_PATH + _TILES_PATH.
- _do_enumerate enumerates bbox tile coords client-side and posts
chunked inventory requests (5000-entry cap per the contract).
- _download_one_tile parses tile_id_str into (z,x,y) and fetches
the slippy-map URL.
- Common GET / POST retry+auth ladder consolidated into _send_request.
- New module helpers: _enumerate_bbox_tile_coords,
_tile_center_latlon, _tile_size_meters_at, _format_tile_id_str,
_parse_tile_id_str, _chunk_iter.
- _DEFAULT_ESTIMATED_TILE_BYTES (50 KiB) replaces the inventory-side
estimatedBytes field the v1.0.0 contract dropped.
Tests:
- 14/14 unit tests in tests/unit/c11_tile_manager/test_tile_downloader.py
rewritten for the new POST inventory + slippy-map GET handler.
_StubTileWriter rekeyed by call-index (the downloader now derives
lat/lon from the slippy-map coord, so fixtures can't fabricate
arbitrary positions).
- New Tier-2 smoke at tests/e2e/satellite_provider/test_smoke.py:
validates inventory POST schema + drives HttpTileDownloader against
the real service. Gated by RUN_REPLAY_E2E=1 + tier2.
Compose / env:
- e2e-runner SATELLITE_PROVIDER_URL switched from mock-sat:5100 to
https://satellite-provider:8080; TLS_INSECURE + Bearer JWT env +
depends_on satellite-provider added.
- .env.test.example documents SATELLITE_PROVIDER_API_KEY + dev TLS
bypass security note.
- scripts/mint_dev_jwt.py mints HS256 dev JWTs from env / .env.test.
- pyjwt added to dev extras.
Tracker hygiene:
- AZ-777 row in _dependencies_table.md bumped 5pt -> 8pt to match
the 2026-05-21 override decision log.
Code review: PASS_WITH_WARNINGS (3 medium/low findings, all deferred
to later AZ-777 phases) -- see batch_104_review.md. Batch report at
batch_104_cycle3_report.md.
Co-authored-by: Cursor <cursoragent@cursor.com>
237 lines
10 KiB
YAML
237 lines
10 KiB
YAML
# Tier-2 e2e harness — Jetson Orin Nano (JetPack 6.x, L4T R36.x).
|
|
#
|
|
# AZ-615: companion compose file to `docker-compose.test.yml` that runs
|
|
# the full Reality Gate on a CUDA-capable host. Used via `ssh jetson-e2e
|
|
# "docker compose -f docker-compose.test.jetson.yml up ..."` driven by
|
|
# `scripts/run-tests-jetson.sh`.
|
|
#
|
|
# Difference vs. docker-compose.test.yml:
|
|
# * `runtime: nvidia` + `gpus: all` on `e2e-runner` so the SUT can
|
|
# resolve `model.half().cuda()` against the Orin GPU.
|
|
# * `GPS_DENIED_TIER=2` — turns OFF the auto-skip for `@pytest.mark.tier2`
|
|
# ACs (see tests/conftest.py:31-44). The heavy ACs (AC-1, AC-2, AC-3,
|
|
# AC-5, AC-6) actually run.
|
|
# * Builds from `tests/e2e/Dockerfile.jetson` (l4t-pytorch base).
|
|
# * Companion / db / mock-sat continue to come from the root
|
|
# `docker-compose.yml` via `extends:` (same as Colima) — they have ARM64
|
|
# tags via the existing build pipeline.
|
|
#
|
|
# AZ-688 (sibling of AZ-616): the real satellite-provider .NET service is
|
|
# defined inline below (services.satellite-provider + services.satellite-
|
|
# provider-postgres). `run-tests-jetson.sh` rsyncs `../satellite-provider/`
|
|
# to a sibling directory on the Jetson so the build context resolves
|
|
# identically on the workstation and on the Jetson.
|
|
#
|
|
# Why inline instead of `include: ../satellite-provider/docker-compose.yml`:
|
|
# Compose's `include:` rejects same-name service overrides ("conflicts with
|
|
# imported resource"). We need to customize the api service (healthcheck,
|
|
# network alias, internal-only ports) so the upstream compose's verbatim
|
|
# `include:` doesn't work. Inline is cleaner than the multi-`-f` ordering
|
|
# games required to make overlay precedence work.
|
|
#
|
|
# `mock-sat` remains in the graph for now — AZ-692 retires it once the
|
|
# gps-denied client (AZ-691) lands.
|
|
|
|
services:
|
|
# ------------------------------------------------------------------
|
|
# Init services (profiles: [setup]) — NOT started by the default
|
|
# `docker compose up`. They are invoked explicitly as one-shot jobs
|
|
# via `docker compose run --rm --profile setup <service>` before the
|
|
# main harness run:
|
|
#
|
|
# 1. db-migrate — applies Alembic migrations so companion's
|
|
# FreshnessGate / PostgresFilesystemStore find their tables.
|
|
# (AZ-618 ordering gap: build_pre_constructed queries the DB
|
|
# before the composition root can call apply_migrations.)
|
|
#
|
|
# 2. tile-init — writes a minimal valid HNSW32 FAISS descriptor
|
|
# index into the tile-data volume so FaissDescriptorIndex._load()
|
|
# succeeds during build_pre_constructed.
|
|
# (AZ-618 gap: the production provisioning pipeline normally
|
|
# writes the index; in the test harness it must pre-exist.)
|
|
#
|
|
# They are in profile "setup" so they do not participate in the
|
|
# default `docker compose up` and do not trip --abort-on-container-exit.
|
|
# ------------------------------------------------------------------
|
|
db-migrate:
|
|
profiles: ["setup"]
|
|
image: gps-denied-onboard/e2e-runner:jetson
|
|
entrypoint: ["alembic"]
|
|
command: ["upgrade", "head"]
|
|
working_dir: /opt/project
|
|
volumes:
|
|
- .:/opt/project:ro
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
restart: "no"
|
|
|
|
tile-init:
|
|
profiles: ["setup"]
|
|
image: gps-denied-onboard/e2e-runner:jetson
|
|
entrypoint: ["python3"]
|
|
command: ["/opt/project/scripts/mk_test_faiss_fixture.py"]
|
|
volumes:
|
|
- .:/opt/project:ro
|
|
- tile-data:/var/lib/gps-denied/tiles
|
|
restart: "no"
|
|
|
|
# companion and operator-orchestrator are intentionally absent from
|
|
# the Jetson e2e test harness.
|
|
#
|
|
# Every test in tests/e2e/replay/ invokes the ``gps-denied-replay``
|
|
# console-script directly as a subprocess and does not call the
|
|
# companion or operator-orchestrator HTTP APIs. Including either
|
|
# service caused the harness to abort before any test could run:
|
|
#
|
|
# * companion crashes at startup because live-mode requires a
|
|
# production-provisioned C7 inference engine (PyTorch FP16 or
|
|
# TensorRT) that is absent from the test environment. This is the
|
|
# pre-existing AZ-618 gap (build_pre_constructed fails before the
|
|
# composition root can apply_migrations + engine artifacts).
|
|
# * operator-orchestrator crashed for the same C7 inference reason.
|
|
#
|
|
# When the AZ-618 epic ships the full airborne boot-up in a sandboxed
|
|
# environment (Phase E / engine stubs), companion can be re-added here.
|
|
|
|
db:
|
|
extends:
|
|
file: docker-compose.yml
|
|
service: db
|
|
|
|
e2e-runner:
|
|
build:
|
|
context: .
|
|
dockerfile: tests/e2e/Dockerfile.jetson
|
|
image: gps-denied-onboard/e2e-runner:jetson
|
|
# nvidia-container-runtime exposes the Tegra GPU + libcuda mounts.
|
|
# Without this block the container starts but `torch.cuda.is_available()`
|
|
# returns False and every tier2 AC errors at `.cuda()`.
|
|
runtime: nvidia
|
|
deploy:
|
|
resources:
|
|
reservations:
|
|
devices:
|
|
- driver: nvidia
|
|
count: all
|
|
capabilities: [gpu]
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
satellite-provider:
|
|
condition: service_healthy
|
|
environment:
|
|
# Same FullSystemConfig env block as Colima — see comments in
|
|
# docker-compose.test.yml for the per-var rationale.
|
|
GPS_DENIED_FC_PROFILE: ardupilot_plane
|
|
# Tier-2 turns OFF the `tier2` / `gpu` auto-skip in tests/conftest.py
|
|
# so the heavy ACs in tests/e2e/replay/test_derkachi_1min.py actually
|
|
# execute. This is the WHOLE POINT of the Jetson harness.
|
|
GPS_DENIED_TIER: "2"
|
|
DB_URL: postgresql://gps_denied:dev@db:5432/gps_denied
|
|
# AZ-777 Phase 1: e2e-runner consumes the real parent-suite
|
|
# satellite-provider .NET service over its compose-DNS name. The
|
|
# dev TLS cert is self-signed against `localhost`, so the suite-
|
|
# internal probe must skip cert verification — see SECURITY note
|
|
# in `.env.test.example`. Production deploys ship a real CA-issued
|
|
# cert and MUST set SATELLITE_PROVIDER_TLS_INSECURE="0" (or omit it).
|
|
SATELLITE_PROVIDER_URL: https://satellite-provider:8080
|
|
SATELLITE_PROVIDER_TLS_INSECURE: "1"
|
|
SATELLITE_PROVIDER_API_KEY: ${SATELLITE_PROVIDER_API_KEY:?SATELLITE_PROVIDER_API_KEY must be set via .env.test — see scripts/mint_dev_jwt.py}
|
|
# AZ-777 Phase 1 also forwards the JWT triple so the smoke test
|
|
# can mint its own dev token in-container as a fallback when
|
|
# SATELLITE_PROVIDER_API_KEY is rotated mid-session.
|
|
JWT_SECRET: ${JWT_SECRET}
|
|
JWT_ISSUER: ${JWT_ISSUER}
|
|
JWT_AUDIENCE: ${JWT_AUDIENCE}
|
|
COMPANION_URL: http://companion:8080
|
|
CAMERA_CALIBRATION_PATH: /opt/tests/fixtures/calibration/adti26.json
|
|
LOG_LEVEL: INFO
|
|
LOG_SINK: console
|
|
INFERENCE_BACKEND: pytorch_fp16
|
|
FDR_PATH: /var/lib/gps-denied/fdr
|
|
TILE_CACHE_PATH: /var/lib/gps-denied/tiles
|
|
MAVLINK_SIGNING_KEY: /opt/tests/fixtures/mavlink_signing/dev_key
|
|
RUN_REPLAY_E2E: "1"
|
|
# Replay-mode build flags (Invariant 9). See identical block in
|
|
# docker-compose.test.yml — all three are required for the
|
|
# composition root to construct the replay strategies. The
|
|
# original harness only set BUILD_REPLAY_SINK_JSONL because
|
|
# every Reality-Gate run died at auto-sync before the other
|
|
# two flags were checked.
|
|
BUILD_VIDEO_FILE_FRAME_SOURCE: "ON"
|
|
BUILD_TLOG_REPLAY_ADAPTER: "ON"
|
|
BUILD_REPLAY_SINK_JSONL: "ON"
|
|
BUILD_FAISS_INDEX: "ON"
|
|
volumes:
|
|
- ./tests:/opt/tests:ro
|
|
- ./_docs/00_problem/input_data:/opt/_docs/00_problem/input_data:ro
|
|
- fdr-data:/var/lib/gps-denied/fdr
|
|
- tile-data:/var/lib/gps-denied/tiles
|
|
|
|
# AZ-688: real satellite-provider .NET service. Mirrors the upstream
|
|
# compose at ../satellite-provider/docker-compose.yml with three
|
|
# deliberate customizations:
|
|
# * service name = `satellite-provider` (clearer than the upstream's
|
|
# generic `api`) so AZ-692's client uses https://satellite-provider:8080
|
|
# * TCP-level healthcheck via bash /dev/tcp so other services can
|
|
# `depends_on: service_healthy`. The base image
|
|
# (mcr.microsoft.com/dotnet/aspnet:10.0, debian-12-slim) ships
|
|
# bash and /dev/tcp is a bash builtin; no extra package needed.
|
|
# * no host port mappings — internal-only access via compose DNS;
|
|
# keeps host ports free for nested e2e runs.
|
|
satellite-provider:
|
|
build:
|
|
context: ../satellite-provider
|
|
dockerfile: SatelliteProvider.Api/Dockerfile
|
|
image: gps-denied-onboard/satellite-provider:dev
|
|
container_name: gps-denied-e2e-satellite-provider
|
|
environment:
|
|
ASPNETCORE_ENVIRONMENT: Development
|
|
ASPNETCORE_URLS: https://+:8080
|
|
ASPNETCORE_Kestrel__Certificates__Default__Path: /app/certs/api.pfx
|
|
ASPNETCORE_Kestrel__Certificates__Default__Password: satellite-dev-cert
|
|
ConnectionStrings__DefaultConnection: Host=satellite-provider-postgres;Port=5432;Database=satelliteprovider;Username=postgres;Password=postgres
|
|
MapConfig__ApiKey: ${GOOGLE_MAPS_API_KEY:-}
|
|
# Suite JWT contract — see _docs/10_auth.md. Sourced from .env.test
|
|
# via run-tests-jetson.sh; the API fails fast at startup if any of
|
|
# the three are missing or whitespace-only.
|
|
JWT_SECRET: ${JWT_SECRET:?JWT_SECRET must be set via .env.test}
|
|
JWT_ISSUER: ${JWT_ISSUER:?JWT_ISSUER must be set via .env.test}
|
|
JWT_AUDIENCE: ${JWT_AUDIENCE:?JWT_AUDIENCE must be set via .env.test}
|
|
volumes:
|
|
- ../satellite-provider/certs/api.pfx:/app/certs/api.pfx:ro
|
|
- ../satellite-provider/tiles:/app/tiles
|
|
- ../satellite-provider/ready:/app/ready
|
|
- ../satellite-provider/logs:/app/logs
|
|
healthcheck:
|
|
test: ["CMD", "bash", "-c", "exec 3<>/dev/tcp/127.0.0.1/8080"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 30s
|
|
depends_on:
|
|
satellite-provider-postgres:
|
|
condition: service_healthy
|
|
|
|
satellite-provider-postgres:
|
|
image: postgres:16
|
|
container_name: gps-denied-e2e-satellite-provider-postgres
|
|
environment:
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_PASSWORD: postgres
|
|
POSTGRES_DB: satelliteprovider
|
|
volumes:
|
|
- satellite-provider-postgres-data:/var/lib/postgresql/data
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
|
interval: 5s
|
|
timeout: 5s
|
|
retries: 5
|
|
|
|
volumes:
|
|
db-data: {}
|
|
fdr-data: {}
|
|
tile-data: {}
|
|
satellite-provider-postgres-data: {}
|