Commit Graph

9 Commits

Author SHA1 Message Date
Oleksandr Bezdieniezhnykh b15454b9a9 [AZ-777] Phase 1 hotfix (z/x/y) + Phase 2 Derkachi seed + ops
Phase 1 hotfix:
- C11 HttpTileDownloader adapted to satellite-provider v2.0.0
  z/x/y inventory contract (bulk POST keyed by slippy-map coords).
- Unit tests rewritten to exercise the new inventory schema.
- E2E smoke test updated to match the v2.0.0 wire.

Phase 2 (Derkachi seed + smoke-validated on Jetson):
- tests/fixtures/derkachi_c6/{README,bbox.yaml,seed_region.py}
  drives POST /api/satellite/region against satellite-provider
  with Google Maps as the imagery source. Smoke run produced
  4 regions, 175 tiles, inventory 32/32.
- scripts/mint_dev_jwt.py + run-tests-jetson.sh auto-mint and
  export SATELLITE_PROVIDER_API_KEY using JWT_SECRET / JWT_ISSUER
  / JWT_AUDIENCE env vars (no host port mappings; e2e-runner
  reaches SP via internal docker network only).

Spec amendment: AZ-777 todo spec updated to record the
Google Maps imagery source decision and STOP-gate state.

AZ-777 Phase 3+ work is superseded by Epic AZ-835 (see next
commit).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 17:39:21 +03:00
Oleksandr Bezdieniezhnykh 811b04e605 [AZ-777] Phase 1: wire e2e-runner to real satellite-provider + C11 contract adapt
Adapt C11 HttpTileDownloader to the AZ-505 v1.0.0 tile-inventory
contract (POST /api/satellite/tiles/inventory + GET /tiles/{z}/{x}/{y})
and wire the Jetson e2e harness against the real parent-suite
satellite-provider service. Closes Phase 1 of 5 for AZ-777; STOP
gate before Phase 2 (Derkachi catalog seed).

C11 changes:
- _LIST_PATH / _GET_PATH replaced with _INVENTORY_PATH + _TILES_PATH.
- _do_enumerate enumerates bbox tile coords client-side and posts
  chunked inventory requests (5000-entry cap per the contract).
- _download_one_tile parses tile_id_str into (z,x,y) and fetches
  the slippy-map URL.
- Common GET / POST retry+auth ladder consolidated into _send_request.
- New module helpers: _enumerate_bbox_tile_coords,
  _tile_center_latlon, _tile_size_meters_at, _format_tile_id_str,
  _parse_tile_id_str, _chunk_iter.
- _DEFAULT_ESTIMATED_TILE_BYTES (50 KiB) replaces the inventory-side
  estimatedBytes field the v1.0.0 contract dropped.

Tests:
- 14/14 unit tests in tests/unit/c11_tile_manager/test_tile_downloader.py
  rewritten for the new POST inventory + slippy-map GET handler.
  _StubTileWriter rekeyed by call-index (the downloader now derives
  lat/lon from the slippy-map coord, so fixtures can't fabricate
  arbitrary positions).
- New Tier-2 smoke at tests/e2e/satellite_provider/test_smoke.py:
  validates inventory POST schema + drives HttpTileDownloader against
  the real service. Gated by RUN_REPLAY_E2E=1 + tier2.

Compose / env:
- e2e-runner SATELLITE_PROVIDER_URL switched from mock-sat:5100 to
  https://satellite-provider:8080; TLS_INSECURE + Bearer JWT env +
  depends_on satellite-provider added.
- .env.test.example documents SATELLITE_PROVIDER_API_KEY + dev TLS
  bypass security note.
- scripts/mint_dev_jwt.py mints HS256 dev JWTs from env / .env.test.
- pyjwt added to dev extras.

Tracker hygiene:
- AZ-777 row in _dependencies_table.md bumped 5pt -> 8pt to match
  the 2026-05-21 override decision log.

Code review: PASS_WITH_WARNINGS (3 medium/low findings, all deferred
to later AZ-777 phases) -- see batch_104_review.md. Batch report at
batch_104_cycle3_report.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 14:52:39 +03:00
Oleksandr Bezdieniezhnykh 21a7784682 [AZ-701] Fix Jetson e2e harness infrastructure blockers
- gtsam_isam2_estimator: shim for gtsam>=4.3a0 aarch64 pre-release
  where IncrementalFixedLagSmoother/FixedLagSmootherKeyTimestampMap
  moved from gtsam_unstable to gtsam
- inference_factory: eager import of c7_inference package so
  register_component_block runs before config.components is read
- docker-compose.test.jetson.yml: remove companion and
  operator-orchestrator (not needed by replay CLI tests and crash
  in test env due to AZ-618 live-mode deps); add db-migrate and
  tile-init setup-profile services for Alembic migrations and FAISS
  fixture provisioning; update e2e-runner depends_on to db only
- scripts/mk_test_faiss_fixture.py: generate minimal HNSW32 FAISS
  descriptor index into the tile-data volume for the test harness

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 19:01:36 +03:00
Oleksandr Bezdieniezhnykh a7b3e60716 [autodev] Update Jetson test environment and satellite-provider integration
ci/woodpecker/push/02-build-push Pipeline failed
- Added `.env.test` to `.gitignore` to exclude test environment variables.
- Enhanced `docker-compose.test.jetson.yml` to include the real satellite-provider .NET service and its PostgreSQL database, replacing the mock service.
- Updated test execution policy to mandate all tests run exclusively on Jetson hardware, deprecating the previous two-tier model.
- Revised documentation in `_docs/LESSONS.md`, `_docs/02_document/tests/environment.md`, and `_docs/04_deploy/ci_cd_pipeline.md` to reflect the new testing strategy and environment setup.
- Improved `run-tests-jetson.sh` script to ensure proper environment variable handling and satellite-provider integration.

This commit aligns the testing framework with production environments, enhancing reliability and coverage.
2026-05-20 13:22:51 +03:00
Oleksandr Bezdieniezhnykh bf13549b32 [autodev] Update configuration and documentation for cycle-1
ci/woodpecker/push/02-build-push Pipeline failed
- Enhanced `.env.example` with detailed CMake build flags and replay-mode strategy flags for development and CI environments.
- Updated `.gitignore` to include a new deploy rollback bookmark.
- Revised `_docs/_autodev_state.md` to reflect the current task status and steps.
- Added new lessons to `_docs/LESSONS.md` regarding testing and architectural improvements.
- Documented changes in `_docs/02_document/deployment/ci_cd_pipeline.md` to reflect the relaxed OpenCV version pin.
- Updated test data documentation in `_docs/02_document/tests/test-data.md` to clarify fixture usage and paths.

This commit continues the cycle-1 documentation sync and addresses various configuration updates for improved clarity and functionality.
2026-05-20 08:05:35 +03:00
Oleksandr Bezdieniezhnykh b7012d2787 [AZ-615] run-tests-jetson: resolve ~ before quoted heredoc cd
REMOTE_DIR defaults to ~/gps-denied-onboard. rsync expands the
leading tilde server-side, but the later 'bash -s <<EOF' heredoc
embeds the value literally inside cd "$REMOTE_DIR" -- and bash does
NOT expand ~ inside double quotes, so the heredoc step bails out
with 'No such file or directory'. Resolve any leading ~ against the
remote $HOME up-front so the value is safe to double-quote in both
contexts.

The previous successful Jetson runs (tasks 2388 / 915484) were
one-off ssh commands that never hit this code path; this commit
makes the script actually work end-to-end.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 09:04:43 +03:00
Oleksandr Bezdieniezhnykh d62df9ad15 [AZ-615] run-tests-jetson: BSD rsync compat (no --info=progress2)
macOS ships BSD rsync, which doesn't support GNU's --info=progress2.
Drop the flag (added --stats so we still get a summary at the end)
and document the LFS-pointer pre-smudge requirement that bit during
the first end-to-end attempt.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 07:46:44 +03:00
Oleksandr Bezdieniezhnykh 9c13ab3bd0 [AZ-615] [AZ-617] Add Jetson e2e harness + tier2 marks
C7 inference (PytorchFp16Runtime / TensorRTRuntime / OnnxTrtEpRuntime)
is CUDA-only by design — `model.half().cuda()` is hard-wired with no
CPU fallback. The Colima/Tier-1 smoke harness can never exercise C3
matcher or C7 inference. Once AZ-614 fixes the tlog time-base mismatch
and the pipeline reaches those stages, Colima runs would hard-fail at
`.cuda()` instead of cleanly skipping.

This commit lays down the Jetson companion harness and wires the
existing `tier2` auto-skip:

  * tests/e2e/Dockerfile.jetson  — l4t-pytorch:r36.4.0-pth2.3-py3 base,
    same /opt layout as the Colima image so AC-4 AST scan + bind mounts
    work identically. Built ON the Jetson via run-tests-jetson.sh.
  * docker-compose.test.jetson.yml — mirrors docker-compose.test.yml
    but with `runtime: nvidia`, GPU device exposure, and
    GPS_DENIED_TIER=2 (turns OFF the tier2 auto-skip).
  * scripts/run-tests-jetson.sh — rsync → ssh build → ssh up,
    exit-code-from e2e-runner so the local exit code reflects the
    remote test verdict. No credentials in the repo; uses
    `ssh jetson-e2e` alias resolved via ~/.ssh/config.
  * _docs/03_implementation/jetson_harness_setup.md — one-time SSH
    key + alias + sshd hardening + GPU verification steps. Documents
    the smoke vs. Reality Gate split + the GPS_DENIED_TIER switch.

AZ-617 (mark heavy ACs with tier2): adds @pytest.mark.tier2 to AC-1,
AC-2, AC-3, AC-5, AC-6 in tests/e2e/replay/test_derkachi_1min.py.
Reuses the existing tier2 marker + auto-skip in tests/conftest.py
(scope revision documented as a comment on AZ-617). AC-4a/4b/AC-7/AC-9
stay unmarked — they don't touch CUDA.

Defers to follow-up Jira:

  * AZ-614 — Derkachi tlog synth time-base mismatch (unblocks tier2 ACs
    actually reaching the GPU stage on the Jetson)
  * AZ-616 — replace mock-sat with real ../satellite-provider service

Not run yet: the harness needs operator-side SSH setup to come online
before scripts/run-tests-jetson.sh can be executed end-to-end. Setup
steps documented in jetson_harness_setup.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 01:57:23 +03:00
Oleksandr Bezdieniezhnykh b12db61444 [AZ-263] Bootstrap: repo skeleton + Docker + CI + Alembic + Tier-1 tests
Implements the AZ-263 / E-BOOT initial structure task:

- Python src/-layout package `gps_denied_onboard/` with per-component
  interface stubs (14 components), type-only DTOs under `_types/`,
  shared helpers under `helpers/` (R14 LightGlue ownership), structured
  JSON logging, runtime composition root with env-var fail-fast gate,
  healthcheck module shared by Docker and CI smoke.
- CMake top-level + `cmake/{build_options,dependencies,strategies}.cmake`
  with the BUILD_* per-binary flags (ADR-002) and pinned external git
  refs for OKVIS2 / VINS-Mono / GTSAM / FAISS / OpenCV >=4.12.0.
- Three Dockerfiles (companion-tier1, operator-tooling,
  mock-suite-sat-service) + two compose files (dev + Tier-1 test).
- Four GitHub Actions workflows: ci.yml (lint/unit/integration/dual
  binary build/SBOM diff/security), ci-tier2.yml (self-hosted Jetson
  AC-bound NFTs), release.yml, cve-rescan.yml.
- Two CI gate scripts: `ci/sbom_diff.py` (deployment SBOM subset +
  R02 exclusion), `ci/opencv_pin_gate.py` (>=4.12.0 enforcement,
  D-CROSS-CVE-1).
- Alembic-driven Postgres 16 initial migration `0001_initial.py`
  mirroring satellite-provider tiles + flights + sector_classifications
  + manifests + engine_cache_entries (data_model.md s 2).
- Tier-1 test scaffolding: 95 passing unit tests covering every AC,
  per-component smoke tests, structured logging JSON output check,
  env-var gate check, healthcheck import check. Two CI-gated tests
  (cmake configure, actionlint) skip locally with explicit reasons.
- Batch report + code review report under `_docs/03_implementation/`.

Verdict: PASS_WITH_WARNINGS (two Low findings, both informational).
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 01:00:28 +03:00