[autodev] Step 11 outcome — local Tier-1 green, reality gate deferred

Local Tier-1 pytest suite: 3343 pass / 88 skip / 0 fail across 12 chunks.

Docker harness SUT Reality Gate UNMET — both Tier-1 docker harnesses
(scripts/run-tests.sh and e2e/docker/run-tier1.sh) have pre-existing
drift that prevents them from running end-to-end. Findings:

  H-1..H-3 (fixed in 6ce3158): dockerfile rename, fdr-output tmpfs cap,
                               e2e-results bind dir + gitignore.
  H-4..H-6 (deferred): three SITL/MAVLink Docker Hub images don't exist
                       (ardupilot/mavproxy, ardupilot/ardupilot-sitl,
                       inavflight/inav-sitl). environment.md spec was
                       written against aspirational image names.
  H-7..H-8 (deferred): tests/e2e/Dockerfile entrypoint points at empty
                       scenarios dir + doesn't install the SUT package.
  H-9 (deferred): tile-cache-fixture seeder missing (relates to AZ-595).

Plus a regression caught and fixed mid-run: pytest-csv autoload
conflicts with our custom --csv flag (commit eb6dc17). Also surfaced a
false-positive batch-89 test-result report; proposed preventive
meta-rule pending user approval.

Step 11 marked status=blocked pending harness rehabilitation tickets
(payloads recorded in _docs/_process_leftovers/). Full outcome report:
_docs/03_implementation/run_tests_step11_report.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-17 20:30:19 +03:00
parent 6ce31587d4
commit c4e4063650
3 changed files with 301 additions and 3 deletions
@@ -0,0 +1,117 @@
# Leftover — E2E Tier-1 harness rehabilitation tickets deferred
- **Timestamp**: 2026-05-17T17:30:00Z
- **What was blocked**: Jira ticket creation for the harness drift surfaced during Step 11
- **Reason for blockage**: same session as the csv_reporter fix; user skipped the
structured Q&A so I did not pause for tracker writes. Full findings live in
`_docs/03_implementation/run_tests_step11_report.md`; this leftover records the
tickets that need filing.
## Pending tracker writes — replay on next /autodev
### Epic
```yaml
type: Epic
summary: "E2E Tier-1 harness rehabilitation"
description: |
Surfaced during /autodev Step 11 (Run Tests) cycle 1 on 2026-05-17. Both
Tier-1 docker harnesses (top-level scripts/run-tests.sh and the fuller
e2e/docker/run-tier1.sh) had pre-existing drift preventing them from
running end-to-end. Local pytest suite is green (3343/88/0); SUT Reality
Gate is unmet until at least the bootstrap harness can run
tests/e2e/replay/ with RUN_REPLAY_E2E=1. Full report:
_docs/03_implementation/run_tests_step11_report.md
linked_to: AZ-595, AZ-444 # related but distinct: tile-cache fixtures, Tier-2 hw loop
```
### Story: H-7 — Bootstrap runner entrypoint
```yaml
type: Story
summary: "[Bug] tests/e2e/Dockerfile entrypoint points at empty scenarios dir"
description: |
Current entrypoint: `pytest -q /opt/tests/e2e/scenarios` (empty in repo).
Real tests are in `tests/e2e/replay/` (test_derkachi_1min.py, etc.).
Fix: change entrypoint to /opt/tests/e2e/ (let pytest discover both
scenarios and replay).
story_points: 1
```
### Story: H-8 — Install SUT in runner image
```yaml
type: Story
summary: "[Bug] tests/e2e e2e-runner image doesn't install gps-denied-onboard"
description: |
Image is python:3.10-slim with only pytest+requests+pyyaml. The replay
tests need `gps-denied-replay` console script on PATH. Either:
- COPY pyproject.toml + src/ and pip install -e ".[dev]", or
- Build a wheel in a separate stage and pip install it.
Verify the resulting image: `which gps-denied-replay`.
story_points: 3
```
### Story: H-4..H-6 — SITL/MAVLink images choice
```yaml
type: Story
summary: "[Decision] Choose SITL strategy for e2e/docker harness"
description: |
environment.md specifies ardupilot/ardupilot-sitl:plane-stable,
inavflight/inav-sitl:9.0.0, ardupilot/mavproxy:latest. All MISSING from
Docker Hub. Options:
a) Switch to community images (radarku/ardupilot-sitl etc.)
b) Build SITLs from source in a separate stage
c) Strip SITL services and mark SITL-bound scenarios skip(reason="sitl-unavailable")
Track 1 doesn't depend on this; Track 2 does.
story_points: 5
```
### Story: MAVProxy local image
```yaml
type: Story
summary: "[Story] Replace ardupilot/mavproxy:latest with local pip-MAVProxy Dockerfile"
description: |
Image doesn't exist on Docker Hub. Wrap `pip install MAVProxy` in a
python:3.10-slim Dockerfile in e2e/fixtures/mavproxy/. Update compose
to use the local build.
story_points: 1
```
### Story: H-9 — Tile-cache fixture builder
```yaml
type: Story
summary: "Link H-9 to AZ-595 / tile-cache fixture seeder"
description: |
e2e/docker/docker-compose.test.yml declares tile-cache-fixture as an
empty named volume. Track 2 cannot run without seeded tiles. AZ-595
exists and owns this; verify scope alignment, add a link.
story_points: 2
```
### Story: H-1..H-3 — fixes already committed
```yaml
type: Story
summary: "[Bug] e2e/docker harness drift (already fixed in commit 6ce3158)"
description: |
Fixed in this session: dockerfile rename, fdr-output tmpfs cap, e2e-results
dir + gitignore. Ticket is just for tracking — already in dev branch.
story_points: 1
status_after_create: "Done"
```
### Bug: csv_reporter --csv collision (already committed)
```yaml
type: Bug
summary: "[Bug] csv_reporter --csv flag collides with pytest-csv autoload"
description: |
See _docs/_process_leftovers/2026-05-17_csv_reporter_pytest_csv_conflict.md
Fix already in commit eb6dc17.
linked_to: AZ-446
story_points: 2
status_after_create: "Done"
```
## Replay obligation
Next /autodev should:
1. Open Jira, create the Epic + Stories above (link Epic to AZ-595 and AZ-444).
2. Update the Epic with the actual issue keys once created.
3. Delete this leftover entry on success.