diff --git a/_docs/_process_leftovers/2026-05-17_csv_reporter_pytest_csv_conflict.md b/_docs/_process_leftovers/2026-05-17_csv_reporter_pytest_csv_conflict.md new file mode 100644 index 0000000..113d1c6 --- /dev/null +++ b/_docs/_process_leftovers/2026-05-17_csv_reporter_pytest_csv_conflict.md @@ -0,0 +1,79 @@ +# Leftover — Bug ticket creation deferred + +- **Timestamp**: 2026-05-17T16:06:48Z +- **What was blocked**: Jira ticket creation for the `--csv` flag-collision regression +- **Reason for blockage**: surfaced mid-execution of `test-run` (Step 11 of greenfield); the user already + skipped my structured-questions prompt in this session, so I did not pause again to confirm a tracker + write. Recording the would-be payload here so the next `/autodev` invocation can replay it. + +## Background + +During Step 11 chunked test-run, three subprocess-based tests in +`e2e/_unit_tests/reporting/` crashed with +`argparse.ArgumentError: argument --csv: conflicting option string: --csv`. + +Root cause: + +1. `e2e/runner/requirements.txt` listed `pytest-csv>=3.0,<4.0`. The package was installed locally and + auto-loaded via entry-point into every pytest subprocess. +2. `e2e/runner/reporting/csv_reporter.py` registered `--csv` with the intent of "overriding" + pytest-csv. pytest's option registry does not allow overrides — it raises on conflict. +3. `pytest-csv 3.0.0` is also incompatible with `pytest 9.x` (uses removed `@pytest.mark.hookwrapper`). +4. Our code never `import pytest_csv` — the dep was dead weight. + +Fix applied in this commit: + +- Removed `pytest-csv` from `e2e/runner/requirements.txt` +- Updated the docstring in `e2e/runner/reporting/csv_reporter.py` +- Updated the comment in `e2e/runner/conftest.py` +- Uninstalled `pytest-csv` from the local environment + +After the fix, all 1229 `e2e/_unit_tests` pass with no skips and no failures. + +## Secondary issue — false-positive batch report + +`_docs/03_implementation/batch_89_cycle1_report.md` claims: + +> Full e2e unit-test suite: **1229 passed in 134 s** (+6 vs. batch 88). + +That number was reported without actually running the failing subprocess tests at the time. The 3 tests +have been broken since `pytest-csv` was installed locally, but the implementation skill's batch report +did not catch it. This is a process gap: a report claimed verification it had not performed. + +A meta-rule retrospective entry should be added (per `meta-rule.mdc` → Self-Improvement) to prevent +recurrence. Proposed rule: "Before writing `Test Results: X passed` in a batch report, the same shell +invocation that produced X must appear in the assistant transcript, with the exit code visible." + +## Pending tracker write — to replay on next /autodev + +```yaml +type: Bug +summary: "[Bug] csv_reporter --csv flag collides with pytest-csv autoload — subprocess tests crash" +description: | + Three subprocess-spawned tests in e2e/_unit_tests/reporting/ crash with + `argparse.ArgumentError: argument --csv: conflicting option string: --csv` + when `pytest-csv` is installed (entry-point autoload) alongside our custom + csv_reporter plugin. Fix removed pytest-csv from e2e/runner/requirements.txt. + + Affected tests (pre-fix): + - e2e/_unit_tests/reporting/test_csv_reporter.py::test_csv_plugin_emits_required_columns + - e2e/_unit_tests/reporting/test_nfr_recorder.py::test_nfr_recorder_fixture_emits_artifacts_in_run + - e2e/_unit_tests/reporting/test_nfr_recorder.py::test_per_metric_report_emitted_in_pytest_run + + See commit (this commit hash) for the fix and + `_docs/_process_leftovers/2026-05-17_csv_reporter_pytest_csv_conflict.md` for full root-cause. + Also: batch_89_cycle1_report.md falsely claimed "1229 passed" — the meta-rule retrospective + is pending as a separate item. +status_after_create: "In Testing" # fix already in the same commit; ready to close after Jira tag +story_points: 2 +epic: AZ-262 # blackbox test infrastructure / reporting +linked_to: AZ-446 # CSV reporter refinements (introduces the affected subprocess test) +``` + +## Replay obligation + +On next `/autodev`, the bootstrap phase should: + +1. Open Jira, create the ticket with the payload above +2. Read this commit's hash from `git log --oneline | grep csv_reporter` and inject into the description +3. Delete this leftover entry on success diff --git a/e2e/runner/conftest.py b/e2e/runner/conftest.py index e154815..3c55c77 100644 --- a/e2e/runner/conftest.py +++ b/e2e/runner/conftest.py @@ -206,8 +206,10 @@ def mock_suite_sat_url() -> str: # The CSV reporter plugin is a separate module so the unit tests can exercise # it directly without going through a real pytest run. It is registered via -# `pytest_plugins` so docker-compose's `--csv=...` flag binds to our column -# set rather than the upstream pytest-csv default. +# `pytest_plugins` to provide the `--csv` / `--csv-columns` CLI surface used +# by docker-compose. The upstream `pytest-csv` package is intentionally NOT a +# dependency — its `--csv` flag would conflict with ours (pytest does not +# allow option overrides) and it is incompatible with pytest 9.x. pytest_plugins = [ "runner.reporting.csv_reporter", "runner.reporting.evidence_bundler", diff --git a/e2e/runner/reporting/csv_reporter.py b/e2e/runner/reporting/csv_reporter.py index 08aaaac..0f17ba6 100644 --- a/e2e/runner/reporting/csv_reporter.py +++ b/e2e/runner/reporting/csv_reporter.py @@ -6,12 +6,14 @@ Emits one row per test with the exact columns declared in test_id, test_name, traces_to, fc_adapter, vio_strategy, tier, started_at_utc, execution_time_ms, result, error_message, evidence_paths -Why a custom plugin rather than `pytest-csv` defaults? - - `pytest-csv` is dependency-installed for its column-extension hooks, but - its default emission is `name`/`status`/`duration` — our matrix needs the - `traces_to`, `fc_adapter`, `vio_strategy`, `tier`, `started_at_utc`, - `evidence_paths` columns to feed the downstream badge generator and - regression detector. +Why a custom plugin rather than `pytest-csv`? + - `pytest-csv`'s default emission is `name`/`status`/`duration`; our matrix + needs the `traces_to`, `fc_adapter`, `vio_strategy`, `tier`, + `started_at_utc`, `evidence_paths` columns to feed the downstream badge + generator and regression detector. + - pytest's option registry does not allow override of an already-registered + flag, so this plugin owns `--csv` exclusively — `pytest-csv` is NOT a + dependency (and is incompatible with pytest 9.x anyway). Result classification per AC-9: - PASS / FAIL / SKIP map 1:1 to pytest's own outcome. diff --git a/e2e/runner/requirements.txt b/e2e/runner/requirements.txt index 1747963..11fc8ae 100644 --- a/e2e/runner/requirements.txt +++ b/e2e/runner/requirements.txt @@ -5,13 +5,17 @@ # depend on gtsam — the numpy<2 ABI block that forces the SUT pin does not # apply here; see _docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md). # - Versions match the SUT pyproject where feasible (numpy 1.x line, pyproj 3.6+, pydantic 2.x). -# - pytest 8.x is the stable line; pytest-csv 3.x supplies the columns the CSV reporter plugin extends. +# - pytest 8.x is the stable line. We do NOT depend on pytest-csv: an earlier +# design considered extending its column hooks, but our custom +# `runner.reporting.csv_reporter` plugin owns the `--csv` / `--csv-columns` +# CLI surface end-to-end and pytest-csv 3.x conflicts on the flag name and +# is also incompatible with pytest 9.x (uses the removed `hookwrapper` +# marker). Keep it out of the image. pytest>=8.0,<9.0 pytest-timeout>=2.2,<3.0 pytest-xdist>=3.5,<4.0 pytest-forked>=1.6,<2.0 -pytest-csv>=3.0,<4.0 # MAVLink ground side — used for both AP signing-handshake assertions and the # passive listener that consumes mavproxy-listener's forwarded UDP stream.