[AZ-899] [AZ-900] [AZ-901] Baseline doc + retro gate + EVIDENCE_OUT fix

AZ-899: create _docs/02_document/architecture_compliance_baseline.md
seeded with 0 violations and the 2026-05-20 structural snapshot facts
(15 inventory entries, 0 import cycles, 5 contract files). Documents
the append-on-violation / mark-resolved-on-fix / snapshot-refresh
protocol so cumulative reviews can emit Baseline Delta sections.
Closes cycle-1 retro Top-3 #3 (third attempt).

AZ-900: codify LESSONS 2026-05-26 [process] in
.cursor/skills/autodev/flows/existing-code.md - Re-Entry After
Completion now hosts a Previous-Cycle Retro Existence Gate that
BLOCKS the cycle increment if no _docs/06_metrics/retro_*.md file
dated within [cycle_start, cycle_end] exists. Skipped on
state.cycle == 1. Presents Choose A (author retro) / B (stub +
leftover) / C (abort). state.md - Session Boundaries gains a
cross-reference bullet.

AZ-901: fix e2e/runner/conftest.py:56 EVIDENCE_OUT default - host
pytest now resolves <repo_root>/e2e-results/evidence/ instead of
/e2e-results/evidence (container-only path; crashed on macOS / non-
root Linux). Docker + Jetson harnesses unaffected (they pass
--evidence-out explicitly). Verified locally: 24 SKIPPED, exit 0,
evidence written. Closes leftover 2026-05-26_evidence_out_default_path.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-26 17:18:54 +03:00
parent 940066bee2
commit aa8b9f2ee9
8 changed files with 190 additions and 54 deletions
@@ -0,0 +1,78 @@
# Land `architecture_compliance_baseline.md` (cycle-3 retro #3, third try)
**Task**: AZ-899_architecture_compliance_baseline
**Name**: Create `_docs/02_document/architecture_compliance_baseline.md` so cumulative reviews can emit `## Baseline Delta` rows
**Description**: Cycle-1 retro Top-3 Improvement Action #3, repeated in cycle-3 retro Top-3 #3. The file has been unmade across cycles 2 and 3, leaving cumulative reviews unable to quantify carried-over / resolved / newly-introduced architecture violations per cycle. Seed the baseline from `_docs/06_metrics/structure_2026-05-20.md` with `0` violations, freeze the snapshot semantics, and wire the existing-code flow's Step 2 to reference it.
**Complexity**: 1 SP
**Dependencies**: None (operates on existing artifact `_docs/06_metrics/structure_2026-05-20.md`)
**Component**: documentation only — no source code change
**Tracker**: AZ-899 (https://denyspopov.atlassian.net/browse/AZ-899)
**Epic**: (none — cycle-4 process housekeeping)
## Problem
Cycle-3 retro § Structural Metrics:
> `_docs/02_document/architecture_compliance_baseline.md` **still does not exist** — cycle-1 retro Top-3 Improvement Action #3 was NOT delivered in cycles 2 or 3.
Without a baseline, cumulative reviews log "`_docs/02_document/architecture_compliance_baseline.md` does NOT exist → no Baseline Delta section emitted". Structural regressions (new cycles in the import graph, newly-introduced violations) therefore cannot be quantified across cycles — only verified pairwise per batch.
## Outcome
- Cumulative-review reports starting from cycle-4 batch 1 emit a `## Baseline Delta` section that quantifies new vs. resolved vs. carried-over architecture violations.
- Cycle-end retros can compare structural deltas across cycles using a single canonical baseline document instead of re-deriving from the previous cycle's snapshot.
## Scope
### Included
- Create `_docs/02_document/architecture_compliance_baseline.md` seeded with **0** violations.
- Reference `_docs/06_metrics/structure_2026-05-20.md` as the source-of-truth snapshot from which the baseline was derived.
- Document the file's update protocol: a new violation found in a cumulative review is appended (with batch ID, severity, finding ID); a resolution is recorded by marking the row `RESOLVED in batch <ID>`.
- Document the snapshot-refresh trigger: any cycle that materially changes structure (component count, cross-component edges, new contracts) re-snapshots via `python -m gps_denied_onboard.tools.structure_snapshot` (or equivalent existing script — verify before reference).
### Excluded
- Refactoring source code to fix violations — none currently exist.
- Adding new component scaffolding — out of scope.
- Modifying `code-review` or `retrospective` skills — they already reference the file; the only change needed is making the referenced file exist.
## Acceptance Criteria
**AC-1: Baseline file exists with 0 violations**
Given a fresh repo checkout
When `ls _docs/02_document/architecture_compliance_baseline.md` runs
Then the file exists and its `## Violations` section is explicitly empty (or marked "None at baseline")
**AC-2: Baseline references the structural snapshot**
Given the baseline file
When read
Then it includes a `## Source` section pointing at `_docs/06_metrics/structure_2026-05-20.md` and lists the structural facts (15 components, 0 import cycles, 5 contract files) that establish the "0 violations" claim
**AC-3: Update protocol documented**
Given the baseline file
When read
Then it includes an `## Update Protocol` section describing append-on-violation, mark-resolved-on-fix, and the snapshot-refresh trigger
**AC-4: Cumulative-review hook verified**
Given the baseline file in place
When the cycle-4 first cumulative-review report is generated
Then the report emits a `## Baseline Delta` section (even if empty: "0 new, 0 resolved, 0 carried-over")
## Constraints
- File format: markdown, matches the structure of `_docs/06_metrics/structure_2026-05-20.md` style.
- No source code change permitted under this ticket — strictly documentation.
## Risks & Mitigation
**Risk 1: Future violations slip past the baseline**
- *Risk*: A cumulative review finds a violation but the reviewer forgets to append it to the baseline.
- *Mitigation*: The `code-review` skill (referenced in cycle-3 retro Suggested Updates) should be updated separately to auto-append; this ticket only delivers the baseline file. The follow-up belongs in cycle 5 if needed.
## References
- Cycle-3 retro: `_docs/06_metrics/retro_2026-05-26.md` § Top 3 Improvement Actions #3
- Cycle-1 retro: `_docs/06_metrics/retro_2026-05-20.md` § Top 3 Improvement Actions #3 (original)
- Source snapshot: `_docs/06_metrics/structure_2026-05-20.md`
- Existing-code flow Step 2: `.cursor/skills/autodev/flows/existing-code.md` § "Step 2 — Architecture Baseline Scan"
@@ -0,0 +1,82 @@
# Autodev: gate Step-9 entry on previous-cycle retro existence
**Task**: AZ-900_autodev_retro_existence_gate
**Name**: Codify the LESSONS rule — autodev must block cycle-N+1 Step 9 entry if `retro_<YYYY-MM-DD>.md` for cycle N is absent
**Description**: Cycle-3 retro Top-3 Improvement Action #2 and 2026-05-26 LESSONS entry both call for codifying a Re-Entry After Completion gate that verifies the previous cycle's retro file exists before incrementing the cycle counter. Cycle-2 retro was never filed; the orchestrator silently advanced to cycle 3 and all cycle-1 retro Top-3 actions sat invisible. This ticket codifies the gate in `.cursor/skills/autodev/flows/existing-code.md` § Re-Entry After Completion.
**Complexity**: 1 SP
**Dependencies**: None
**Component**: `.cursor/skills/autodev/flows/existing-code.md` (workflow doc only)
**Tracker**: AZ-900 (https://denyspopov.atlassian.net/browse/AZ-900)
**Epic**: (none — cycle-4 process housekeeping)
## Problem
LESSONS 2026-05-26 [process] entry:
> Cycle-2 retro was never filed. The autodev orchestrator silently auto-chained from cycle-2 Step 17 (if it ran at all) straight into cycle-3 Step 9 without producing `retro_<cycle2-date>.md`. As a result, cycle-1 retro's Top-3 Improvement Actions sat invisible across cycle 2 and were re-discovered, all three still undelivered, only at cycle-3 close.
Cycle-3 retro Top-3 #2 echoes the same recommendation.
The fix is a one-line check in the flow file that BLOCKS Step 9 entry for cycle N+1 unless `_docs/06_metrics/retro_<YYYY-MM-DD>.md` for cycle N exists.
## Outcome
- Future cycle-N → cycle-(N+1) transitions are gated: the autodev orchestrator refuses to enter Step 9 of cycle N+1 if no retro file exists for cycle N.
- Missing retros are surfaced at the session boundary, not 6 weeks later at the next cycle's close.
## Scope
### Included
- Edit `.cursor/skills/autodev/flows/existing-code.md` § "Re-Entry After Completion" to add a gate: before incrementing `cycle`, glob `_docs/06_metrics/retro_*.md` and verify a file dated after the cycle-N start exists.
- Define the BLOCK behavior: if absent, present a Choose A/B/C block:
- **A)** Author the missing retro now (invoke `.cursor/skills/retrospective/SKILL.md` in cycle-end mode)
- **B)** Stub a backfilled retro and proceed (with a leftover entry filed for proper backfill)
- **C)** Abort and ask the user
- Add a corresponding bullet to `.cursor/skills/autodev/state.md` § "Session Boundaries" pointing at the new gate.
### Excluded
- Retroactively writing cycle-2 retro (separate ticket if user wants it; cycle-3 retro already covers cycle-2 trend deltas where data is on disk).
- Adding similar gates to greenfield or meta-repo flows (only `existing-code` has the cycle counter).
- Per-step retro check inside cycles (this gate fires only at the cycle boundary).
## Acceptance Criteria
**AC-1: Flow file gate exists**
Given `.cursor/skills/autodev/flows/existing-code.md`
When the "Re-Entry After Completion" section is read
Then it contains a step `Verify previous cycle's retro exists` BEFORE the cycle increment
**AC-2: Choose A/B/C block specified**
Given the gate triggers (no retro file found)
When the documented behavior is consulted
Then it specifies the three options (A: author now, B: stub + leftover, C: abort) with the standard Choose format
**AC-3: state.md cross-reference**
Given `.cursor/skills/autodev/state.md`
When the "Session Boundaries" section is read
Then it mentions the new retro-existence gate or links to the flow file's gate
**AC-4: Discovery rule**
Given the gate
When the file pattern is documented
Then the glob is unambiguous: `_docs/06_metrics/retro_*.md` with a date matching cycle-N's date range; the date-range derivation is explicit (cycle N start = last `implementation_report_*_cycle{N-1}.md` date; cycle N end = today)
## Constraints
- Pure workflow doc change — no source code, no tests.
- Must not break the existing greenfield-Done → existing-code Phase-B transition (greenfield → existing-code is a one-shot flow change with no retro requirement on first entry, since there is no previous cycle).
## Risks & Mitigation
**Risk 1: False positive on greenfield→existing-code transition**
- *Risk*: First cycle of an existing-code flow shouldn't require a previous-cycle retro.
- *Mitigation*: Gate condition includes `state.cycle > 1` — cycle 1 has no previous cycle.
## References
- LESSONS 2026-05-26 [process] entry: `_docs/LESSONS.md` § 2026-05-26 [process]
- Cycle-3 retro Top-3 #2: `_docs/06_metrics/retro_2026-05-26.md`
- Flow file: `.cursor/skills/autodev/flows/existing-code.md` § "Re-Entry After Completion"
- State management: `.cursor/skills/autodev/state.md` § "Session Boundaries"
@@ -0,0 +1,85 @@
# Fix `EVIDENCE_OUT` default path — workspace-relative, not container-only
**Task**: AZ-901_evidence_out_default_path_fix
**Name**: Change `e2e/runner/conftest.py:56` `EVIDENCE_OUT` default from `/e2e-results/evidence` to a workspace-relative path so Tier-1 host runs don't crash
**Description**: Closes leftover `_docs/_process_leftovers/2026-05-26_evidence_out_default_path.md`. Cycle-3 Step 15 (Performance Test) surfaced this: the default path `/e2e-results/evidence` is the container mount inside the Tier-1 Docker harness; a developer Mac/Linux workstation invoking `python -m pytest e2e/tests/performance/` directly hits `OSError: [Errno 30] Read-only file system: '/e2e-results'` (macOS) or `PermissionError` (Linux). Workaround today: `EVIDENCE_OUT="$(pwd)/e2e-results/..." pytest ...`. Fix: resolve a workspace-relative default when neither `--evidence-out` nor `EVIDENCE_OUT` is set.
**Complexity**: 1 SP
**Dependencies**: None
**Component**: `e2e/runner/conftest.py`
**Tracker**: AZ-901 (https://denyspopov.atlassian.net/browse/AZ-901)
**Epic**: (none — cycle-4 process housekeeping)
## Problem
`e2e/runner/conftest.py:56`:
```python
default=os.environ.get("EVIDENCE_OUT", "/e2e-results/evidence")
```
The default `/e2e-results/evidence` is a container-mount path. Tier-1 Docker harness and the Tier-2 Jetson runner pass `--evidence-out` explicitly, so they're fine. Host-direct `python -m pytest e2e/tests/performance/` invocations (developer machine, no Docker) hit `nfr_recorder.pytest_sessionfinish` which tries `mkdir(evidence_dir)` and crashes.
## Outcome
- Developer can run `python -m pytest e2e/tests/performance/` on a Mac/Linux workstation without setting `EVIDENCE_OUT` and without crashing.
- Docker / Jetson runners continue to work unchanged (they pass `--evidence-out` explicitly).
## Scope
### Included
- Modify `e2e/runner/conftest.py:56` to resolve a workspace-relative default when `EVIDENCE_OUT` is unset.
- Proposed: `default=os.environ.get("EVIDENCE_OUT", str(Path(__file__).resolve().parents[2] / "e2e-results" / "evidence"))`
- Verify Docker compose files and Jetson scripts that pass `--evidence-out` still work (they should — they override the default).
- Verify `.gitignore` ignores `e2e-results/` at repo root (probably already does — confirm before commit).
- Delete the leftover file `_docs/_process_leftovers/2026-05-26_evidence_out_default_path.md` once the fix lands and the verification AC passes.
### Excluded
- The "lazy fallback inside the recorder" alternative shape — staying with the workspace-relative-default shape for simplicity (Option 1 from the leftover file).
- Refactoring `nfr_recorder.pytest_sessionfinish` — the writer code is fine; only the default path is wrong.
- Adding new evidence-out related env vars or CLI flags.
## Acceptance Criteria
**AC-1: Host-direct pytest works without EVIDENCE_OUT**
Given a clean workspace on macOS or Linux
When `python -m pytest e2e/tests/performance/ -v --tb=short` runs (no `EVIDENCE_OUT` env var, no `--evidence-out` flag)
Then pytest exits 0, evidence is written under `<workspace_root>/e2e-results/evidence/`, and no `OSError` / `PermissionError` is raised
**AC-2: Docker harness unchanged**
Given the Tier-1 Docker compose (`docker-compose.test.jetson.yml`)
When the e2e suite runs inside the container
Then `--evidence-out` is still passed and evidence lands at the container mount path `/e2e-results/evidence/` (no behavioral change)
**AC-3: Jetson harness unchanged**
Given `scripts/run-tests-jetson.sh`
When invoked
Then it still passes `--evidence-out` to pytest and evidence is collected per the existing protocol
**AC-4: gitignore covers workspace-relative path**
Given the fix in place
When a host-direct run produces `<workspace_root>/e2e-results/`
Then `git status` does NOT show `e2e-results/` as untracked (already covered by `.gitignore`, or `.gitignore` is updated as part of this ticket)
**AC-5: Leftover deleted**
Given the fix lands and ACs 14 pass
When `ls _docs/_process_leftovers/2026-05-26_evidence_out_default_path.md`
Then the file does not exist
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|-------------|-----------------|
| AC-1 | Run `pytest e2e/tests/performance/` without env vars on host | Exit 0, evidence at `<workspace_root>/e2e-results/evidence/` |
## Constraints
- Backward-compatible — existing callers passing `--evidence-out` or setting `EVIDENCE_OUT` see no change.
- No new dependencies; uses `pathlib.Path` which `conftest.py` already imports (verify before commit).
## References
- Leftover file: `_docs/_process_leftovers/2026-05-26_evidence_out_default_path.md`
- Cycle-3 Step 15 perf report: `_docs/06_metrics/perf_2026-05-26_cycle3-tier1-probe.md` § "Findings worth tracking" item 3
- Conftest: `e2e/runner/conftest.py:56`