satellite-provider/_docs/06_metrics/retro_2026-05-12_cycle3.md

# Retrospective — Cycle 3 (2026-05-12)

**Tasks**: AZ-491 (consolidate JWT test-mint helpers, 3 SP) + AZ-492 (perf harness PT-07 + PT-08 + JWT-attach, 5 SP) + AZ-493 (integration test DB reset hook, 3 SP) + AZ-494 (JWT iss/aud validation, 3 SP, Option B) + AZ-495 (doc folder convention, 2 SP) + AZ-496 (bump AspNetCore 8.0.25, 2 SP)
**Mode**: cycle-end (autodev Step 17)
**Previous retro**: `retro_2026-05-11_cycle2.md`

## 1. Implementation Metrics

| Metric | Cycle 3 | Δ vs cycle 2 |
|--------|---------|--------------|
| Tasks implemented | **6** (AZ-491, AZ-492, AZ-493, AZ-494, AZ-495, AZ-496) | +4 |
| Batches executed | **5** | +3 |
| Avg tasks / batch | 1.2 | -1.3 (cycle 2 = 1.0 tasks/batch but only 2 batches) |
| Total complexity delivered | **18 SP** | +8 (cycle 2 = 10 SP) |
| Avg complexity / batch | 3.6 SP | -1.4 |
| Tasks at-or-below 5 SP cap | **6 of 6** (100%) | +4 (cycle 2 = 1 of 2) |
| Tasks above cap | 0 | -1 (cycle 2 had AZ-488 at 8 SP) |
| Cumulative reviews | **2** (after batches 01-03, after batches 04-05) | +1 (cycle 2 had 0) |

**Sequencing**: batches ordered to respect Cycle-2 retro's Action 1 (Action 1 → AZ-491 first) so subsequent perf and DB-reset work could consume the consolidated factory. AZ-494 was sequenced LAST because it depended on AZ-491's TestSupport + AZ-492's perf-harness env-var pass-through.

## 2. Quality Metrics

| Metric | Cycle 3 | Δ vs cycle 2 |
|--------|---------|--------------|
| Code review pass rate | **5/5 = 100%** (all PASS_WITH_WARNINGS) | unchanged |
| Code review findings — Critical | 0 | unchanged |
| Code review findings — High | 0 | unchanged |
| Code review findings — Medium | 0 | unchanged |
| Code review findings — Low | **7 distinct** (batch-01 F1 + F2 + batch-02 L1 + batch-03 F1 + batch-04 L1 + L2 + batch-05 L1) | +2 |
| Code review FAIL count | 0 | unchanged |
| Cumulative review findings | 4 Low (carry-overs from per-batch) + 1 operational gate (AZ-494 prod iss/aud) | new metric |
| Security audit verdict | **PASS_WITH_WARNINGS** | unchanged |
| Security findings introduced by cycle 3 | 0 new Medium, 3 new Low (D4 test-only, F-DBR-2 test-only, F-PERF-1 operator-CLI), 2 Informational (F-AUTH-3 test-runner log, F-AUTH-4 DEV-ONLY by design), 1 False Positive (F-DBR-1) | **better** (cycle 2 had 2 new Medium + 4 new Low + 1 Info) |
| Security findings RESOLVED by cycle 3 | **F-AUTH-2 (Medium)** by AZ-494; **D1 (Medium)** + **D3 (Low)** by AZ-496 | first cycle with explicit resolved-finding count |

## 3. Structural Metrics (snapshot: `structure_2026-05-12_cycle3.md`)

| Metric | Cycle 3 | Δ vs cycle 2 |
|--------|---------|--------------|
| .NET projects (csproj) | **9** | +1 (new: `SatelliteProvider.TestSupport`) |
| Cross-project edges (ProjectReference) | grew by 2 (TestSupport ← Tests, TestSupport ← IntegrationTests) | +2 |
| Cycles in project graph | 0 | unchanged |
| Public API symbols in TestSupport | 3 (`JwtTokenFactory`, `IntegrationTestResetGuard`, internal helpers) | new |
| New Architecture violations | **0** | unchanged |
| Resolved Architecture violations | F-AUTH-2 (auth contract gap), D1+D3 (supply-chain drift), PT-07/PT-08 deferred-since-cycle-2 | **best metric of cycle** — first net-negative architecture delta cycle |
| Net Architecture delta | **-3** (resolved minus new) | improvement |
| Contract coverage % | unchanged (no new public API surfaces this cycle) | n/a |

A snapshot was written to `_docs/06_metrics/structure_2026-05-12_cycle3.md` (see Step 1 self-verification).

## 4. Efficiency Metrics

| Metric | Cycle 3 | Δ vs cycle 2 |
|--------|---------|--------------|
| Blocked tasks (during implementation) | **1 of 6** — AZ-494 blocked on cross-team input (admin-team iss/aud values), resolved by user choosing Option B (plumbing only, fail-fast at deploy) | new pattern |
| Tasks completed first attempt (no post-review fix commits) | **5 of 6** — only AZ-491 had a follow-on batch-02 fix (`fix:` commit) for the env-var save/restore pattern between iss/aud overrides; the rest landed clean | **best cycle on record** (cycle 1 = 0 of 1; cycle 2 = 0 of 2) |
| Tasks requiring multiple post-code-review fix commits | 0 | -2 (cycle 2 had 2 of 2) |
| Most-findings batch | batch 04 (AZ-492, 2 Low: L1 proxy measurement + L2 dupe JPEG factory) | batch 02 in cycle 2 |
| Cumulative-review-only findings | 0 — every cumulative-review item was already raised at the per-batch level | new metric; the cumulative review acted as confirmation, not discovery |
| Step-15 (Perf Test) execution | **SKIPPED** (user skipped the gate question; recorded as leftover) | unchanged vs cycle 2 (also skipped) |
| Step-14 (Security Audit) — net findings improvement | **+1** (3 Resolved, 3 new Low test-only) | net-zero in cycle 2 |

## 5. Patterns Identified

### Pattern 1 — Action items from prior retros directly drove the cycle scope and DID resolve

Cycle 2 retro identified three Top-3 actions:
- **Action 1** (consolidate JWT test-mint helpers) → drove **AZ-491**. Verdict: **resolved**. The Phase-6 code-review rule added during AZ-491 review (the AZ-491 batch produced both the consolidation AND the rule that prevents recurrence) immediately fired on AZ-492 batch-04's L2 finding (dupe JPEG factory), proving the rule is doing useful work.
- **Action 2** (perf harness work as real task) → drove **AZ-492**. Verdict: **resolved**. PT-07 + PT-08 now runnable; the leftover from cycle 2 was deleted.
- **Action 3** (DB-state reset) → drove **AZ-493**. Verdict: **resolved**. Wall-clock workaround removed; two-guard model unit-tested.

This is the first cycle where retro actions translated directly to next-cycle work AND closed. Pattern is healthy. Keep doing it.

### Pattern 2 — One AC (AZ-494 AC-7) requires cross-repo work; workspace-boundary rule fired correctly

AZ-494 AC-7 says "Suite contract reflects reality" — i.e. update `suite/_docs/10_auth.md`. That file lives in a different workspace. The autodev correctly did NOT write across workspaces; instead, the batch report + cumulative review + deploy report all flagged the cross-repo write as deferred operational work.

**Insight**: ACs that require cross-repo writes should be split — one AC per workspace — and tagged with the workspace they target. Otherwise an AC tagged "deferred" on technicality looks like incomplete work in the traceability matrix when it's actually the right outcome.

### Pattern 3 — Spec-vs-reality drift on measurement / sentinel paths repeated twice

- **AZ-492 L1**: spec said "per-item gate cost < 50 ms" without specifying the measurement path; harness produced a derived proxy because direct measurement requires server-side instrumentation outside AZ-492's scope.
- **AZ-493 F1**: spec prescribed "DB name contains `_test`" as Guard 2; reality uses `Database=satelliteprovider` and rename requires user confirmation; the implementation substituted Host allowlist as an equivalent guard.

Both cases were caught and documented during code review, but the underlying pattern is the same: **ACs that prescribe a specific measurement or sentinel mechanism should also prescribe (or explicitly defer) the path for collecting / enforcing it, so the implementation has a clear bound between "follow the spec" and "substitute an equivalent".**

### Pattern 4 — Option-B forcing functions (fail-fast in prod) elegantly handle cross-team blockers

AZ-494 was blocked on the admin team's iss/aud values. Rather than freeze the work, Option B (user choice) shipped the validation code with empty `appsettings.json` values, guaranteeing prod fails at startup until the real values are supplied. This:
- Unblocks cycle 3 implementation immediately.
- Forces the cross-team conversation to happen before prod deploy, not after.
- Leaves no silently-broken state ("validation off in prod") possible.

This is a generalizable pattern: **for cross-team blockers, prefer a fail-fast scaffold over a defer-the-whole-task decision.** Worth adding as a lesson.

### Pattern 5 — NU1902 warning surfaced AT build, was triaged inline, accepted as cycle-3 D4

The new TestSupport project's `System.IdentityModel.Tokens.Jwt 7.0.3` pin triggered NU1902 (moderate severity) at every restore — visible in the build log 9 times. The batch-02 review (AZ-491) recorded this; the cycle-3 security audit recorded it again as D4; the deploy report carries it as a follow-up.

The path from "compiler warning during test runs" → "tracked security finding with a remediation owner" worked correctly. Worth keeping as a positive example: build-side warnings should ride into the security audit, not be silenced by the agent.

### Pattern 6 — `dotnet format whitespace --verify-no-changes` ran transparently; no format-drift commits

Format drift caused two extra fix commits in cycle 2 (CS0104 fix, JwtFactory net8.0 fix). Cycle 3 had ZERO format-drift fixes — `dotnet format` ran as the first step of `run-tests.sh`, was visible in the log, and all 5 batches passed it without intervention. The earlier hardening (cycle 2 added format check to the test runner) pays off here.

## 6. Comparison vs. previous retro

| Metric                               | Cycle 1        | Cycle 2        | Cycle 3        |
|--------------------------------------|----------------|----------------|----------------|
| Tasks implemented                    | 1              | 2              | 6              |
| Batches                              | 1              | 2              | 5              |
| Critical/High review findings        | 0              | 0              | 0              |
| New Medium review findings           | 0              | 0              | **0**          |
| New Low review findings              | 3              | 6 (5 distinct) | **7**          |
| Code review pass rate                | 100% (1/1)     | 100% (2/2)     | **100% (5/5)** |
| Tasks completed first attempt        | 0 of 1         | 0 of 2         | **5 of 6**     |
| New Medium security findings         | 2              | 2              | **0**          |
| Resolved security findings           | 0              | 0              | **3** (F-AUTH-2, D1, D3) |
| Net Architecture delta               | n/a (baseline) | +0             | **-3**         |
| Step-15 (Perf) executed              | N/A            | SKIPPED        | SKIPPED        |
| Step-15 leftover present at retro    | N/A            | YES (PT-07)    | YES (cycle-3 perf-harness execution) |

### Did the cycle-2 actions land?

- **Cycle 2 Action 1 (consolidate JWT mint helpers)** — landed as **AZ-491**. Phase-6 code-review rule added simultaneously. Rule fired in AZ-492 batch-04 (L2 dupe JPEG factory) as designed. **Verdict**: full implementation + recurrence prevention.
- **Cycle 2 Action 2 (perf harness as real feature)** — landed as **AZ-492**. PT-07 + PT-08 are now runnable. Cycle-2 leftover file deleted. **Verdict**: full implementation. (Execution is a separate cycle-3 leftover, see Action 3 below.)
- **Cycle 2 Action 3 (DB reset between runs)** — landed as **AZ-493**. AZ-488 wall-clock workaround removed; two-guard model in place. **Verdict**: full implementation.

This is the **first cycle in which all prior retro actions translated to closed work**. Cycle 1 actions had mixed outcomes; cycle 2 actions were the input to cycle 3.

## 7. Top 3 Improvement Actions (ranked by impact)

### Action 1 — Execute the cycle-3 perf harness against the deployed `dev` image to convert the cycle-3 perf-execution leftover into PT-07/PT-08 baseline numbers

**Why this is the highest impact**: PT-07 and PT-08 are NOW runnable (AZ-492 closed cycle 2's biggest backlog item). The harness is sitting unexercised because Step 15 was skipped. The next time anyone makes a change that touches the region read or upload paths, the perf gate must compare against *something* — and right now there is no recent baseline.

**Action**: at the start of cycle 4 (or as a one-off "ops" task), run `./scripts/run-performance-tests.sh` against the deployed `dev`-tier image. Record the PT-01..PT-06 / PT-07 / PT-08 numbers in `_docs/06_metrics/perf_<YYYY-MM-DD>.md` (mirrors `perf_2026-05-11_cycle1_az484.md`). Use those numbers as the new baseline.

**Cost**: ~30 minutes (script run + record + delete leftover file). Possibly tracked as a 2-SP cycle-4 PBI rather than ad-hoc.

### Action 2 — Bump `System.IdentityModel.Tokens.Jwt` 7.0.3 → 7.1.2+ (or to 8.0.x) in `SatelliteProvider.TestSupport` to clear the NU1902 build noise

**Why**: 9 NU1902 hits in every test build log degrade the signal-to-noise ratio of CI output and add a low-grade "is this PR adding new vulnerabilities?" cognitive load during review. The fix is one csproj edit; the trade-off is just verifying test compat with the newer version. Test-only finding, NOT production-reachable, but cleaning it up is cheap.

**Action**: 2 SP PBI in cycle 4. Recommended pin: `7.1.2` to stay on the 7.x line that JwtBearer 8.0.25 transitively depends on. Verify all `JwtTokenFactory` unit + integration tests still pass.

### Action 3 — Make "ACs requiring cross-repo writes" structurally visible during task decomposition

**Why**: AZ-494 AC-7 was correctly deferred to a cross-repo write, but the traceability matrix shows AC-7 as `◐ deferred` which looks like missed work. Future cycles will accumulate more cross-repo ACs as the satellite-provider's contract surface grows; without structure, the traceability matrix gradually fills with `◐` markers that mean "different thing each time".

**Action**:
- Add a `workspace:` field to task spec ACs (where applicable): `workspace: satellite-provider` (default) or `workspace: suite`.
- In `new-task/SKILL.md` Step 6 (AC writing), prompt: "if this AC requires changes outside this workspace, tag with `workspace: <name>` and confirm with user that a cross-repo PBI will be filed".
- In the traceability matrix, render cross-workspace ACs in a separate section ("Cross-Workspace Follow-ups") rather than mixed with in-workspace ACs.

**Cost**: 2 SP — small skill rule addition + traceability template update + one-time backfill of AZ-494 AC-7.

## 8. Recommended Rule / Skill updates

1. **Add to `coderule.mdc`**: "When a task's success depends on cross-team input that may not be available in-cycle, prefer an Option-B forcing function (ship the validation/scaffolding with prod-empty config that fails-fast at deploy) over deferring the entire task. The fail-fast contract makes the cross-team conversation impossible to skip." (Justified by Pattern 4.)
2. **Add to `coderule.mdc` (or new `task-spec.mdc`)**: "ACs that prescribe a specific measurement or sentinel mechanism (e.g. 'per-item latency < 50ms', 'guard fires if DB name contains _test') should also prescribe — or explicitly defer — the path for collecting / enforcing it, so the implementation has a clear bound between 'follow the spec' and 'substitute an equivalent.'" (Justified by Pattern 3.)
3. **Update `.cursor/skills/new-task/SKILL.md`** (Step 6): introduce optional `workspace:` field on each AC; warn when an AC implies cross-repo work. (Justified by Action 3.)
4. **Keep the Phase-6 dupe-helper rule** added to `.cursor/skills/code-review/SKILL.md` during AZ-491 review — it fired correctly in AZ-492 batch-04 and is a low-cost / high-signal check. No change needed.

## 9. Decision items carried over (operator)

- **Admin team iss/aud confirmation** (deploy R1 from cycle-3 deploy report) — required before promoting beyond `dev`. Tracked in `deploy_cycle3.md`.
- **Cross-repo doc** (deploy R2 / Pattern 2 / Action 3) — `suite/_docs/10_auth.md` paragraph addition. Tracked in `deploy_cycle3.md`.
- **Cycle-1 hardening backlog** (S1, S2, S4, I1, I3, I5) — pre-public-network items, unchanged. Tracked in cycle-1 retro + `security_report.md`.

## 10. What this retro says about process maturity

Cycle 3 is the first cycle that:
- Closes more security findings than it opens (-3 Medium-or-above).
- Lands all prior-retro action items.
- Has every task at-or-below the 5-SP cap.
- Achieves a >80% first-attempt-completion rate (5/6 = 83%).
- Has zero format-drift fix commits.
- Has zero code-review FAIL verdicts.

The process is converging. The next areas of friction are (a) cross-repo coordination (Action 3), (b) following through on the perf-harness execution that the harness work itself enabled (Action 1), and (c) ongoing test-side supply-chain hygiene (Action 2). None of these are systemic; all are concrete cycle-4 PBI candidates.