2 Commits

Author SHA1 Message Date
Oleksandr Bezdieniezhnykh d2e50489df Merge branch 'stage' into main
ci/woodpecker/push/build-arm Pipeline was successful
2026-04-22 01:40:07 +03:00
Oleksandr Bezdieniezhnykh 132b1f4279 Merge branch 'dev' into stage
ci/woodpecker/push/build-arm Pipeline was successful
2026-04-22 01:37:06 +03:00
152 changed files with 433 additions and 9805 deletions
-1
View File
@@ -39,7 +39,6 @@ alwaysApply: true
- When you think you are done with changes, run the full test suite. Every failure in tests that cover code you modified or that depend on code you modified is a **blocking gate**. For pre-existing failures in unrelated areas, report them to the user but do not block on them. Never silently ignore or skip a failure without reporting it. On any blocking failure, stop and ask the user to choose one of:
- **Investigate and fix** the failing test or source code
- **Remove the test** if it is obsolete or no longer relevant
- **Iterative-skill exception**: when an iterative loop skill is active (e.g. autodev / `implement/SKILL.md` batch loop, `refactor/SKILL.md` batch loop), the skill governs full-suite cadence — typically focused tests per task/batch and a single full-suite gate at the very end of the implementation phase, NOT after each batch. "Done with changes" means done with the entire implementation phase the skill is running, not done with one batch. Do not run the full suite per batch unless the skill explicitly says to.
- Do not rename any databases or tables or table columns without confirmation. Avoid such renaming if possible.
- Make sure we don't commit binaries, create and keep .gitignore up to date and delete binaries after you are done with the task
-41
View File
@@ -1,41 +0,0 @@
---
description: "Use chunked writes (Write + StrReplace marker pattern) for large generated files, especially after a monolithic Write fails"
alwaysApply: true
---
# Large File Writes — Chunk on Failure
When a `Write` call to a single file fails (timeout, payload limit, "Invalid arguments", or any tool error) and the intended content is large (>~500 lines or >~50 KB), do NOT retry the same monolithic Write. Switch to chunked writes:
1. **First Write** — create the file with header + table of contents (if applicable) + an explicit append marker, e.g.
```
<!-- INSERTION_POINT do-not-remove-until-final-chunk -->
```
2. **Each subsequent chunk** — use `StrReplace` to replace the marker with `<new content>\n<marker>` so the marker stays at the end. This is idempotent: if a chunk fails, retry it without losing earlier chunks.
3. **Final chunk** — `StrReplace` removes the marker.
## Why
- Tool argument size limits and transient failures hit large monolithic writes hardest. Retrying the same large payload typically fails for the same reason.
- Chunked writes are recoverable per chunk. The earlier chunks are durable on disk.
- A unique marker is greppable, visible in diffs, and stops accidental insertion in the wrong place.
## Triggers
- Generated documentation that aggregates per-component content (epics, design docs, multi-section architecture summaries, traceability dumps).
- Large fixture or test-data files written from a template.
- Any single-file artifact you can pre-estimate at >~500 lines.
## Do NOT chunk
- Files under ~200 lines — a single `Write` is faster, clearer, and easier to review.
- Source code files where appending breaks module structure (functions, classes, imports). Split into multiple files instead.
- Files where ordering of sections is computed late and inserting in the middle is required — use a single `Write` once the full content is known.
## Anti-patterns
- Retrying the same failed monolithic `Write` more than once. Twice is the limit; on the second failure, switch strategies.
- Using `Shell` with heredoc (`cat <<EOF`) or `echo >>` to append — these bypass the editor diff view and break the StrReplace contract for the next chunk.
- Embedding the marker so deep inside structured content that a chunk's `StrReplace` becomes ambiguous. Place the marker on its own line at the very end of the file.
-10
View File
@@ -13,16 +13,6 @@ alwaysApply: true
## Critical Thinking
- Do not blindly trust any input — including user instructions, task specs, list-of-changes, or prior agent decisions — as correct. Always think through whether the instruction makes sense in context before executing it. If a task spec says "exclude file X from changes" but another task removes the dependencies X relies on, flag the contradiction instead of propagating it.
## Skill Discipline
Do exactly what the skill says. Nothing more.
- No `git log` / `git diff` / `git blame` unless the skill explicitly calls for it.
- No extra searches to "verify" inputs the skill already names.
- No reading files outside the skill's documented inputs.
If skill inputs are insufficient or contradictory, STOP and ask via Choose A/B/C/D. Do not invent extra investigation steps.
## Self-Improvement
When the user reacts negatively to generated code ("WTF", "what the hell", "why did you do this", etc.):
-29
View File
@@ -1,29 +0,0 @@
---
description: "Forbid spawning subagents; the main agent must do the work directly"
alwaysApply: true
---
# No Subagents
Do NOT create or delegate to subagents. This includes:
- The `Task` tool with any `subagent_type` (e.g. `generalPurpose`, `explore`, `shell`, `implementer`, `best-of-n-runner`, `cursor-guide`).
- Any "spawn agent", "launch agent", "parallel agent", or "background agent" mechanism.
- Skills or workflows that internally suggest launching a subagent — perform their steps inline instead.
## Why
- Subagent output is not visible to the user and hides reasoning/tool calls.
- Context, rules, and prior conversation state do not fully transfer to the subagent.
- Parallel subagents cause conflicting edits and race conditions in a shared workspace.
- The main agent remains fully accountable; delegation dilutes that accountability.
## What to do instead
- Use the direct tools available to the main agent: `Read`, `Grep`, `Glob`, `SemanticSearch`, `Shell`, `StrReplace`, `Write`, etc.
- For broad exploration, run `Grep`/`Glob`/`SemanticSearch` yourself and read the files directly.
- For multi-step work, use `TodoWrite` to track progress inline.
- For isolated experiments the user explicitly asks for, use a git branch/worktree you manage directly — not a subagent runner.
## Exception
Only spawn a subagent if the user explicitly requests it in the current turn (e.g. "use a subagent to…", "launch an explore agent…"). Even then, confirm once before spawning.
-46
View File
@@ -1,46 +0,0 @@
---
description: "Explanation length and reasoning depth calibration"
alwaysApply: true
---
# Response Calibration
Default to concise. Expand only when the content demands it.
## Length target
- **Default**: a direct answer in ~310 lines. Short paragraphs or a tight bullet list.
- **Expand when**: the question involves trade-offs across multiple options, a migration/architectural decision, a security/data-loss risk, or the user explicitly asks for depth ("explain in detail", "walk me through", "why").
- **Shrink when**: the user asks for "shorter", "simpler", "TL;DR", "one line", or similar. Do not re-inflate in later turns unless they ask a new deeper question.
## Completeness floor
Short ≠ incomplete. Every response must still:
- Answer the actual question asked (not a reframed version).
- State the key constraint or reason *once*, not repeatedly.
- Flag a real caveat if one exists (data loss, breaking change, wrong-OS, security). One sentence is enough.
- Not drop a step from an action sequence. If there are 5 steps, list 5 — but without narration between them.
If the honest answer truly needs more space (e.g. trade-off matrix, multi-option decision), write more — but lead with the recommendation or direct answer, then the detail.
## Structure
- One direct sentence first. Then supporting detail.
- Prefer bullets over prose for enumerations, comparisons, or step lists.
- Drop section headers for anything under ~15 lines.
- No "Summary" / "Conclusion" sections unless the response is genuinely long.
## Reasoning depth (internal)
- Match thinking to the problem, not the length of the answer.
- Factual / "where is X used" / single-file edit → minimal thinking, go straight to tools.
- Trade-off / refactor / debugging 3+ hypotheses deep → full thinking budget.
- Do not pad thinking to look thorough. Do not skip thinking on genuinely ambiguous problems to look fast.
## Anti-patterns to avoid
- Restating the question back to the user.
- Multi-paragraph preambles before the answer.
- Exhaustive "alternatives considered" sections when the user didn't ask for alternatives.
- Recapping what was just done at the end of every tool-using turn ("Done. I have edited the file…") — a one-line confirmation is enough.
- Speculative "you might also want to…" paragraphs. Offer follow-ups as a single short sentence, or not at all.
-38
View File
@@ -1,38 +0,0 @@
---
description: "Standards for creating and maintaining Cursor skills"
globs: [".cursor/skills/**"]
---
# Skill Building
## When To Create A Skill
- Create a skill for repeatable, bounded workflows that benefit from a reusable process.
- Do not create a skill for a one-off task, vague goal, or workflow that still needs product decisions.
- Start small; evolve the skill when repeated use reveals clearer steps, constraints, or checks.
## Skill Contract
- `SKILL.md` must define a clear `name` and a proactive `description` that explains when the skill should be used.
- State expected inputs, constraints, workflow steps, and final output shape.
- Make trigger conditions explicit enough that the agent can recognize intent without an exact command.
- Base instructions on observable project evidence; do not invite fabrication or unsupported assumptions.
## Keep The Core Lean
- Keep `SKILL.md` concise and under the repo's `.cursor/` size guidance.
- Move detailed standards, examples, and background knowledge into `references/`.
- Put reusable output shapes in `templates/` or other skill-local assets instead of embedding them in the main instructions.
- Keep one primary responsibility per skill; use an orchestrator skill only when multiple existing skills must run in a defined order.
## Deterministic Work
- Use scripts for mechanical steps that are repeatable, parameterized, and safer outside the model's reasoning.
- Scripts must expose explicit inputs, avoid hidden side effects, and fail loudly on errors.
- Do not use scripts to bypass review, hide destructive behavior, or hardcode secrets.
## Quality Proof
- Include realistic examples, checklists, or eval-style scenarios that define what good output looks like.
- Cover common failure cases such as missing sections, leftover placeholders, hallucinated facts, unsafe actions, or malformed output.
- Review skill changes against those checks before treating the skill as ready.
## Security Review
- Treat third-party skills like untrusted code until reviewed.
- Inspect scripts, dependencies, references, secret handling, network calls, and destructive commands before use.
- Prefer local, project-scoped assets and dependencies; document any external dependency the skill requires.
+3 -6
View File
@@ -14,14 +14,11 @@ alwaysApply: true
- Issue types: Epic, Story, Task, Bug, Subtask
## Tracker Availability Gate
- If Jira MCP returns **Unauthorized**, **errored**, **connection refused**, **timeout**, a non-2xx status code, an empty body, or any response shape that does not clearly confirm the requested change: **STOP IMMEDIATELY** — no automatic retry, no silent continuation. Surface the full raw error/response to the user verbatim and notify via the Choose A/B/C/D format documented in `.cursor/skills/autodev/protocols.md`.
- A minimal `{"success": true}` body with no echoed issue state is NOT a confirmed transition. When a transition's success matters (status moves, ticket creation, blocking link), follow it with a read-back call (`getJiraIssue` or equivalent) and confirm the new state matches what you asked for. If the read-back disagrees → STOP and ASK.
- Do NOT loop "retry up to N times before asking". One call, one verification. On failure, the user decides whether to retry.
- If Jira MCP returns **Unauthorized**, **errored**, **connection refused**, or any non-success response: **STOP** tracker operations and notify the user via the Choose A/B/C/D format documented in `.cursor/skills/autodev/protocols.md`.
- The user may choose to:
- **Retry the same operation** — once, after the user authorizes it. If it fails again, surface both responses.
- **Retry authentication** — preferred when the failure looks like an auth/credentials problem; the tracker remains the source of truth.
- **Retry authentication** — preferred; the tracker remains the source of truth.
- **Continue in `tracker: local` mode** — only when the user explicitly accepts this option. In that mode all tasks keep numeric prefixes and a `Tracker: pending` marker is written into each task header. The state file records `tracker: local`. The mode is NOT silent — the user has been asked and has acknowledged the trade-off.
- Do NOT auto-fall-back to `tracker: local` without a user decision. Do not pretend a write succeeded. Do not paper over an opaque response by moving on. If the user is unreachable (e.g., non-interactive run), stop and wait.
- Do NOT auto-fall-back to `tracker: local` without a user decision. Do not pretend a write succeeded. If the user is unreachable (e.g., non-interactive run), stop and wait.
- When the tracker becomes available again, any `Tracker: pending` tasks should be synced — this is done at the start of the next `/autodev` invocation via the Leftovers Mechanism below.
## Leftovers Mechanism (non-user-input blockers only)
+4 -14
View File
@@ -3,7 +3,7 @@ name: autodev
description: |
Auto-chaining orchestrator that drives the full BUILD-SHIP workflow from problem gathering through deployment.
Detects current project state from _docs/ folder, resumes from where it left off, and flows through
problem → research → plan → test specs → decompose → implement → tests → docs sync → deploy without manual skill invocation.
problem → research → plan → decompose → implement → deploy without manual skill invocation.
Maximizes work per conversation by auto-transitioning between skills.
Trigger phrases:
- "autodev", "auto", "start", "continue"
@@ -52,7 +52,7 @@ Determine which flow to use (check in order — first match wins):
After selecting the flow, apply its detection rules (first match wins) to determine the current step.
**Note**: the meta-repo flow uses a different artifact layout — its source of truth is `_docs/_repo-config.yaml`, not `_docs/NN_*/` folders. After Step 2.5 it also produces `_docs/glossary.md` and a `## Architecture Vision` section in the cross-cutting architecture doc identified by `docs.cross_cutting`. Other detection rules assume the BUILD-SHIP artifact layout; they don't apply to meta-repos.
**Note**: the meta-repo flow uses a different artifact layout — its source of truth is `_docs/_repo-config.yaml`, not `_docs/NN_*/` folders. Other detection rules assume the BUILD-SHIP artifact layout; they don't apply to meta-repos.
## Execution Loop
@@ -67,9 +67,8 @@ B3. Read state — `_docs/_autodev_state.md` (if it exists).
B4. Read File Index — `state.md`, `protocols.md`, and the active flow file.
### Resolve (once per invocation, after Bootstrap)
R1. Reconcile state — verify state file against `_docs/` contents; probe `<workspace-root>/../docs`
(parent suite `docs/` — see `state.md` → "State File Rules" #4); on disagreement,
trust the folders and update the state file (rules: `state.md` → "State File Rules" #4).
R1. Reconcile state — verify state file against `_docs/` contents; on disagreement, trust the folders
and update the state file (rules: `state.md` → "State File Rules" #4).
After this step, `state.step` / `state.status` are authoritative.
R2. Resolve flow — see §Flow Resolution above.
R3. Resolve current step — when a state file exists, `state.step` drives detection.
@@ -113,15 +112,6 @@ Do NOT modify, skip, or abbreviate any part of the sub-skill's workflow. The aut
The state file (`_docs/_autodev_state.md`) is a minimal pointer — only the current step. See `state.md` for the authoritative template, field semantics, update rules, and worked examples. Do not restate the schema here — `state.md` is the single source of truth.
**Conciseness rule (authoritative).** The state file MUST stay short. Acceptable content per field:
- `name` — the step title from the active flow's Step Reference Table. That's it.
- `sub_step.name` — kebab-case identifier from the active sub-skill. That's it.
- `sub_step.detail`**leave empty (`""`) by default.** Add a one-line note ONLY when the next-session resumer cannot infer where to pick up from `phase` + `name` + on-disk artifacts alone (e.g. `"batch 2 of 4"`, `"blocked on D-PROJ-2 reply"`, `"variant 1b"`). NEVER use `detail` as a changelog, recap, or summary of completed work — those facts belong in the relevant `_docs/` artifact (glossary, traceability matrix, leftovers folder, retro report, etc.) and in git history.
- **Total file size target: <30 lines.** If you're tempted to write more, you're using the wrong artifact — write in `_docs/` instead.
Multi-line `detail` blobs that recap what was just completed are a smell. The state file is a *pointer*, not a logbook.
## Trigger Conditions
This skill activates when the user wants to:
@@ -13,7 +13,7 @@ A first-time run executes Phase A then Phase B; every subsequent invocation re-e
| Step | Name | Sub-Skill | Internal SubSteps |
|------|------|-----------|-------------------|
| 1 | Document | document/SKILL.md | Steps 07 incl. inline 2.5 (module-layout) and 4.5 (glossary + arch vision) |
| 1 | Document | document/SKILL.md | Steps 18 |
| 2 | Architecture Baseline Scan | code-review/SKILL.md (baseline mode) | Phase 1 + Phase 7 |
| 3 | Test Spec | test-spec/SKILL.md | Phases 14 |
| 4 | Code Testability Revision | refactor/SKILL.md (guided mode) | Phases 07 (conditional) |
@@ -53,8 +53,6 @@ Action: An existing codebase without documentation was detected. Read and execut
The document skill's Step 2.5 produces `_docs/02_document/module-layout.md`, which is required by every downstream step that assigns file ownership (`/implement` Step 4, `/code-review` Phase 7, `/refactor` discovery). If this file is missing after Step 1 completes (e.g., a pre-existing `_docs/` dir predates the 2.5 addition), re-invoke `/document` in resume mode — it will pick up at Step 2.5.
The document skill's Step 4.5 produces `_docs/02_document/glossary.md` and prepends a confirmed `## Architecture Vision` section to `architecture.md`. Both are user-confirmed artifacts; downstream skills (refactor, decompose, new-task) treat them as authoritative for terminology and structural intent. If `glossary.md` is missing after Step 1 (pre-existing `_docs/` dir from before the 4.5 addition), re-invoke `/document` in resume mode — it will pick up at Step 4.5 without redoing module/component analysis.
---
**Step 2 — Architecture Baseline Scan**
@@ -152,17 +150,15 @@ If `_docs/02_tasks/` subfolders have some task files already (e.g., refactoring
---
**Step 6 — Implement Tests**
Condition (folder fallback): `_docs/02_tasks/todo/` contains test task files AND `_dependencies_table.md` exists AND `_docs/03_implementation/implementation_report_tests.md` does not exist.
Condition (folder fallback): `_docs/02_tasks/todo/` contains task files AND `_dependencies_table.md` exists AND `_docs/03_implementation/implementation_report_tests.md` does not exist.
State-driven: reached by auto-chain from Step 5.
Action: Invoke `.cursor/skills/implement/SKILL.md` with task selection context **Test implementation**.
Action: Read and execute `.cursor/skills/implement/SKILL.md`
The implement skill reads only test tasks from `_docs/02_tasks/todo/` and implements them.
The implement skill reads test tasks from `_docs/02_tasks/todo/` and implements them.
If `_docs/03_implementation/` has batch reports, the implement skill detects completed tasks and continues.
For folder fallback, **test task files** means `*_test_infrastructure.md` plus task specs whose `**Component**` or `**Epic**` identifies `Blackbox Tests`.
---
**Step 7 — Run Tests**
+65 -217
View File
@@ -1,6 +1,6 @@
# Greenfield Workflow
Workflow for new projects built from scratch. Flows linearly: Problem → Research → Plan → UI Design (if applicable) → Test Spec → Decompose → Implement + Product Completeness Gate → Code Testability Revision → Decompose Tests → Implement Tests → Run Tests → Test-Spec Sync → Update Docs → Security Audit (optional) → Performance Test (optional) → Deploy → Retrospective.
Workflow for new projects built from scratch. Flows linearly: Problem → Research → Plan → UI Design (if applicable) → Decompose → Implement → Run Tests → Security Audit (optional) → Performance Test (optional) → Deploy → Retrospective.
## Step Reference Table
@@ -10,19 +10,13 @@ Workflow for new projects built from scratch. Flows linearly: Problem → Resear
| 2 | Research | research/SKILL.md | Mode A: Phase 14 · Mode B: Step 08 |
| 3 | Plan | plan/SKILL.md | Step 16 + Final |
| 4 | UI Design | ui-design/SKILL.md | Phase 08 (conditional — UI projects only) |
| 5 | Test Spec | test-spec/SKILL.md | Phases 14 |
| 6 | Decompose | decompose/SKILL.md (implementation task decomposition) | Step 1 + Step 1.5 + Step 2 + Step 4 |
| 7 | Implement | implement/SKILL.md | Batch loop + Product Implementation Completeness Gate |
| 8 | Code Testability Revision | refactor/SKILL.md (guided mode) | Phases 07 (conditional) |
| 9 | Decompose Tests | decompose/SKILL.md (tests-only) | Step 1t + Step 3 + Step 4 |
| 10 | Implement Tests | implement/SKILL.md | (batch-driven, no fixed sub-steps) |
| 11 | Run Tests | test-run/SKILL.md | Steps 14 |
| 12 | Test-Spec Sync | test-spec/SKILL.md (cycle-update mode) | Phase 2 + Phase 3 (scoped) |
| 13 | Update Docs | document/SKILL.md (task mode) | Task Steps 05 |
| 14 | Security Audit | security/SKILL.md | Phase 15 (optional) |
| 15 | Performance Test | test-run/SKILL.md (perf mode) | Steps 15 (optional) |
| 16 | Deploy | deploy/SKILL.md | Step 17 |
| 17 | Retrospective | retrospective/SKILL.md (cycle-end mode) | Steps 14 |
| 5 | Decompose | decompose/SKILL.md | Step 14 |
| 6 | Implement | implement/SKILL.md | (batch-driven, no fixed sub-steps) |
| 7 | Run Tests | test-run/SKILL.md | Steps 14 |
| 8 | Security Audit | security/SKILL.md | Phase 15 (optional) |
| 9 | Performance Test | test-run/SKILL.md (perf mode) | Steps 15 (optional) |
| 10 | Deploy | deploy/SKILL.md | Step 17 |
| 11 | Retrospective | retrospective/SKILL.md (cycle-end mode) | Steps 14 |
## Detection Rules
@@ -86,12 +80,12 @@ If `_docs/02_document/` exists but is incomplete (has some artifacts but no `FIN
---
**Step 4 — UI Design (conditional)**
Condition (folder fallback): `_docs/02_document/architecture.md` exists AND `_docs/02_document/tests/traceability-matrix.md` does not exist.
Condition (folder fallback): `_docs/02_document/architecture.md` exists AND `_docs/02_tasks/todo/` does not exist or has no task files.
State-driven: reached by auto-chain from Step 3.
Action: Read and execute `.cursor/skills/ui-design/SKILL.md`. The skill runs its own **Applicability Check**, which handles UI project detection and the user's A/B choice. It returns one of:
- `outcome: completed` → mark Step 4 as `completed`, auto-chain to Step 5 (Test Spec).
- `outcome: completed` → mark Step 4 as `completed`, auto-chain to Step 5 (Decompose).
- `outcome: skipped, reason: not-a-ui-project` → mark Step 4 as `skipped`, auto-chain to Step 5.
- `outcome: skipped, reason: user-declined` → mark Step 4 as `skipped`, auto-chain to Step 5.
@@ -99,162 +93,34 @@ The autodev no longer inlines UI detection heuristics — they live in `ui-desig
---
**Step 5 — Test Spec**
Condition (folder fallback): `_docs/02_document/FINAL_report.md` exists AND `_docs/02_document/architecture.md` exists AND `_docs/02_document/tests/traceability-matrix.md` does not exist.
State-driven: reached by auto-chain from Step 4 (completed or skipped).
**Step 5 — Decompose**
Condition: `_docs/02_document/` contains `architecture.md` AND `_docs/02_document/components/` has at least one component AND `_docs/02_tasks/todo/` does not exist or has no task files
Action: Read and execute `.cursor/skills/test-spec/SKILL.md`.
This step converts the greenfield problem statement, acceptance criteria, solution, architecture, component docs, and UI design artifacts (if any) into test specifications before implementation begins. The test spec should cover unit, integration, blackbox, and e2e scenarios where those levels are applicable to the project.
---
**Step 6 — Decompose**
Condition: `_docs/02_document/` contains `architecture.md` AND `_docs/02_document/components/` has at least one component AND `_docs/02_document/tests/traceability-matrix.md` exists AND `_docs/02_tasks/todo/` does not exist or has no implementation task files.
Action: Invoke `.cursor/skills/decompose/SKILL.md` for **implementation task decomposition**. The greenfield flow selects the implementation entrypoint before handing off: Bootstrap Structure, Module Layout, Component Task Decomposition, and Cross-Task Verification.
Do not invoke Blackbox Test Task Decomposition from Step 6. Test tasks are intentionally deferred to Step 9 (Decompose Tests) so the first implementation batch stays focused on product functionality and Step 8 can revise testability before test task files exist.
Action: Read and execute `.cursor/skills/decompose/SKILL.md`
If `_docs/02_tasks/` subfolders have some task files already, the decompose skill's resumability handles it.
---
**Step 7 — Implement**
Condition: `_docs/02_tasks/todo/` contains implementation task files AND `_dependencies_table.md` exists AND `_docs/03_implementation/` does not contain a valid product implementation report.
**Step 6 — Implement**
Condition: `_docs/02_tasks/todo/` contains task files AND `_dependencies_table.md` exists AND `_docs/03_implementation/` does not contain any `implementation_report_*.md` file
Action: Invoke `.cursor/skills/implement/SKILL.md` with task selection context **Product implementation**.
The implement skill must run its **Product Implementation Completeness Gate** before it writes any final product implementation report. This gate compares completed product task specs, architecture/component promises, and actual source code so scaffold-only implementations cannot advance to Step 8. A final product implementation report without `_docs/03_implementation/implementation_completeness_cycle[N]_report.md` is incomplete and must not be treated as Step 7 completion.
Action: Read and execute `.cursor/skills/implement/SKILL.md`
If `_docs/03_implementation/` has batch reports, the implement skill detects completed tasks and continues. The FINAL report filename is context-dependent — see implement skill documentation for naming convention.
For folder fallback, **implementation task files** means task specs that are not test-only specs: exclude `*_test_infrastructure.md` and task specs whose `**Component**` or `**Epic**` identifies `Blackbox Tests`.
For folder fallback, a **product implementation report** is any `_docs/03_implementation/implementation_report_*.md` file except `_docs/03_implementation/implementation_report_tests.md` and refactor reports. It is valid for greenfield progression only when:
- the matching `_docs/03_implementation/implementation_completeness_cycle[N]_report.md` exists,
- that completeness report does not contain unresolved `FAIL` classifications, and
- `_docs/02_tasks/todo/` contains no pending implementation task files.
If a product report exists but any of those validity checks fail, treat product implementation as incomplete and stay in Step 7.
---
**Step 8Code Testability Revision**
Condition (folder fallback): `_docs/03_implementation/` contains a valid product implementation report, `_docs/03_implementation/implementation_completeness_cycle[N]_report.md` exists without unresolved `FAIL` classifications, `_docs/04_refactoring/01-testability-refactoring/testability_assessment.md` does not exist, `_docs/04_refactoring/01-testability-refactoring/testability_changes_summary.md` does not exist, `_docs/03_implementation/implementation_report_tests.md` does not exist, and `_docs/02_tasks/todo/` does not contain test task files.
State-driven: reached by auto-chain from Step 7.
**Purpose**: verify the newly built code can be exercised by the planned tests before writing the test suite. Greenfield code should be testable by design; this step catches accidental hardcoded paths, singletons, direct external service construction, or other implementation choices that would make meaningful tests impossible.
**Scope — MINIMAL, SURGICAL fixes**: this is not a general refactor. It is the smallest set of changes required to make the implemented code runnable under tests.
**Allowed changes** in this phase:
- Replace hardcoded URLs / file paths / credentials / magic numbers with env vars or constructor arguments.
- Extract narrow interfaces for components that need stubbing in tests.
- Add optional constructor parameters for dependency injection; default to the existing behavior so callers do not break.
- Wrap global singletons in thin accessors that tests can override.
- Split a function ONLY when necessary to stub one of its collaborators — do not split for clarity alone.
**NOT allowed** in this phase (defer to a later refactor task):
- Renaming public APIs.
- Moving code between files unless strictly required for isolation.
- Changing algorithms or business logic.
- Restructuring module boundaries or rewriting layers.
Action: Analyze the codebase against the test specs to determine whether the code can be tested as-is.
1. Read `_docs/02_document/tests/traceability-matrix.md` and all test scenario files in `_docs/02_document/tests/`.
2. For each test scenario, check whether the code under test can be exercised in isolation. Look for:
- Hardcoded file paths or directory references
- Hardcoded configuration values (URLs, credentials, magic numbers)
- Global mutable state that cannot be overridden
- Tight coupling to external services without abstraction
- Missing dependency injection or non-configurable parameters
- Direct file system operations without path configurability
- Inline construction of heavy dependencies (models, clients)
3. If ALL scenarios are testable as-is:
- Create `_docs/04_refactoring/01-testability-refactoring/`
- Write `_docs/04_refactoring/01-testability-refactoring/testability_assessment.md` with the scenarios reviewed and outcome "Code is testable — no changes needed"
- Mark Step 8 as `completed` with outcome "Code is testable — no changes needed"
- Auto-chain to Step 9 (Decompose Tests)
4. If testability issues are found:
- Create `_docs/04_refactoring/01-testability-refactoring/`
- Write `list-of-changes.md` in that directory using the refactor skill template (`.cursor/skills/refactor/templates/list-of-changes.md`), with:
- **Mode**: `guided`
- **Source**: `autodev-greenfield-testability-analysis`
- One change entry per testability issue found (change ID, file paths, problem, proposed change, risk, dependencies). Each entry must fit the allowed-changes list above; reject entries that drift into full refactor territory and log them under "Deferred refactor candidates" instead.
- Invoke the refactor skill in **guided mode**: read and execute `.cursor/skills/refactor/SKILL.md` with the `list-of-changes.md` as input
- Phase 3 (Safety Net) is skipped for this testability run because the test suite has not been implemented yet
- After execution, surface `RUN_DIR/testability_changes_summary.md` to the user via the Choose format (accept / request follow-up) before auto-chaining
- Copy or save the accepted summary as `_docs/04_refactoring/01-testability-refactoring/testability_changes_summary.md` so folder fallback can detect Step 8 completion
- Mark Step 8 as `completed`
- Auto-chain to Step 9 (Decompose Tests)
---
**Step 9 — Decompose Tests**
Condition (folder fallback): `_docs/02_document/tests/traceability-matrix.md` exists AND workspace contains source code files AND `_docs/03_implementation/` contains a valid product implementation report AND `_docs/03_implementation/implementation_completeness_cycle[N]_report.md` exists without unresolved `FAIL` classifications AND (`_docs/04_refactoring/01-testability-refactoring/testability_assessment.md` exists OR `_docs/04_refactoring/01-testability-refactoring/testability_changes_summary.md` exists) AND (`_docs/02_tasks/todo/` does not exist or has no test task files) AND `_docs/03_implementation/implementation_report_tests.md` does not exist.
State-driven: reached by auto-chain from Step 8.
Action: Read and execute `.cursor/skills/decompose/SKILL.md` in **tests-only mode** (pass `_docs/02_document/tests/` as input). The decompose skill will:
1. Run Step 1t (test infrastructure bootstrap)
2. Run Step 3 (blackbox/e2e-capable test task decomposition)
3. Run Step 4 (cross-verification against test coverage)
If `_docs/02_tasks/` subfolders have some task files already, the decompose skill's resumability handles it — it appends test tasks alongside existing completed implementation tasks.
---
**Step 10 — Implement Tests**
Condition (folder fallback): `_docs/02_tasks/todo/` contains test task files AND `_dependencies_table.md` exists AND `_docs/03_implementation/implementation_report_tests.md` does not exist.
State-driven: reached by auto-chain from Step 9.
Action: Invoke `.cursor/skills/implement/SKILL.md` with task selection context **Test implementation**.
The implement skill reads only test tasks from `_docs/02_tasks/todo/` and implements them.
If `_docs/03_implementation/` has batch reports, the implement skill detects completed test tasks and continues.
For folder fallback, **test task files** means `*_test_infrastructure.md` plus task specs whose `**Component**` or `**Epic**` identifies `Blackbox Tests`.
---
**Step 11 — Run Tests**
Condition (folder fallback): `_docs/03_implementation/implementation_report_tests.md` exists.
State-driven: reached by auto-chain from Step 10.
**Step 7Run Tests**
Condition (folder fallback): `_docs/03_implementation/` contains an `implementation_report_*.md` file.
State-driven: reached by auto-chain from Step 6.
Action: Read and execute `.cursor/skills/test-run/SKILL.md`
Verifies the implemented unit, integration, blackbox, and e2e tests pass before proceeding to spec and documentation sync. This is a hard product gate, not a harness-smoke gate: e2e/blackbox tests must exercise the actual implemented system through public runtime boundaries and compare actual outputs against `_docs/00_problem/input_data/expected_results/results_report.md` or referenced machine-readable expected-result files. Stubs are allowed only for external systems outside the product boundary; missing internal product implementation must fail or block the gate and send the flow back to Implement.
---
**Step 12Test-Spec Sync**
State-driven: reached by auto-chain from Step 11. Requires `_docs/02_document/tests/traceability-matrix.md` to exist — if missing, mark Step 12 `skipped` (see Action below).
Action: Read and execute `.cursor/skills/test-spec/SKILL.md` in **cycle-update mode**. Pass the completed implementation task specs, completed test task specs, and implementation reports as inputs.
The skill appends implementation-learned acceptance criteria, scenarios, and NFR updates to the existing test-spec files without rewriting unaffected sections. If `traceability-matrix.md` is missing, mark Step 12 as `skipped` — the next `/test-spec` full run will regenerate it.
After completion, auto-chain to Step 13 (Update Docs).
---
**Step 13 — Update Docs**
State-driven: reached by auto-chain from Step 12 (completed or skipped). Requires `_docs/02_document/` to contain existing documentation — if missing, mark Step 13 `skipped` (see Action below).
Action: Read and execute `.cursor/skills/document/SKILL.md` in **Task mode**. Pass all completed implementation and test task spec files plus the implementation reports.
The document skill in Task mode updates affected module docs, component docs, system-level docs, and test documentation without redoing full discovery, verification, or problem extraction.
If `_docs/02_document/` does not contain existing docs, mark Step 13 as `skipped`.
After completion, auto-chain to Step 14 (Security Audit).
---
**Step 14 — Security Audit (optional)**
State-driven: reached by auto-chain from Step 13 (completed or skipped).
**Step 8Security Audit (optional)**
State-driven: reached by auto-chain from Step 7.
Action: Apply the **Optional Skill Gate** (`protocols.md` → "Optional Skill Gate") with:
- question: `Run security audit before deploy?`
@@ -262,12 +128,12 @@ Action: Apply the **Optional Skill Gate** (`protocols.md` → "Optional Skill Ga
- option-b-label: `Skip — proceed directly to deploy`
- recommendation: `A — catches vulnerabilities before production`
- target-skill: `.cursor/skills/security/SKILL.md`
- next-step: Step 15 (Performance Test)
- next-step: Step 9 (Performance Test)
---
**Step 15 — Performance Test (optional)**
State-driven: reached by auto-chain from Step 14 (completed or skipped).
**Step 9 — Performance Test (optional)**
State-driven: reached by auto-chain from Step 8.
Action: Apply the **Optional Skill Gate** (`protocols.md` → "Optional Skill Gate") with:
- question: `Run performance/load tests before deploy?`
@@ -275,30 +141,30 @@ Action: Apply the **Optional Skill Gate** (`protocols.md` → "Optional Skill Ga
- option-b-label: `Skip — proceed directly to deploy`
- recommendation: `A or B — base on whether acceptance criteria include latency, throughput, or load requirements`
- target-skill: `.cursor/skills/test-run/SKILL.md` in **perf mode** (the skill handles runner detection, threshold comparison, and its own A/B/C gate on threshold failures)
- next-step: Step 16 (Deploy)
- next-step: Step 10 (Deploy)
---
**Step 16 — Deploy**
State-driven: reached by auto-chain from Step 15 (after Step 15 is completed or skipped).
**Step 10 — Deploy**
State-driven: reached by auto-chain from Step 9 (after Step 9 is completed or skipped).
Action: Read and execute `.cursor/skills/deploy/SKILL.md`.
After the deploy skill completes successfully, mark Step 16 as `completed` and auto-chain to Step 17 (Retrospective).
After the deploy skill completes successfully, mark Step 10 as `completed` and auto-chain to Step 11 (Retrospective).
---
**Step 17 — Retrospective**
State-driven: reached by auto-chain from Step 16.
**Step 11 — Retrospective**
State-driven: reached by auto-chain from Step 10.
Action: Read and execute `.cursor/skills/retrospective/SKILL.md` in **cycle-end mode**. This closes the cycle's feedback loop by folding metrics into `_docs/06_metrics/retro_<date>.md` and appending the top-3 lessons to `_docs/LESSONS.md`.
After retrospective completes, mark Step 17 as `completed` and enter "Done" evaluation.
After retrospective completes, mark Step 11 as `completed` and enter "Done" evaluation.
---
**Done**
State-driven: reached by auto-chain from Step 17. (Sanity check: `_docs/04_deploy/` should contain all expected artifacts — containerization.md, ci_cd_pipeline.md, environment_strategy.md, observability.md, deployment_procedures.md, deploy_scripts.md.)
State-driven: reached by auto-chain from Step 11. (Sanity check: `_docs/04_deploy/` should contain all expected artifacts — containerization.md, ci_cd_pipeline.md, environment_strategy.md, observability.md, deployment_procedures.md, deploy_scripts.md.)
Action: Report project completion with summary. Then **rewrite the state file** so the next `/autodev` invocation enters the feature-cycle loop in the existing-code flow:
@@ -325,65 +191,47 @@ On the next invocation, Flow Resolution rule 1 reads `flow: existing-code` and r
| Research (2) | Auto-chain → Research Decision (ask user: another round or proceed?) |
| Research Decision → proceed | Auto-chain → Plan (3) |
| Plan (3) | Auto-chain → UI Design detection (4) |
| UI Design (4, done or skipped) | Auto-chain → Test Spec (5) |
| Test Spec (5) | Auto-chain → Decompose (6) |
| Decompose (6) | **Session boundary** — suggest new conversation before Implement |
| Implement (7) | Auto-chain only after Product Implementation Completeness Gate passes → Code Testability Revision (8) |
| Code Testability Revision (8) | Auto-chain → Decompose Tests (9) |
| Decompose Tests (9) | **Session boundary** — suggest new conversation before Implement Tests |
| Implement Tests (10) | Auto-chain → Run Tests (11) |
| Run Tests (11, all pass) | Auto-chain → Test-Spec Sync (12) |
| Test-Spec Sync (12, done or skipped) | Auto-chain → Update Docs (13) |
| Update Docs (13, done or skipped) | Auto-chain → Security Audit choice (14) |
| Security Audit (14, done or skipped) | Auto-chain → Performance Test choice (15) |
| Performance Test (15, done or skipped) | Auto-chain → Deploy (16) |
| Deploy (16) | Auto-chain → Retrospective (17) |
| Retrospective (17) | Report completion; rewrite state to existing-code flow, step 9 |
| UI Design (4, done or skipped) | Auto-chain → Decompose (5) |
| Decompose (5) | **Session boundary** — suggest new conversation before Implement |
| Implement (6) | Auto-chain → Run Tests (7) |
| Run Tests (7, all pass) | Auto-chain → Security Audit choice (8) |
| Security Audit (8, done or skipped) | Auto-chain → Performance Test choice (9) |
| Performance Test (9, done or skipped) | Auto-chain → Deploy (10) |
| Deploy (10) | Auto-chain → Retrospective (11) |
| Retrospective (11) | Report completion; rewrite state to existing-code flow, step 9 |
## Status Summary — Step List
Flow name: `greenfield`. Render using the banner template in `protocols.md` → "Banner Template (authoritative)". No header-suffix, current-suffix, or footer-extras — all empty for this flow.
| # | Step Name | Extra state tokens (beyond the shared set) |
|---|-----------------------------|--------------------------------------------|
| 1 | Problem | — |
| 2 | Research | `DONE (N drafts)` |
| 3 | Plan | — |
| 4 | UI Design | — |
| 5 | Test Spec | — |
| 6 | Decompose | `DONE (N tasks)` |
| 7 | Implement | `IN PROGRESS (batch M of ~N)` |
| 8 | Code Testability Revision | — |
| 9 | Decompose Tests | `DONE (N tasks)` |
| 10 | Implement Tests | `IN PROGRESS (batch M)` |
| 11 | Run Tests | `DONE (N passed, M failed)` |
| 12 | Test-Spec Sync | — |
| 13 | Update Docs | — |
| 14 | Security Audit | — |
| 15 | Performance Test | — |
| 16 | Deploy | — |
| 17 | Retrospective | — |
| # | Step Name | Extra state tokens (beyond the shared set) |
|---|--------------------|--------------------------------------------|
| 1 | Problem | — |
| 2 | Research | `DONE (N drafts)` |
| 3 | Plan | — |
| 4 | UI Design | — |
| 5 | Decompose | `DONE (N tasks)` |
| 6 | Implement | `IN PROGRESS (batch M of ~N)` |
| 7 | Run Tests | `DONE (N passed, M failed)` |
| 8 | Security Audit | — |
| 9 | Performance Test | — |
| 10 | Deploy | — |
| 11 | Retrospective | — |
All rows also accept the shared state tokens (`DONE`, `IN PROGRESS`, `NOT STARTED`, `FAILED (retry N/3)`); rows 4, 12, 13, 14, 15 additionally accept `SKIPPED`.
All rows also accept the shared state tokens (`DONE`, `IN PROGRESS`, `NOT STARTED`, `FAILED (retry N/3)`); rows 4, 8, 9 additionally accept `SKIPPED`.
Row rendering format (step-number column is right-padded to 2 characters for alignment):
```
Step 1 Problem [<state token>]
Step 2 Research [<state token>]
Step 3 Plan [<state token>]
Step 4 UI Design [<state token>]
Step 5 Test Spec [<state token>]
Step 6 Decompose [<state token>]
Step 7 Implement [<state token>]
Step 8 Code Testability Rev. [<state token>]
Step 9 Decompose Tests [<state token>]
Step 10 Implement Tests [<state token>]
Step 11 Run Tests [<state token>]
Step 12 Test-Spec Sync [<state token>]
Step 13 Update Docs [<state token>]
Step 14 Security Audit [<state token>]
Step 15 Performance Test [<state token>]
Step 16 Deploy [<state token>]
Step 17 Retrospective [<state token>]
Step 1 Problem [<state token>]
Step 2 Research [<state token>]
Step 3 Plan [<state token>]
Step 4 UI Design [<state token>]
Step 5 Decompose [<state token>]
Step 6 Implement [<state token>]
Step 7 Run Tests [<state token>]
Step 8 Security Audit [<state token>]
Step 9 Performance Test [<state token>]
Step 10 Deploy [<state token>]
Step 11 Retrospective [<state token>]
```
+32 -314
View File
@@ -5,8 +5,7 @@ Workflow for **meta-repositories** — repos that aggregate multiple components
This flow differs fundamentally from `greenfield` and `existing-code`:
- **No problem/research/plan phases** — meta-repos don't build features, they coordinate existing ones
- **No test spec / run tests** — the meta-repo has no code to test
- **`implement` is scoped to suite-level work only** — cross-repo concerns, repo/folder renames, suite-root infra additions (e.g., `.gitmodules`, `_infra/`, suite `e2e/`). Per-component implementation lives in each component's own workspace `/autodev` cycle. The meta-repo's implement step (Step 3.5) executes only when `_docs/tasks/todo/` is non-empty AND the user explicitly opts in; placement is **before** the sync skills so subsequent Doc/E2E/CICD sync propagates the post-implementation state.
- **No test spec / implement / run tests** — the meta-repo has no code to test
- **No `_docs/00_problem/` artifacts** — documentation target is `_docs/*.md` unified docs, not per-feature `_docs/NN_feature/` folders
- **Primary artifact is `_docs/_repo-config.yaml`** — generated by `monorepo-discover`, read by every other step
@@ -16,11 +15,8 @@ This flow differs fundamentally from `greenfield` and `existing-code`:
|------|------|-----------|-------------------|
| 1 | Discover | monorepo-discover/SKILL.md | Phase 110 |
| 2 | Config Review | (human checkpoint, no sub-skill) | — |
| 2.5 | Glossary & Architecture Vision | (inline, no sub-skill) | Steps 15 |
| 3 | Status | monorepo-status/SKILL.md | Sections 15 |
| 3.5 | Suite Implement | implement/SKILL.md (suite-level invocation context) | Steps 114 + 16 (Step 14.5 + Step 15 skipped); conditional on `_docs/tasks/todo/` non-empty AND user opt-in |
| 4 | Document Sync | monorepo-document/SKILL.md | Phase 17 (conditional on doc drift) |
| 4.5 | Integration Test Sync | monorepo-e2e/SKILL.md | Phase 16 (conditional on suite-e2e drift; skipped if `suite_e2e:` block absent in config) |
| 5 | CICD Sync | monorepo-cicd/SKILL.md | Phase 17 (conditional on CI drift) |
| 6 | Loop | (auto-return to Step 3 on next invocation) | — |
@@ -62,121 +58,17 @@ Action: This is a **hard session boundary**. The skill cannot proceed until a hu
══════════════════════════════════════
```
- If user picks A → verify `confirmed_by_user: true` is now set in the config. If still `false`, re-ask. If true, auto-chain to **Step 2.5 (Glossary & Architecture Vision)**.
- If user picks A → verify `confirmed_by_user: true` is now set in the config. If still `false`, re-ask. If true, auto-chain to **Step 3 (Status)**.
- If user picks B → mark Step 2 as `in_progress`, update state file, end the session. Tell the user to invoke `/autodev` again after reviewing.
**Do NOT auto-flip `confirmed_by_user`.** Only the human does that.
---
**Step 2.5 — Glossary & Architecture Vision** (one-shot)
Condition (folder fallback): `_docs/_repo-config.yaml` exists AND `confirmed_by_user: true` AND (`_docs/glossary.md` does NOT exist OR the cross-cutting architecture doc identified in `docs.cross_cutting` does NOT contain a `## Architecture Vision` section).
State-driven: reached by auto-chain from Step 2 (user picked A).
**Goal**: Capture meta-repo-wide terminology and the user's architecture vision **once**, after the config is confirmed but before any sync skill runs. Without this, `monorepo-document` will faithfully propagate per-component changes but never surface a unified mental model of the meta-repo to the user, and the AI will keep re-inferring the same project terminology on every invocation.
**Why inline (no sub-skill)**: `monorepo-discover` is hard-guarded to write only `_repo-config.yaml`; `monorepo-document` only edits *existing* docs. Glossary and architecture-vision creation is a first-time, user-confirmed write that crosses both guarantees, so it lives directly in the flow.
**Inputs**:
- `_docs/_repo-config.yaml` (component list, doc map, conventions, assumptions log)
- Cross-cutting docs listed under `docs.cross_cutting` (existing architecture doc, if any)
- Each component's `primary_doc` (read-only, for terminology + responsibility extraction)
- Root `README.md` if `repo.root_readme` is referenced
**Outputs**:
- `_docs/glossary.md` (or `<docs.root>/glossary.md` if `docs.root``_docs/`) — NEW
- The cross-cutting architecture doc updated in place: a `## Architecture Vision` section is prepended (or merged into an existing "Vision" / "Overview" heading)
- One new entry appended to `_docs/_repo-config.yaml` under `assumptions_log:` recording the run
- A new top-level config entry: `glossary_doc: <path>` so future `monorepo-status` and `monorepo-document` runs treat the glossary as a known cross-cutting doc
**Procedure**:
1. **Draft glossary** from `_repo-config.yaml` + each component's primary doc. Include:
- Component codenames as they appear in the config (`name` field) and any rename pairs the user noted in `unresolved:` resolutions
- Domain terms that recur across ≥2 component docs
- Acronyms / abbreviations
- Convention names from `conventions:` (e.g., commit prefix, deployment tier names)
- Stakeholder personas if cross-cutting docs reference them
Each entry: one-line definition + source (`source: components.<name>.primary_doc` or `source: _repo-config.yaml conventions`). Skip generic terms.
2. **Draft architecture vision** from the meta-repo perspective:
- **One paragraph**: what the system as a whole is, what each component contributes, the runtime topology (one binary / N services / N clients + 1 server / hybrid), how components communicate (REST / gRPC / queue / DB-shared / file-shared)
- **Components & responsibilities** (one-line each), pulled directly from `_repo-config.yaml` `components:` list
- **Cross-cutting concerns ownership**: which doc owns which concern (auth, schema, deployment, etc.) — pulled from `docs.cross_cutting[].owns`
- **Architectural principles / non-negotiables** the user has implied across components (e.g., "all components share a single Postgres", "submodules own their own CI", "deployment is per-tier, not per-component")
- **Open questions / structural drift signals**: components missing from `docs.cross_cutting`, components in registry but not in config (registry mismatch), or contradictions between component primary docs
3. **Present condensed view** to the user (NOT the full draft files):
```
══════════════════════════════════════
REVIEW: Meta-Repo Glossary + Architecture Vision
══════════════════════════════════════
Glossary (N terms drafted from config + component docs):
- <Term>: <one-line definition>
- ...
Architecture Vision — meta-repo level:
<one-paragraph synopsis>
Components / responsibilities:
- <component>: <one-line>
- ...
Cross-cutting ownership:
- <concern> → <doc>
- ...
Principles / non-negotiables:
- <principle>
- ...
Open questions / drift signals:
- <q1>
- <q2>
══════════════════════════════════════
A) Looks correct — write the files
B) Add / correct entries (provide diffs)
C) Resolve open questions / drift signals first
══════════════════════════════════════
Recommendation: pick C if drift signals exist;
otherwise B if components or principles
don't match your intent; A only when
the inferred vision is exactly right.
══════════════════════════════════════
```
4. **Iterate**:
- On B → integrate the user's diffs/additions, re-present, loop until A.
- On C → ask the listed open questions in one batch, integrate answers, re-present.
- **Do NOT proceed to step 5 until the user picks A.**
5. **Save**:
- Write `_docs/glossary.md` (alphabetical) with `**Status**: confirmed-by-user` + date.
- Update the cross-cutting architecture doc identified in `docs.cross_cutting` (or create one at `_docs/00_architecture.md` if none exists and the user's option-B input named one): prepend `## Architecture Vision` with the confirmed paragraph + components + ownership + principles. Preserve every existing H2 below verbatim.
- Append to `_docs/_repo-config.yaml`:
- Top-level `glossary_doc: <path-relative-to-repo-root>` (sibling of `docs.root`)
- New `assumptions_log:` entry: `{ date: <today>, skill: autodev-meta-repo Step 2.5, run_notes: "Captured glossary + architecture vision", assumptions: [...] }`
- Do NOT flip any `confirmed: false` → `confirmed: true` in the config; this step writes its own confirmed artifact, it does not retroactively confirm config inferences.
**Self-verification**:
- [ ] Every glossary entry traces to either the config or a component primary doc
- [ ] Every component listed in the vision matches a `components:` entry in the config
- [ ] All open questions are answered or explicitly deferred (with the user's acknowledgement)
- [ ] The cross-cutting architecture doc still contains every H2 it had before this step
- [ ] User picked option A on the latest condensed view
**Idempotency**: if both `_docs/glossary.md` exists AND the architecture doc already has a `## Architecture Vision` section, this step is **skipped on re-invocation**. To refresh, the user invokes `/autodev` after deleting `glossary.md` (or running `monorepo-discover` with structural changes that justify a re-confirmation).
After completion, auto-chain to **Step 3 (Status)**.
---
**Step 3 — Status**
Condition (folder fallback): `_docs/_repo-config.yaml` exists AND `confirmed_by_user: true` AND (`_docs/glossary.md` exists OR `glossary_doc:` is recorded in the config).
State-driven: reached by auto-chain from Step 2.5, or entered on any re-invocation after a completed cycle.
Condition (folder fallback): `_docs/_repo-config.yaml` exists AND `confirmed_by_user: true`.
State-driven: reached by auto-chain from Step 2 (user picked A), or entered on any re-invocation after a completed cycle.
Action: Read and execute `.cursor/skills/monorepo-status/SKILL.md`.
@@ -186,16 +78,11 @@ The status report identifies:
- Registry/config mismatches
- Unresolved questions
Based on the report, auto-chain branches in this evaluation order (first match wins):
Based on the report, auto-chain branches:
1. **Registry mismatch** (new components not in config, or config component not in registry) → present the Choose format below FIRST. After the user resolves it (A: refresh discover, B: onboard, C: continue with mismatch acknowledged), proceed to the next rule. This rule has priority because a stale config would mislead Step 3.5's ownership-envelope synthesis and any sync skill's component scope.
2. **Pre-routing gate (Step 3.5 detection)** — check `_docs/tasks/todo/` for suite-level task files (`*.md` excluding files starting with `_`). If ≥1 task is present, auto-chain to **Step 3.5 (Suite Implement)**. After Step 3.5 returns (regardless of A/B outcome), the post-implement re-status applies rules 36 below to the post-implementation state.
3. If **doc drift** found → auto-chain to **Step 4 (Document Sync)**
4. Else if **CI drift** (only) found → auto-chain to **Step 5 (CICD Sync)**
5. Else if **suite-e2e drift** (only) found → auto-chain to **Step 4.5 (Integration Test Sync)** (only when `suite_e2e:` block exists in config)
6. Else → **workflow done for this cycle**.
**Registry mismatch Choose format** (rule 1):
- If **doc drift** found → auto-chain to **Step 4 (Document Sync)**
- Else if **CI drift** (only) found → auto-chain to **Step 5 (CICD Sync)**
- Else if **registry mismatch** found (new components not in config) → present Choose format:
```
══════════════════════════════════════
@@ -212,134 +99,7 @@ Based on the report, auto-chain branches in this evaluation order (first match w
══════════════════════════════════════
```
When rule 6 fires (no drift, no todo tasks), report "No drift. Meta-repo is in sync." and end the cycle. Loop waits for next invocation.
---
**Step 3.5 — Suite Implement**
Condition (folder fallback): `_docs/tasks/todo/` exists AND contains ≥1 file matching `*.md` excluding files starting with `_` (e.g., `_dependencies_table.md` is excluded by convention).
State-driven: reached by auto-chain from Step 3 when the pre-routing gate detected todo tasks. Inserted **before** the sync skills (Step 4 / 4.5 / 5) by deliberate design: implementing renames + cross-repo edits first means the subsequent sync skills propagate the actual landed state rather than the pre-change state, avoiding a second cycle to fix downstream drift.
**Skip condition**: `_docs/tasks/todo/` is empty, missing, or contains only `_*` files. In that case Step 3.5 is skipped entirely and the cycle proceeds with Step 3's existing drift-based routing.
**Goal**: Execute suite-level implementation tasks — cross-repo concerns (e.g., `autopilot` + `ui` + suite `e2e/` cutover in a coordinated change-set), folder renames (e.g., `git mv flights missions` + `.gitmodules` edit + `_infra/` path refs), and suite-root infrastructure additions (e.g., `_infra/dev/docker-compose.dev.yml`). Per-component implementation work stays in each component's own workspace `/autodev` cycle.
**Why this exists**: the meta-repo's existing sync skills (`monorepo-document`, `monorepo-cicd`, `monorepo-e2e`) only **propagate** changes that already landed. They cannot **execute** a task spec. Without Step 3.5, suite-level tickets like AZ-543 (B4 repo rename) or AZ-506 (new dev compose) have no flow path forward — they require operator action outside autodev.
**Inputs**:
- `_docs/tasks/todo/*.md` (excluding `_*`) — task specs in the existing format (`Task` / `Component` / `Dependencies` / `Acceptance criteria` headers)
- `_docs/_repo-config.yaml` — `components[].path` list, used to compute the suite-level OWNED envelope (workspace root EXCLUDING any path under a component's folder)
- `_docs/tasks/_dependencies_table.md` — synthesized by this step if missing (see Procedure)
- `_docs/tasks/_suite_module_layout.md` — synthesized by this step if missing (see Procedure)
**Procedure**:
1. **Detection (already done by Step 3 pre-routing gate)**. List task files in `_docs/tasks/todo/` (excluding `_*`). If 0 → skip Step 3.5. If ≥1 → continue.
2. **Present Choose**:
```
══════════════════════════════════════
DECISION REQUIRED: <N> suite-level task(s) in _docs/tasks/todo/
══════════════════════════════════════
Task(s) detected:
- AZ-XXX: <title> (deps: <list or "—">)
- AZ-YYY: <title> (deps: <list or "—">)
...
A) Run implement skill on these task(s) now (then continue to Doc / E2E / CICD sync)
B) Skip implement this cycle — continue to Doc / E2E / CICD sync without executing tasks
C) Pause — review the tasks before deciding (end session, no state changes)
══════════════════════════════════════
Recommendation: A — running implement BEFORE syncs means subsequent
sync skills propagate the post-implementation state.
B is appropriate when tasks are blocked on user input
or external coordination. C when the tasks themselves
need owner clarification before execution.
══════════════════════════════════════
```
3. **On user A — Pre-flight**:
a. **Working tree clean check**. Run `git status --porcelain`. If non-empty, surface to the user with a Choose A/B/C identical to the implement skill's prerequisite gate (commit/stash manually; agent commits as `chore: WIP pre-implement`; abort).
b. **Synthesize `_docs/tasks/_dependencies_table.md`** if missing. Parse each in-scope task's `Dependencies:` field. Write a minimal table of the form:
```markdown
# Suite-Level Task Dependencies
| Task ID | Depends on | Notes |
|---------|------------|-------|
| AZ-XXX | (none) | — |
| AZ-YYY | AZ-XXX | — |
```
If a task lists a dependency that is neither in `todo/` nor `done/`, log a warning in the synthesized file but do not block — implement skill's Step 1 (Parse) will surface the issue if it actually blocks execution.
c. **Synthesize `_docs/tasks/_suite_module_layout.md`** if missing. Default content:
```markdown
# Suite-Level Module Layout (synthetic)
Generated by autodev meta-repo Step 3.5. The suite root has no per-feature decomposition; ownership is defined at the component-boundary level only.
## Per-Component Mapping
| Component | Owns | Imports from |
|-----------|----------------------------------|--------------|
| suite | (workspace root) excluding any path listed under `_repo-config.yaml.components[].path` | (read-only) every component's primary doc + `_docs/*.md` |
Suite-level tasks operate on: `.gitmodules`, `_infra/**`, `_docs/**` (excluding `_docs/tasks/_*` regenerated files), root `README.md`, `e2e/**` (suite e2e harness only).
Forbidden paths for suite-level tasks: `<component>/**` for every component listed in `_repo-config.yaml.components[].path` — those edits live in the component's own workspace `/autodev` cycle.
```
d. **Prepare invocation context**:
```
suite_level: true
TASKS_DIR: _docs/tasks/
module_layout_path: _docs/tasks/_suite_module_layout.md
```
4. **Invoke implement skill**. Read and execute `.cursor/skills/implement/SKILL.md` with the prepared context. The skill's "Suite-level invocation context" subsection (added in tandem with this flow change) honors the three flags above and skips:
- Step 14.5 (cumulative code review) — no `architecture_compliance_baseline.md` exists at the suite level; cross-task drift is captured by the next `monorepo-status` cycle instead.
- Step 15 (Product Implementation Completeness Gate) — the gate's inputs (`_docs/02_document/architecture.md`, `system-flows.md`, `components/*/description.md`) do not exist in the meta-repo artifact layout. Suite tasks are infrastructure / coordination work, not feature implementation.
All other implement skill steps (114, 16) execute unchanged. Tracker integration (Step 5: In Progress, Step 12: In Testing) runs normally.
5. **Post-implement re-status**. After the implement skill completes (last batch committed, all originally-todo tasks moved to `_docs/tasks/done/`), silently re-run Step 3's drift detection logic — do NOT re-render the full Status report; just re-evaluate the drift signals against the post-implementation tree. Then auto-chain per the post-implementation drift findings:
- Doc drift → Step 4 (Document Sync)
- Suite-e2e drift only → Step 4.5
- CI drift only → Step 5
- No drift → cycle complete
Note: the post-implement re-status is exactly why Step 3.5 is placed before sync. A repo rename will typically introduce doc + CI drift; the next invocation of Step 4 / Step 5 catches it on the same cycle.
6. **On user B (skip)** → mark Step 3.5 `skipped` in state file. Apply Step 3's original drift-based routing (compute from the pre-Step-3.5 Status report).
7. **On user C (pause)** → end session. Update state to `step: 3.5, status: in_progress, sub_step: {phase: 0, name: awaiting-task-review, detail: "<N> tasks pending review"}`. Tell the user to invoke `/autodev` again after deciding. **Do NOT modify any files** — pre-flight has not run yet.
**Self-verification** (executed before invoking implement):
- [ ] Working tree is clean (or user explicitly chose B in the WIP-stash sub-Choose)
- [ ] `_docs/tasks/_dependencies_table.md` exists (synthesized if it didn't)
- [ ] `_docs/tasks/_suite_module_layout.md` exists (synthesized if it didn't)
- [ ] All in-scope task files have a `Component:` field (skip + report any that don't — don't guess ownership)
- [ ] Tracker availability gate satisfied per `protocols.md` (or `tracker: local` previously chosen)
**Failure handling**:
- If implement returns FAILED → standard Failure Handling (`protocols.md`): retry up to 3 times, then escalate.
- If implement is interrupted mid-batch → next invocation re-detects via the implement skill's resumability protocol (read latest `_docs/03_implementation/suite_batch_*.md`). Step 3.5 itself is reentrant: on re-entry, if `todo/` still has tasks, it presents the Choose again with the remaining set.
- **Half-applied state risk** (acknowledged): if implement is interrupted between commits, the working tree is clean at the last commit boundary but the in-flight batch is lost. The user is responsible for inspecting and re-invoking. This is intentional — automated rollback of suite-level renames + `.gitmodules` edits is more dangerous than a human-driven recovery.
**Idempotency**: if `_docs/tasks/todo/` becomes empty after this step (all tasks moved to `done/`), the next `/autodev` invocation skips Step 3.5 entirely and proceeds with normal Status → sync flow.
- Else → **workflow done for this cycle**. Report "No drift. Meta-repo is in sync." Loop waits for next invocation.
---
@@ -355,28 +115,6 @@ The skill:
3. Applies doc edits
4. Skips any component with unconfirmed mapping (M5), reports
After completion:
- If the status report ALSO flagged suite-e2e drift → auto-chain to **Step 4.5 (Integration Test Sync)**
- Else if the status report ALSO flagged CI drift → auto-chain to **Step 5 (CICD Sync)**
- Else → end cycle, report done
---
**Step 4.5 — Integration Test Sync**
State-driven: reached by auto-chain from Step 3 (when status report flagged suite-e2e drift and no doc drift) or from Step 4 (when both doc and suite-e2e drift were flagged).
**Skip condition**: if `_docs/_repo-config.yaml` has no `suite_e2e:` block, this step is skipped entirely — there's no harness to sync. The status report should not flag suite-e2e drift in that case; if it does, that's a status-skill bug.
Action: Read and execute `.cursor/skills/monorepo-e2e/SKILL.md` with scope = components flagged by status.
The skill:
1. Verifies every path under `suite_e2e.*` exists (binary fixtures excepted — see the skill's Phase 1)
2. Classifies each flagged change against the suite-e2e impact table
3. Applies edits to `e2e/docker-compose.suite-e2e.yml`, `e2e/fixtures/init.sql`, `e2e/fixtures/expected_detections.json` metadata, and `e2e/runner/tests/*.spec.ts` selectors as needed
4. Bumps baseline `fixture_version` with a `-stale` suffix and appends a `_docs/_process_leftovers/` entry whenever the detection model revision changes (binary fixture cannot be regenerated automatically)
5. Reports synced files; does not run the suite e2e itself
After completion:
- If the status report ALSO flagged CI drift → auto-chain to **Step 5 (CICD Sync)**
- Else → end cycle, report done
@@ -385,11 +123,11 @@ After completion:
**Step 5 — CICD Sync**
State-driven: reached by auto-chain from Step 3 (when status report flagged CI drift and no doc/suite-e2e drift), Step 4, or Step 4.5.
State-driven: reached by auto-chain from Step 3 (when status report flagged CI drift and no doc drift) or from Step 4 (when both doc and CI drift were flagged).
Action: Read and execute `.cursor/skills/monorepo-cicd/SKILL.md` with scope = components flagged by status.
After completion, end cycle. Report files updated across doc, suite-e2e, and CI sync.
After completion, end cycle. Report files updated across both doc and CI sync.
---
@@ -418,24 +156,14 @@ After onboarding completes, the config is updated. Auto-chain back to **Step 3 (
| Completed Step | Next Action |
|---------------|-------------|
| Discover (1) | Auto-chain → Config Review (2) |
| Config Review (2, user picked A, confirmed_by_user: true) | Auto-chain → Glossary & Architecture Vision (2.5) |
| Config Review (2, user picked A, confirmed_by_user: true) | Auto-chain → Status (3) |
| Config Review (2, user picked B) | **Session boundary** — end session, await re-invocation |
| Glossary & Architecture Vision (2.5) | Auto-chain → Status (3) |
| Status (3, todo tasks present) | Auto-chain → Suite Implement (3.5) — pre-routing gate fires before drift-based routing |
| Status (3, no todo tasks, doc drift) | Auto-chain → Document Sync (4) |
| Status (3, no todo tasks, suite-e2e drift only) | Auto-chain → Integration Test Sync (4.5) |
| Status (3, no todo tasks, CI drift only) | Auto-chain → CICD Sync (5) |
| Status (3, no todo tasks, no drift) | **Cycle complete** — end session, await re-invocation |
| Status (3, doc drift) | Auto-chain → Document Sync (4) |
| Status (3, CI drift only) | Auto-chain → CICD Sync (5) |
| Status (3, no drift) | **Cycle complete** — end session, await re-invocation |
| Status (3, registry mismatch) | Ask user (A: discover, B: onboard, C: continue) |
| Suite Implement (3.5, user picked A, success) | Silent re-status; auto-chain per post-implementation drift (Step 4 / 4.5 / 5 / cycle complete) |
| Suite Implement (3.5, user picked B) | Mark `skipped`; auto-chain per Step 3's original drift findings |
| Suite Implement (3.5, user picked C) | **Session boundary** — end session, await re-invocation |
| Suite Implement (3.5, FAILED ×3) | Standard Failure Handling escalation (`protocols.md`) |
| Document Sync (4) + suite-e2e drift pending | Auto-chain → Integration Test Sync (4.5) |
| Document Sync (4) + CI drift only pending | Auto-chain → CICD Sync (5) |
| Document Sync (4) + no further drift | **Cycle complete** |
| Integration Test Sync (4.5) + CI drift pending | Auto-chain → CICD Sync (5) |
| Integration Test Sync (4.5) + no CI drift | **Cycle complete** |
| Document Sync (4) + CI drift pending | Auto-chain → CICD Sync (5) |
| Document Sync (4) + no CI drift | **Cycle complete** |
| CICD Sync (5) | **Cycle complete** |
## Status Summary — Step List
@@ -450,40 +178,30 @@ Flow-specific slot values:
Config: _docs/_repo-config.yaml [confirmed_by_user: <true|false>, last_updated: <date>]
```
| # | Step Name | Extra state tokens (beyond the shared set) |
|---|------------------------------------|--------------------------------------------|
| 1 | Discover | — |
| 2 | Config Review | `IN PROGRESS (awaiting human)` |
| 2.5 | Glossary & Architecture Vision | `SKIPPED (already captured)` |
| 3 | Status | `DONE (no drift)`, `DONE (N drifts)` |
| 3.5 | Suite Implement | `DONE (N tasks)`, `SKIPPED (no todo tasks)`, `SKIPPED (user picked B)`, `IN PROGRESS (batch M of ~N)`, `IN PROGRESS (awaiting-task-review)` |
| 4 | Document Sync | `DONE (N docs)`, `SKIPPED (no doc drift)` |
| 4.5 | Integration Test Sync | `DONE (N files)`, `SKIPPED (no suite-e2e drift)`, `SKIPPED (no suite_e2e config block)` |
| 5 | CICD Sync | `DONE (N files)`, `SKIPPED (no CI drift)` |
| # | Step Name | Extra state tokens (beyond the shared set) |
|---|------------------|--------------------------------------------|
| 1 | Discover | — |
| 2 | Config Review | `IN PROGRESS (awaiting human)` |
| 3 | Status | `DONE (no drift)`, `DONE (N drifts)` |
| 4 | Document Sync | `DONE (N docs)`, `SKIPPED (no doc drift)` |
| 5 | CICD Sync | `DONE (N files)`, `SKIPPED (no CI drift)` |
All rows accept the shared state tokens (`DONE`, `IN PROGRESS`, `NOT STARTED`, `FAILED (retry N/3)`); rows 2.5, 3.5, 4, 4.5, and 5 additionally accept `SKIPPED`.
All rows accept the shared state tokens (`DONE`, `IN PROGRESS`, `NOT STARTED`, `FAILED (retry N/3)`); rows 4 and 5 additionally accept `SKIPPED`.
Row rendering format:
```
Step 1 Discover [<state token>]
Step 2 Config Review [<state token>]
Step 2.5 Glossary & Architecture Vision [<state token>]
Step 3 Status [<state token>]
Step 3.5 Suite Implement [<state token>]
Step 4 Document Sync [<state token>]
Step 4.5 Integration Test Sync [<state token>]
Step 5 CICD Sync [<state token>]
Step 1 Discover [<state token>]
Step 2 Config Review [<state token>]
Step 3 Status [<state token>]
Step 4 Document Sync [<state token>]
Step 5 CICD Sync [<state token>]
```
## Notes for the meta-repo flow
- **Session boundaries**: Step 2 (Config Review pending), Step 2.5 (one-shot glossary/vision review), and Step 3.5 (when user picks C "Pause"). Step 3.5's A/B picks do NOT cross a session boundary — they auto-chain to syncs in the same session.
- **No session boundary except Step 2**: unlike existing-code flow (which has boundaries around decompose), meta-repo flow only pauses at config review. Syncing is fast enough to complete in one session.
- **Cyclical, not terminal**: no "done forever" state. Each invocation completes a drift cycle; next invocation starts fresh.
- **Tracker integration scope**: this flow does NOT create Jira/ADO tickets in its sync skills (Status / Document Sync / E2E / CICD). Step 3.5 (Suite Implement) IS tracker-integrated — it transitions existing tickets In Progress → In Testing per the implement skill's standard tracker handling. Suite-level tickets are authored manually by the operator (typically as children of an Epic that spans multiple components, like AZ-539); the flow doesn't auto-create them.
- **Per-component vs. suite-level work**:
- Tickets that touch component source code (`<component>/src/**`) belong in that component's own workspace `/autodev` cycle. The meta-repo flow does NOT execute them.
- Tickets that touch suite-root paths only (`.gitmodules`, `_infra/**`, suite `e2e/**`, root `README.md`, suite `_docs/**` outside `tasks/_*`) are eligible for Step 3.5.
- Tickets that span both (e.g., AZ-550 B11 consumer cutover, which touches `autopilot/`, `ui/`, AND suite `e2e/`) are NOT executable from a single workspace by design — split the ticket so the suite-level slice can run in Step 3.5 and the component slices run in their owning workspaces.
- **No tracker integration**: this flow does NOT create Jira/ADO tickets. Maintenance is not a feature — if a feature-level ticket spans the meta-repo's concerns, it lives in the per-component workspace.
- **Onboarding is opt-in**: never auto-onboarded. User must explicitly request.
- **Failure handling**: uses the same retry/escalation protocol as other flows (see `protocols.md`).
+4 -6
View File
@@ -110,11 +110,9 @@ Before entering a step from this table for the first time in a session, verify t
| Flow | Step | Sub-Step | Tracker Action |
|------|------|----------|----------------|
| greenfield | Plan | Step 6 — Epics | Create epics for each component |
| greenfield | Decompose | Implementation decomposition Step 1 + Step 2Product tasks | Create ticket per product task, link to epic |
| greenfield | Decompose Tests | Step 1t + Step 3 — All test tasks | Create ticket per task, link to epic |
| greenfield | Decompose | Step 1 + Step 2 + Step 3All tasks | Create ticket per task, link to epic |
| existing-code | Decompose Tests | Step 1t + Step 3 — All test tasks | Create ticket per task, link to epic |
| existing-code | New Task | Step 7 — Ticket | Create ticket per task, link to epic |
| meta-repo | Suite Implement | Step 3.5 — implement skill Step 5 / Step 12 | Transition existing tickets In Progress → In Testing per implement skill (does NOT create new tickets — operator authors them) |
### State File Marker
@@ -140,7 +138,7 @@ One retry ladder covers all failure modes: explicit failure returned by a sub-sk
Treat the sub-skill as **failed** when ANY of the following is observed:
- The sub-skill explicitly returns a failed result (including blocked tasks, auto-fix loop exhaustion, prerequisite violations).
- The sub-skill explicitly returns a failed result (including blocked subagents, auto-fix loop exhaustion, prerequisite violations).
- **Stuck signals**: the same artifact is rewritten 3+ times without meaningful change; the sub-skill re-asks a question that was already answered; no new artifact has been saved despite active execution.
### Retry ladder
@@ -293,7 +291,7 @@ For steps that produce `_docs/` artifacts (problem, research, plan, decompose, d
## Debug Protocol
When the implement skill's auto-fix loop fails (code review FAIL after 2 auto-fix attempts) or a task reports a blocker, the user is asked to intervene. This protocol guides the debugging process. (Retry budget and escalation are covered by Failure Handling above; this section is about *how* to diagnose once the user has been looped in.)
When the implement skill's auto-fix loop fails (code review FAIL after 2 auto-fix attempts) or an implementer subagent reports a blocker, the user is asked to intervene. This protocol guides the debugging process. (Retry budget and escalation are covered by Failure Handling above; this section is about *how* to diagnose once the user has been looped in.)
### Structured Debugging Workflow
@@ -389,7 +387,7 @@ The banner shell is defined here once. Each flow file contributes only its step-
where `<state token>` comes from the state-token set defined per row in the flow's step-list table.
- `<current-suffix>` — optional, flow-specific. The existing-code flow appends ` (cycle <N>)` when `state.cycle > 1`; other flows leave it empty.
- `Retry:` row — omit entirely when `retry_count` is 0. Include it with `<N>/3` otherwise.
- `<footer-extras>` — optional, flow-specific. The meta-repo flow adds a `Config:` line with `_docs/_repo-config.yaml` state; other flows leave it empty unless **parent suite docs** apply: if `<workspace-root>/../docs` exists and is a directory, append `Suite docs (parent): <absolute path>` on its own line (or `Suite docs (parent): absent` is **not** required — omit when missing). This line is orthogonal to flow-specific footer lines; both may appear.
- `<footer-extras>` — optional, flow-specific. The meta-repo flow adds a `Config:` line with `_docs/_repo-config.yaml` state; other flows leave it empty.
### State token set (shared)
+2 -15
View File
@@ -13,7 +13,7 @@ The autodev persists its position to `_docs/_autodev_state.md`. This is a lightw
## Current Step
flow: [greenfield | existing-code | meta-repo]
step: [1-17 for greenfield, 1-17 for existing-code, 1-6 for meta-repo (incl. fractional 2.5 and 3.5), or "done"]
step: [1-11 for greenfield, 1-17 for existing-code, 1-6 for meta-repo, or "done"]
name: [step name from the active flow's Step Reference Table]
status: [not_started / in_progress / completed / skipped / failed]
sub_step:
@@ -82,19 +82,6 @@ retry_count: 0
cycle: 1
```
```
flow: meta-repo
step: 3.5
name: Suite Implement
status: in_progress
sub_step:
phase: 7
name: batch-loop
detail: "AZ-543 batch 1 of 1; suite-level"
retry_count: 0
cycle: 1
```
```
flow: existing-code
step: 10
@@ -113,7 +100,7 @@ cycle: 3
1. **Create** on the first autodev invocation (after state detection determines Step 1)
2. **Update** after every change — this includes: batch completion, sub-step progress, step completion, session boundary, failed retry, or any meaningful state transition. The state file must always reflect the current reality.
3. **Read** as the first action on every invocation — before folder scanning
4. **Cross-check**: verify against actual `_docs/` folder contents. If they disagree, trust the folder structure and update the state file. **Parent suite `docs/`**: on every invocation, also probe `<workspace-root>/../docs` (the parent directorys `docs` folder — typical suite-level shared documentation next to a component repo). If it exists, mention it in the Status Summary footer per `protocols.md`; use it only as supplemental reading context unless a flow step explicitly ties detection to it. It never replaces workspace `_docs/` for step detection by default.
4. **Cross-check**: verify against actual `_docs/` folder contents. If they disagree, trust the folder structure and update the state file
5. **Never delete** the state file
6. **Retry tracking**: increment `retry_count` on each failed auto-retry; reset to `0` on success. If `retry_count` reaches 3, set `status: failed`
7. **Failed state on re-entry**: if `status: failed` with `retry_count: 3`, do NOT auto-retry — present the issue to the user first
+2 -2
View File
@@ -209,7 +209,7 @@ Bug, Spec-Gap, Security, Performance, Maintainability, Style, Scope, Architectur
The `/implement` skill invokes this skill after each batch completes:
1. Collects changed files from all tasks implemented in the batch
1. Collects changed files from all implementer agents in the batch
2. Passes task spec paths + changed files to this skill
3. If verdict is FAIL — presents findings to user (BLOCKING), user fixes or confirms
4. If verdict is PASS or PASS_WITH_WARNINGS — proceeds automatically (findings shown as info)
@@ -221,7 +221,7 @@ The `/implement` skill invokes this skill after each batch completes:
| Input | Type | Source | Required |
|-------|------|--------|----------|
| `task_specs` | list of file paths | Task `.md` files from `_docs/02_tasks/todo/` for the current batch | Yes |
| `changed_files` | list of file paths | Files modified by the tasks in the batch (from `git diff`) | Yes |
| `changed_files` | list of file paths | Files modified by implementer agents (from `git diff` or agent reports) | Yes |
| `batch_number` | integer | Current batch number (for report naming) | Yes |
| `project_restrictions` | file path | `_docs/00_problem/restrictions.md` | If exists |
| `solution_overview` | file path | `_docs/01_solution/solution.md` | If exists |
+24 -27
View File
@@ -2,8 +2,8 @@
name: decompose
description: |
Decompose planned components into atomic implementable tasks with bootstrap structure plan.
Workflow entrypoints: implementation task decomposition, single component decomposition, and tests-only decomposition.
The invoking flow decides which entrypoint to run; this skill executes that selected sequence.
4-step workflow: bootstrap structure plan, component task decomposition, blackbox test task decomposition, and cross-task verification.
Supports full decomposition (_docs/ structure), single component mode, and tests-only mode.
Trigger phrases:
- "decompose", "decompose features", "feature decomposition"
- "task decomposition", "break down components"
@@ -20,7 +20,7 @@ Decompose planned components into atomic, implementable task specs with a bootst
## Core Principles
- **Atomic tasks**: each task does one thing; if it exceeds 5 complexity points, split it
- **Atomic tasks**: each task does one thing; if it exceeds 8 complexity points, split it
- **Behavioral specs, not implementation plans**: describe what the system should do, not how to build it
- **Flat structure**: all tasks are tracker-ID-prefixed files in TASKS_DIR — no component subdirectories
- **Save immediately**: write artifacts to disk after each task; never accumulate unsaved work
@@ -30,15 +30,14 @@ Decompose planned components into atomic, implementable task specs with a bootst
## Context Resolution
Resolve the selected entrypoint from the invocation context before any other logic runs. The caller decides whether this is implementation, single component, or tests-only decomposition; this skill only executes the selected sequence.
Determine the operating mode based on invocation before any other logic runs.
**Implementation task decomposition** (default; selected by flows before invoking this skill):
**Default** (no explicit input file provided):
- DOCUMENT_DIR: `_docs/02_document/`
- TASKS_DIR: `_docs/02_tasks/`
- TASKS_TODO: `_docs/02_tasks/todo/`
- Reads from: `_docs/00_problem/`, `_docs/01_solution/`, DOCUMENT_DIR
- Produces only implementation tasks. Blackbox/e2e test task files are produced only when the invoking flow selects tests-only decomposition.
**Single component mode** (provided file is within `_docs/02_document/` and inside a `components/` subdirectory):
@@ -56,24 +55,24 @@ Resolve the selected entrypoint from the invocation context before any other log
- TESTS_DIR: `DOCUMENT_DIR/tests/`
- Reads from: `_docs/00_problem/`, `_docs/01_solution/`, TESTS_DIR
Announce the selected entrypoint and resolved paths to the user before proceeding.
Announce the detected mode and resolved paths to the user before proceeding.
### Step Applicability by Mode
| Step | File | Implementation | Single | Tests-only |
|------|------|:--------------:|:------:|:----------:|
| Step | File | Default | Single | Tests-only |
|------|------|:-------:|:------:|:----------:|
| 1 Bootstrap Structure | `steps/01_bootstrap-structure.md` | ✓ | — | — |
| 1t Test Infrastructure | `steps/01t_test-infrastructure.md` | — | — | ✓ |
| 1.5 Module Layout | `steps/01-5_module-layout.md` | ✓ | — | — |
| 2 Task Decomposition | `steps/02_task-decomposition.md` | ✓ | ✓ | — |
| 3 Blackbox Test Tasks | `steps/03_blackbox-test-decomposition.md` | | — | ✓ |
| 3 Blackbox Test Tasks | `steps/03_blackbox-test-decomposition.md` | | — | ✓ |
| 4 Cross-Verification | `steps/04_cross-verification.md` | ✓ | — | ✓ |
## Input Specification
### Required Files
**Implementation task decomposition:**
**Default:**
| File | Purpose |
|------|---------|
@@ -81,11 +80,10 @@ Announce the selected entrypoint and resolved paths to the user before proceedin
| `_docs/00_problem/restrictions.md` | Constraints and limitations |
| `_docs/00_problem/acceptance_criteria.md` | Measurable acceptance criteria |
| `_docs/01_solution/solution.md` | Finalized solution |
| `DOCUMENT_DIR/architecture.md` | Architecture from plan/document skill (must contain a `## Architecture Vision` H2 — confirmed user intent) |
| `DOCUMENT_DIR/glossary.md` | Project terminology (confirmed by user in plan Phase 2a.0 or document Step 4.5). Use it to keep task names, component references, and AC wording consistent with the user's vocabulary |
| `DOCUMENT_DIR/architecture.md` | Architecture from plan skill |
| `DOCUMENT_DIR/system-flows.md` | System flows from plan skill |
| `DOCUMENT_DIR/components/[##]_[name]/description.md` | Component specs from plan skill |
| `DOCUMENT_DIR/tests/` | Optional product acceptance context from test-spec skill; do not create test task files from it in this entrypoint |
| `DOCUMENT_DIR/tests/` | Blackbox test specs from plan skill |
**Single component mode:**
@@ -112,7 +110,7 @@ Announce the selected entrypoint and resolved paths to the user before proceedin
### Prerequisite Checks (BLOCKING)
**Implementation task decomposition:**
**Default:**
1. DOCUMENT_DIR contains `architecture.md` and `components/`**STOP if missing**
2. Create TASKS_DIR and TASKS_TODO if they do not exist
@@ -146,8 +144,6 @@ TASKS_DIR/
**Naming convention**: Each task file is initially saved in `TASKS_TODO/` with a temporary numeric prefix (`[##]_[short_name].md`). After creating the work item ticket, rename the file to use the work item ticket ID as prefix (`[TRACKER-ID]_[short_name].md`). For example: `todo/01_initial_structure.md``todo/AZ-42_initial_structure.md`.
If tracker availability fails, follow `.cursor/rules/tracker.mdc` before continuing. Only when the user explicitly chooses `tracker: local` may the numeric prefix remain; in that mode set `Tracker: pending` and `Epic: pending` in the task header and keep the task eligible for later tracker sync.
### Save Timing
| Step | Save immediately after | Filename |
@@ -169,11 +165,11 @@ If TASKS_DIR subfolders already contain task files:
## Progress Tracking
At the start of execution, create a TodoWrite with all applicable steps for the selected entrypoint (see Step Applicability table). Update status as each step/component completes.
At the start of execution, create a TodoWrite with all applicable steps for the detected mode (see Step Applicability table). Update status as each step/component completes.
## Workflow
### Step 1: Bootstrap Structure Plan (implementation mode only)
### Step 1: Bootstrap Structure Plan (default mode only)
Read and follow `steps/01_bootstrap-structure.md`.
@@ -185,25 +181,25 @@ Read and follow `steps/01t_test-infrastructure.md`.
---
### Step 1.5: Module Layout (implementation mode only)
### Step 1.5: Module Layout (default mode only)
Read and follow `steps/01-5_module-layout.md`.
---
### Step 2: Task Decomposition (implementation and single component modes)
### Step 2: Task Decomposition (default and single component modes)
Read and follow `steps/02_task-decomposition.md`.
---
### Step 3: Blackbox Test Task Decomposition (tests-only mode only)
### Step 3: Blackbox Test Task Decomposition (default and tests-only modes)
Read and follow `steps/03_blackbox-test-decomposition.md`.
---
### Step 4: Cross-Task Verification (implementation and tests-only modes)
### Step 4: Cross-Task Verification (default and tests-only modes)
Read and follow `steps/04_cross-verification.md`.
@@ -211,7 +207,7 @@ Read and follow `steps/04_cross-verification.md`.
- **Coding during decomposition**: this workflow produces specs, never code
- **Over-splitting**: don't create many tasks if the component is simple — 1 task is fine
- **Tasks exceeding 5 points**: split them; no task should be too complex for a single implementer
- **Tasks exceeding 8 points**: split them; no task should be too complex for a single implementer
- **Cross-component tasks**: each task belongs to exactly one component
- **Skipping BLOCKING gates**: never proceed past a BLOCKING marker without user confirmation
- **Creating git branches**: branch creation is an implementation concern, not a decomposition one
@@ -224,7 +220,7 @@ Read and follow `steps/04_cross-verification.md`.
| Situation | Action |
|-----------|--------|
| Ambiguous component boundaries | ASK user |
| Task complexity exceeds 5 points after splitting | ASK user |
| Task complexity exceeds 8 points after splitting | ASK user |
| Missing component specs in DOCUMENT_DIR | ASK user |
| Cross-component dependency conflict | ASK user |
| Tracker epic not found for a component | ASK user for Epic ID |
@@ -236,14 +232,15 @@ Read and follow `steps/04_cross-verification.md`.
┌────────────────────────────────────────────────────────────────┐
│ Task Decomposition (Multi-Mode) │
├────────────────────────────────────────────────────────────────┤
│ CONTEXT: Invoke the selected entrypoint (implementation / single / tests-only) │
│ CONTEXT: Resolve mode (default / single component / tests-only) │
│ │
IMPLEMENTATION TASK DECOMPOSITION:
DEFAULT MODE:
│ 1. Bootstrap Structure → steps/01_bootstrap-structure.md │
│ [BLOCKING: user confirms structure] │
│ 1.5 Module Layout → steps/01-5_module-layout.md │
│ [BLOCKING: user confirms layout] │
│ 2. Component Tasks → steps/02_task-decomposition.md │
│ 3. Blackbox Tests → steps/03_blackbox-test-decomposition.md │
│ 4. Cross-Verification → steps/04_cross-verification.md │
│ [BLOCKING: user confirms dependencies] │
│ │
@@ -26,7 +26,7 @@ For each component (or the single provided component):
4. Do not create tasks for other components — only tasks for the current component
5. Each task should be atomic, containing 1 API or a list of semantically connected APIs
6. Write each task spec using `templates/task.md`
7. Estimate complexity per task (1, 2, 3, 5 points); no task should exceed 5 points — split if it does
7. Estimate complexity per task (1, 2, 3, 5, 8 points); no task should exceed 8 points — split if it does
8. Note task dependencies (referencing tracker IDs of already-created dependency tasks, e.g., `AZ-42_initial_structure`)
9. **Cross-cutting rule**: if a concern spans ≥2 components (logging, config loading, auth/authZ, error envelope, telemetry, feature flags, i18n), create ONE shared task under the cross-cutting epic. Per-component tasks declare it as a dependency and consume it; they MUST NOT re-implement it locally. Duplicate local implementations are an `Architecture` finding (High) in code-review Phase 7 and a `Maintainability` finding in Phase 6.
10. **Shared-models / shared-API rule**: classify the task as shared if ANY of the following is true:
@@ -43,32 +43,16 @@ For each component (or the single provided component):
Consumers read the contract file, not the producer's task spec. This prevents interface drift when the producer's implementation detail leaks into consumers.
11. **Immediately after writing each task file**: create a work item ticket, link it to the component's epic, write the work item ticket ID and Epic ID back into the task header, then rename the file from `todo/[##]_[short_name].md` to `todo/[TRACKER-ID]_[short_name].md`.
## Runtime Completeness Decomposition Gate
Before Step 2 is considered complete, scan `architecture.md`, `system-flows.md`, component descriptions, and the solution for named internal runtime capabilities and dependencies. Examples include BASALT/OpenVINS/Kimera, FAISS, DINOv2, ONNX/TensorRT, ALIKED/DISK, LightGlue, RANSAC, PostGIS, MAVLink emission, FDR rollover, and any "A-Z" user-visible pipeline.
For every named internal capability:
1. Ensure at least one implementation task explicitly owns the production integration or production algorithm.
2. Do not treat "define protocol", "create adapter boundary", "add deterministic fallback", "create scaffold", or "prepare native bridge" as implementation of the capability unless the architecture explicitly says the real capability is out of scope.
3. If a capability needs external hardware/data to verify, still create the production implementation task. Verification may be hardware-gated later; implementation must not be omitted.
4. Add a `## Runtime Completeness` section to any affected task with:
- named capability/dependency,
- production code that must exist,
- allowed external stubs, if any,
- unacceptable substitutes such as fake/deterministic/internal stubs.
## Self-verification (per component)
- [ ] Every task is atomic (single concern)
- [ ] No task exceeds 5 complexity points
- [ ] No task exceeds 8 complexity points
- [ ] Task dependencies reference correct tracker IDs
- [ ] Tasks cover all interfaces defined in the component spec
- [ ] No tasks duplicate work from other components
- [ ] Every task has a work item ticket linked to the correct epic
- [ ] Every shared-models / shared-API task has a contract file at `_docs/02_document/contracts/<component>/<name>.md` and a `## Contract` section linking to it
- [ ] Every cross-cutting concern appears exactly once as a shared task, not N per-component copies
- [ ] Every named internal runtime capability has a production implementation task, not only an interface/scaffold/fallback task
## Save action
@@ -1,4 +1,4 @@
# Step 3: Blackbox Test Task Decomposition (tests-only mode only)
# Step 3: Blackbox Test Task Decomposition (default and tests-only modes)
**Role**: Professional Quality Assurance Engineer
**Goal**: Decompose blackbox test specs into atomic, implementable task specs.
@@ -6,6 +6,7 @@
## Numbering
- In default mode: continue sequential numbering from where Step 2 left off.
- In tests-only mode: start from 02 (01 is the test infrastructure bootstrap from Step 1t).
## Steps
@@ -13,26 +14,21 @@
1. Read all test specs from `DOCUMENT_DIR/tests/` (`blackbox-tests.md`, `performance-tests.md`, `resilience-tests.md`, `security-tests.md`, `resource-limit-tests.md`)
2. Group related test scenarios into atomic tasks (e.g., one task per test category or per component under test)
3. Each task should reference the specific test scenarios it implements and the environment/test-data specs
4. Add a **System Under Test Boundary** section to every e2e/blackbox test task:
- The test must drive the product through public runtime boundaries and compare actual outputs to `_docs/00_problem/input_data/expected_results/results_report.md` and any referenced machine-readable expected-result files.
- Stubs are allowed only for external systems outside the product boundary: flight controller/SITL, QGC observer, satellite-provider/Suite service, physical Jetson hardware, physical camera, licensed public datasets, and network services.
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are not allowed for internal product modules that the scenario is meant to validate, such as VIO, safety/anchor wrapper, satellite retrieval, anchor verification, tile manager, MAVLink output adapter, or FDR.
- If an internal module is not implemented, the test must fail/block as missing product implementation; it must not pass by replacing that module with a test stub.
5. Dependencies:
4. Dependencies:
- In default mode: blackbox test tasks depend on the component implementation tasks they exercise
- In tests-only mode: blackbox test tasks depend on the test infrastructure bootstrap task (Step 1t)
6. Write each task spec using `templates/task.md`
7. Estimate complexity per task (1, 2, 3, 5 points); no task should exceed 5 points — split if it does
8. Note task dependencies (referencing tracker IDs of already-created dependency tasks)
9. **Immediately after writing each task file**: create a work item ticket under the "Blackbox Tests" epic, write the work item ticket ID and Epic ID back into the task header, then rename the file from `todo/[##]_[short_name].md` to `todo/[TRACKER-ID]_[short_name].md`.
5. Write each task spec using `templates/task.md`
6. Estimate complexity per task (1, 2, 3, 5, 8 points); no task should exceed 8 points — split if it does
7. Note task dependencies (referencing tracker IDs of already-created dependency tasks)
8. **Immediately after writing each task file**: create a work item ticket under the "Blackbox Tests" epic, write the work item ticket ID and Epic ID back into the task header, then rename the file from `todo/[##]_[short_name].md` to `todo/[TRACKER-ID]_[short_name].md`.
## Self-verification
- [ ] Every scenario from `tests/blackbox-tests.md` is covered by a task
- [ ] Every scenario from `tests/performance-tests.md`, `tests/resilience-tests.md`, `tests/security-tests.md`, and `tests/resource-limit-tests.md` is covered by a task
- [ ] No task exceeds 5 complexity points
- [ ] Dependencies correctly reference the test infrastructure task
- [ ] No task exceeds 8 complexity points
- [ ] Dependencies correctly reference the dependency tasks (component tasks in default mode, test infrastructure in tests-only mode)
- [ ] Every task has a work item ticket linked to the "Blackbox Tests" epic
- [ ] Every e2e/blackbox task forbids internal product stubs/fakes and requires comparison against expected-results artifacts
## Save action
@@ -1,4 +1,4 @@
# Step 4: Cross-Task Verification (implementation and tests-only modes)
# Step 4: Cross-Task Verification (default and tests-only modes)
**Role**: Professional software architect and analyst
**Goal**: Verify task consistency and produce `_dependencies_table.md`.
@@ -8,20 +8,17 @@
1. Verify task dependencies across all tasks are consistent
2. Check no gaps:
- In implementation mode: every product interface in `architecture.md` has implementation task coverage
- In default mode: every interface in `architecture.md` has tasks covering it
- In tests-only mode: every test scenario in `traceability-matrix.md` is covered by a task
- In implementation mode: every named internal runtime capability/dependency from architecture, solution, system flows, and component descriptions has a production implementation task, not only an interface/scaffold/fallback task
- In tests-only mode: every e2e/blackbox task has a System Under Test Boundary section that forbids stubbing internal product modules and requires comparison to expected-results artifacts
3. Check no overlaps: tasks don't duplicate work
4. Check no circular dependencies in the task graph
5. Produce `_dependencies_table.md` using `templates/dependencies-table.md`
## Self-verification
### Implementation mode
### Default mode
- [ ] Every product interface in `architecture.md` is covered by at least one implementation task
- [ ] Every named internal runtime capability has a production implementation task
- [ ] Every architecture interface is covered by at least one task
- [ ] No circular dependencies in the task graph
- [ ] Cross-component dependencies are explicitly noted in affected task specs
- [ ] `_dependencies_table.md` contains every task with correct dependencies
@@ -29,7 +26,6 @@
### Tests-only mode
- [ ] Every test scenario from `traceability-matrix.md` "Covered" entries has a corresponding task
- [ ] Every e2e/blackbox task validates actual product behavior and allows stubs only for external systems
- [ ] No circular dependencies in the task graph
- [ ] Test task dependencies reference the test infrastructure bootstrap
- [ ] `_dependencies_table.md` contains every task with correct dependencies
@@ -28,4 +28,4 @@ Use this template after cross-task verification. Save as `TASKS_DIR/_dependencie
- Dependencies column lists tracker IDs (e.g., "AZ-43, AZ-44") or "None"
- No circular dependencies allowed
- Tasks should be listed in recommended execution order
- The `/implement` skill reads this table to compute dependency-aware batches; task execution remains sequential
- The `/implement` skill reads this table to compute parallel batches
@@ -1,6 +1,6 @@
# Module Layout Template
The module layout is the **authoritative file-ownership map** used by the `/implement` skill to assign OWNED / READ-ONLY / FORBIDDEN files to each task. It is derived from `_docs/02_document/architecture.md` and the component specs at `_docs/02_document/components/`, and it follows the target language's standard project-layout conventions.
The module layout is the **authoritative file-ownership map** used by the `/implement` skill to assign OWNED / READ-ONLY / FORBIDDEN files to implementer subagents. It is derived from `_docs/02_document/architecture.md` and the component specs at `_docs/02_document/components/`, and it follows the target language's standard project-layout conventions.
Save as `_docs/02_document/module-layout.md`. This file is produced by the decompose skill (Step 1.5 module layout) and consumed by the implement skill (Step 4 file ownership). Task specs remain purely behavioral — they do NOT carry file paths. The layout is the single place where component → filesystem mapping lives.
@@ -104,4 +104,4 @@ The implement skill's Step 4 (File Ownership) reads this file and, for each task
3. Set READ-ONLY = the Public API files of every component listed in `Imports from`, plus `shared/*` Public API files.
4. Set FORBIDDEN = every other component's Owns glob.
Execution inside a batch is already sequential (one task at a time). This mapping is still required because it enforces scope discipline per task — preventing a task from drifting into files that belong to another component.
If two tasks in the same batch map to the same component, the implement skill schedules them sequentially (one implementer at a time for that component) to avoid file conflicts on shared internal files.
+3 -2
View File
@@ -11,7 +11,7 @@ Save as `TASKS_DIR/[##]_[short_name].md` initially, then rename to `TASKS_DIR/[T
**Task**: [TRACKER-ID]_[short_name]
**Name**: [short human name]
**Description**: [one-line description of what this task delivers]
**Complexity**: [1|2|3|5] points
**Complexity**: [1|2|3|5|8] points
**Dependencies**: [AZ-43_shared_models, AZ-44_db_migrations] or "None"
**Component**: [component name for context]
**Tracker**: [TASK-ID]
@@ -102,7 +102,8 @@ Consumers MUST read that file — not this task spec — to discover the interfa
- 2 points: Non-trivial, low complexity, minimal coordination
- 3 points: Multi-step, moderate complexity, potential alignment needed
- 5 points: Difficult, interconnected logic, medium-high risk
- 8+ points: Too complex — split into smaller tasks
- 8 points: High difficulty, high ambiguity or coordination, multiple components
- 13 points: Too complex — split into smaller tasks
## Output Guidelines
@@ -26,8 +26,7 @@
- Application components under test
- Test runner container (black-box, no internal imports)
- Isolated database with seed data
- All tests runnable via `docker compose -f docker-compose.test.yml up --abort-on-container-exit --exit-code-from e2e-runner`
- See the Woodpecker two-workflow contract in [`../templates/ci_cd_pipeline.md`](../templates/ci_cd_pipeline.md) — the test runner entry point defined here becomes the first step of `.woodpecker/01-test.yml`.
- All tests runnable via `docker compose -f docker-compose.test.yml up --abort-on-container-exit`
7. Define image tagging strategy: `<registry>/<project>/<component>:<git-sha>` for CI, `latest` for local dev only
## Self-verification
@@ -85,140 +85,3 @@ Save as `_docs/04_deploy/ci_cd_pipeline.md`.
| Deploy success | [Slack] | [team] |
| Deploy failure | [Slack/email + PagerDuty] | [on-call] |
```
---
## Reference Implementation: Woodpecker CI two-workflow contract
Use this when the project's CI is **Woodpecker** and the test layout follows the autodev e2e contract from [`../../decompose/templates/test-infrastructure-task.md`](../../decompose/templates/test-infrastructure-task.md) (an `e2e/` folder containing `Dockerfile`, `docker-compose.test.yml`, `conftest.py`, `requirements.txt`, `mocks/`, `fixtures/`, `tests/`).
The contract is **two workflows in `.woodpecker/`**, scheduled on the same agent label, with the build workflow gated on a successful test run:
- `.woodpecker/01-test.yml` — runs the e2e contract, publishes `results/report.csv` as an artifact, fails the pipeline on any test failure.
- `.woodpecker/02-build-push.yml``depends_on: [01-test]`. Builds the image, tags it `${CI_COMMIT_BRANCH}-${TAG_SUFFIX}`, pushes it to the registry. Skipped automatically if test failed.
The agent label is parameterized via `matrix:` so a single workflow file fans out across architectures: `labels: platform: ${PLATFORM}` routes each matrix entry to the matching agent. Both workflows for a repo must use the same matrix so test and build run on the same machine and share Docker layer cache. New architectures = new matrix entries; never new files.
### Multi-arch matrix conventions
| Variable | Meaning | Typical values |
|----------|---------|----------------|
| `PLATFORM` | Woodpecker agent label — selects which physical machine runs the entry. | `arm64`, `amd64` |
| `TAG_SUFFIX` | Image tag suffix appended after the branch name. | `arm`, `amd` |
| `DOCKERFILE` *(only when arches need different Dockerfiles)* | Path to the Dockerfile for this entry. | `Dockerfile`, `Dockerfile.jetson` |
Most repos use the same `Dockerfile` for both arches (multi-arch base images handle the rest), so `DOCKERFILE` can be omitted from the matrix and hardcoded in the build command. Repos with split per-arch Dockerfiles (e.g., `detections` uses `Dockerfile.jetson` on Jetson with TensorRT/CUDA-on-L4T) declare `DOCKERFILE` as a matrix var.
When only one architecture is currently in use, keep the matrix block with a single entry and the second entry commented out — adding a new arch is then a one-line uncomment, not a structural change.
### `.woodpecker/01-test.yml`
```yaml
when:
event: [push, pull_request, manual]
branch: [dev, stage, main]
matrix:
include:
- PLATFORM: arm64
TAG_SUFFIX: arm
# - PLATFORM: amd64
# TAG_SUFFIX: amd
labels:
platform: ${PLATFORM}
steps:
- name: e2e
image: docker
commands:
- cd e2e
- docker compose -f docker-compose.test.yml up --abort-on-container-exit --exit-code-from e2e-runner --build
- docker compose -f docker-compose.test.yml down -v
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- name: report
image: docker
when:
status: [success, failure]
commands:
- test -f e2e/results/report.csv && cat e2e/results/report.csv || echo "no report"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
```
Notes:
- `--abort-on-container-exit` shuts the whole compose down as soon as ANY service exits, so a crashed dependency surfaces immediately instead of hanging the runner.
- `--exit-code-from e2e-runner` ensures the pipeline's exit code reflects the test runner's, not the SUT's.
- The `report` step runs on `[success, failure]` so the report is always published; without this the CSV is lost on red builds.
- `down -v` between runs drops mock state and DB volumes — every test run starts clean.
### `.woodpecker/02-build-push.yml`
```yaml
when:
event: [push, manual]
branch: [dev, stage, main]
depends_on:
- 01-test
matrix:
include:
- PLATFORM: arm64
TAG_SUFFIX: arm
# - PLATFORM: amd64
# TAG_SUFFIX: amd
labels:
platform: ${PLATFORM}
steps:
- name: build-push
image: docker
environment:
REGISTRY_HOST:
from_secret: registry_host
REGISTRY_USER:
from_secret: registry_user
REGISTRY_TOKEN:
from_secret: registry_token
commands:
- echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" -u "$REGISTRY_USER" --password-stdin
- export TAG=${CI_COMMIT_BRANCH}-${TAG_SUFFIX}
- export BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
- |
docker build -f Dockerfile \
--build-arg CI_COMMIT_SHA=$CI_COMMIT_SHA \
--label org.opencontainers.image.revision=$CI_COMMIT_SHA \
--label org.opencontainers.image.created=$BUILD_DATE \
--label org.opencontainers.image.source=$CI_REPO_URL \
-t $REGISTRY_HOST/azaion/<service>:$TAG .
- docker push $REGISTRY_HOST/azaion/<service>:$TAG
volumes:
- /var/run/docker.sock:/var/run/docker.sock
```
Notes:
- `depends_on: [01-test]` is enforced by Woodpecker — a failed `01-test` (any matrix entry) skips this workflow.
- The build workflow does NOT trigger on `pull_request` events: PRs get test signal only; pushes to `dev`/`stage`/`main` produce images. Avoids polluting the registry with PR images.
- Replace `<service>` with the actual service name (matches the registry namespace pattern `azaion/<service>`).
- For repos with split per-arch Dockerfiles, add `DOCKERFILE: Dockerfile.jetson` (or similar) to the matrix entry and substitute `${DOCKERFILE}` for `Dockerfile` in the `docker build -f` line.
### Variations by stack
The contract is language-agnostic because the runner is `docker compose`. The Dockerfile inside `e2e/` selects the test framework:
| Stack | `e2e/Dockerfile` runs |
|-------|----------------------|
| Python | `pytest --csv=/results/report.csv -v` |
| .NET | `dotnet test --logger:"trx;LogFileName=/results/report.trx"` (convert to CSV in a final step if needed) |
| Node/UI | `npm test -- --reporters=default --reporters=jest-junit --outputDirectory=/results` |
| Rust | `cargo test --no-fail-fast -- --format json > /results/report.json` |
When the repo has **only unit tests** (no `e2e/docker-compose.test.yml`), drop the compose orchestration and run the native test command directly inside a stack-appropriate image. Keep the same two-workflow split — `01-test.yml` runs unit tests, `02-build-push.yml` is unchanged.
### Manual-trigger override (test infrastructure not yet validated)
If a repo ships a complete `e2e/` layout but the test fixtures are not yet validated end-to-end (e.g., expected-results data is still being authored), gate `01-test.yml` on `event: [manual]` only and add a TODO comment pointing to the unblocking task. The `02-build-push.yml` workflow drops its `depends_on` clause for the manual-only window — an explicit and reversible exception, not a permanent split.
@@ -31,7 +31,6 @@ _docs/
│ ├── components.md
│ └── flows/
├── 04_verification_log.md # Step 4
├── glossary.md # Step 4.5 (confirmed-by-user)
├── FINAL_report.md # Step 7
└── state.json # Resumability
```
@@ -50,7 +49,6 @@ Maintained in `DOCUMENT_DIR/state.json` for resumability:
"modules_remaining": ["services/auth", "api/endpoints"],
"module_batch": 1,
"components_written": [],
"step_4_5_glossary_vision": "not_started",
"last_updated": "2026-03-21T14:00:00Z"
}
```
+2 -102
View File
@@ -15,7 +15,7 @@ Covers three related modes that share the same 8-step pipeline:
## Progress Tracking
Create a TodoWrite with all steps (0 through 7, including the inline Step 2.5 Module Layout Derivation and Step 4.5 Glossary & Architecture Vision). Update status as each step completes.
Create a TodoWrite with all steps (0 through 7). Update status as each step completes.
## Steps
@@ -251,107 +251,7 @@ Apply corrections inline to the documents that need them.
**BLOCKING**: Present verification summary to user. Do NOT proceed until user confirms corrections are acceptable or requests additional fixes.
---
### Step 4.5: Glossary & Architecture Vision (BLOCKING)
**Role**: Software architect + business analyst
**Goal**: Reconcile the AI's verified understanding of the codebase with the user's intended terminology and architecture vision. Existing-code projects often carry domain language and structural intent that is invisible from code alone (synonyms, deprecated names, modules that are "supposed to" be split, components the user thinks of as one logical unit even though they live in two folders). This step makes that intent explicit before any downstream skill (refactor, decompose, new-task) acts on the docs.
**When this step runs**:
- Always, after Step 4 (Verification Pass) — for Full and Resume modes.
- **Skipped** in Focus Area mode (the glossary/vision is system-wide; running it on a partial scan would produce a partial glossary). Resume the user once a full pass exists.
**Inputs** (already on disk after Step 4):
- `DOCUMENT_DIR/architecture.md`, `system-flows.md`, `data_model.md`, `deployment/*`
- `DOCUMENT_DIR/components/*/description.md`
- `DOCUMENT_DIR/modules/*.md`
- `DOCUMENT_DIR/04_verification_log.md` (so the AI knows which doc parts are confirmed vs. flagged)
**Outputs**:
- `DOCUMENT_DIR/glossary.md` (NEW)
- `DOCUMENT_DIR/architecture.md` updated in place: a new `## Architecture Vision` section is prepended (or merged into an existing "Overview" / "Vision" heading if already present); existing technical sections are preserved verbatim
**Procedure**:
1. **Draft glossary** from verified docs:
- Domain entities, processes, roles named in module/component docs
- Acronyms / abbreviations
- Internal codenames (project, service, model names) that recur in the codebase
- Synonym pairs the AI noticed (e.g., the codebase uses "flight" but module comments say "mission")
- Stakeholder personas if any docs reference them
Each entry: one-line definition + source reference (`source: components/03_flights/description.md`). Skip generic CS/industry terms.
2. **Draft architecture vision** as the AI currently understands the codebase:
- **One paragraph**: what the system is, who runs it, the runtime topology shape (monolith / services / pipeline / library / hybrid), and the dominant pattern (e.g., "submodule-based meta-repo with REST + SSE between UI and backend").
- **Components & responsibilities** (one-line each), pulled from `components/*/description.md`.
- **Major data flows** (one or two sentences each), pulled from `system-flows.md`.
- **Architectural principles / non-negotiables** the AI inferred from the code (e.g., "DB-driven config", "all UI traffic via REST + SSE only", "no per-component shared state"). Mark each with `inferred-from: <source>`.
- **Open questions / drift signals**: places where the code disagrees with itself, or where the AI cannot tell intent from implementation (e.g., two components doing similar work — is that legacy duplication or deliberate?).
3. **Present condensed view** to the user (NOT the full draft files — a synopsis only):
```
══════════════════════════════════════
REVIEW: Glossary + Architecture Vision (existing code)
══════════════════════════════════════
Glossary (N terms drafted from verified docs):
- <Term>: <one-line definition>
- ...
Architecture Vision — as inferred from the codebase:
<one-paragraph synopsis>
Components / responsibilities:
- <component>: <one-line>
- ...
Principles / non-negotiables (inferred):
- <principle> [inferred-from: <source>]
- ...
Open questions / drift signals:
- <q1>
- <q2>
══════════════════════════════════════
A) Inferred vision matches my intent — write the files
B) Add / correct entries (provide diffs — terms, components,
principles, or rename pairs)
C) Resolve the open questions / drift signals first
══════════════════════════════════════
Recommendation: pick C if any drift signals exist;
otherwise B if the vision misses
project-specific intent; A only when
the inferred vision is exactly right.
══════════════════════════════════════
```
4. **Iterate**:
- On B → integrate the user's diffs/additions, re-present, loop until A.
- On C → ask the listed open questions in one batch (M4-style), integrate answers, re-present.
- **Do NOT proceed to step 5 until the user picks A.**
5. **Save**:
- Write `DOCUMENT_DIR/glossary.md`, alphabetical, with a top-line `**Status**: confirmed-by-user` and the date.
- Update `DOCUMENT_DIR/architecture.md`:
- If a `## Architecture Vision` (or `## Vision` / `## Overview`) section already exists at the top, replace its body with the confirmed paragraph + components + principles.
- Otherwise, insert `## Architecture Vision` as the first H2 after the title; preserve every existing H2 below.
- Do NOT delete or re-order existing technical sections (Tech Stack, Deployment Model, Data Model, NFRs, ADRs).
6. **Update `state.json`**: mark `step_4_5_glossary_vision: confirmed`. Resume on rerun must skip this step unless the user explicitly invokes `/document --refresh-vision`.
**Self-verification**:
- [ ] Every glossary entry traces to at least one file under `DOCUMENT_DIR/`
- [ ] Every component listed in the vision matches a folder under `DOCUMENT_DIR/components/`
- [ ] All open questions are answered or explicitly deferred (with the user's acknowledgement)
- [ ] `architecture.md` still contains all H2 sections it had before this step
- [ ] User picked option A on the latest condensed view
**BLOCKING**: Do NOT proceed to the session boundary / Step 5 until both files are saved and the user has picked A.
---
**Session boundary**: After Step 4.5 is confirmed, suggest a session break before proceeding to the synthesis steps (57). These steps produce different artifact types and benefit from fresh context:
**Session boundary**: After verification is confirmed, suggest a session break before proceeding to the synthesis steps (57). These steps produce different artifact types and benefit from fresh context:
```
══════════════════════════════════════
+56 -161
View File
@@ -1,59 +1,41 @@
---
name: implement
description: |
Implement tasks sequentially with dependency-aware batching and integrated code review.
Orchestrate task implementation with dependency-aware batching, parallel subagents, and integrated code review.
Reads flat task files and _dependencies_table.md from TASKS_DIR, computes execution batches via topological sort,
implements tasks one at a time in dependency order, runs code-review skill after each batch, and loops until done.
launches up to 4 implementer subagents in parallel, runs code-review skill after each batch, and loops until done.
Use after /decompose has produced task files.
Trigger phrases:
- "implement", "start implementation", "implement tasks"
- "execute tasks"
- "run implementers", "execute tasks"
category: build
tags: [implementation, batching, code-review]
tags: [implementation, orchestration, batching, parallel, code-review]
disable-model-invocation: true
---
# Implementation Runner
# Implementation Orchestrator
Implement all tasks produced by the `/decompose` skill. This skill reads task specs, computes execution order, writes the code and tests for each task **sequentially** (no subagents, no parallel execution), validates results via the `/code-review` skill, and escalates issues.
Orchestrate the implementation of all tasks produced by the `/decompose` skill. This skill is a **pure orchestrator** — it does NOT write implementation code itself. It reads task specs, computes execution order, delegates to `implementer` subagents, validates results via the `/code-review` skill, and escalates issues.
For each task the main agent receives a task spec, analyzes the codebase, implements the feature, writes tests, and verifies acceptance criteria — then moves on to the next task.
The `implementer` agent is the specialist that writes all the code — it receives a task spec, analyzes the codebase, implements the feature, writes tests, and verifies acceptance criteria.
## Core Principles
- **Sequential execution**: implement one task at a time. Do NOT spawn subagents and do NOT run tasks in parallel. (See `.cursor/rules/no-subagents.mdc`.)
- **Dependency-aware ordering**: tasks run only when all their dependencies are satisfied
- **Batching for review, not parallelism**: tasks are grouped into batches so `/code-review` and commits operate on a coherent unit of work — all tasks inside a batch are still implemented one after the other
- **Orchestrate, don't implement**: this skill delegates all coding to `implementer` subagents
- **Dependency-aware batching**: tasks run only when all their dependencies are satisfied
- **Max 4 parallel agents**: never launch more than 4 implementer subagents simultaneously
- **File isolation**: no two parallel agents may write to the same file
- **Integrated review**: `/code-review` skill runs automatically after each batch
- **Completeness before testing**: product implementation is not done until code is checked against task outcomes, included scope, architecture/component promises, named runtime dependencies, and unresolved scaffold/native placeholders — not just task AC tests
- **Runtime dependency reality**: production code cannot satisfy a task by exposing only a protocol, fake runner, deterministic fallback, or "native bridge" placeholder when the task/architecture promises a concrete internal capability such as BASALT VIO, FAISS retrieval, LightGlue matching, or a full A-Z localization pipeline. Stubs are allowed only for external systems and tests.
- **Auto-start**: batches start immediately — no user confirmation before a batch
- **Auto-start**: batches launch immediately — no user confirmation before a batch
- **Gate on failure**: user confirmation is required only when code review returns FAIL
- **Commit per batch**: after each batch is confirmed, commit. Ask the user whether to push to remote unless the user previously opted into auto-push for this session.
## Context Resolution
- TASKS_DIR: `_docs/02_tasks/`
- Task files: selected `*.md` files in `TASKS_DIR/todo/` (excluding files starting with `_`)
- Task files: all `*.md` files in `TASKS_DIR/todo/` (excluding files starting with `_`)
- Dependency table: `TASKS_DIR/_dependencies_table.md`
### Task Selection Context
The invoking flow decides which task category this run should execute. The implement skill must honor that selected context instead of consuming every file in `todo/`.
| Context | Selected task files |
|---------|---------------------|
| Product implementation | Task specs that are not test-only and not refactoring specs |
| Test implementation | `*_test_infrastructure.md` plus task specs whose `Component` or `Epic` identifies `Blackbox Tests` |
| Refactoring | Task specs whose filename or task ID includes `_refactor_` |
If no explicit context is provided, infer it from the active autodev step:
- greenfield Step 7 or existing-code Step 10 → Product implementation
- greenfield Step 10 or existing-code Step 6 → Test implementation
- refactor Phase 4 → Refactoring
Unselected task files remain in `TASKS_DIR/todo/` for their later flow step.
### Task Lifecycle Folders
```
@@ -64,31 +46,9 @@ TASKS_DIR/
└── done/ ← completed tasks (moved here after implementation)
```
### Suite-level invocation context (meta-repo flow)
When invoked from `.cursor/skills/autodev/flows/meta-repo.md` Step 3.5 (or any caller that supplies the same context envelope), the skill receives:
```
suite_level: true
TASKS_DIR: <override> # e.g., _docs/tasks/ (vs. default _docs/02_tasks/)
module_layout_path: <override> # e.g., _docs/tasks/_suite_module_layout.md
```
When `suite_level: true` is present, the following gate adjustments apply — and ONLY these. All other steps (114, 16) execute unchanged:
1. **TASKS_DIR override** is honored throughout the skill (Step 1 Parse, Step 13 Archive, Step 15 input paths if it ran). Default `_docs/02_tasks/` is replaced by the supplied path.
2. **module_layout_path override** is read instead of the hardcoded `_docs/02_document/module-layout.md` in Step 4 (Assign File Ownership). The supplied file uses the same `Per-Component Mapping` schema. If both the override and the hardcoded path are missing, behavior is unchanged from default mode (STOP and instruct).
3. **Step 14.5 (Cumulative Code Review) — SKIPPED**. The meta-repo has no `_docs/02_document/architecture_compliance_baseline.md`; cross-task drift is captured by the next `monorepo-status` cycle instead.
4. **Step 15 (Product Implementation Completeness Gate) — SKIPPED**. The gate's hard inputs (`_docs/02_document/architecture.md`, `system-flows.md`, `components/*/description.md`) do not exist in the meta-repo artifact layout. Suite-level tasks are infrastructure / coordination work (renames, cross-repo edits, suite-root infra additions), not feature implementation; the equivalent completeness signal is the next `monorepo-status` drift report (which the meta-repo flow re-runs immediately after Step 3.5 returns).
5. **Final report filename**: `_docs/03_implementation/suite_implementation_report_{run_name}.md` (in addition to the existing feature/test/refactor variants). Batch reports follow `_docs/03_implementation/suite_batch_{NN}_report.md`.
6. **Tracker integration** (Step 5: In Progress, Step 12: In Testing) runs unchanged — suite-level tickets follow the same tracker rules as any other.
Without `suite_level: true`, none of these adjustments apply and the skill runs exactly as documented in default mode.
## Prerequisite Checks (BLOCKING)
1. `TASKS_DIR/todo/` exists and contains at least one task file for the selected context **STOP if missing**
- Exception for Product implementation re-entry: if no selected product tasks remain in `todo/`, but the active autodev state is Step 7 or the latest product completeness report is missing/invalid/contains `FAIL`, skip directly to Step 15 (Product Implementation Completeness Gate). This gate may create remediation tasks and return to Step 1. Do not write a final implementation report from this state.
1. `TASKS_DIR/todo/` exists and contains at least one task file — **STOP if missing**
2. `_dependencies_table.md` exists — **STOP if missing**
3. At least one task is not yet completed — **STOP if all done**
4. **Working tree is clean** — run `git status --porcelain`; the output must be empty.
@@ -96,16 +56,16 @@ Without `suite_level: true`, none of these adjustments apply and the skill runs
- A) Commit or stash stray changes manually, then re-invoke `/implement`
- B) Agent commits stray changes as a single `chore: WIP pre-implement` commit and proceeds
- C) Abort
- Rationale: each batch ends with a commit. Unrelated uncommitted changes would get silently folded into batch commits otherwise.
- Rationale: implementer subagents edit files in parallel and commit per batch. Unrelated uncommitted changes get silently folded into batch commits otherwise.
- This check is repeated at the start of each batch iteration (see step 6 / step 14 Loop).
## Algorithm
### 1. Parse
- Read selected task `*.md` files from `TASKS_DIR/todo/` (excluding files starting with `_`)
- Read all task `*.md` files from `TASKS_DIR/todo/` (excluding files starting with `_`)
- Read `_dependencies_table.md` — parse into a dependency graph (DAG)
- Validate: no circular dependencies in the selected task graph, all referenced selected-task dependencies exist or are already completed in `TASKS_DIR/done/`
- Validate: no circular dependencies, all referenced dependencies exist
### 2. Detect Progress
@@ -118,23 +78,22 @@ Without `suite_level: true`, none of these adjustments apply and the skill runs
- Topological sort remaining tasks
- Select tasks whose dependencies are ALL satisfied (completed)
- A batch is simply a coherent group of tasks for review + commit. Within the batch, tasks are implemented sequentially in topological order.
- Cap the batch size at a reasonable review scope (default: 4 tasks)
- If a ready task depends on any task currently being worked on in this batch, it must wait for the next batch
- Cap the batch at 4 parallel agents
- If the batch would exceed 20 total complexity points, suggest splitting and let the user decide
### 4. Assign File Ownership
The authoritative file-ownership map is `_docs/02_document/module-layout.md` (produced by the decompose skill's Step 1.5), unless `suite_level: true` was supplied in the invocation context — in which case the `module_layout_path` override is read instead (see "Suite-level invocation context" above). Task specs are purely behavioral — they do NOT carry file paths. Derive ownership from the layout, not from the task spec's prose.
The authoritative file-ownership map is `_docs/02_document/module-layout.md` (produced by the decompose skill's Step 1.5). Task specs are purely behavioral — they do NOT carry file paths. Derive ownership from the layout, not from the task spec's prose.
For each task in the batch:
- Read the task spec's **Component** field.
- Look up the component in `_docs/02_document/module-layout.md` → Per-Component Mapping.
- Set **OWNED** = the component's `Owns` glob (the files this task is allowed to write).
- Set **OWNED** = the component's `Owns` glob (exclusive write for the duration of the batch).
- Set **READ-ONLY** = Public API files of every component in the component's `Imports from` list, plus all `shared/*` Public API files.
- Set **FORBIDDEN** = every other component's `Owns` glob, and every other component's internal (non-Public API) files.
- If the task is a shared / cross-cutting task (lives under `shared/*`), OWNED = that shared directory; READ-ONLY = nothing; FORBIDDEN = every component directory.
Since execution is sequential, there is no parallel-write conflict to resolve; ownership here is a **scope discipline** check — it stops a task from drifting into unrelated components even when alone.
- If two tasks in the same batch map to the same component or overlapping `Owns` globs, schedule them sequentially instead of in parallel.
If `_docs/02_document/module-layout.md` is missing or the component is not found:
- STOP the batch.
@@ -143,30 +102,31 @@ If `_docs/02_document/module-layout.md` is missing or the component is not found
### 5. Update Tracker Status → In Progress
For each task in the batch, transition its ticket status to **In Progress** via the configured work item tracker (see `protocols.md` for tracker detection) before starting work. If `tracker: local`, skip this step. If a tracker operation fails unexpectedly, follow `.cursor/rules/tracker.mdc`.
For each task in the batch, transition its ticket status to **In Progress** via the configured work item tracker (see `protocols.md` for tracker detection) before launching the implementer. If `tracker: local`, skip this step.
### 6. Implement Tasks Sequentially
### 6. Launch Implementer Subagents
**Per-batch dirty-tree re-check**: before starting the batch, run `git status --porcelain`. On the first batch this is guaranteed clean by the prerequisite check. On subsequent batches, the previous batch ended with a commit so the tree should still be clean. If the tree is dirty at this point, STOP and surface the dirty files to the user using the same A/B/C choice as the prerequisite check. The most likely causes are a failed commit in the previous batch, a user who edited files mid-loop, or a pre-commit hook that re-wrote files and was not captured.
**Per-batch dirty-tree re-check**: before launching subagents, run `git status --porcelain`. On the first batch this is guaranteed clean by the prerequisite check. On subsequent batches, the previous batch ended with a commit so the tree should still be clean. If the tree is dirty at this point, STOP and surface the dirty files to the user using the same A/B/C choice as the prerequisite check. The most likely causes are a failed commit in the previous batch, a user who edited files mid-loop, or a pre-commit hook that re-wrote files and was not captured.
For each task in the batch **in topological order, one at a time**:
1. Read the task spec file.
2. Respect the file-ownership envelope computed in Step 4 (OWNED / READ-ONLY / FORBIDDEN).
3. Implement the feature and write/update tests for every acceptance criterion in the spec. Tests for internal product behavior must exercise the production implementation path. If a test cannot run in the current environment (e.g., TensorRT requires GPU), the test must still exist and skip/block with a clear prerequisite reason, but that skip does not make missing production code complete.
4. Run the relevant tests locally before moving on to the next task in the batch. If tests fail, fix in-place — do not defer.
5. Capture a short per-task status line (files changed, tests pass/fail, any blockers) for the batch report.
For each task in the batch, launch an `implementer` subagent with:
- Path to the task spec file
- List of files OWNED (exclusive write access)
- List of files READ-ONLY
- List of files FORBIDDEN
- **Explicit instruction**: the implementer must write or update tests that validate each acceptance criterion in the task spec. If a test cannot run in the current environment (e.g., TensorRT requires GPU), the test must still be written and skip with a clear reason.
Do NOT spawn subagents and do NOT attempt to implement two tasks simultaneously, even if they touch disjoint files. See `.cursor/rules/no-subagents.mdc`.
Launch all subagents immediately — no user confirmation.
### 7. Collect Status
### 7. Monitor
- After all tasks in the batch are finished, aggregate the per-task status lines into a structured batch status.
- If any task reported "Blocked", log the blocker with the failing task's ID and continue — the batch report will surface it.
- Wait for all subagents to complete
- Collect structured status reports from each implementer
- If any implementer reports "Blocked", log the blocker and continue with others
**Stuck detection** — while implementing a task, watch for these signals in your own progress:
- The same file has been rewritten 3+ times without tests going green → stop, mark the task Blocked, and move to the next task in the batch (the user will be asked at the end of the batch).
- You have tried 3+ distinct approaches without evidence-driven progress → stop, mark Blocked, move on.
- Do NOT loop indefinitely on a single task. Record the blocker and proceed.
**Stuck detection** — while monitoring, watch for these signals per subagent:
- Same file modified 3+ times without test pass rate improving → flag as stuck, stop the subagent, report as Blocked
- Subagent has not produced new output for an extended period → flag as potentially hung
- If a subagent is flagged as stuck, do NOT let it continue looping — stop it and record the blocker in the batch report
### 8. AC Test Coverage Verification
@@ -179,8 +139,8 @@ Before code review, verify that every acceptance criterion in each task spec has
- **Not covered**: no test exists for this AC
If any AC is **Not covered**:
- This is a **BLOCKING** failure — the missing test must be written before proceeding
- Go back to the offending task, add tests for the specific ACs that lack coverage, then re-run this check
- This is a **BLOCKING** failure — the implementer must write the missing test before proceeding
- Re-launch the implementer with the specific ACs that need tests
- If the test cannot run in the current environment (GPU required, platform-specific, external service), the test must still exist and skip with `pytest.mark.skipif` or `pytest.skip()` explaining the prerequisite
- A skipped test counts as **Covered** — the test exists and will run when the environment allows
@@ -229,22 +189,18 @@ Track `auto_fix_attempts` and `escalated_findings` in the batch report for retro
### 12. Update Tracker Status → In Testing
After the batch is committed (and pushed if the user approved pushing), transition the ticket status of each task in the batch to **In Testing** via the configured work item tracker. If `tracker: local`, skip this step. If a tracker operation fails unexpectedly, follow `.cursor/rules/tracker.mdc`.
After the batch is committed and pushed, transition the ticket status of each task in the batch to **In Testing** via the configured work item tracker. If `tracker: local`, skip this step.
### 13. Archive Completed Tasks
Move each completed task file from `TASKS_DIR/todo/` to `TASKS_DIR/done/`.
For product implementation, this archive means "batch implementation accepted." The Product Implementation Completeness Gate can still require follow-up remediation tasks before the feature is complete; it does not move original task files back to `todo/`.
### 14. Loop
- Go back to step 2 until all tasks in `todo/` are done
### 14.5. Cumulative Code Review (every K batches)
**Skipped entirely when `suite_level: true`** (see "Suite-level invocation context" above) — the meta-repo has no `architecture_compliance_baseline.md` to evaluate against; cross-task drift is captured by the next `monorepo-status` cycle.
- **Trigger**: every K completed batches (default `K = 3`; configurable per run via a `cumulative_review_interval` knob in the invocation context)
- **Purpose**: per-batch review (Step 9) catches batch-local issues; cumulative review catches issues that only appear when tasks are combined — architecture drift, cross-task inconsistency, duplicate symbols introduced across different batches, contracts that drifted across producer/consumer batches
- **Scope**: the union of files changed since the **last** cumulative review (or since the start of the run if this is the first)
@@ -260,81 +216,22 @@ For product implementation, this archive means "batch implementation accepted."
- **Interaction with Auto-Fix Gate**: Architecture findings (new category from code-review Phase 7) always escalate per the implement auto-fix matrix; they cannot silently auto-fix
- **Resumability**: if interrupted, the next invocation checks for the latest `cumulative_review_batches_*.md` and computes the changed-file set from batch reports produced after that review
### 15. Product Implementation Completeness Gate
### 15. Final Test Run
Run this gate after all **product implementation** tasks are complete and before writing any final product implementation report or allowing autodev to proceed to testability/test decomposition. Skip this gate when (a) the remaining context is explicitly test implementation or refactoring (as determined by the task files and report filename rules), OR (b) `suite_level: true` was supplied in the invocation context (the gate's inputs do not exist in the meta-repo artifact layout — see "Suite-level invocation context" above).
**Goal**: catch the failure mode where narrow tests validate scaffold behavior while the task's actual outcome, included scope, architecture promise, or named integration remains unimplemented.
Inputs:
- Completed product task specs from `_docs/02_tasks/done/` for the current cycle
- `_docs/02_document/architecture.md`
- `_docs/02_document/system-flows.md`
- Relevant `_docs/02_document/components/*/description.md` files
- Current source code under each completed task's ownership envelope
- Batch reports and code-review reports for the current cycle
For each completed product task:
1. Read these sections from the task spec: `Description`, `Outcome`, `Scope / Included`, `Acceptance Criteria`, `Non-Functional Requirements`, `Constraints`, and explicit named technologies or integrations.
2. Compare those promises against actual source code, not only tests or report prose.
3. Search the task's owned component files for unresolved implementation markers: `placeholder`, `stub`, `reserved`, `TODO`, `NotImplemented`, `pass`, `deterministic`, `fake`, `mock`, `scaffold`, `native bridge`, and empty native/readme-only integration directories. Ignore test fixtures/mocks only when they are under test-owned paths and not used as production behavior.
4. Verify that each named runtime dependency in the task promise is integrated as production behavior, not merely represented by an interface. Examples: if a task promises FAISS, DINOv2, BASALT, LightGlue, OpenCV, RANSAC, a database, cloud service, or hardware SDK, the production code must either call that dependency or contain an adapter that loads and executes the real dependency package. A deterministic fallback, fake runner, empty `native/` package, or "bridge to be supplied later" is **FAIL** unless the task itself explicitly scoped the dependency out before implementation started.
5. Distinguish internal implementation from external prerequisites:
- Internal product capabilities (VIO, anchor verification, cache retrieval, safety wrapper, FDR, MAVLink emission) must be implemented in production code before the task can pass.
- External systems/hardware/data (Jetson device, physical camera, ArduPilot process, QGC, third-party service credentials, unavailable licensed dataset) may be `BLOCKED` only when production code exists and the missing prerequisite is outside the product boundary.
6. Verify tests exercise the real implementation path where local prerequisites exist. Environment-gated tests may skip only with an explicit prerequisite reason; they do not make missing production code complete.
7. For any architecture promise that describes an end-to-end user outcome, verify there is an executable production pipeline connecting the relevant components. Isolated component contracts and test-only harness orchestration are not enough.
8. Classify each task:
- **PASS**: task promises are implemented or explicitly out of scope in the task itself.
- **BLOCKED**: production code exists but cannot be fully verified due to external hardware/data/license/runtime prerequisites; the blocker is explicit and tests report blocked/skipped with reason.
- **FAIL**: promised production behavior is missing, only scaffolded, or only represented in tests/reports.
Save the audit to `_docs/03_implementation/implementation_completeness_cycle[N]_report.md` with:
- Per-task classification
- Evidence files/symbols checked
- Any unresolved scaffold/native placeholders
- Any named promised technologies not integrated
- Required remediation task suggestions, each sized to 5 points or less
Gate:
- If every product task is `PASS` or `BLOCKED` with explicit prerequisite evidence, continue to Final Test Run.
- If any product task is `FAIL`, STOP. Do not write the final product implementation report and do not proceed to any downstream autodev step. Completed original task files remain in `done/`; the missing work is represented by remediation tasks. Present a Choose block:
- A) Create remediation tasks now and return to implementation
- B) Mark the missing behavior explicitly out of scope in task/docs, then re-run this gate
- C) Abort for manual correction
- Recommendation must normally be A unless the user deliberately accepts reduced scope.
Remediation task creation:
1. For each `FAIL`, create one or more task specs using `.cursor/skills/decompose/templates/task.md`; each remediation task must be sized at 5 points or less.
2. Save each task to `_docs/02_tasks/todo/` with a short name prefixed by `remediate_`.
3. Set **Component** to the failed task's component and set **Dependencies** to the failed task ID plus any remediation prerequisites.
4. Create or defer tracker tickets using the same tracker rules as decompose/new-task: if tracker is available, create tickets immediately; if the user explicitly chose `tracker: local`, keep numeric prefixes with `Tracker: pending` / `Epic: pending`.
5. Append the remediation tasks to `_docs/02_tasks/_dependencies_table.md`.
6. Return to Step 1 (Parse) in **Product implementation** context. The final product implementation report can be written only after remediation tasks complete and this gate reruns without `FAIL`.
### 16. Final Test Run
- After all batches are complete, run the full test suite once unless the invoking flow's immediate next step is `Run Tests`.
- If the next flow step is `Run Tests`, record a handoff in the final implementation report and let `.cursor/skills/test-run/SKILL.md` own the full-suite gate to avoid duplicate full runs.
- When this step does run, read and execute `.cursor/skills/test-run/SKILL.md` (detect runner, run suite, diagnose failures, present blocking choices).
- Test failures are a **blocking gate** — do not proceed until the test-run skill completes with a user decision.
- When tests pass, report final summary.
- After all batches are complete, run the full test suite once
- Read and execute `.cursor/skills/test-run/SKILL.md` (detect runner, run suite, diagnose failures, present blocking choices)
- Test failures are a **blocking gate** — do not proceed until the test-run skill completes with a user decision
- When tests pass, report final summary
## Batch Report Persistence
After each batch completes, save the batch report to `_docs/03_implementation/batch_[NN]_cycle[N]_report.md` for feature implementation (or `batch_[NN]_report.md` for test/refactor runs). Create the directory if it doesn't exist. For product implementation, produce the FINAL implementation report only after the Product Implementation Completeness Gate passes. For test and refactor implementation, produce the FINAL report after all selected tasks complete and the full-suite gate is either run or handed off per Step 16. The filename depends on context:
After each batch completes, save the batch report to `_docs/03_implementation/batch_[NN]_cycle[N]_report.md` for feature implementation (or `batch_[NN]_report.md` for test/refactor runs). Create the directory if it doesn't exist. When all tasks are complete, produce a FINAL implementation report with a summary of all batches. The filename depends on context:
- **Test implementation** (tasks from test decomposition): `_docs/03_implementation/implementation_report_tests.md`
- **Feature implementation**: `_docs/03_implementation/implementation_report_{feature_slug}_cycle{N}.md` where `{feature_slug}` is derived from the batch task names (e.g., `implementation_report_core_api_cycle2.md`) and `{N}` is the current `state.cycle` from `_docs/_autodev_state.md`. If `state.cycle` is absent (pre-migration), default to `cycle1`.
- **Refactoring**: `_docs/03_implementation/implementation_report_refactor_{run_name}.md`
- **Suite-level** (when `suite_level: true` was supplied — see "Suite-level invocation context" above): `_docs/03_implementation/suite_implementation_report_{run_name}.md`. Batch reports use `_docs/03_implementation/suite_batch_{NN}_report.md`. `{run_name}` is derived from the batch task IDs (e.g., `suite_implementation_report_az543_az549_az550.md`).
Determine the context from the task files being implemented: if all tasks have test-related names or belong to a test epic, use the tests filename; if `suite_level: true` was supplied, use the suite filename; otherwise derive the feature slug from the component names and append the cycle suffix.
Determine the context from the task files being implemented: if all tasks have test-related names or belong to a test epic, use the tests filename; otherwise derive the feature slug from the component names and append the cycle suffix.
Batch report filenames must also include the cycle counter when running feature implementation: `_docs/03_implementation/batch_{NN}_cycle{N}_report.md` (test and refactor runs may use the plain `batch_{NN}_report.md` form since they are not cycle-scoped).
@@ -367,10 +264,9 @@ After each batch, produce a structured report:
| Situation | Action |
|-----------|--------|
| Same task rewritten 3+ times without green tests | Mark Blocked, continue batch, escalate at batch end |
| Implementer fails same approach 3+ times | Stop it, escalate to user |
| Task blocked on external dependency (not in task list) | Report and skip |
| File ownership violated (task wrote outside OWNED) | ASK user |
| Product completeness gate finds missing promised implementation | STOP — create remediation tasks or get explicit user scope reduction |
| File ownership conflict unresolvable | ASK user |
| Test failure after final test run | Delegate to test-run skill — blocking gate |
| All tasks complete | Report final summary, suggest final commit |
| `_dependencies_table.md` missing | STOP — run `/decompose` first |
@@ -385,8 +281,7 @@ Each batch commit serves as a rollback checkpoint. If recovery is needed:
## Safety Rules
- Never start a task whose dependencies are not yet completed
- Never run tasks in parallel and never spawn subagents — see `.cursor/rules/no-subagents.mdc`
- If a task is flagged as stuck, stop working on it and report — do not let it loop indefinitely
- Always run the Product Implementation Completeness Gate before final product reports
- Always run or hand off the full test suite after all batches complete (step 16)
- Never launch tasks whose dependencies are not yet completed
- Never allow two parallel agents to write to the same file
- If a subagent fails or is flagged as stuck, stop it and report — do not let it loop indefinitely
- Always run the full test suite after all batches complete (step 15)
@@ -3,31 +3,29 @@
## Topological Sort with Batch Grouping
The `/implement` skill uses a topological sort to determine execution order,
then groups tasks into batches for code review and commit. Execution within a
batch is **sequential** — see `.cursor/rules/no-subagents.mdc`.
then groups tasks into batches for parallel execution.
## Algorithm
1. Build adjacency list from `_dependencies_table.md`
2. Compute in-degree for each task node
3. Initialize the ready set with all nodes that have in-degree 0
3. Initialize batch 0 with all nodes that have in-degree 0
4. For each batch:
a. Select up to 4 tasks from the ready set (default batch size cap)
b. Implement the selected tasks one at a time in topological order
c. When all tasks in the batch complete, remove them from the graph and
decrement in-degrees of dependents
d. Add newly zero-in-degree nodes to the ready set
a. Select up to 4 tasks from the ready set
b. Check file ownership — if two tasks would write the same file, defer one to the next batch
c. Launch selected tasks as parallel implementer subagents
d. When all complete, remove them from the graph and decrement in-degrees of dependents
e. Add newly zero-in-degree nodes to the next batch's ready set
5. Repeat until the graph is empty
## Ordering Inside a Batch
## File Ownership Conflict Resolution
Tasks inside a batch are executed in topological order — a task is only
started after every task it depends on (inside the batch or in a previous
batch) is done. When two tasks have the same topological rank, prefer the
lower-numbered (more foundational) task first.
When two tasks in the same batch map to overlapping files:
- Prefer to run the lower-numbered task first (it's more foundational)
- Defer the higher-numbered task to the next batch
- If both have equal priority, ask the user
## Complexity Budget
Each batch should not exceed 20 total complexity points.
If it does, split the batch and let the user choose which tasks to include.
The budget exists to keep the per-batch code review scope reviewable.
+1 -2
View File
@@ -129,8 +129,7 @@ If `_docs/_repo-config.yaml` already exists:
- Entries removed (component removed from registry)
4. **Ask the user** whether to apply the diff.
5. If applied, **preserve `confirmed: true` flags** for entries that still match — don't reset human-approved mappings.
6. **Preserve user-owned top-level keys verbatim**: `glossary_doc:` (written by autodev meta-repo Step 2.5) and any `assumptions_log:` entries are NEVER edited or removed by this skill. Carry them through unchanged. If the file referenced by `glossary_doc:` no longer exists on disk, surface as an `unresolved:` question — do not auto-clear the field.
7. If user declines, stop — leave config untouched.
6. If user declines, stop — leave config untouched.
### Phase 8: Batch question checkpoint (M4)
@@ -15,8 +15,6 @@ Propagates component changes into the unified documentation set. Strictly scoped
| Root `README.md` **only** if `_repo-config.yaml` lists it as a doc target (e.g., services table) | Install scripts (`ci-*.sh`) → use `monorepo-cicd` |
| Docs index (`_docs/README.md` or similar) cross-reference tables | Component-internal docs (`<component>/README.md`, `<component>/docs/*`) |
| Cross-cutting docs listed in `docs.cross_cutting` | `_docs/_repo-config.yaml` itself (only `monorepo-discover` and `monorepo-onboard` write it) |
| Body of cross-cutting docs **except** the `## Architecture Vision` section (preserved verbatim — owned by autodev meta-repo Step 2.5) | The file at `glossary_doc:` (user-confirmed; only autodev meta-repo Step 2.5 rewrites it). New project terms surfaced during sync are reported back to the user, not silently appended |
| `## Architecture Vision` body — read-only, may be referenced for terminology consistency but never edited | — |
If a component change requires CI/env updates too, tell the user to also run `monorepo-cicd`. This skill does NOT cross domains.
@@ -168,8 +166,6 @@ Append to `_docs/_repo-config.yaml` under `assumptions_log:`:
- Change `confirmed_by_user` or any `confirmed: <bool>` flag
- Auto-commit or push
- Guess a mapping not in the config
- Edit `glossary_doc:` (the file recorded under the config's `glossary_doc:` key)
- Edit the `## Architecture Vision` section of any cross-cutting doc; if a sync would conflict with that section, surface the conflict to the user and skip — do not silently rewrite user-confirmed content
## Edge cases
-152
View File
@@ -1,152 +0,0 @@
---
name: monorepo-e2e
description: Syncs the suite-level integration e2e harness (`e2e/docker-compose.suite-e2e.yml`, fixtures, Playwright runner) when component contracts drift in ways that affect the cross-service scenario. Reads `_docs/_repo-config.yaml` to know which suite-e2e artifacts are in play. Touches ONLY suite-e2e files — never per-component CI, docs, or component internals. Use when a component changes a port, env var, public API endpoint, DB schema column, or detection model that the suite e2e exercises.
---
# Monorepo Suite-E2E
Propagates component changes into the suite-level integration e2e harness. Strictly scoped — never edits docs, component internals, per-component CI configs, or the production deploy compose.
## Scope — explicit
| In scope | Out of scope |
| -------- | ------------ |
| `e2e/docker-compose.suite-e2e.yml` (overlay, healthchecks, seed services) | Production `_infra/deploy/<target>/docker-compose.yml``monorepo-cicd` owns it |
| `e2e/fixtures/init.sql` (seeded rows that the spec depends on) | Component DB migrations — owned by each component |
| `e2e/fixtures/expected_detections.json` (detection baseline) | Detection model itself — owned by `detections/` |
| `e2e/runner/tests/*.spec.ts` selector / contract-driven edits | New scenarios (user-driven, not drift-driven) |
| `e2e/runner/Dockerfile` / `package.json` Playwright version bumps | Net-new e2e infrastructure (use `monorepo-onboard` or initial scaffolding) |
| `.woodpecker/suite-e2e.yml` (suite-level pipeline) | Per-component `.woodpecker/01-test.yml` / `02-build-push.yml``monorepo-cicd` owns those |
| Suite-e2e leftover entries under `_docs/_process_leftovers/` | Per-component leftovers — owned by each component |
If a component change needs doc updates too, tell the user to also run `monorepo-document`. If it needs production-deploy or per-component CI updates, run `monorepo-cicd`. This skill **only** updates the suite-e2e surface.
## Preconditions (hard gates)
1. `_docs/_repo-config.yaml` exists.
2. Top-level `confirmed_by_user: true`.
3. `suite_e2e.*` section is populated in config (see "Required config block" below). If absent, abort and ask the user to extend the config via `monorepo-discover`.
4. Components-in-scope have confirmed contract mappings (port, public API path, DB tables touched), OR user explicitly approves inferred ones.
## Required config block
This skill expects `_docs/_repo-config.yaml` to carry:
```yaml
suite_e2e:
overlay: e2e/docker-compose.suite-e2e.yml
fixtures:
init_sql: e2e/fixtures/init.sql
baseline_json: e2e/fixtures/expected_detections.json
binary_fixtures:
- e2e/fixtures/sample.mp4
- e2e/fixtures/model.tar.gz
runner:
dockerfile: e2e/runner/Dockerfile
package_json: e2e/runner/package.json
spec_dir: e2e/runner/tests
pipeline: .woodpecker/suite-e2e.yml
scenario:
description: "Upload video → detect → overlays → dataset → DB persistence"
components_exercised:
- ui
- annotations
- detections
- postgres-local
api_contracts:
- component: ui
path: /api/admin/auth/login
- component: annotations
path: /api/annotations/media/batch
- component: annotations
path: /api/annotations/media/{id}/annotations
db_tables:
- media
- annotations
- detection
- detection_classes
model_pin:
detections_repo_path: <path-to-model-config-or-classes-source>
classes_source: annotations/src/Database/DatabaseMigrator.cs
```
If `suite_e2e:` is missing the skill **stops** — it does not invent a default mapping.
## Mitigations (M1M7)
- **M1** Separation: this skill only touches suite-e2e files; no production deploy compose, no per-component CI, no docs, no component internals.
- **M3** Factual vs. interpretive: port, env var, API path, DB column — FACTUAL, read from the components' code. Whether a baseline still matches the model — DEFERRED to the user (the skill flags drift, never silently re-records).
- **M4** Batch questions at checkpoints.
- **M5** Skip over guess: a component change that doesn't map cleanly to one of the in-scope artifacts → skip and report.
- **M6** Assumptions footer + append to `_repo-config.yaml` `assumptions_log`.
- **M7** Drift detection: verify every path under `suite_e2e.*` exists on disk; stop if not.
## Workflow
### Phase 1: Drift check (M7)
Verify every file listed under `suite_e2e.*` (excluding `binary_fixtures`, which are gitignored) exists on disk. Missing file → stop and ask:
- Run `monorepo-discover` to refresh, OR
- Skip the missing artifact (recorded in report)
For `binary_fixtures` paths that are absent (expected — they live in S3/LFS), check whether `expected_detections.json._meta.video_sha256` is still a `TBD-...` placeholder. If yes, surface this as a known leftover (`_docs/_process_leftovers/2026-04-22_suite-e2e-binary-fixtures.md`) and continue.
### Phase 2: Determine scope
Same as `monorepo-cicd` Phase 2 — ask the user, or auto-detect. For **auto-detect**, flag commits that touch suite-e2e-relevant concerns:
| Commit pattern | Suite-e2e impact |
| -------------- | ---------------- |
| New port exposed by `<component>` | Healthcheck override may change in `e2e/docker-compose.suite-e2e.yml` |
| New required env var on `<component>` | `e2e/docker-compose.suite-e2e.yml` `e2e-runner` env block + `init.sql` seed |
| Public API path renamed / removed | Spec selector / API call path in `e2e/runner/tests/*.spec.ts` |
| DB schema column renamed in a `db_tables` entry | `init.sql` column reference + spec `pg.query` text |
| New required DB table referenced by spec | `init.sql` insert block (skip if owned by component migration) |
| Detection model rev change in `detections/` | `expected_detections.json` `_meta.model.revision` + flag baseline as stale |
| New canonical detection class added | `expected_detections.json._meta` annotation |
Present the flagged list; confirm.
### Phase 3: Classify changes per component
| Change type | Target suite-e2e files |
| ----------- | ---------------------- |
| Port / env var change | `e2e/docker-compose.suite-e2e.yml` |
| API path / contract change | `e2e/runner/tests/*.spec.ts` |
| DB schema reference change | `e2e/fixtures/init.sql` and spec SQL queries |
| Model / class catalog change | `e2e/fixtures/expected_detections.json` (mark `_meta.fixture_version` bump + leftover entry for binary refresh) |
| Playwright dependency drift | `e2e/runner/package.json` + `e2e/runner/Dockerfile` |
| Suite scenario steps gone stale | **Stop and ask** — scenario edits are user-driven, not drift-driven |
### Phase 4: Apply edits
Edit each in-scope file. After each batch, run `ReadLints` on touched files. Do NOT run the suite e2e itself — that's a downstream pipeline operation, not a sync-skill responsibility.
For `expected_detections.json`: when the model revision changes, the skill **does not** re-record the baseline — the binary fixture cannot be regenerated from the dev environment. Instead:
1. Set `_meta.model.revision` to the new revision.
2. Set `_meta.fixture_version` to a new bumped version with a `-stale` suffix (e.g., `0.2.0-stale`).
3. Append a new entry to `_docs/_process_leftovers/` describing the required re-record.
4. Leave `expected.by_class` untouched — the spec's tolerance check will fail loudly until the binary refresh lands.
### Phase 5: Update assumptions log
Append a new `assumptions_log:` entry to `_docs/_repo-config.yaml` recording:
- Date, components in scope, which suite-e2e files were touched
- Any inferred contract mappings still tagged `confirmed: false`
- Any leftover entries created
### Phase 6: Report
Render a Choose-format summary of the synced files, surface any `_process_leftovers/` entries created, and end. Do NOT auto-commit.
## Self-verification
- [ ] No file outside `e2e/`, `.woodpecker/suite-e2e.yml`, or `_docs/_process_leftovers/` was edited
- [ ] `_docs/_repo-config.yaml` `suite_e2e:` block was not silently mutated except for `assumptions_log` append
- [ ] `expected_detections.json` was not re-recorded (only metadata bumped + leftover added)
- [ ] Every spec edit traces to a flagged commit pattern in Phase 2
- [ ] `ReadLints` clean on every touched file
## Failure handling
Same retry / escalation protocol as `monorepo-cicd` — see `protocols.md`. The most common failure mode is the binary-fixture leftover (sample.mp4 missing or SHA-mismatched); this skill does not attempt to resolve it, only surfaces it.
-4
View File
@@ -59,8 +59,6 @@ Mark each as `complete` / `partial` / `missing` and explain.
- Every component in `components:` appears in the registry — flag mismatches
- Every `docs.root` file cross-referenced in config exists on disk — flag missing
- Every `ci.orchestration_files` and `ci.install_scripts` exists — flag missing
- `glossary_doc:` (if recorded in config) points to a file that exists on disk — flag missing
- The cross-cutting architecture doc identified by `docs.cross_cutting` contains a `## Architecture Vision` section — flag missing (signals the meta-repo flow's Step 2.5 was skipped or the section was removed)
### Section 5: Unresolved questions
@@ -115,8 +113,6 @@ In registry, not in config: [list or "(none)"]
In config, not in registry: [list or "(none)"]
Config-referenced docs missing: [list or "(none)"]
Config-referenced CI files missing: [list or "(none)"]
glossary_doc: [path or "not recorded — run /autodev to capture"]
Architecture Vision section: [present | missing in <doc>]
═══════════════════════════════════════════════════
Unresolved questions
+4 -5
View File
@@ -75,7 +75,7 @@ Record the description verbatim for use in subsequent steps.
**Role**: Technical analyst
**Goal**: Determine whether deep research is needed.
Read the user's description and the existing codebase documentation from DOCUMENT_DIR (architecture.md including its `## Architecture Vision` section, glossary.md, components/, system-flows.md). Use `glossary.md` to keep the new task's name, acceptance-criteria wording, and component references aligned with the user's confirmed vocabulary; flag the task to the user if the request appears to violate an Architecture Vision principle, do not silently allow it.
Read the user's description and the existing codebase documentation from DOCUMENT_DIR (architecture.md, components/, system-flows.md).
**Consult LESSONS.md**: if `_docs/LESSONS.md` exists, read it and look for entries in categories `estimation`, `architecture`, `dependencies` that might apply to the task under consideration. If a relevant lesson exists (e.g., "estimation: auth-related changes historically take 2x estimate"), bias the classification and recommendation accordingly. Note in the output which lessons (if any) were applied.
@@ -134,8 +134,7 @@ The `<task_slug>` is a short kebab-case name derived from the feature descriptio
**Goal**: Determine where and how to insert the new functionality, and whether existing tests cover the new requirements.
1. Read the codebase documentation from DOCUMENT_DIR:
- `architecture.md` — overall structure (the `## Architecture Vision` H2 is user-confirmed intent and must not be violated by the new task without explicit approval)
- `glossary.md` — project terminology; reuse the user's vocabulary in task names, AC, and component references
- `architecture.md` — overall structure
- `components/` — component specs
- `system-flows.md` — data flows (if exists)
- `data_model.md` — data model (if exists)
@@ -282,7 +281,7 @@ Present using the Choose format for each decision that has meaningful alternativ
- Update **Epic** field: `[EPIC-ID]`
3. Rename the file from `[##]_[short_name].md` to `[TICKET-ID]_[short_name].md`
If the work item tracker is not authenticated or unavailable, follow `.cursor/rules/tracker.mdc` before continuing. Only if the user explicitly chooses `tracker: local`:
If the work item tracker is not authenticated or unavailable (`tracker: local`):
- Keep the numeric prefix
- Set **Tracker** to `pending`
- Set **Epic** to `pending`
@@ -337,7 +336,7 @@ After the user chooses **Done**:
| Research skill hits a blocker | Follow research skill's own escalation rules |
| Codebase analysis reveals conflicting architectures | **ASK** user which pattern to follow |
| Complexity exceeds 5 points | **WARN** user and suggest splitting into multiple tasks |
| Work item tracker MCP unavailable | Follow `.cursor/rules/tracker.mdc`; do not continue in local mode unless the user explicitly chooses it |
| Work item tracker MCP unavailable | **WARN**, continue with local-only task files |
## Trigger Conditions
+3 -6
View File
@@ -69,7 +69,7 @@ Capture any new questions, findings, or insights that arise during test specific
### Step 2: Solution Analysis
Read and follow `steps/02_solution-analysis.md`. The step opens with **Phase 2a.0: Glossary & Architecture Vision** (BLOCKING) — drafts `_docs/02_document/glossary.md` and a one-paragraph architecture vision, presents the condensed view to the user, iterates until confirmed, then proceeds into the architecture, data-model, and deployment phases. The confirmed vision becomes the first `## Architecture Vision` H2 of `architecture.md`.
Read and follow `steps/02_solution-analysis.md`.
---
@@ -107,7 +107,6 @@ Read and follow `steps/07_quality-checklist.md`.
- **Coding during planning**: this workflow produces documents, never code
- **Multi-responsibility components**: if a component does two things, split it
- **Skipping BLOCKING gates**: never proceed past a BLOCKING marker without user confirmation
- **Skipping the glossary/vision gate (Phase 2a.0)**: drafting `architecture.md` from raw `solution.md` without confirming terminology and vision means the AI's mental model is not aligned with the user's; every downstream artifact will inherit that drift
- **Diagrams without data**: generate diagrams only after the underlying structure is documented
- **Copy-pasting problem.md**: the architecture doc should analyze and transform, not repeat the input
- **Vague interfaces**: "component A talks to component B" is not enough; define the method, input, output
@@ -138,10 +137,8 @@ Read and follow `steps/07_quality-checklist.md`.
│ │
│ 1. Blackbox Tests → test-spec/SKILL.md │
│ [BLOCKING: user confirms test coverage] │
│ 2. Solution Analysis → glossary + vision, architecture,
data model, deployment
│ [BLOCKING 2a.0: user confirms glossary + vision] │
│ [BLOCKING 2a: user confirms architecture] │
│ 2. Solution Analysis → architecture, data model, deployment
[BLOCKING: user confirms architecture]
│ 3. Component Decomp → component specs + interfaces │
│ [BLOCKING: user confirms components] │
│ 4. Review & Risk → risk register, iterations │
@@ -4,105 +4,20 @@
**Goal**: Produce `architecture.md`, `system-flows.md`, `data_model.md`, and `deployment/` from the solution draft
**Constraints**: No code, no component-level detail yet; focus on system-level view
### Phase 2a.0: Glossary & Architecture Vision (BLOCKING)
**Role**: Software architect + business analyst
**Goal**: Align the AI's mental model of the project with the user's intent BEFORE drafting `architecture.md`. Capture domain terminology and the user's high-level architecture vision so every downstream artifact (architecture, components, flows, tests, epics) is grounded in confirmed user intent — not in AI inference.
**Inputs**:
- `_docs/00_problem/problem.md`, `acceptance_criteria.md`, `restrictions.md`
- `_docs/00_problem/input_data/*`
- `_docs/01_solution/solution.md` (and any earlier `solution_draft*.md` siblings)
- Any blackbox-test findings produced in Step 1
**Outputs**:
- `_docs/02_document/glossary.md` (NEW)
- A confirmed "Architecture Vision" paragraph + bullet list held in working memory and used as the spine of Phase 2a's `architecture.md`
**Procedure**:
1. **Draft glossary** — extract project-specific terminology from inputs (NOT generic software terms). Include:
- Domain entities, processes, and roles
- Acronyms / abbreviations
- Internal codenames or product names
- Synonym pairs in active use (e.g., "flight" vs. "mission")
- Stakeholder personas referenced in problem.md
Each entry: one-line definition, plus a parenthetical source (`source: problem.md`, `source: solution.md §3`).
Skip terms that have a single well-known industry meaning (REST, JSON, etc.).
2. **Draft architecture vision** — synthesize from inputs:
- **One paragraph**: what the system is, who uses it, the shape of the runtime topology (monolith / services / pipeline / library / hybrid).
- **Components & responsibilities** (one-line each). At this stage these are *intent-level*, not the formal decomposition that Step 3 produces.
- **Major data flows** (one or two sentences each).
- **Architectural principles / non-negotiables** the user has implied (e.g., "DB-driven config", "no per-component state outside Redis", "all UI traffic via REST + SSE only").
- **Open architectural questions** the AI cannot resolve from inputs alone.
3. **Present condensed view** to the user (NOT the full draft files — a synopsis only):
```
══════════════════════════════════════
REVIEW: Glossary + Architecture Vision
══════════════════════════════════════
Glossary (N terms drafted):
- <Term>: <one-line definition>
- ...
Architecture Vision:
<one-paragraph synopsis>
Components / responsibilities:
- <component>: <one-line>
- ...
Principles / non-negotiables:
- <principle>
- ...
Open questions (AI could not resolve):
- <q1>
- <q2>
══════════════════════════════════════
A) Looks correct — write glossary.md, use vision for Phase 2a
B) I want to add / correct entries (provide diffs)
C) Answer the open questions first, then re-present
══════════════════════════════════════
Recommendation: pick C if open questions exist, otherwise A
══════════════════════════════════════
```
4. **Iterate**:
- On B → integrate the user's diffs/additions, re-present the condensed view, loop until A.
- On C → ask the listed open questions one round (M4-style batch), integrate answers, re-present.
- **Do NOT proceed to step 5 until the user picks A.**
5. **Save**:
- Write `_docs/02_document/glossary.md` with terms in alphabetical order. Include a top-line `**Status**: confirmed-by-user` and the date.
- Hold the confirmed vision (paragraph + components + principles) in working memory; Phase 2a will materialize it into `architecture.md` and **must** preserve every confirmed principle and component intent verbatim.
**Self-verification**:
- [ ] Every glossary entry traces to at least one input file (no invented terms)
- [ ] Every component listed in the vision is one the inputs reference
- [ ] All open questions are either answered or explicitly deferred (with the user's acknowledgement)
- [ ] User picked option A on the latest condensed view
**BLOCKING**: Do NOT proceed to Phase 2a until `glossary.md` is saved and the user has confirmed the architecture vision.
### Phase 2a: Architecture & Flows
1. Read all input files thoroughly
2. Incorporate findings, questions, and insights discovered during Step 1 (blackbox tests)
3. **Apply confirmed vision from Phase 2a.0**: the architecture document must include a top-level `## Architecture Vision` section that contains the user-confirmed paragraph, components, and principles verbatim. The rest of `architecture.md` (tech stack, deployment model, NFRs, ADRs) builds on top of that section, never contradicts it
4. Research unknown or questionable topics via internet; ask user about ambiguities
5. Document architecture using `templates/architecture.md` as structure
6. Document system flows using `templates/system-flows.md` as structure
3. Research unknown or questionable topics via internet; ask user about ambiguities
4. Document architecture using `templates/architecture.md` as structure
5. Document system flows using `templates/system-flows.md` as structure
**Self-verification**:
- [ ] `architecture.md` opens with a `## Architecture Vision` section matching Phase 2a.0
- [ ] Architecture covers all capabilities mentioned in solution.md
- [ ] System flows cover all main user/system interactions
- [ ] No contradictions with problem.md, restrictions.md, or the confirmed vision
- [ ] No contradictions with problem.md or restrictions.md
- [ ] Technology choices are justified
- [ ] Blackbox test findings are reflected in architecture decisions
- [ ] Every term used in `architecture.md` that is project-specific appears in `glossary.md`
**Save action**: Write `architecture.md` and `system-flows.md`
@@ -58,4 +58,4 @@ Do NOT create minimal epics with just a summary and short description. The epic
8. **Create "Blackbox Tests" epic** — this epic will parent the blackbox test tasks created by the `/decompose` skill. It covers implementing the test scenarios defined in `tests/`.
**Save action**: Epics created via the configured tracker MCP. Also saved locally in `epics.md` with ticket IDs. If tracker availability fails, follow `.cursor/rules/tracker.mdc`; only if the user explicitly chooses `tracker: local`, save locally only with pending tracker markers.
**Save action**: Epics created via the configured tracker MCP. Also saved locally in `epics.md` with ticket IDs. If `tracker: local`, save locally only.
+1 -1
View File
@@ -133,4 +133,4 @@ Link to architecture.md and relevant component spec.]
- `component` — a normal per-component epic
- `cross-cutting` — a shared concern that spans ≥2 components
- `tests` — the blackbox-tests epic (always exactly one)
- Complexity points for child issues follow the project standard: 1, 2, 3, 5. Do not create issues above 5 points — split them.
- Complexity points for child issues follow the project standard: 1, 2, 3, 5, 8. Do not create issues above 5 points — split them.
-2
View File
@@ -181,8 +181,6 @@ Categorized measurable criteria with markdown headers and bullet points:
Every criterion must have a measurable value. Vague criteria like "should be fast" are not acceptable — push for "less than 400ms end-to-end".
**AC must be design-independent**: describe testable outcomes only — no libraries, algorithms, params, or design choices. Implementation follows AC, never reverse. (IEEE 830 / Atlassian / GitScrum)
### input_data/
At least one file. Options:
+3 -5
View File
@@ -24,8 +24,6 @@ Phase details live in `phases/` — read the relevant file before executing each
- **Save immediately**: write artifacts to disk after each phase
- **Delegate execution**: all code changes go through the implement skill via task files
- **Ask, don't assume**: when scope or priorities are unclear, STOP and ask the user
- **Exact-fit recommendations**: do not recommend a replacement pattern, library, service, architecture, algorithm, or "modern approach" merely because it improves structure or solves a similar class of problem. It must fit confirmed product constraints, acceptance criteria, operating context, integration boundaries, and current code realities. Otherwise reject it, mark it experimental, or ask the user before adding it to the roadmap.
- **Per-mode API capability verification on replacements**: when a refactor proposes replacing or adding a library/SDK/framework/service that exposes multiple modes or configurations, pin the exact mode the refactored code will use (inputs, outputs, runtime) and verify *that mode* via mandatory `context7` lookup plus a saved Minimum Viable Example before promoting the recommendation to `Selected`. Capability claims at the category level ("supports A, B, C modes") must be cross-checked against the literal mode enumeration — `A, B → A+B` style conflations are the recurring silent-failure path.
## Context Resolution
@@ -59,7 +57,7 @@ Create REFACTOR_DIR and RUN_DIR if missing. If a RUN_DIR with the same name alre
Both modes produce `RUN_DIR/list-of-changes.md` (template: `templates/list-of-changes.md`). Both modes then convert that file into task files in TASKS_DIR during Phase 2.
**Guided mode cleanup**: after `RUN_DIR/list-of-changes.md` is created from the input file, delete the original input file only if it lives outside `RUN_DIR`. If the provided file is already the canonical `RUN_DIR/list-of-changes.md`, keep it as the audit record.
**Guided mode cleanup**: after `RUN_DIR/list-of-changes.md` is created from the input file, delete the original input file to avoid duplication.
## Workflow
@@ -81,10 +79,10 @@ Both modes produce `RUN_DIR/list-of-changes.md` (template: `templates/list-of-ch
- "refactor [specific target]" → skip phase 1 if docs exist
- Default → all phases
**Testability-run specifics** (guided mode invoked by autodev existing-code Step 4 or greenfield Step 8):
**Testability-run specifics** (guided mode invoked by autodev existing-code flow Step 4):
- Run name is `01-testability-refactoring`.
- Phase 3 (Safety Net) is skipped by design — no tests exist yet. Compensating control: the `list-of-changes.md` gate in Phase 1 must be reviewed and approved by the user before Phase 4 runs.
- Scope is MINIMAL and surgical; reject change entries that drift into full refactor territory (see the invoking flow's testability step for allowed/disallowed lists). Flagged entries go to `RUN_DIR/deferred_to_refactor.md` for the next optional full-refactor step or backlog consideration.
- Scope is MINIMAL and surgical; reject change entries that drift into full refactor territory (see existing-code flow Step 4 for allowed/disallowed lists). Flagged entries go to `RUN_DIR/deferred_to_refactor.md` for Step 8 (optional full refactor) consideration.
- After Phase 4 (Execution) completes, write `RUN_DIR/testability_changes_summary.md` as Phase 4.5. Format: one bullet per applied change.
```markdown
# Testability Changes Summary ({{run_name}})
@@ -95,7 +95,7 @@ Also copy to project standard locations:
**Critical step — do not skip.** Before producing the change list, cross-reference documented business flows against actual implementation. This catches issues that static code inspection alone misses.
1. **Read documented flows**: Load `DOCUMENT_DIR/system-flows.md`, `DOCUMENT_DIR/architecture.md` (paying special attention to its `## Architecture Vision` section — that's the user-confirmed structural intent), `DOCUMENT_DIR/glossary.md`, `DOCUMENT_DIR/module-layout.md`, every file under `DOCUMENT_DIR/contracts/`, and `SOLUTION_DIR/solution.md` (whichever exist). Extract every documented business flow, data path, architectural decision, module ownership boundary, and contract shape. Any refactor change that contradicts a confirmed Architecture Vision principle must either be rejected or surfaced to the user before being added to `list-of-changes.md` — those principles are not refactor targets without explicit user approval.
1. **Read documented flows**: Load `DOCUMENT_DIR/system-flows.md`, `DOCUMENT_DIR/architecture.md`, `DOCUMENT_DIR/module-layout.md`, every file under `DOCUMENT_DIR/contracts/`, and `SOLUTION_DIR/solution.md` (whichever exist). Extract every documented business flow, data path, architectural decision, module ownership boundary, and contract shape.
2. **Trace each flow through code**: For every documented flow (e.g., "video batch processing", "image tiling", "engine initialization"), walk the actual code path line by line. At each decision point ask:
- Does the code match the documented/intended behavior?
+4 -29
View File
@@ -7,29 +7,14 @@
## 2a. Deep Research
1. Analyze current implementation patterns
2. Extract the **Project Constraint Matrix** from `problem.md`, `restrictions.md`, `acceptance_criteria.md`, current architecture/docs, and actual code constraints. Include required inputs/outputs, operating context, lifecycle assumptions, integration boundaries, non-functional targets, and hard disqualifiers.
3. Research modern approaches for similar systems
4. For each alternative pattern/library/service/architecture/algorithm, research intrinsic implementation constraints: required inputs/outputs, runtime assumptions, supported deployment modes, resource needs, operational limits, licensing/security constraints, and known failure reports.
**API Capability Verification — Per-Mode (MANDATORY, BLOCKING for proposed replacements)**
When a refactor recommendation replaces (or adds) a library/SDK/framework/service, the same per-mode verification used by `/research` Step 2 applies — selecting a replacement on category fit alone is the same silent-failure path. For every replacement candidate that has multiple modes or configurations:
1. **Pin the exact mode/configuration** the refactored code will use, in one explicit sentence. Inputs (data shapes, sensor counts, payloads, rates), outputs (per `acceptance_criteria.md` and contract files), runtime (matching the project's deployment).
2. **Run `context7` (or equivalent docs lookup)** for the candidate. **Mandatory for every replacement library/SDK/framework candidate**, not optional. Minimum three queries per candidate: mode enumeration, project's exact mode (with input/output shapes), disqualifier probe ("does this mode produce the required output? are there published limitations on this runtime?"). Append URLs to `RUN_DIR/analysis/research_findings.md` references section.
3. **Save a Minimum Viable Example (MVE)** for the pinned mode under `RUN_DIR/analysis/mve_evidence.md` with: source, inputs in example, outputs in example, project inputs, project outputs required, match assessment ✅/⚠️/❌. If no official example covers the project's exact configuration, the recommendation cannot be `Selected` based on category fit alone — it must be `Experimental only` (with required-evidence note) or `Rejected`.
4. **Treat "the same library in a different mode" as a different recommendation.** If the project's pinned mode is `<X>` but the only documented evidence covers `<Y>`, do not silently soften the description. Open a separate recommendation row, with its own MVE, fit assessment, and disqualifiers.
5. **Common silent-failure pattern**: a fact summary paraphrases docs as "supports A, B, C, D modes" when the docs actually mean "supports A; B; C and D as separate orthogonal modes" — no `A+B` combination exists. Cross-check paraphrased capability claims against the literal mode enumeration.
5. Identify what could be done differently
6. Suggest improvements only when they fit the Project Constraint Matrix. A cleaner or more modern approach that violates product constraints must be marked `Rejected` or `Experimental only`, not added as a roadmap recommendation.
2. Research modern approaches for similar systems
3. Identify what could be done differently
4. Suggest improvements based on state-of-the-art practices
Write `RUN_DIR/analysis/research_findings.md`:
- Current state analysis: patterns used, strengths, weaknesses
- Alternative approaches per component: current vs alternative, pros/cons, migration effort
- Prioritized recommendations: quick wins + strategic improvements
- Constraint-fit table: recommendation, **pinned mode/config**, constraints checked, **API capability evidence (MVE link)**, evidence, mismatches/disqualifiers, status (`Selected` / `Rejected` / `Experimental only` / `Needs user decision`)
- For every recommendation that replaces or adds a library/SDK/framework, append a **Restrictions × Candidate-Mode sub-matrix** that walks every numbered line of `restrictions.md` and `acceptance_criteria.md` against the candidate's pinned mode, marking each cell ✅ Pass / ❌ Fail / ❓ Verify / N/A with cited evidence. A recommendation cannot be `Selected` while any cell is ❌ or ❓.
## 2b. Solution Assessment & Hardening Tracks
@@ -37,7 +22,6 @@ Write `RUN_DIR/analysis/research_findings.md`:
2. Identify weak points in codebase, map to specific code areas
3. Perform gap analysis: acceptance criteria vs current state
4. Prioritize changes by impact and effort
5. Reject or escalate any proposed refactor that improves code structure while weakening required behavior, integration contracts, runtime constraints, safety/security posture, or acceptance criteria
Present optional hardening tracks for user to include in the roadmap:
@@ -63,9 +47,6 @@ Write `RUN_DIR/analysis/refactoring_roadmap.md`:
- Gap analysis: what's missing, what needs improvement
- Phased roadmap: Phase 1 (critical fixes), Phase 2 (major improvements), Phase 3 (enhancements)
- Selected hardening tracks and their items
- Applicability gate: each roadmap item must state constraint fit, mismatches, required evidence, and status (`Selected` / `Rejected` / `Experimental only` / `Needs user decision`)
**BLOCKING applicability gate**: Before 2c and 2d, every recommendation in the roadmap must be `Selected`. Items marked `Rejected` are excluded. Items marked `Experimental only` or `Needs user decision` require a user decision before task creation.
## 2c. Create Epic
@@ -74,7 +55,7 @@ Create a work item tracker epic for this refactoring run:
1. Epic name: the RUN_DIR name (e.g., `01-testability-refactoring`)
2. Create the epic via configured tracker MCP
3. Record the Epic ID — all tasks in 2d will be linked under this epic
4. If tracker is unavailable, follow `.cursor/rules/tracker.mdc`; only use `PENDING` placeholders if the user explicitly chooses `tracker: local`
4. If tracker unavailable, use `PENDING` placeholder and note for later
## 2d. Task Decomposition
@@ -98,12 +79,6 @@ Convert the finalized `RUN_DIR/list-of-changes.md` into implementable task files
**Self-verification**:
- [ ] All acceptance criteria are addressed in gap analysis
- [ ] Recommendations are grounded in actual code, not abstract
- [ ] Every recommendation has been checked against the Project Constraint Matrix
- [ ] No recommendation violates product restrictions, acceptance criteria, documented architecture decisions, or actual code integration boundaries
- [ ] Every replacement library/SDK/framework recommendation has a pinned mode/config, a saved MVE in `mve_evidence.md`, and a Restrictions × Candidate-Mode sub-matrix with no ❌ or ❓ cells
- [ ] `context7` (or equivalent) was consulted for every replacement library/SDK/framework recommendation
- [ ] Paraphrased capability claims have been cross-checked against the literal mode-enumeration evidence (no `A, B → A+B` style conflation)
- [ ] Rejected and experimental approaches are documented but not converted into implementation tasks without user approval
- [ ] Roadmap phases are prioritized by impact
- [ ] Epic created and all tasks linked to it
- [ ] Every entry in list-of-changes.md has a corresponding task file in TASKS_DIR
@@ -10,7 +10,7 @@
- All `[TRACKER-ID]_refactor_*.md` files are present
- Each task file has valid header fields (Task, Name, Description, Complexity, Dependencies)
2. Verify `TASKS_DIR/_dependencies_table.md` includes the refactoring tasks
3. Verify all tests pass (safety net from Phase 3 is green), unless this is a testability run where Phase 3 was intentionally skipped
3. Verify all tests pass (safety net from Phase 3 is green)
4. If any check fails, go back to the relevant phase to fix
## 4b. Delegate to Implement Skill
@@ -21,9 +21,9 @@ The implement skill will:
1. Parse task files and dependency graph from TASKS_DIR
2. Detect already-completed tasks (skip non-refactoring tasks from prior workflow steps)
3. Compute execution batches for the refactoring tasks
4. Implement tasks sequentially in topological order (no subagents, no parallelism)
4. Launch implementer subagents (up to 4 in parallel)
5. Run code review after each batch
6. Commit per batch and push only when the user approved pushing
6. Commit and push per batch
7. Update work item ticket status
Do NOT modify, skip, or abbreviate any part of the implement skill's workflow. The refactor skill is delegating execution, not optimizing it.
@@ -47,7 +47,7 @@ After the implement skill completes:
For each successfully completed refactoring task:
1. Transition the work item ticket status to **Done** via the configured tracker MCP
2. If tracker is unavailable, follow `.cursor/rules/tracker.mdc`; if the user explicitly chose `tracker: local`, note the pending status transitions in `RUN_DIR/execution_log.md`
2. If tracker unavailable, note the pending status transitions in `RUN_DIR/execution_log.md`
For any failed or blocked tasks, leave their status as-is (the implement skill already set them to In Testing or blocked).
@@ -32,7 +32,7 @@ For each component doc affected:
## 7d. Update System-Level Documentation
If structural changes were made (new modules, removed modules, changed interfaces):
1. Update `_docs/02_document/architecture.md` if architecture changed — but **never edit the `## Architecture Vision` section**. That section is user-confirmed (plan Phase 2a.0 / document Step 4.5); if a refactor invalidates a vision principle, surface it to the user and let them update the vision themselves before continuing. Update only the technical sections below the Vision H2.
1. Update `_docs/02_document/architecture.md` if architecture changed
2. Update `_docs/02_document/system-flows.md` if flow sequences changed
3. Update `_docs/02_document/diagrams/components.md` if component relationships changed
@@ -23,7 +23,6 @@ Save as `RUN_DIR/list-of-changes.md`. Produced during Phase 1 (Discovery).
- **Problem**: [what makes this problematic / untestable / coupled]
- **Change**: [what to do — behavioral description, not implementation steps]
- **Rationale**: [why this change is needed]
- **Constraint Fit**: [which product constraints / acceptance criteria / integration boundaries this preserves; or "Rejected — violates ..."]
- **Risk**: [low | medium | high]
- **Dependencies**: [other change IDs this depends on, or "None"]
@@ -32,7 +31,6 @@ Save as `RUN_DIR/list-of-changes.md`. Produced during Phase 1 (Discovery).
- **Problem**: [description]
- **Change**: [description]
- **Rationale**: [description]
- **Constraint Fit**: [description]
- **Risk**: [low | medium | high]
- **Dependencies**: [C01, or "None"]
```
@@ -46,8 +44,6 @@ Save as `RUN_DIR/list-of-changes.md`. Produced during Phase 1 (Discovery).
- **File(s)** must reference actual files verified to exist in the codebase
- **Problem** describes the current state, not the desired state
- **Change** describes what the system should do differently — behavioral, not prescriptive
- **Constraint Fit** proves the change preserves confirmed product requirements, restrictions, acceptance criteria, architecture decisions, and integration contracts
- Do not include changes whose only benefit is structural cleanliness if they weaken required behavior or violate constraints; record those as rejected in analysis instead
- **Dependencies** reference other change IDs within this list; cross-run dependencies use tracker IDs
- In guided mode, the input file entries are validated against actual code and enriched with file paths, risk, and dependencies before writing
- In automatic mode, entries are derived from Phase 1 component analysis and Phase 2 research findings
-21
View File
@@ -30,27 +30,6 @@ Transform vague topics raised by users into high-quality, deliverable research r
- **Internet-first investigation** — do not rely on training data for factual claims; search the web extensively for every sub-question, rephrase queries when results are thin, and keep searching until you have converging evidence from multiple independent sources
- **Multi-perspective analysis** — examine every problem from at least 3 different viewpoints (e.g., end-user, implementer, business decision-maker, contrarian, domain expert, field practitioner); each perspective should generate its own search queries
- **Question multiplication** — for each sub-question, generate multiple reformulated search queries (synonyms, related terms, negations, "what can go wrong" variants, practitioner-focused variants) to maximize coverage and uncover blind spots
- **Component option breadth** — for every component area, build a broad option landscape before selecting. Search direct candidates, adjacent-domain alternatives, commercial/open-source variants, classical/simple baselines, current SOTA, and "do not use" failure cases. A component may not be narrowed to one candidate until alternatives have been searched and rejected with evidence.
- **Component research depth** — for every serious component candidate, go beyond discovery pages. Read official docs, repository/license files, issue discussions, benchmarks, deployment guides, version/platform requirements, security notes, maintenance signals, and real-world failure reports. Extract evidence for inputs/outputs, lifecycle assumptions, runtime/storage/latency fit, integration boundaries, licensing, operational risks, and unsupported scenarios before assigning any selection status.
- **Exact-fit component selection** — never select a component, tool, library, service, architecture pattern, or algorithm merely because it solves a similar class of problem. It must be proven compatible with the project's explicit operating context, constraints, required inputs/outputs, non-functional requirements, lifecycle assumptions, and acceptance criteria. If fit is unproven or mismatched, mark it `Rejected`, `Experimental only`, or escalate for user decision before it can shape the solution.
- **Per-mode API capability verification** *(applies only to technical-component selection — see Research Output Class below)* — when a candidate library/SDK/framework/service exposes multiple modes or configurations, *the candidate is not a single thing*. Pin the exact mode the project will use (one explicit sentence: inputs, outputs, runtime), and verify *that mode* against the project's required inputs/outputs via official docs (mandatory `context7` lookup) plus a saved Minimum Viable Example. Capability claims at the category level ("supports X, Y, Z modes") must be cross-checked against the literal mode enumeration before being treated as project-applicable. Two modes of one library are two distinct candidates for the purposes of the Component Applicability Gate. Does not apply to non-technical research (concept comparison, market/policy investigation, knowledge organization, etc.).
## Research Output Class (BLOCKING — set in Step 1)
Before applying any of the technical-component gates (per-mode API capability verification, Component Applicability Gate, Restrictions × Candidate-Mode sub-matrix, MVE evidence, mandatory `context7` lookup), classify the research output into one of two classes. Record the decision in `00_question_decomposition.md` once, near the top, so every downstream step honors it.
| Class | What the output recommends or selects | Examples | Technical-component gates apply? |
|-------|---------------------------------------|----------|----------------------------------|
| **Technical-component selection** | One or more libraries, SDKs, frameworks, services, protocols, data formats, infrastructure patterns, algorithms, or APIs that will be implemented or operated against | "Pick a vector database", "Compare auth-token strategies for our API", "Should we use Kafka or RabbitMQ?", architecture / tech-stack / migration drafts (Mode A, Mode B) | **Yes — all gates active** |
| **Non-technical investigation** | Concept comparisons, knowledge organization, root-cause investigation of an event, market/policy/regulatory/social analysis, literature review, decision support without committing to specific tooling | "Why did adoption stall in Q3?", "Compare phenomenology vs constructivism", "Map regulatory landscape for X", "What do practitioners say about onboarding under remote-first orgs?" | **No — skip API/MVE/sub-matrix gates; the rest of the 8-step engine still applies** |
How to decide:
1. Inspect the question and the input files (`problem.md`, `restrictions.md`, `acceptance_criteria.md`, or the standalone input file).
2. If the deliverable will name specific software/services/protocols that someone will then build with or operate, it is **Technical-component selection**.
3. If the deliverable is a report, comparison, or recommendation that does not commit to specific tooling, it is **Non-technical investigation**.
4. **Mixed runs are valid.** Some research questions have a non-technical core but include one technical sub-question (or vice versa). In that case classify per component area within the run, not the run as a whole, and note in `00_question_decomposition.md` which component areas trigger the technical-component gates.
When the run is purely **Non-technical investigation**, the rest of the research engine — question decomposition, perspective rotation, exhaustive web search, fact extraction, comparison framework, reasoning chain, validation, deliverable formatting — still applies in full. The sections that get skipped are explicitly the technical gates listed in the table above.
## Context Resolution
@@ -27,26 +27,13 @@
- [ ] Iterative deepening completed: follow-up questions from initial findings were searched
- [ ] No sub-question relies solely on training data without web verification
## Component Option Breadth
- [ ] `00_question_decomposition.md` contains a Component Option Search Plan
- [ ] Every component area was searched across simple baseline, established production, open-source, commercial/vendor, current SOTA, adjacent-domain, no-build/defer, and known-bad options where applicable
- [ ] Every component area has at least 3 realistic candidates, or a documented explanation of why broad searches found fewer
- [ ] Each lead candidate has official/source-of-truth evidence plus independent validation when available
- [ ] Each component area includes at least one baseline/fallback option and at least one rejected or experimental option when possible
- [ ] Alternative names, synonyms, and neighboring-domain terms were searched before declaring the option landscape complete
- [ ] Licensing, runtime, platform, maintenance, and unsupported-scenario searches were performed for every lead, fallback, and rejected candidate
## Mode A Specific
- [ ] Phase 1 completed: AC assessment was presented to and confirmed by user
- [ ] AC assessment consistent: Solution draft respects the (possibly adjusted) acceptance criteria and restrictions
- [ ] Competitor analysis included: Existing solutions were researched
- [ ] All components have comparison tables: Each component lists alternatives with tools, advantages, limitations, security, cost
- [ ] Component options are broad: component tables include baseline, production, open-source, commercial/vendor, SOTA/research, adjacent-domain, defer/no-build, and disqualified options where applicable
- [ ] Tools/libraries verified: Suggested tools actually exist and work as described
- [ ] Component fit matrix completed: `06_component_fit_matrix.md` (or `06_component_fit_matrix/` if split) exists and every selected component/tool/pattern is marked `Selected`
- [ ] No field-adjacent substitution: no selected candidate is chosen only because it solves a similar class of problem while failing the project's explicit constraints
- [ ] Testing strategy covers AC: Tests map to acceptance criteria
- [ ] Tech stack documented (if Phase 3 ran): `tech_stack.md` has evaluation tables, risk assessment, and learning requirements
- [ ] Security analysis documented (if Phase 4 ran): `security_analysis.md` has threat model and per-component controls
@@ -58,9 +45,6 @@
- [ ] New draft is self-contained: Written as if from scratch, no "updated" markers
- [ ] Performance column included: Mode B comparison tables include performance characteristics
- [ ] Previous draft issues addressed: Every finding in the table is resolved in the new draft
- [ ] Existing selected components were challenged against a broad alternative landscape before being kept
- [ ] Existing component fit audited: every old and new component/tool/pattern was checked against `restrictions.md`, `acceptance_criteria.md`, and the Project Constraint Matrix
- [ ] Rejected/experimental candidates are not lead recommendations unless the user explicitly accepted the risk
## Timeliness Check (High-Sensitivity Domain BLOCKING)
@@ -80,7 +64,7 @@ When the research topic has Critical or High sensitivity level:
## Target Audience Consistency Check (BLOCKING)
- [ ] Research boundary clearly defined: `00_question_decomposition.md` has clear population/geography/timeframe/level boundaries
- [ ] Every source has target audience annotated in `01_source_registry.md` (or category files under `01_source_registry/` if split)
- [ ] Every source has target audience annotated in `01_source_registry.md`
- [ ] Mismatched sources properly handled (excluded, annotated, or marked reference-only)
- [ ] No audience confusion in fact cards: Every fact has target audience consistent with research boundary
- [ ] No audience confusion in the report: Policies/research/data cited have consistent target audiences
@@ -92,33 +76,3 @@ When the research topic has Critical or High sensitivity level:
- [ ] Cited facts have corresponding statements in the original text (no over-interpretation)
- [ ] Source publication/update dates annotated; technical docs include version numbers
- [ ] Unverifiable information annotated `[limited source]` and not sole support for core conclusions
## Exact-Fit Validation (BLOCKING)
- [ ] Project Constraint Matrix extracted from problem context before component selection
- [ ] Component fit matrix includes `Component Area`, `Option Family`, and `Pinned Mode/Config` columns
- [ ] Every selected component/tool/library/service/pattern/algorithm has evidence for required inputs/outputs and integration boundaries
- [ ] Every selected candidate has evidence for the operating context and lifecycle assumptions it must support
- [ ] Every selected candidate has evidence for non-functional targets that are binding for the project
- [ ] Known unsupported scenarios and failure reports were searched for every selected candidate
- [ ] Mismatches are recorded as disqualifiers, not softened into generic limitations
- [ ] Any candidate with unproven fit is marked `Experimental only` or escalated for user decision
- [ ] Any candidate with documented constraint conflict is marked `Rejected`
## API Capability Verification (BLOCKING)
**Applicability**: this checklist applies only when the run is classified as **Technical-component selection** (see SKILL.md → Research Output Class). For non-technical research (concept comparison, market/policy investigation, root-cause analysis, knowledge organization), skip this checklist entirely and note the skip in `05_validation_log.md`. For mixed runs, apply only to technical component areas.
For every lead candidate that is a library/SDK/framework/service:
- [ ] The exact mode/configuration the project will use is pinned in one explicit sentence (inputs, outputs, runtime); no vague "supports X" language
- [ ] `context7` (or equivalent docs lookup) was run for the candidate, with at least 3 queries: mode enumeration, project's exact mode, disqualifier probe
- [ ] All consulted URLs from context7 / official docs are appended to `01_source_registry.md` (or files under `01_source_registry/` if split)
- [ ] A Minimum Viable Example (MVE) was saved for the pinned mode in `02_fact_cards.md` / `02_fact_cards/` (or `02_mve_evidence.md`) with: source, inputs in example, outputs in example, project inputs, project outputs required, match assessment ✅/⚠️/❌
- [ ] When the MVE inputs or outputs do not exactly match the project's, the mismatch is cited from the official docs (not inferred), and the candidate is `Experimental only` or `Rejected`
- [ ] When a library has multiple modes, each project-relevant mode appears as its own candidate row (not a single library row that softens across modes)
- [ ] Restrictions × Candidate-Modes sub-matrix in `06_component_fit_matrix.md` (or files under `06_component_fit_matrix/` if split) is filled for every lead candidate, with one row per numbered restriction and per numbered acceptance criterion
- [ ] Sub-matrix uses ✅ / ❌ / ❓ / N/A only — no free-form prose substitutes
- [ ] No `Selected` candidate has any ❌ or ❓ cell in its sub-matrix
- [ ] "Validation gate required" footnotes are explicitly classified as either *API capability* (must be resolved here) or *runtime quality* (may be carried forward)
- [ ] Paraphrased capability claims in fact cards have been cross-checked against the literal mode-enumeration evidence (no `mono, inertial → mono-inertial` style conflation)
@@ -89,7 +89,7 @@ Value Translation:
## Source Registry Entry Template
For each source consulted, immediately append to `01_source_registry.md` (or the appropriate category file under `01_source_registry/` if the artifact has been split — see splittable-artifacts convention in `steps/00_project-integration.md`):
For each source consulted, immediately append to `01_source_registry.md`:
```markdown
## Source #[number]
- **Title**: [source title]
@@ -57,49 +57,22 @@ RESEARCH_DIR/
├── 03_comparison_framework.md # Step 4 output: selected framework and populated data
├── 04_reasoning_chain.md # Step 6 output: fact → conclusion reasoning
├── 05_validation_log.md # Step 7 output: use-case validation results
├── 06_component_fit_matrix.md # Step 7.5 output: component exact-fit gate
└── raw/ # Raw source archive (optional)
├── source_1.md
└── source_2.md
```
#### Splittable artifacts — Layout convention
The following three artifacts MAY equivalently be a **folder** of the same base name when the single-file form has grown unwieldy (typically ≳ 1000 lines or ≳ 200 KB):
- `01_source_registry.md``01_source_registry/`
- `02_fact_cards.md``02_fact_cards/`
- `06_component_fit_matrix.md``06_component_fit_matrix/`
When using the folder form:
- Place a `00_summary.md` index file at the folder root with a short common summary table and the cross-cutting status the single-file form would have carried in its preamble.
- Split per-entry content into category files (e.g. one file per sub-question or per component): `SQ1_*.md`, `C1_*.md`, etc. Keep entry numbering global across the folder so cross-references like "Source #42" still resolve to exactly one place.
- Cross-references from outside the folder may point at either `01_source_registry/00_summary.md` (for the index) or directly at the relevant category file.
```
RESEARCH_DIR/01_source_registry/ # split form (when single-file is too large)
├── 00_summary.md # index + investigation status + compact source table
├── SQ1_existing_systems.md # category file
├── SQ2_canonical_pipeline.md # category file
├── C1_vio.md # per-component file
└── ...
```
Throughout the rest of this skill (other steps, references, templates), the singular `XX.md` form is used as a logical name; treat each occurrence as applying equally to the folder form when the artifact has been split.
### Save Timing & Content
| Step | Save immediately after completion | Filename |
|------|-----------------------------------|----------|
| Mode A Phase 1 | AC & restrictions assessment tables | `00_ac_assessment.md` |
| Step 0-1 | Question type classification + sub-question list | `00_question_decomposition.md` |
| Step 2 | Each consulted source link, tier, summary | `01_source_registry.md` *(splittable, see convention)* |
| Step 3 | Each fact card (statement + source + confidence) | `02_fact_cards.md` *(splittable, see convention)* |
| Step 2 | Each consulted source link, tier, summary | `01_source_registry.md` |
| Step 3 | Each fact card (statement + source + confidence) | `02_fact_cards.md` |
| Step 4 | Selected comparison framework + initial population | `03_comparison_framework.md` |
| Step 6 | Reasoning process for each dimension | `04_reasoning_chain.md` |
| Step 7 | Validation scenarios + results + review checklist | `05_validation_log.md` |
| Step 7.5 | Component exact-fit gate and selection status | `06_component_fit_matrix.md` *(splittable, see convention)* |
| Step 8 | Complete solution draft | `OUTPUT_DIR/solution_draft##.md` |
### Save Principles
@@ -117,12 +90,11 @@ Throughout the rest of this skill (other steps, references, templates), the sing
|------|---------|----------------|
| `00_ac_assessment.md` | AC & restrictions assessment (Mode A only) | After Phase 1 completion |
| `00_question_decomposition.md` | Question type, sub-question list | After Step 0-1 completion |
| `01_source_registry.md` *(splittable)* | All source links and summaries | Continuously updated during Step 2 |
| `02_fact_cards.md` *(splittable)* | Extracted facts and sources | Continuously updated during Step 3 |
| `01_source_registry.md` | All source links and summaries | Continuously updated during Step 2 |
| `02_fact_cards.md` | Extracted facts and sources | Continuously updated during Step 3 |
| `03_comparison_framework.md` | Selected framework and populated data | After Step 4 completion |
| `04_reasoning_chain.md` | Fact → conclusion reasoning | After Step 6 completion |
| `05_validation_log.md` | Use-case validation and review | After Step 7 completion |
| `06_component_fit_matrix.md` *(splittable)* | Exact-fit matrix for every proposed component/tool/pattern with status `Selected` / `Rejected` / `Experimental only` / `Needs user decision` | Before Step 8 deliverable formatting |
| `OUTPUT_DIR/solution_draft##.md` | Complete solution draft | After Step 8 completion |
| `OUTPUT_DIR/tech_stack.md` | Tech stack evaluation and decisions | After Phase 3 (optional) |
| `OUTPUT_DIR/security_analysis.md` | Threat model and security controls | After Phase 4 (optional) |
@@ -6,9 +6,7 @@ Triggered when no `solution_draft*.md` files exist in OUTPUT_DIR, or when the us
**Role**: Professional software architect
> **AC must be design-independent**: describe testable outcomes only — no libraries, algorithms, params, or design choices. Implementation follows AC, never reverse. (IEEE 830 / Atlassian / GitScrum)
A focused preliminary research pass **before** the main solution research. The goal is to validate that the acceptance criteria and restrictions are realistic before designing a solution around them. Any revision proposed in this phase must respect the design-independence rule above — propose AC changes as outcome/budget edits, not as implementation prescriptions.
A focused preliminary research pass **before** the main solution research. The goal is to validate that the acceptance criteria and restrictions are realistic before designing a solution around them.
**Input**: All files from INPUT_DIR (or INPUT_FILE in standalone mode)
@@ -75,18 +73,16 @@ Full 8-step research methodology. Produces the first solution draft.
**Task** (drives the 8-step engine):
1. Research existing/competitor solutions for similar problems — search broadly across industries and adjacent domains, not just the obvious competitors
2. Research the problem thoroughly — all possible ways to solve it, split into components; search for how different fields approach analogous problems
3. Derive a **Project Constraint Matrix** before evaluating component options. Extract exact constraints from `problem.md`, `restrictions.md`, `acceptance_criteria.md`, input data notes, and the Phase 1 AC assessment. Include required inputs/outputs, operating context, runtime envelope, data availability, lifecycle boundaries, non-functional targets, integration boundaries, security constraints, and explicit out-of-scope decisions.
4. For each component, research all possible solutions and find the most efficient state-of-the-art approaches — use multiple query variants and perspectives from Step 1
5. For each promising approach, search for real-world deployment experience: success stories, failure reports, lessons learned, and practitioner opinions
6. Search for contrarian viewpoints — who argues against the common approaches and why? What failure modes exist?
7. Verify that suggested tools/libraries actually exist and work as described — check official repos, latest releases, and community health (stars, recent commits, open issues)
8. For every candidate component/tool/library/service/pattern/algorithm, prove exact fit against the Project Constraint Matrix. A field-adjacent solution is not selectable unless its documented implementation assumptions match the project's constraints. Mismatches must be recorded as disqualifiers and the candidate marked `Rejected`, `Experimental only`, or `Needs user decision`.
9. Include security considerations in each component analysis
10. Provide rough cost estimates for proposed solutions
3. For each component, research all possible solutions and find the most efficient state-of-the-art approaches — use multiple query variants and perspectives from Step 1
4. For each promising approach, search for real-world deployment experience: success stories, failure reports, lessons learned, and practitioner opinions
5. Search for contrarian viewpoints — who argues against the common approaches and why? What failure modes exist?
6. Verify that suggested tools/libraries actually exist and work as described — check official repos, latest releases, and community health (stars, recent commits, open issues)
7. Include security considerations in each component analysis
8. Provide rough cost estimates for proposed solutions
Be concise in formulating. The fewer words, the better, but do not miss any important details.
**Save action**: Write `RESEARCH_DIR/06_component_fit_matrix.md` (or its split-folder equivalent under `RESEARCH_DIR/06_component_fit_matrix/`, per the splittable-artifacts convention in `00_project-integration.md`) before the final draft, then write `OUTPUT_DIR/solution_draft##.md` using template: `templates/solution_draft_mode_a.md`
**Save action**: Write `OUTPUT_DIR/solution_draft##.md` using template: `templates/solution_draft_mode_a.md`
---
@@ -10,25 +10,18 @@ Full 8-step research methodology applied to assessing and improving an existing
**Task** (drives the 8-step engine):
1. Read the existing solution draft thoroughly
2. Derive or refresh the **Project Constraint Matrix** from all files in INPUT_DIR. Include required inputs/outputs, operating context, runtime envelope, data availability, lifecycle boundaries, non-functional targets, integration boundaries, security constraints, and explicit out-of-scope decisions.
3. Audit every component/decision in the existing draft against the Project Constraint Matrix before researching alternatives:
- If a component's documented implementation assumptions match the project constraints, keep it eligible and record evidence.
- If fit is unproven, mark it `Experimental only` until evidence is found.
- If constraints conflict, mark it `Rejected` and search for alternatives.
- If rejecting it changes product behavior or risk materially, escalate for user decision.
4. Research in internet extensively — for each component/decision in the draft, search for:
2. Research in internet extensively — for each component/decision in the draft, search for:
- Known problems and limitations of the chosen approach
- What practitioners say about using it in production
- Better alternatives that may have emerged recently
- Common failure modes and edge cases
- How competitors/similar projects solve the same problem differently
5. Search specifically for contrarian views: "why not [chosen approach]", "[chosen approach] criticism", "[chosen approach] failure"
6. Identify security weak points and vulnerabilities — search for CVEs, security advisories, and known attack vectors for each technology in the draft
7. Identify performance bottlenecks — search for benchmarks, load test results, and scalability reports
8. For each identified weak point, search for multiple solution approaches and compare them
9. For every revised candidate, prove exact fit against the Project Constraint Matrix. Do not select field-adjacent or "similar problem" options unless their intrinsic implementation constraints match the project.
10. Based on findings, form a new solution draft in the same format
3. Search specifically for contrarian views: "why not [chosen approach]", "[chosen approach] criticism", "[chosen approach] failure"
4. Identify security weak points and vulnerabilities — search for CVEs, security advisories, and known attack vectors for each technology in the draft
5. Identify performance bottlenecks — search for benchmarks, load test results, and scalability reports
6. For each identified weak point, search for multiple solution approaches and compare them
7. Based on findings, form a new solution draft in the same format
**Save action**: Write `RESEARCH_DIR/06_component_fit_matrix.md` (or its split-folder equivalent under `RESEARCH_DIR/06_component_fit_matrix/`, per the splittable-artifacts convention in `00_project-integration.md`) before the final draft, then write `OUTPUT_DIR/solution_draft##.md` (incremented) using template: `templates/solution_draft_mode_b.md`
**Save action**: Write `OUTPUT_DIR/solution_draft##.md` (incremented) using template: `templates/solution_draft_mode_b.md`
**Optional follow-up**: After Mode B completes, the user can request Phase 3 (Tech Stack Consolidation) or Phase 4 (Security Deep Dive) using the revised draft. These phases work identically to their Mode A descriptions in `steps/01_mode-a-initial-research.md`.
@@ -40,7 +40,6 @@ Key principle: Critical-sensitivity topics (AI/LLMs, blockchain) require sources
- "What existing/competitor solutions address this problem?"
- "What are the component parts of this problem?"
- "For each component, what are the state-of-the-art solutions?"
- "For each component, what are the practical alternatives across simple baseline, established production option, open-source option, commercial option, current SOTA, adjacent-domain option, and no-build/defer option?"
- "What are the security considerations per component?"
- "What are the cost implications of each approach?"
@@ -49,7 +48,6 @@ Key principle: Critical-sensitivity topics (AI/LLMs, blockchain) require sources
- "What are the security vulnerabilities in the proposed architecture?"
- "Where are the performance bottlenecks?"
- "What solutions exist for each identified issue?"
- "For each component already selected in the draft, what alternatives should be considered before keeping, replacing, or rejecting it?"
**General sub-question patterns** (use when applicable):
- **Sub-question A**: "What is X and how does it work?" (Definition & mechanism)
@@ -86,27 +84,6 @@ For **each sub-question**, generate **at least 3-5 search query variants** befor
Record all planned queries in `00_question_decomposition.md` alongside each sub-question.
#### Component Option Breadth (MANDATORY)
Before Step 2, identify the component areas implied by the problem and create a search plan for options in each area. A component area is any replaceable tool, library, model, service, algorithm, data format, protocol, infrastructure pattern, or validation approach that could materially affect the solution.
For every component area, generate search queries for these option families unless clearly not applicable:
- **Simple baseline**: low-complexity classical or manual approach that can serve as a fallback or regression baseline.
- **Established production option**: mature library/service/pattern with field usage.
- **Open-source candidate**: permissive-license option with inspectable implementation and community history.
- **Commercial/vendor option**: paid or vendor-supported option, including SDK/platform constraints.
- **Current SOTA / research option**: recent model, paper, or benchmark leader that may be promising but immature.
- **Adjacent-domain option**: solution from a neighboring domain with similar constraints.
- **No-build / defer option**: whether the component can be avoided, simplified, or moved out of scope.
- **Known bad option**: candidate or family that appears attractive but has documented failure modes or disqualifiers.
For each component area, record:
- Candidate names and option families to search.
- At least 5 query variants covering alternatives, comparisons, limitations, licensing, runtime/scale, and exact project constraints.
- The minimum evidence needed to mark a candidate `Selected`, `Rejected`, `Experimental only`, or `Needs user decision`.
Add this as a "Component Option Search Plan" section in `00_question_decomposition.md`.
**Research Subject Boundary Definition (BLOCKING - must be explicit)**:
When decomposing questions, you must explicitly define the **boundaries of the research subject**:
@@ -117,9 +94,6 @@ When decomposing questions, you must explicitly define the **boundaries of the r
| **Geography** | Which region is being studied? | Chinese universities vs US universities vs global |
| **Timeframe** | Which period is being studied? | Post-2020 vs full historical picture |
| **Level** | Which level is being studied? | Undergraduate vs graduate vs vocational |
| **Operating context** | What exact environment, lifecycle phase, and runtime conditions must the solution support? | In-flight embedded runtime vs offline post-processing; production web traffic vs admin batch job |
| **Required interfaces** | What inputs, outputs, protocols, data shapes, and ownership boundaries are fixed? | One camera vs stereo rig; REST API vs message queue; local file boundary vs service API |
| **Non-functional envelope** | What latency, throughput, storage, memory, availability, safety, security, cost, and maintainability targets are binding? | <400 ms p95, 8 GB RAM, 99.9% availability, reversible migrations |
**Common mistake**: User asks about "university classroom issues" but sources include policies targeting "K-12 students" — mismatched target populations will invalidate the entire research.
@@ -142,11 +116,9 @@ Record the audit result in `00_question_decomposition.md` as a "Completeness Aud
- Summary of relevant problem context from INPUT_DIR
- Classified question type and rationale
- **Research subject boundary definition** (population, geography, timeframe, level)
- **Project Constraint Matrix summary** (operating context, required interfaces, non-functional envelope, lifecycle assumptions, and hard disqualifiers extracted from input files)
- List of decomposed sub-questions
- **Chosen perspectives** (at least 3 from the Perspective Rotation table) with rationale
- **Search query variants** for each sub-question (at least 3-5 per sub-question)
- **Component Option Search Plan** (component areas, option families, candidate names, query variants, required evidence)
- **Completeness audit** (taxonomy cross-reference + domain discovery results)
4. Write TodoWrite to track progress
@@ -160,7 +132,7 @@ Tier sources by authority, **prioritize primary sources** (L1 > L2 > L3 > L4). C
**Tool Usage**:
- Use `WebSearch` for broad searches; `WebFetch` to read specific pages
- Use the `context7` MCP server (`resolve-library-id` then `query-docs` / `get-library-docs`) for up-to-date library/framework documentation. **Mandatory per lead candidate** — see "API Capability Verification" below.
- Use the `context7` MCP server (`resolve-library-id` then `get-library-docs`) for up-to-date library/framework documentation
- Always cross-verify training data claims against live sources for facts that may have changed (versions, APIs, deprecations, security advisories)
- When citing web sources, include the URL and date accessed
@@ -173,77 +145,17 @@ Do not stop at the first few results. The goal is to build a comprehensive evide
- Consult at least **2 different source tiers** per sub-question (e.g., L1 official docs + L4 community discussion)
- If initial searches yield fewer than 3 relevant sources for a sub-question, **broaden the search** with alternative terms, related domains, or analogous problems
**Minimum search effort per component area**:
- Search every option family from the "Component Option Search Plan" before choosing a lead candidate.
- For each lead, fallback, or rejected candidate, search at least one official/source-of-truth page and at least one independent validation source when available.
- Search `"[component] alternatives"`, `"[candidate] vs [alternative]"`, `"[candidate] limitations"`, `"[candidate] license"`, `"[candidate] production"`, and `"[candidate] [binding project constraint]"`.
- If fewer than 3 realistic candidates are found for a component area, explicitly document why the landscape is narrow and search adjacent domains before accepting that result.
- Include at least one simple baseline and one "do not use" or disqualified candidate per component area when possible; these prevent false confidence in the selected option.
**Candidate implementation-limit searches (MANDATORY)**:
For every component/tool/library/service/pattern/algorithm that may be selected or recommended, search for its intrinsic implementation constraints. Do not rely on product category labels, marketing summaries, or examples from a different operating context. Include query variants for:
- Official supported inputs/outputs, protocols, data formats, and deployment modes
- Required hardware/runtime/platform/version constraints
- Timing, throughput, memory, storage, synchronization, and scaling assumptions
- Lifecycle assumptions: offline vs online, batch vs real time, development vs production, single tenant vs multi tenant, local vs networked
- Known unsupported scenarios, limitations, issue reports, production failures, and workarounds
- Licensing, security, maintenance, and community-health constraints
- Exact phrases from the project's restrictions and acceptance criteria combined with the candidate name
**API Capability Verification — Per-Mode (MANDATORY, BLOCKING for lead candidates)**:
**Applicability**: this section applies only when the run is classified as **Technical-component selection** in the SKILL's Research Output Class section, and only to lead candidates that are libraries/SDKs/frameworks/services/protocols/data formats with multiple modes or configurations. For non-technical research (concept comparison, market/policy investigation, knowledge organization, root-cause analysis without tooling commitments), skip this entire sub-section and continue with the rest of Step 2 — the broader candidate implementation-limit search above is sufficient. State the skip explicitly once in `02_fact_cards.md` (or in `02_fact_cards/00_summary.md` if split): `API Capability Verification: not applicable — this run is a Non-technical investigation, no library/SDK/service candidates`.
Most libraries/SDKs/services expose **multiple modes or configurations** (e.g., monocular vs stereo VO, sync vs async API, batch vs streaming inference, write-through vs write-behind cache). Selecting a candidate "because it supports X" without pinning *which mode* the project will use, and *whether that exact mode produces the required outputs from the required inputs*, is the most common silent-failure path in research. A library can support a class of problem in mode A while being unusable for the project's specific configuration in mode B.
For every lead candidate that is a library/SDK/framework/service with multiple modes or configurations, do the following — in this order, before marking the candidate `Selected`:
1. **Pin the exact mode/configuration the project will use.**
Derived from the Project Constraint Matrix: which inputs are available (sensor count, sensor types, data shapes, rates), which outputs are required (per `acceptance_criteria.md` and contract files), which hardware/runtime is fixed (per `restrictions.md`). Write this as a single sentence: "We will use `<library>` in `<mode/config>` with inputs `<list>` and expect outputs `<list>` on `<runtime>`." Do not progress past this step on a vague mode description.
2. **Run `context7` (or equivalent docs lookup) for the candidate** — this is **mandatory for every lead library/SDK/framework candidate**, not optional. Minimum three queries per candidate:
1. *Mode enumeration*: "What modes/configurations does `<library>` support? List every value of the mode/config enum and what each requires as input."
2. *Project's exact mode*: "Show a minimum runnable example of `<library>` in `<the pinned mode>` with `<the project's input shape>`. What does it produce?"
3. *Disqualifier probe*: "Does `<library>` `<the pinned mode>` produce `<the required output>`? Are there published limitations of `<the pinned mode>` for `<the project's runtime/hardware>`?"
For services without context7 coverage, use official docs site + WebFetch on the API reference page + the project's example/tutorial directory in the source repo. Append every consulted URL to `01_source_registry.md` (or the appropriate category file under `01_source_registry/` if split — see splittable-artifacts convention in `00_project-integration.md`).
3. **Save a Minimum Viable Example (MVE) for the pinned mode.**
Append to `02_fact_cards.md` / `02_fact_cards/` (or a sibling `02_mve_evidence.md`) at least one block per lead library candidate with:
```markdown
## MVE — <library> in <pinned mode>
- **Source**: <official URL or context7 reference, with date>
- **Inputs in the example**: <e.g., 2 calibrated cameras + IMU at 200 Hz>
- **Outputs in the example**: <e.g., 6-DoF pose with covariance>
- **Project inputs**: <e.g., 1 camera + IMU at 200 Hz>
- **Project outputs required**: <e.g., 6-DoF pose with metric translation>
- **Match assessment**: ✅ exact match / ⚠️ partial (specify dimension) / ❌ mismatch (specify dimension)
- **If ⚠️ or ❌**: cite the official-docs sentence that establishes the mismatch.
```
If no official example covers the project's exact configuration → the candidate cannot be marked `Selected` based on category fit alone. Status must be `Experimental only` (with required-evidence note) or `Rejected` (when the docs explicitly disqualify the configuration).
4. **Bind every numbered Restriction and Acceptance Criterion to the candidate's pinned mode.**
For each numbered line in `restrictions.md` and `acceptance_criteria.md`, decide one of: `Pass` (the pinned mode satisfies it with cited evidence), `Fail` (the pinned mode contradicts it with cited evidence), `Verify` (no evidence either way; deeper investigation required), `N/A` (the line is irrelevant to this component area). Record this in `02_fact_cards.md` (or the candidate's per-component file under `02_fact_cards/` if split) under the candidate's MVE block. The structural matrix in Step 7.5 reads from these bindings.
5. **Treat "the same library in a different mode" as a different candidate.**
If the project's pinned mode is `Monocular` but the only documented evidence covers `Stereo`, do not silently soften "rotation only" into "rotation + translation". Open a separate candidate row for the Monocular mode, with its own MVE, fit assessment, and disqualifiers. Two modes of one library are two distinct candidates for the purposes of this gate.
**Common silent-failure pattern this guards against**: a fact card paraphrases the docs as "supports A, B, C, D modes" when the docs actually mean "supports A; B; C and D as separate orthogonal modes". A category-level "Selected" decision then carries through every downstream artifact, masking that the project's required A+B combination does not exist as a single mode.
**Search broadening strategies** (use when results are thin):
- Try adjacent fields: if researching "drone indoor navigation", also search "robot indoor navigation", "warehouse AGV navigation"
- Try different communities: academic papers, industry whitepapers, military/defense publications, hobbyist forums
- Try different geographies: search in English + search for European/Asian approaches if relevant
- Try historical evolution: "history of X", "evolution of X approaches", "X state of the art 2024 2025"
- Try failure analysis: "X project failure", "X post-mortem", "X recall", "X incident report"
- Try disqualifier probes: "X unsupported", "X limitations", "X requirements", "X with [project constraint]", "X without [required input]", "X real-time [target]", "X production failure"
**Search saturation rule**: Continue searching until new queries stop producing substantially new information. If the last 3 searches only repeat previously found facts, the sub-question is saturated.
**Save action**:
For each source consulted, **immediately** append to `01_source_registry.md` (or the appropriate category file under `01_source_registry/` if split) using the entry template from `references/source-tiering.md`.
For each source consulted, **immediately** append to `01_source_registry.md` using the entry template from `references/source-tiering.md`.
---
@@ -273,7 +185,7 @@ Transform sources into **verifiable fact cards**:
- ❓ Low: Inference or from unofficial sources
**Save action**:
For each extracted fact, **immediately** append to `02_fact_cards.md` (or the appropriate category file under `02_fact_cards/` if split):
For each extracted fact, **immediately** append to `02_fact_cards.md`:
```markdown
## Fact #[number]
- **Statement**: [specific fact description]
@@ -282,7 +194,6 @@ For each extracted fact, **immediately** append to `02_fact_cards.md` (or the ap
- **Target Audience**: [which group this fact applies to, inherited from source or further refined]
- **Confidence**: ✅/⚠️/❓
- **Related Dimension**: [corresponding comparison dimension]
- **Fit Impact**: [supports selection / disqualifies / makes experimental / needs user decision]
```
**Target audience in fact statements**:
@@ -318,7 +229,7 @@ After initial fact extraction, review what you have found and identify **knowled
- Failure cases and edge conditions
- Recent developments that may change the picture
4. **Update artifacts**: Append new sources to `01_source_registry.md`, new facts to `02_fact_cards.md` (use the appropriate category files under `01_source_registry/` and `02_fact_cards/` if split)
4. **Update artifacts**: Append new sources to `01_source_registry.md`, new facts to `02_fact_cards.md`
**Exit criteria**: Proceed to Step 4 when:
- Every sub-question has at least 3 facts with at least one from L1/L2
@@ -24,18 +24,6 @@ Write to `03_comparison_framework.md`:
| ... | | | |
```
**Required exact-fit dimensions for component/tool decisions**:
When the output selects or recommends a component, tool, library, service, architecture pattern, or algorithm, the framework MUST include these dimensions unless explicitly not applicable:
- Option family (`Simple baseline`, `Established production`, `Open-source`, `Commercial/vendor`, `Current SOTA`, `Adjacent-domain`, `No-build/defer`, `Known bad`)
- Required inputs/outputs and ownership boundaries
- Operating context and lifecycle fit
- Non-functional envelope fit
- Implementation assumptions and hard disqualifiers
- Evidence quality and source tier
- Selection status (`Selected`, `Rejected`, `Experimental only`, `Needs user decision`)
For each component area, include multiple candidates in the initial population. Do not present only the preferred option unless the investigation found no realistic alternatives; if so, state the searches that proved the narrow landscape.
---
### Step 5: Reference Point Baseline Alignment
@@ -109,8 +97,6 @@ Validate conclusions against a typical scenario:
- [ ] Are there any important dimensions missed?
- [ ] Is there any over-extrapolation?
- [ ] Are conclusions actionable/verifiable?
- [ ] Does every selected component/tool/pattern match the Project Constraint Matrix?
- [ ] Are mismatches marked as disqualifiers instead of hidden as generic "limitations"?
**Save action**:
Write to `05_validation_log.md`:
@@ -142,66 +128,6 @@ If using Y: [expected behavior]
---
### Step 7.5: Component Applicability Gate (BLOCKING)
**Applicability**: this gate applies only when the run is classified as **Technical-component selection** in the SKILL's Research Output Class section. For non-technical research (concept comparison, market/policy investigation, root-cause analysis without tooling, knowledge organization), skip this entire step and proceed to Step 8 — there are no components to gate. State the skip once in `05_validation_log.md`: `Step 7.5 (Component Applicability Gate): not applicable — Non-technical investigation`. For mixed runs (some component areas technical, some not), apply this gate only to the technical component areas; the non-technical ones do not produce 7.5 rows.
Before finalizing the solution draft, build an exact-fit matrix for every component/tool/library/service/pattern/algorithm that is selected, recommended, rejected, or treated as a fallback. Free-form prose in a "Project Constraints Checked" column is **not sufficient** — mismatches hide inside rationale text. The matrix must be structured per restriction and per acceptance criterion.
#### 7.5.1 Top-level Component Fit Matrix
```markdown
# Component Fit Matrix
| Component Area | Candidate | Pinned Mode/Config | Option Family | Intended Role | API Capability Evidence | Mismatches / Disqualifiers | Status | Decision Rationale |
|----------------|-----------|--------------------|---------------|---------------|-------------------------|----------------------------|--------|--------------------|
| [area] | [name] | [exact mode/config the project will use, copied verbatim from the MVE block in Step 2] | [family] | [role] | MVE: [link to MVE block in `02_fact_cards.md` / `02_fact_cards/` or `02_mve_evidence.md`]; docs: [Source #] | [none / list] | Selected / Rejected / Experimental only / Needs user decision | [why] |
```
The new **Pinned Mode/Config** column is mandatory. A row without a pinned mode is incomplete. The new **API Capability Evidence** column links to the Minimum Viable Example saved during Step 2's API Capability Verification — without an MVE link the candidate cannot be `Selected`.
#### 7.5.2 Restrictions × Candidate-Modes Sub-Matrix (MANDATORY)
For each lead candidate row in the top-level matrix, append a structured cross-check that walks every numbered line of `restrictions.md` and `acceptance_criteria.md` against the candidate's **pinned mode/config**.
```markdown
## Sub-Matrix — <Candidate Name> in <Pinned Mode>
| Restriction / AC | Candidate-mode behavior | Result | Evidence |
|------------------|-------------------------|--------|----------|
| R1: <verbatim line from restrictions.md> | <how the pinned mode behaves under this restriction> | ✅ Pass / ❌ Fail / ❓ Verify / N/A | [Fact # / Source # / MVE link] |
| R2: ... | ... | ... | ... |
| ... | ... | ... | ... |
| AC-1.1: <verbatim line from acceptance_criteria.md> | <how the pinned mode satisfies (or contradicts) this AC's measurable target> | ✅ / ❌ / ❓ / N/A | [Fact # / Source # / MVE link] |
| AC-1.2: ... | ... | ... | ... |
| ... | ... | ... | ... |
```
Cell semantics:
- ✅ **Pass** — the candidate's pinned mode satisfies this line, with cited official-doc or MVE evidence.
- ❌ **Fail** — the candidate's pinned mode contradicts this line, with cited evidence. Even one ❌ disqualifies the candidate from `Selected` status.
- ❓ **Verify** — no evidence yet either way; further investigation required (loops back to Step 2 / Step 3.5). A row left ❓ at the end of analysis blocks the candidate.
- **N/A** — the line is irrelevant to this component area (state why in one phrase).
A candidate row may not be marked `Selected` while any cell is ❌ or ❓.
#### 7.5.3 Decision Rules
- `Selected` is allowed only when (a) the top-level row has an MVE link, (b) the sub-matrix has zero ❌, (c) the sub-matrix has zero ❓, and (d) the candidate's documented implementation assumptions match the project's explicit constraints and acceptance criteria.
- `Experimental only` is required when a candidate might work but lacks proof for the exact operating context (e.g., MVE exists for a similar configuration but not the exact one).
- `Rejected` is required when documented assumptions conflict with project constraints (any sub-matrix row is ❌ with cited evidence).
- `Needs user decision` is required when a mismatch changes scope, cost, safety, product behavior, or acceptance criteria — and the user has not yet been consulted.
- Each component area must include at least one selected or fallback-safe option, plus the most credible rejected/experimental alternatives discovered during web research.
- A component area with only one candidate is incomplete unless `00_question_decomposition.md` documents the broader searches and why they yielded no realistic alternatives.
- A candidate may not appear as the lead solution in Step 8 unless this gate marks it `Selected`.
- "Validation gate required" footnotes are not equivalent to `Selected`. If the validation gate concerns API capability (does the mode produce the required output?), that is a Step-2 / Step-7.5 question and must be resolved here, not deferred to runtime. Only validation gates concerning *runtime quality* (e.g., "does this VO converge on this terrain class?") may be carried forward as `Selected with runtime gate`.
**Save action**: Write `06_component_fit_matrix.md` (or, when split, the equivalent files under `06_component_fit_matrix/` — typically `00_summary.md` for the top-level matrix plus per-component sub-matrix files) containing both 7.5.1 (top-level) and 7.5.2 (per-candidate sub-matrices).
**BLOCKING**: If any lead candidate has ❌, ❓, `Experimental only`, `Rejected`, or `Needs user decision` status, do not silently proceed. Ask the user or choose a different selected candidate.
---
### Step 8: Deliverable Formatting
Make the output **readable, traceable, and actionable**.
@@ -213,8 +139,8 @@ Integrate all intermediate artifacts. Write to `OUTPUT_DIR/solution_draft##.md`
Sources to integrate:
- Extract background from `00_question_decomposition.md`
- Reference key facts from `02_fact_cards.md` (or files under `02_fact_cards/` if split)
- Reference key facts from `02_fact_cards.md`
- Organize conclusions from `04_reasoning_chain.md`
- Generate references from `01_source_registry.md` (or files under `01_source_registry/` if split)
- Generate references from `01_source_registry.md`
- Supplement with use cases from `05_validation_log.md`
- For Mode A: include AC assessment from `00_ac_assessment.md`
@@ -10,21 +10,12 @@
[Architecture solution that meets restrictions and acceptance criteria.]
> **Applicability** — the table columns `Pinned Mode/Config` and `API Capability Evidence` apply only to technical-component runs (per SKILL.md → Research Output Class). For non-technical research outputs (concept comparison, market/policy report, investigation answer), this Architecture section may be replaced with a comparison/analysis section that does not use these columns; or the columns may be marked `N/A` per row when the row describes a non-technical "component" (a process, a policy, an organizational construct). For mixed runs, fill the columns only on rows that describe libraries/SDKs/frameworks/services/protocols/data formats/algorithms.
### Component: [Component Name]
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|-----------|-------------|-------------|----------|------|-------------------------|-----|
| [Option 1] | [lib/platform] | [exact mode/config used: inputs, outputs, runtime] | [pros] | [cons] | [intrinsic requirements] | [security] | [cost] | MVE: [link to MVE block]; docs: [Source #] | [Selected / Rejected / Experimental only / Needs user decision — cite exact-fit evidence and disqualifiers] |
| [Option 2] | [lib/platform] | [exact mode/config used] | [pros] | [cons] | [intrinsic requirements] | [security] | [cost] | MVE: [link]; docs: [Source #] | [Selected / Rejected / Experimental only / Needs user decision] |
**Exact-fit evidence**:
- Project constraints checked: [inputs/outputs, operating context, lifecycle, NFRs, acceptance criteria]
- Evidence: [Fact # / Source #]
- Disqualifiers: [none or list]
- Restrictions × Candidate-Modes sub-matrix: see `06_component_fit_matrix.md` (or `06_component_fit_matrix/` if split) § <Candidate Name>
- API capability gates: ✅ MVE saved / ⚠️ partial — see disqualifiers / ❌ no MVE — candidate is Experimental only or Rejected
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|-----------|-------------|-------------|----------|------|-----|
| [Option 1] | [lib/platform] | [pros] | [cons] | [reqs] | [security] | [cost] | [fit assessment] |
| [Option 2] | [lib/platform] | [pros] | [cons] | [reqs] | [security] | [cost] | [fit assessment] |
[Repeat per component]
@@ -13,21 +13,12 @@
[Architecture solution that meets restrictions and acceptance criteria.]
> **Applicability** — the table columns `Pinned Mode/Config` and `API Capability Evidence` apply only to technical-component runs (per SKILL.md → Research Output Class). For non-technical assessment outputs (e.g., reassessing a policy approach, comparing organizational designs), this Architecture section may be replaced with the assessment content that does not use these columns; or the columns may be marked `N/A` per row for non-technical "components". For mixed runs, fill the columns only on rows that describe libraries/SDKs/frameworks/services/protocols/data formats/algorithms.
### Component: [Component Name]
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|-----------|-------------|-------------|----------|------------|-------------------------|-----|
| [Option 1] | [lib/platform] | [exact mode/config used: inputs, outputs, runtime] | [pros] | [cons] | [intrinsic requirements] | [security] | [perf] | MVE: [link to MVE block]; docs: [Source #] | [Selected / Rejected / Experimental only / Needs user decision — cite exact-fit evidence and disqualifiers] |
| [Option 2] | [lib/platform] | [exact mode/config used] | [pros] | [cons] | [intrinsic requirements] | [security] | [perf] | MVE: [link]; docs: [Source #] | [Selected / Rejected / Experimental only / Needs user decision] |
**Exact-fit evidence**:
- Project constraints checked: [inputs/outputs, operating context, lifecycle, NFRs, acceptance criteria]
- Evidence: [Fact # / Source #]
- Disqualifiers: [none or list]
- Restrictions × Candidate-Modes sub-matrix: see `06_component_fit_matrix.md` (or `06_component_fit_matrix/` if split) § <Candidate Name>
- API capability gates: ✅ MVE saved / ⚠️ partial — see disqualifiers / ❌ no MVE — candidate is Experimental only or Rejected
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|-----------|-------------|-------------|----------|------------|-----|
| [Option 1] | [lib/platform] | [pros] | [cons] | [reqs] | [security] | [perf] | [fit assessment] |
| [Option 2] | [lib/platform] | [pros] | [cons] | [reqs] | [security] | [perf] | [fit assessment] |
[Repeat per component]
+2 -13
View File
@@ -22,7 +22,7 @@ test-run has two modes. The caller passes the mode explicitly; if missing, defau
| Mode | Scope | Typical caller | Input artifacts |
|------|-------|---------------|-----------------|
| `functional` (default) | Unit / integration / blackbox tests — correctness | autodev Steps that verify after Implement Tests or Implement | `scripts/run-tests.sh`, `_docs/02_document/tests/environment.md`, `_docs/02_document/tests/blackbox-tests.md` |
| `perf` | Performance / load / stress / soak tests — latency, throughput, error-rate thresholds | autodev greenfield Step 15, existing-code Step 15 (pre-deploy) | `scripts/run-performance-tests.sh`, `_docs/02_document/tests/performance-tests.md`, AC thresholds in `_docs/00_problem/acceptance_criteria.md` |
| `perf` | Performance / load / stress / soak tests — latency, throughput, error-rate thresholds | autodev greenfield Step 9, existing-code Step 15 (pre-deploy) | `scripts/run-performance-tests.sh`, `_docs/02_document/tests/performance-tests.md`, AC thresholds in `_docs/00_problem/acceptance_criteria.md` |
Direct user invocation (`/test-run`) defaults to `functional`. If the user says "perf tests", "load test", "performance", or passes a performance scenarios file, run `perf` mode.
@@ -32,17 +32,6 @@ After selecting a mode, read its corresponding workflow below; do not mix them.
## Functional Mode
### 0. System-Under-Test Reality Gate
Before accepting any functional, blackbox, or e2e result as a pass, verify what the tests actually exercised.
1. If `_docs/00_problem/input_data/expected_results/results_report.md` exists, at least one e2e/blackbox run must compare actual product outputs against that mapping or the machine-readable files it references.
2. Stubs are allowed only for external systems outside the product boundary: flight controller/SITL, QGC observer, satellite-provider/Suite service, physical Jetson hardware, physical camera, unavailable licensed datasets, and network services.
3. Stubs, fakes, deterministic fallbacks, monkeypatches, or direct replacement of internal product modules are not allowed for the behavior under test. Internal examples include VIO, safety/anchor wrapper, satellite retrieval, anchor verification, tile manager, MAVLink output adapter, FDR, and the A-Z localization pipeline.
4. If tests pass only because an internal module is fake/scaffolded, classify the run as **failed** with category `missing product implementation`.
5. If a scenario is blocked because external hardware/data is absent, verify the production code path exists before accepting the block as legitimate. Missing internal production code is not an environment block.
6. If the test runner writes CSV/Markdown reports, inspect them. A zero exit code is not enough; blocked/internal-stubbed scenarios still require classification.
### 1. Detect Test Runner
Check in order — first match wins:
@@ -105,7 +94,7 @@ Categorize skips as: **explicit skip (dead code)**, **runtime skip (unreachable)
### 5. Handle Outcome
**All tests pass, zero skipped, and the System-Under-Test Reality Gate passes** → return success to the autodev for auto-chain.
**All tests pass, zero skipped** → return success to the autodev for auto-chain.
**Any test fails or errors** → this is a **blocking gate**. Never silently ignore failures. **Always investigate the root cause before deciding on an action.** Read the failing test code, read the error output, check service logs if applicable, and determine whether the bug is in the test or in the production code.
@@ -95,7 +95,7 @@ Examples:
File: `expected_results/image_01_detections.json`
```json
```json
{
"input": "image_01.jpg",
"expected": {
@@ -119,7 +119,7 @@ File: `expected_results/image_01_detections.json`
]
}
}
```
```
```
---
-34
View File
@@ -1,34 +0,0 @@
# annotations service — production environment template.
# Copy to .env (or set via the container orchestrator) and fill in real values.
# All variables marked REQUIRED cause startup to fail fast when missing.
# CHANGE_ME placeholders MUST be replaced before deploying to Production.
# REQUIRED — Postgres connection. Either a Linq2DB connection string or a
# postgresql://user:pass@host:port/db URL.
DATABASE_URL=postgresql://annotations_user:CHANGE_ME@CHANGE_ME_DB_HOST:5432/azaion
# REQUIRED — JWT verifier configuration. Values MUST match admin's JwtConfig
# in the same environment (admin/secrets/production.public.env shows the same
# Issuer/Audience pair).
JWT_ISSUER=AzaionApi
JWT_AUDIENCE=Annotators/OrangePi/Admins
JWT_JWKS_URL=https://admin.azaion.com/.well-known/jwks.json
# REQUIRED in Production — explicit CORS allow-list. Empty origins +
# AllowAnyOrigin=false aborts startup; AllowAnyOrigin=true is an explicit
# operator opt-in and MUST NOT be used in Production.
CorsConfig__AllowedOrigins__0=https://admin.azaion.com
CorsConfig__AllowedOrigins__1=CHANGE_ME_ANNOTATOR_UI_ORIGIN
CorsConfig__AllowAnyOrigin=false
# REQUIRED — RabbitMQ stream sync (suite-level credentials).
RABBITMQ_HOST=CHANGE_ME_RABBITMQ_HOST
RABBITMQ_STREAM_PORT=5552
RABBITMQ_PRODUCER_USER=azaion_producer
RABBITMQ_PRODUCER_PASS=CHANGE_ME
RABBITMQ_STREAM_NAME=azaion-annotations
# ASP.NET Core — set Production explicitly so the CORS validator's strict gate
# engages. Mirrors admin/secrets/production.public.env.
ASPNETCORE_ENVIRONMENT=Production
ASPNETCORE_URLS=http://+:8080
-4
View File
@@ -36,7 +36,3 @@ ui/dist
*.enc
key-fragment*.bin
images.tar
# E2E / test outputs
test-results/
e2e/e2e-results/
-35
View File
@@ -1,35 +0,0 @@
{
"version": "0.2.0",
"configurations": [
{
// Use IntelliSense to find out which attributes exist for C# debugging
// Use hover for the description of the existing attributes
// For further information visit https://github.com/dotnet/vscode-csharp/blob/main/debugger-launchjson.md.
"name": ".NET Core Launch (web)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/src/bin/Debug/net10.0/Azaion.Annotations.dll",
"args": [],
"cwd": "${workspaceFolder}/src",
"stopAtEntry": false,
// Enable launching a web browser when ASP.NET Core starts. For more information: https://aka.ms/VSCode-CS-LaunchJson-WebBrowser
"serverReadyAction": {
"action": "openExternally",
"pattern": "\\bNow listening on:\\s+(https?://\\S+)"
},
"env": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"sourceFileMap": {
"/Views": "${workspaceFolder}/Views"
}
},
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach"
}
]
}
-41
View File
@@ -1,41 +0,0 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"command": "dotnet",
"type": "process",
"args": [
"build",
"${workspaceFolder}/src/Azaion.Annotations.csproj",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary;ForceNoAlign"
],
"problemMatcher": "$msCompile"
},
{
"label": "publish",
"command": "dotnet",
"type": "process",
"args": [
"publish",
"${workspaceFolder}/src/Azaion.Annotations.csproj",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary;ForceNoAlign"
],
"problemMatcher": "$msCompile"
},
{
"label": "watch",
"command": "dotnet",
"type": "process",
"args": [
"watch",
"run",
"--project",
"${workspaceFolder}/src/Azaion.Annotations.csproj"
],
"problemMatcher": "$msCompile"
}
]
}
-13
View File
@@ -1,13 +0,0 @@
# Azaion.Annotations
.NET REST API for media, annotations, datasets, and settings.
## Documentation
The **canonical** description of this service (HTTP API, media flows, RabbitMQ sync, SSE, settings) is maintained with the rest of the suite:
**[suite/_docs/01_annotations.md](../_docs/01_annotations.md)**
That file is the product and integration reference for this repository. Update it when you change public contracts, queues, or behavior it documents.
If you use a standalone clone without the parent `suite` tree, open `01_annotations.md` from your checkout of the suite `_docs` folder (same content).
-73
View File
@@ -1,73 +0,0 @@
# Azaion.Annotations — Acceptance criteria (retrospective)
> Every criterion has a measurable value and a code/config evidence pointer. **No automated test suite exists in the repo today** (`_docs/02_document/00_discovery.md`), so the criteria below are derived from validation rules, configuration limits, and explicit code branches — they are the contract a future test suite (autodev existing-code Step 3 + Step 6) must encode. Criteria that depend on a Refactor Backlog item landing first are flagged with `[after RB-XX]`.
## Functional — annotation lifecycle (component `01 annotations-rest`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-01 | `POST /annotations` with `image_bytes + detections` for the same payload returns the same `id` on every call. | `id` is `XxHash3.Hash128` (32 hex chars) of the sampled `image_bytes` window — byte-stable. | `Services/AnnotationService.cs` `GenerateAnnotationId(...)` (post RB-04 — currently `XxHash64`). |
| AC-F-02 | A repeat `POST /annotations` for an existing id is a no-op write (no duplicate row, no duplicate file). | DB reads return existing row before insert; file write is `WriteAllBytesAsync` overwriting same bytes. | `Services/AnnotationService.cs`. |
| AC-F-03 | `POST /annotations` writes a YOLO-format label file at `images_dir/<id>.txt` containing one line per detection: `<class_id> <cx> <cy> <w> <h>`. | Exact format with space-separated floats, normalised 0..1, line-per-detection. | `Services/AnnotationService.cs` (label-file write site). |
| AC-F-04 | `POST /annotations` returns HTTP 200 with the persisted entity (id, status, detections). | Response shape mirrors `AnnotationDto`. | `Controllers/AnnotationsController.cs`. |
| AC-F-05 | `[after RB-01]` Every successful `POST/PUT/PATCH/DELETE /annotations/*` emits exactly one SSE event AND inserts exactly one `annotations_queue_records` row, with the correct `QueueOperation` enum. | Created=10, Updated=20, Deleted=40. | `_docs/02_document/architecture.md` ADR-009; `Services/QueueOperation.cs`. |
| AC-F-06 | `[after RB-01]` `DELETE /annotations/{id}` flips the row to `Status=Deleted (40)`, relocates `images_dir/<id>.{jpg,txt}` to `deleted_dir/`, and emits the `Deleted` lifecycle event. | Row count unchanged; files moved; status transitions per `AnnotationStatus`. | `_docs/02_document/architecture.md` ADR-009 + glossary "Soft-delete". |
| AC-F-07 | `[after RB-01 + RB-08]` Soft-deleted rows do not appear in `GET /annotations` or `GET /dataset` results. | Filter `WHERE Status <> 40` enforced at every read path. | RB-01, RB-08 in `_docs/02_document/architecture.md`. |
| AC-F-08 | `[after RB-02]` There is no `silent_detection` column, field, DTO property, or branch in code. | Schema diff + grep produces zero matches. | RB-02. |
## Functional — realtime + sync (component `02 annotations-realtime-sync`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-10 | A connected SSE client receives the lifecycle event for a successful `POST /annotations` within 1 second of the response. | <1s P99 in single-instance, single-pod local run. | `Services/AnnotationEventService.cs`. |
| AC-F-11 | A subscriber that joins **after** the event has been published does not receive it (channel is fire-and-forget). | No backfill replay in `Channel<>`. | ADR-001. |
| AC-F-12 | `FailsafeProducer` consumes a row from `annotations_queue_records` and publishes a MessagePack-gzip frame to the `azaion-annotations` stream within the configured drain interval. | Drain loop interval is the configured cadence; row deletion happens after stream confirm. | `Services/FailsafeProducer.cs`. |
| AC-F-13 | `[after RB-09]` Every wire message carries `(annotation_id, operation, date_time)` so a downstream consumer can dedupe re-deliveries. | Three fields present on the wire schema. | `_docs/02_document/architecture.md` ADR-013. |
## Functional — media (component `03 media`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-20 | `POST /media` accepts a multipart upload, persists the file to the configured directory, and returns the persisted `MediaDto`. | HTTP 200 + JSON body. | `Controllers/MediaController.cs`, `Services/MediaService.cs`. |
| AC-F-21 | `POST /media/batch` accepts N files in one request, writes N rows + N files, and returns N persisted DTOs. | N inputs → N outputs, atomic per file. | same. |
## Functional — dataset (component `04 dataset`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-30 | `GET /dataset` honors filter parameters (mission id, status, class). | Returned rows match filter conditions. | `Controllers/DatasetController.cs`, `Services/DatasetService.cs`. |
| AC-F-31 | `POST /dataset/status/bulk` flips status on N rows in a single SQL statement. | One UPDATE WHERE id IN (…). | `Services/DatasetService.cs`. |
## Functional — settings & metadata (component `05 settings-metadata`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-40 | `PUT /settings/directories` persists changes and triggers `pathResolver.Reset()` so subsequent path lookups reflect the new values. | Verified — `Services/SettingsService.cs:71, 85`. | `Services/SettingsService.cs`. |
| AC-F-41 | `GET /classes` returns the 19 seeded detection classes (ids 018: `ArmorVehicle, Truck, Vehicle, Artillery, Shadow, Trenches, MilitaryMan, TyreTracks, AdditionArmoredTank, Smoke, Plane, Moto, CamouflageNet, CamouflageBranches, Roof, Building, Caponier, Ammo, Protect.Struct`). | 19 rows; ids stable from `DatabaseMigrator`. | `Database/DatabaseMigrator.cs:101-121`. |
| AC-F-42 | `[after RB-06]` `[ADM]` write endpoints exist for `/classes`; the in-memory cache invalidates on write via `Reset()`. | Cache hit ratio observable; cache miss on each write. | RB-06. |
## Functional — auth & platform (component `06 platform`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-50 | A request bearing an ES256 access token issued by admin (`iss = JWT_ISSUER`, `aud = JWT_AUDIENCE`, signature verifies against the JWKS at `JWT_JWKS_URL`, `exp` in the future) reaches the controller. Tokens that fail issuer / audience / signature / lifetime validation, or whose `alg` is not `ES256`, return HTTP 401. | `JwtBearerHandler` defaults + `AddJwtAuth` parameters. | `Auth/JwtExtensions.cs`. |
| AC-F-51 | Annotations does not host any token-issuance or token-refresh endpoint. Long-running callers refresh against admin's `POST /token/refresh` and pass the resulting access token to annotations. | No `[AllowAnonymous]` route except `/health`; `AuthController` removed. | `Program.cs`, suite admin docs. |
| AC-F-52 | Endpoints under policy `ANN` reject callers without that role with HTTP 403. Endpoints under `DATASET` reject non-DATASET callers with HTTP 403. Endpoints under `ADM` reject non-ADM with HTTP 403. | `Authorization` middleware. | `Auth/JwtExtensions.cs`. |
| AC-F-53 | All errors are returned in the `{ error: { code, message, …details } }` envelope. | Single envelope shape across all controllers. | `Middleware/ErrorHandlingMiddleware.cs`, `_docs/02_document/common-helpers/01_http-error-envelope.md`. |
| AC-F-54 | `GET /health` returns HTTP 200 within 5 seconds of process start (Dockerfile `HEALTHCHECK`). | 200 OK on `/health`. | `Dockerfile`, `Program.cs`. |
## Non-functional
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-N-01 | Container boot to `/health` 200 ≤ Docker `HEALTHCHECK` interval/timeout configured in the suite-level orchestrator. | Per `Dockerfile` HEALTHCHECK directive (consult orchestrator config for actual values). | `Dockerfile`. |
| AC-N-02 | `DatabaseMigrator.MigrateAsync()` is idempotent — second boot against the same DB makes no schema changes. | `IF NOT EXISTS` / `ON CONFLICT DO NOTHING` everywhere. | `Database/DatabaseMigrator.cs`. |
| AC-N-03 | `FailsafeProducer` keeps `annotations_queue_records` depth bounded under steady-state lifecycle traffic. | Queue depth metric (to be exposed during Step 14 Observability work). | `Services/FailsafeProducer.cs`. |
| AC-N-04 | The service emits zero unhandled exceptions to clients — every uncaught exception is mapped via `ErrorHandlingMiddleware` into the error envelope. | Middleware terminal handler. | `Middleware/ErrorHandlingMiddleware.cs`. |
| AC-N-05 | Single SSE connection survives ≥ 30 minutes idle with bounded memory (channel is unbounded; growth must come from real traffic, not heartbeats). | Heap stable across 30-minute idle window. | `Services/AnnotationEventService.cs`. |
## Gaps acknowledged
- No measurable latency / throughput targets (P50, P95, P99) are stated anywhere in code. Need to be set during Step 15 (Performance Test).
- No security audit findings yet (Step 14). Items like JWT issuer validation, CORS tightening, and Swagger gating are planned, not yet acceptance criteria.
- No backup / RPO / RTO contract for `images_dir` and `deleted_dir` — the storage layer is treated as durable by assumption.
@@ -1,92 +0,0 @@
# Azaion.Annotations — Input data parameters
> Inventory of every external input the service accepts, with shape, evidence, and validation behavior. Sources: REST DTOs, multipart form fields, env vars, database seed contract.
## REST API inputs
### `POST /annotations`
| Field | Type | Required | Constraint | Evidence |
|-------|------|----------|-----------|----------|
| `image_bytes` | `byte[]` (base64-encoded JSON or multipart) | yes | none enforced; sampled by `XxHash3.Hash128` (per RB-04) for id derivation | `Services/AnnotationService.cs` `GenerateAnnotationId(...)` |
| `mission_id` | `Guid` (post RB-07; today `flight_id`) | yes | foreign-key style, but no FK enforced in schema today | `Entities/AnnotationEntity.cs`, `_docs/02_document/glossary.md` |
| `media_type` | `MediaType` enum | yes | `Image=10` or `Video=20`; integer wire format | `Models/Wire/MediaType.cs` |
| `detections` | `Detection[]` | yes (≥0) | each detection: class id, normalised cx/cy/w/h, confidence | `Models/Dto/DetectionDto.cs` |
| `metadata` | optional payload | no | passes through to row | `AnnotationDto.cs` |
### `PUT /annotations/{id}` and `PATCH /annotations/{id}/status`
Same DTO shape on the body; `id` from path. Status transition values come from the `AnnotationStatus` wire enum: `Pending=10, Accepted=20, Rejected=30, Deleted=40`.
### `DELETE /annotations/{id}`
Path param only. Soft-deletes the row (sets status to `Deleted=40`) and relocates files to `deleted_dir` (per RB-01 + glossary "Soft-delete").
### `GET /annotations` and `/dataset`
Query string filters: `mission_id`, `status`, `class_id`, paging (`offset`, `limit`). Validation is implicit through `[FromQuery]` model binding — no explicit validators visible at controller level.
### `POST /media` and `POST /media/batch`
Multipart form: `IFormFile` / `IFormFileCollection`, `mission_id`, `media_type`. No format whitelist visible at controller layer (verify in Step 14).
### `GET /media/{id}/file`, `GET /media/{id}/thumbnail`
Path param only; returns binary stream.
### Auth endpoints
Annotations no longer hosts `POST /auth/login`, `POST /auth/refresh`, or `POST /auth/register`. Token issuance and refresh are owned by the **admin** service. The only auth-related input on the annotations surface is the `Authorization: Bearer <token>` HTTP header on every non-`/health` request, validated by `JwtBearerHandler` against admin's JWKS:
| Header | Required | Notes |
|--------|----------|-------|
| `Authorization` | yes (everywhere except `/health`) | `Bearer <ES256 JWT>` issued by admin; `iss` / `aud` / `exp` / signature all validated; `alg` pinned to `ES256` |
### `/settings/*`
Each controller binds JSON DTOs from `Models/Dto/*` mirroring the `system_settings`, `directory_settings`, `camera_settings`, `user_settings` shapes in `Database/DatabaseMigrator.cs`.
## Database seed inputs (boot-time)
`DatabaseMigrator` issues `ON CONFLICT DO NOTHING` inserts on:
| Table | Seeded rows |
|-------|-------------|
| `directory_settings` | one row with default paths |
| `system_settings` | one row (today still includes `silent_detection`; removal tracked by RB-02) |
| `detection_classes` | 19 rows (ids 018): `ArmorVehicle, Truck, Vehicle, Artillery, Shadow, Trenches, MilitaryMan, TyreTracks, AdditionArmoredTank, Smoke, Plane, Moto, CamouflageNet, CamouflageBranches, Roof, Building, Caponier, Ammo, Protect.Struct` (`Smoke` and `Plane` share color `#000080` — pre-existing data quirk, fixed by RB-06) |
(There is no `users` table in this service — identity is owned by the admin service.)
Detection class catalog becomes admin-CRUD after RB-06.
## Environment variables (process inputs)
| Name | Required | Default | Purpose |
|------|----------|---------|---------|
| `DATABASE_URL` | yes | none — fail-fast (`ConfigurationResolver`) | Postgres connection; URI form auto-converted to Linq2DB form |
| `JWT_ISSUER` | yes | none — fail-fast | Expected `iss` claim (admin's issuer) |
| `JWT_AUDIENCE` | yes | none — fail-fast | Expected `aud` claim (this service) |
| `JWT_JWKS_URL` | yes | none — fail-fast; HTTPS required | Admin's JWKS endpoint for ES256 key resolution |
| `CorsConfig:AllowedOrigins` | yes (prod, unless `AllowAnyOrigin=true`) | empty | Configured origins for the default CORS policy |
| `CorsConfig:AllowAnyOrigin` | optional | `false` | Explicit opt-in to permissive CORS (validator blocks empty allow-list in `Production` unless this is set) |
| `RABBITMQ_HOST` | optional | `127.0.0.1` | stream broker host |
| `RABBITMQ_STREAM_PORT` | optional | `5552` | stream listener port |
| `RABBITMQ_PRODUCER_USER` | optional | `azaion_producer` | stream auth user |
| `RABBITMQ_PRODUCER_PASS` | optional | `producer_pass` | stream auth pass |
| `AZAION_REVISION` | optional | `unknown` | image build stamp; logged at boot |
| `ASPNETCORE_URLS` | optional | `http://+:8080` | bind address |
| `ASPNETCORE_ENVIRONMENT` | optional | `Production` | bound to ASP.NET host |
## Stream consumer wire format
Outbound (this service is producer-only):
- Stream name: `azaion-annotations`
- Body: gzip(MessagePack(`AnnotationStreamMessage`))
- Schema fields (post RB-09): `annotation_id`, `operation` (`QueueOperation`), `date_time`, payload — see `_docs/02_document/components/02_annotations-realtime-sync/description.md` and ADR-013.
## Cross-references
- Wire enum table: `_docs/02_document/modules/wire-enums.md`
- ER diagram: `_docs/02_document/data_model.md`
- Common error envelope: `_docs/02_document/common-helpers/01_http-error-envelope.md`
@@ -1,173 +0,0 @@
# Expected Results — Azaion.Annotations
Maps every input data item the test corpus exercises against `Azaion.Annotations` to its quantifiable expected result. Tests use this mapping to compare actual system output against known-correct answers.
This contract is **annotations-service-shape**, not detections-service-shape. The same binary fixtures are reused (see `../fixtures.md`), but the expected outputs here describe annotation lifecycle behavior — content-addressed ids, persisted DTOs, label-file writes, SSE delivery, outbox + stream — not bounding-box inference.
## Result Format Legend
| Result Type | When to Use | Example |
|-------------|-------------|---------|
| Exact value | Output must match precisely | `status_code: 200`, `detection_count: 3` |
| Tolerance range | Numeric output with acceptable variance | `latency: 800ms ± 200ms` |
| Threshold | Output must exceed or stay below a limit | `latency ≤ 1000ms` |
| Pattern match | Output must match a string/regex pattern | `id =~ /^[0-9a-f]{32}$/` |
| File reference | Complex output compared against a reference file | `match expected_results/F1_001_response.json` |
| Schema match | Output structure must conform to a schema | `body matches AnnotationDto` |
| Set/count | Output must contain specific items or counts | `detections.length == 3` |
## Comparison Methods
| Method | Description | Tolerance Syntax |
|--------|-------------|-----------------|
| `exact` | Actual == Expected | N/A |
| `numeric_tolerance` | abs(actual - expected) ≤ tolerance | `± <value>` or `± <percent>%` |
| `threshold_min` | actual ≥ threshold | `≥ <value>` |
| `threshold_max` | actual ≤ threshold | `≤ <value>` |
| `regex` | actual matches regex pattern | regex string |
| `substring` | actual contains substring | substring |
| `json_diff` | structural comparison against reference JSON | diff tolerance per field |
| `schema_match` | actual conforms to a JSON schema | N/A |
| `file_exists` | a file at a computed path exists on disk | N/A |
| `file_content` | a file's contents match expected (line-by-line) | exact / regex |
## Global invariants
These hold for every successful response from the service unless explicitly negated by the row's own expected result.
| Invariant | Comparison | Notes |
|-----------|------------|-------|
| Response Content-Type is `application/json` for non-binary endpoints | exact | except `/health`, image/thumbnail file routes, and SSE (`text/event-stream`) |
| Error responses follow the suite envelope `{ error: { code, message, …details } }` | schema_match | `_docs/02_document/common-helpers/01_http-error-envelope.md` |
| `id` fields in annotation responses are 32 lowercase hex chars | regex `^[0-9a-f]{32}$` | derived from `XxHash3.Hash128` (post RB-04) over sampled image bytes |
| Tokens passed by callers are ES256 JWTs issued by admin (3 base64url segments) | regex `^[\w-]+\.[\w-]+\.[\w-]+$` | annotations does not issue tokens; this is the shape it accepts |
| For `[after RB-XX]` rows: skip until the listed Refactor Backlog item lands | — | Phase 3 validation removes them otherwise |
## Input → Expected Result Mapping
### Group F1 — Annotation create (`POST /annotations`)
Each row uses one binary fixture from `fixtures.md` plus a synthetic `detections[]` payload from `requests/F1_<NNN>_request.json`. The class_num values come from the seeded `detection_classes` (ids 018).
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F1-001 | `image_small` + `requests/F1_001_request.json` (1 detection: class_num=10 Plane, normalized bbox) | Single small frame, single detection | HTTP 200; body matches `AnnotationDto`; `body.detections.length == 1`; `body.id =~ /^[0-9a-f]{32}$/` | exact (status), schema_match (body), regex (id) | N/A | `expected_results/F1_001_response.json` |
| F1-002 | Same as F1-001 (re-POST with identical payload) | Idempotency check | HTTP 200; `body.id == <id from F1-001>`; no duplicate row written (verifiable via `GET /annotations/{id}` returning a single row) | exact | N/A | N/A |
| F1-003 | `image_empty_scene` + `requests/F1_003_request.json` (0 detections) | Frame with no detections | HTTP 200; `body.detections.length == 0`; YOLO label file `<images_dir>/<id>.txt` exists with 0 lines | exact (count), file_exists, file_content | N/A | N/A |
| F1-004 | `image_dense01` + `requests/F1_004_request.json` (5 detections: mixed class_nums 0,1,2,9,10) | Dense scene, multiple classes | HTTP 200; `body.detections.length == 5`; YOLO label file has 5 lines, each `<class_num> <cx> <cy> <w> <h>` with normalized floats | exact (count), file_content (regex per line) | N/A | `expected_results/F1_004_response.json` |
| F1-005 | `image_large` + `requests/F1_005_request.json` (3 detections) | Large payload (~7 MB) | HTTP 200; same shape as F1-001; latency `≤ 5000ms` (single-instance dev DB, no concurrent load) | exact, threshold_max | latency ± 1000ms | N/A |
| F1-006 | `video_short01` (mediaType=Video) + `requests/F1_006_request.json` (1 detection at videoTime=00:00:02.000) | Video frame annotation | HTTP 200; `body.id =~ /^[0-9a-f]{32}$/`; `body.videoTime == "00:00:02"` | exact, regex | N/A | N/A |
| F1-007 | `video_short01` + `video_short02` content-distinct + same detections payload | Distinct image bytes → distinct ids | `body_F1-007_a.id != body_F1-007_b.id` | exact (inequality) | N/A | N/A |
### Group F1-N — Annotation create negative cases
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F1-N-001 | `requests/F1_N_001_request.json` (no `image` bytes) | Missing image bytes | HTTP 400 or 422; error envelope present; `error.code` not empty | exact, schema_match | N/A | N/A |
| F1-N-002 | `requests/F1_N_002_request.json` (image bytes present but `mediaType` missing) | Missing required field | HTTP 400 or 422; error envelope | exact, schema_match | N/A | N/A |
| F1-N-003 | `image_small` + valid payload + JWT with policy `DATASET` only | Caller missing ANN policy | HTTP 403; error envelope `error.code` ∈ {`forbidden`, `policy_denied`} | exact, set_contains | N/A | N/A |
| F1-N-004 | `image_small` + valid payload + no `Authorization` header | Unauthenticated | HTTP 401; error envelope | exact | N/A | N/A |
| F1-N-005 | `image_small` + payload with `detections[0].centerX = 1.5` (out of 0..1 range) | Invalid bbox value | HTTP 200 today (no validator) → flag as documented gap; OR HTTP 400/422 if validation lands per SEC-05 | exact (today: 200) | N/A | N/A |
### Group F2 — Annotation listing & detail (`GET /annotations`, `/annotations/{id}`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F2-001 | `GET /annotations?limit=10` after F1-001..F1-004 succeeded | Paginated list | HTTP 200; `body.length == 4`; each item matches `AnnotationListItem` schema | exact (count), schema_match | N/A | N/A |
| F2-002 | `GET /annotations/{id from F1-001}` | Detail of an existing annotation | HTTP 200; `body.id == <id>`; `body.detections.length == 1` | exact | N/A | `expected_results/F1_001_response.json` (same file as F1-001) |
| F2-003 | `GET /annotations/00000000000000000000000000000000` | Nonexistent id | HTTP 404; error envelope; `error.code` matches `/not.?found/i` | exact, regex | N/A | N/A |
| F2-004 | `GET /annotations?missionId=<unknown-guid>` | Filter by mission with no annotations | HTTP 200; `body.length == 0` | exact | N/A | N/A |
### Group F3 — Realtime SSE (`GET /annotations/events`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F3-001 | Subscriber connects to `/annotations/events`, then F1-001 fires | SSE delivery for new annotation | Subscriber receives one event with `data` parsing as `AnnotationEventDto`; `event.operation == "Created"`; `event.annotationId == <id from F1-001>`; latency `≤ 1000ms` | schema_match, exact, threshold_max | latency ± 200ms | N/A |
| F3-002 | F1-001 fires, then subscriber connects | No backfill expected | Subscriber receives 0 events for the historical id within 5s window | exact (count) | N/A | N/A |
| F3-003 | Subscriber connects without `Authorization` header | Unauthenticated SSE | HTTP 401 on the SSE connection establishment | exact | N/A | N/A |
| F3-004 `[after RB-01]` | Subscriber connects, then `PUT /annotations/{id}` updates fields | Lifecycle observability for Update | Subscriber receives event with `event.operation == "Updated"`, payload reflecting the update | exact, schema_match | N/A | N/A |
| F3-005 `[after RB-01]` | Subscriber connects, then `DELETE /annotations/{id}` | Lifecycle observability for Delete (soft-delete) | Subscriber receives event with `event.operation == "Deleted"`; row status flips to `Deleted (40)`; image+label files relocate to `deleted_dir` | exact, file_exists | N/A | N/A |
### Group F4 — Outbox + Stream (`FailsafeProducer`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F4-001 | F1-001 succeeds | Outbox row inserted | After F1-001 returns 200, exactly one new row exists in `annotations_queue_records` with `annotation_id == <id>`, `operation == 10` (Created) | exact | N/A | N/A |
| F4-002 | After F4-001, wait for one drain cycle | Drainer publishes to RabbitMQ stream | Within `drain_interval + 2s`, the row is deleted AND a message lands on stream `azaion-annotations` | exact, threshold_max | N/A | N/A |
| F4-003 | Inspect the published stream message | Message wire format | gzip-decoded MessagePack body deserializes into the documented schema (`annotationId`, `operation`, `dateTime`, payload) | schema_match | N/A | `expected_results/F4_003_stream_message.json` |
| F4-004 `[after RB-09]` | Two F1-001 invocations with the same image bytes | Stream dedupe contract | Stream messages carry `(annotationId, operation, dateTime)`; a downstream consumer can collapse duplicates by that triple | exact, schema_match | N/A | N/A |
| F4-005 | RabbitMQ unreachable, then F1-001 fires | Drainer survives broker outage | Row stays in `annotations_queue_records` (does not get deleted); `FailsafeProducer` does not crash; queue depth grows; HTTP 200 still returned to the original caller | exact, schema_match | N/A | N/A |
### Group F5 — Media upload (`POST /media`, `POST /media/batch`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F5-001 | multipart `POST /media` with `image_small`, `mediaType=Image`, `waypointId=<guid>` | Single media upload | HTTP 200; body matches `MediaListItem`; file exists at `<media_dir>/<media_id>` (extension preserved) | exact, schema_match, file_exists | N/A | N/A |
| F5-002 | multipart `POST /media/batch` with 3 files (`image_small`, `image_dense01`, `image_dense02`) + same waypointId | Batch upload | HTTP 200; `body.length == 3`; 3 distinct `mediaId` values; 3 files on disk | exact, file_exists | N/A | N/A |
| F5-003 | `POST /media` with no `waypointId` | Missing required field | HTTP 400 or 422; error envelope | exact, schema_match | N/A | N/A |
| F5-004 | `POST /media` with caller missing ANN policy | AuthZ check | HTTP 403; error envelope | exact | N/A | N/A |
### Group F6 — Auth verification (Bearer token validation)
Annotations does not host login / refresh / register — those are owned by admin and out-of-scope for this test corpus. The annotations e2e harness runs against an in-stack **mock JWKS issuer** that mints ES256 tokens with the configured `JWT_ISSUER` / `JWT_AUDIENCE`; runtime tokens are minted on demand by the runner (`fixtures/auth/mock_issuer.py` or equivalent) using a private key whose public half lives in the JWKS the service fetches at boot. See `_docs/02_document/tests/test-data.md` and `_docs/02_document/tests/security-tests.md`.
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F6-001 | Any authenticated route called with a freshly minted ES256 token (correct iss / aud / exp) | Happy-path verification | HTTP 200 | exact | N/A | N/A |
| F6-002 | Same route with `iss` mismatched against `JWT_ISSUER` | Issuer rejection | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-003 | Same route with `aud` mismatched against `JWT_AUDIENCE` | Audience rejection | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-004 | Same route with `exp` 1 minute in the past | Expired token | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-005 | Same route with `alg=HS256` and admin's public ES256 key reused as the HMAC key | Algorithm-confusion attack | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-006 | Same route with no `Authorization` header | Anonymous rejection (except `/health`) | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
### Group F7 — Settings & metadata
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F7-001 | `GET /classes` after fresh boot | Detection class catalog | HTTP 200; `body.length == 19`; ids `[0..18]` present; entry where `id == 9` has `name == "Smoke"`; entry where `id == 10` has `name == "Plane"` | exact, set_contains | N/A | `expected_results/F7_001_classes.json` |
| F7-002 | `GET /settings/system` | System settings read | HTTP 200; `body` matches `SystemSettings` shape; `silent_detection` field present today (removed post RB-02) | exact, schema_match | N/A | N/A |
| F7-003 | `PUT /settings/directories` with new `imagesDir` value | PathResolver invariant | HTTP 200; subsequent `GET /settings/directories` returns the new value; `pathResolver.Reset()` invariant — the next `POST /annotations` writes to the new path | exact, file_exists (under new path) | N/A | N/A |
| F7-004 | `PUT /settings/directories` with caller missing ADM policy | AuthZ check | HTTP 403; error envelope | exact | N/A | N/A |
| F7-005 `[after RB-06]` | `POST /classes` (admin CRUD) with caller having ADM policy | New class added | HTTP 200; `GET /classes` returns 20 rows; cache invalidated | exact | N/A | N/A |
### Group F8 — Dataset
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F8-001 | `GET /dataset?status=10` after F1-001..F1-004 | Filter by status `Pending` | HTTP 200; all returned items have `status == 10`; `body.length` matches the count of Pending rows in DB | exact | N/A | N/A |
| F8-002 | `GET /dataset?classNum=10` | Filter by class `Plane` | HTTP 200; every returned item's annotation has at least one detection with `class_num == 10` | exact | N/A | N/A |
| F8-003 | `GET /dataset/class-distribution` | Class distribution | HTTP 200; `body` is an array; each entry has `classNum`, `label`, `color`, `count`; sum of counts equals total detection count | exact, schema_match | N/A | N/A |
| F8-004 | `POST /dataset/status/bulk` with `{ annotationIds: [<id1>, <id2>], status: 20 }` | Bulk status update | HTTP 200; both annotations have `status == 20` after the call (atomic SQL `UPDATE … WHERE id IN (…)`) | exact | N/A | N/A |
| F8-005 `[after RB-08]` | F8-004 path | Lifecycle event emission | Each updated annotation emits a `Updated` SSE event AND inserts an outbox row | exact, schema_match | N/A | N/A |
### Group F9 — Health & boot
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F9-001 | `GET /health` | Health check | HTTP 200; latency `≤ 200ms` | exact, threshold_max | N/A | N/A |
| F9-002 | Container fresh boot, run migrator twice | Migrator idempotence | Second boot makes 0 schema changes (no new tables, no new columns); 0 errors | exact (DDL diff) | N/A | N/A |
## Coverage summary
- **Functional positive**: 26 rows (F1-001..007, F2-001..004, F3-001..005, F4-001..005, F5-001..002, F6-001/004/007, F7-001..003, F8-001..004, F9-001..002).
- **Functional negative**: 12 rows (F1-N-001..005, F2-003, F3-003, F5-003..004, F6-002/003/005/006, F7-004).
- **`[after RB-XX]` rows** (skipped until the backlog item lands): F3-004, F3-005, F4-004, F7-005, F8-005, plus the post-RB-04 hash invariant in F1-001 — 6 deferred rows.
Total: **44 rows**; **38 active today**, **6 deferred behind backlog items**.
## Reference files (to author next)
The rows above reference these reference files in `expected_results/`. They will be authored as part of this skill's Phase 1 input-data analysis if the runner needs them; complex JSON bodies are best captured here once we run F1-001 against a real DB once and capture the response. For the initial spec, the regex/schema_match patterns above are sufficient.
| File | Purpose |
|------|---------|
| `F1_001_response.json` | Reference `AnnotationDto` body for `image_small` + 1 detection |
| `F1_004_response.json` | Reference body for dense scene (5 detections) |
| `F4_003_stream_message.json` | Reference MessagePack-decoded stream payload |
| `F7_001_classes.json` | Reference class catalog (19 rows, ids 018) |
## Open data gaps (raised during this draft)
- **Performance baselines**: `F1-005` and `F9-001` use single-instance latency thresholds (5000ms / 200ms) inferred from the codebase, NOT a contracted SLA. If suite-level perf targets exist, they override these.
- **`F1-N-005` invalid bbox value**: today the service silently accepts out-of-range `centerX`. Documented in `security_approach.md` SEC-05; needs a decision on whether the test should target the current (lenient) or future (validated) behavior.
- **F4-005 outage simulation**: depends on the test harness being able to restart RabbitMQ between cases — operational concern for the runner script (Phase 4).
-41
View File
@@ -1,41 +0,0 @@
# Test Fixtures
Binary fixtures (frame images + videos) live in the **sibling `detections` repo** under `azaion/suite/detections/_docs/00_problem/input_data/`. We do not duplicate them in this repo — the suite layout already collocates the two services and the test runners (Step 4 onward) resolve fixtures via the relative path below.
## Canonical fixture root
```
$SUITE_ROOT/detections/_docs/00_problem/input_data/
```
Where `$SUITE_ROOT` is the parent directory containing both `annotations/` and `detections/`. Test runner scripts (Phase 4) compute this from the running script's location: `dirname "$0"/../../../detections/_docs/00_problem/input_data/`.
## Image fixtures
| Local id | Source path (relative to suite root) | Dimensions | Size | Notes |
|----------|--------------------------------------|------------|------|-------|
| `image_small` | `detections/_docs/00_problem/input_data/image_small.jpg` | 1280 × 720 | ~1.5 MB | Primary single-frame test |
| `image_dense01` | `detections/_docs/00_problem/input_data/image_dense01.jpg` | n/a (~230 KB) | small | Many-detections test |
| `image_dense02` | `detections/_docs/00_problem/input_data/image_dense02.jpg` | n/a (~2.8 MB) | medium | Many-detections + larger payload |
| `image_different_types` | `detections/_docs/00_problem/input_data/image_different_types.jpg` | 900 × 1600 | ~150 KB | Multi-class detection input |
| `image_empty_scene` | `detections/_docs/00_problem/input_data/image_empty_scene.jpg` | 1920 × 1080 | ~2 MB | Zero-detection input |
| `image_large` | `detections/_docs/00_problem/input_data/image_large.JPG` | 6252 × 4168 | ~7 MB | Large payload boundary |
## Video fixtures
| Local id | Source path | Size | Notes |
|----------|-------------|------|-------|
| `video_short01` | `detections/_docs/00_problem/input_data/video_short01.mp4` | ~150 MB | Video annotation flow |
| `video_short02` | `detections/_docs/00_problem/input_data/video_short02.mp4` | ~150 MB | Distinct-bytes second input — for content-addressed-id divergence |
## Synthetic request payloads
Synthetic JSON request bodies (annotation create / update / dataset query / settings update / auth login) live under `_docs/00_problem/input_data/requests/`. They reference image fixtures by `local_id` from the table above; the runner inlines the binary at request time.
## Why path reference, not copy
- The video binaries are ~150 MB each; committing them would bloat this repo.
- Both services live under the same suite, so the relative path is stable.
- The detections team owns the source-of-truth fixtures (frames, videos). The annotations test corpus consumes them with its own contract layer (`expected_results/results_report.md`) — we do not redefine the inputs, only the contract.
If the layout ever diverges (annotations and detections move into different parent directories), `fixtures.md` is the one place to update the path resolution.
-55
View File
@@ -1,55 +0,0 @@
# Azaion.Annotations — Problem statement (retrospective)
> Reverse-engineered from `_docs/02_document/architecture.md`, `system-flows.md`, the per-component specs, and `suite/_docs/01_annotations.md`. Not copied from a real PRD — this is a retrospective synthesis.
## What this system is
`Azaion.Annotations` is the **annotation lifecycle service** of the AZAION suite. It is the single owner of the `annotations` table, the YOLO-format label files on disk, the lifecycle event stream (RabbitMQ + SSE), and the dataset exploration surface that downstream tooling — annotator UIs, the AI training pipeline, the admin sync worker — relies on.
It is a single .NET 10 service backed by PostgreSQL and a content-addressed filesystem cache, packaged as an ARM64 Docker image, deployed by the suite's branch-driven Woodpecker pipeline.
## Problem it solves
The suite's surveillance / detection pipeline produces a continuous stream of detected objects in video frames. Three independent consumers need that same data shaped differently:
1. **Annotator UI** — humans need to review, correct, accept, or reject each detection in near-real-time, frame-by-frame, with the underlying image visible. They need every change another annotator makes to surface immediately on their screen — no refresh button.
2. **AI training pipeline** — needs the *finalized* annotations + image bytes as a durable, replayable feed so it can build training datasets at any cadence.
3. **Suite-level admin worker** — needs an audit-grade record of every state change (who, when, what) for cross-service synchronisation.
Without a dedicated lifecycle service, these consumers would each poll the detection pipeline directly, which (a) doesn't expose lifecycle semantics — only "the model said this", not "a human accepted it", (b) has no notion of soft delete, status transitions, or human authorship, and (c) cannot deliver realtime updates to UIs and durable replay to batch consumers from the same source of truth.
`Azaion.Annotations` solves that three-way mismatch by being **the one place** where annotation state lives, where state transitions are emitted, and where both push (SSE for humans) and durable-pull (RabbitMQ Stream for machines) consumers attach.
## Users (consumer roles)
| Consumer | How they reach the system | What they need |
|----------|---------------------------|----------------|
| Annotator UI (human-facing web app) | REST + SSE, JWT policy `ANN` | List + detail of annotations, mutations, real-time fan-out of every other annotator's edits |
| Dataset Explorer UI | REST under `/dataset`, JWT policy `DATASET` | Filterable read of the current annotation corpus + bulk-status writes |
| Detections service (upstream pipeline) | REST `POST /annotations`, JWT policy `ANN`; long-running tokens are refreshed against admin's `POST /token/refresh` (annotations is verifier-only) | Push raw detections + the original frame image; receive the assigned annotation id |
| AI training pipeline | RabbitMQ Stream consumer (`azaion-annotations`) | Durable, replayable lifecycle events with the full payload |
| Admin sync worker | RabbitMQ Stream consumer (`azaion-annotations`) | Same stream, different consumer offset; cross-service event correlation |
| Suite admin (humans) | REST `[ADM]` endpoints (planned for `/classes` per RB-06; service-account registration is owned by the admin service, not annotations) | Manage detection class catalog, register service accounts (against admin) |
## How it works at a high level
A detection arrives as `POST /annotations` with the original frame as `image_bytes` and a list of YOLO detections in normalised coordinates. The service:
1. **Content-addresses** the image — sampled hashing produces a stable 32-char hex id; identical re-uploads collapse to the same row.
2. **Persists** the image, optionally a media row, the annotation row, and the detection rows in a transactional unit (subject to the agreed Refactor Backlog item RB-03 — today FS + DB + outbox are not yet wrapped together).
3. **Writes** a YOLO `.txt` label file next to the image so the AI training pipeline can ingest the data with no transformation step.
4. **Publishes** an SSE event so every Annotator UI viewing that mission gets the new annotation immediately.
5. **Enqueues** a row in the transactional outbox `annotations_queue_records`. The in-process `FailsafeProducer` drains that outbox into the RabbitMQ stream as a MessagePack-gzip frame, so the AI pipeline and admin worker get a durable copy.
Mutations (Update / UpdateStatus / Delete) follow the same shape — but only after Refactor Backlog item RB-01 lands (today they are silent on both SSE and outbox; that is a known gap, not a design choice). Deletes are *soft*: status flips to `Deleted (40)`, files relocate to a `deleted_dir`, the row stays.
The service also serves the dataset exploration surface (`/dataset/*`), the media upload pipeline (`/media/*`), and the system-metadata catalog (`/settings/*`, `/classes`).
## Cross-references
- Suite-level integration narrative: `suite/_docs/01_annotations.md`
- Architecture vision + 13 ADRs: `_docs/02_document/architecture.md`
- 8 verified system flows F1F8: `_docs/02_document/system-flows.md`
- Component-level specs: `_docs/02_document/components/*/description.md`
- Glossary (canonical terminology): `_docs/02_document/glossary.md`
- README: none in repo (gap noted in `_docs/02_document/00_discovery.md`).
-53
View File
@@ -1,53 +0,0 @@
# Azaion.Annotations — Restrictions
> Only constraints **evidenced in code, configs, or Dockerfiles** are listed. Inferred-but-unverified items are flagged.
## Hardware
| ID | Restriction | Evidence |
|----|-------------|----------|
| HW-01 | Service binary is built for ARM64 only — no AMD64 image is produced. | `.woodpecker/build-arm.yml` (`platforms: linux/arm64`); `Dockerfile` `--arch=$BUILDARCH` driven by `BUILDPLATFORM=linux/arm64`. |
| HW-02 | Local writable filesystem is required at `images_dir` / `videos_dir` / (planned) `deleted_dir`. | `Services/AnnotationService.cs` (`File.WriteAllBytesAsync`), `Services/PathResolver.cs`, `directory_settings` table. |
| HW-03 | Memory pressure scales with the largest single image read into memory by `FailsafeProducer` (re-reads the image to put bytes on the wire). | `Services/FailsafeProducer.cs:138` neighborhood. |
## Software
| ID | Restriction | Evidence |
|----|-------------|----------|
| SW-01 | .NET 10 SDK and runtime — no fallback. | `Dockerfile` `mcr.microsoft.com/dotnet/sdk:10.0`, `aspnet:10.0`. |
| SW-02 | PostgreSQL backend; migrator emits `IF NOT EXISTS`, `ON CONFLICT`, `CREATE TYPE` — Postgres 13+ semantics expected. | `Database/DatabaseMigrator.cs`. |
| SW-03 | RabbitMQ broker with the **streams plugin** enabled — service uses `RabbitMQ.Stream.Client`, not classic queues. | `Services/FailsafeProducer.cs`. |
| SW-04 | Linq2DB ORM, MessagePack with the contractless resolver, gzip wire format. | `Services/FailsafeProducer.cs`. |
| SW-05 | JWT verification is **ES256 over admin's JWKS** (`JWT_JWKS_URL`); `ValidAlgorithms` is pinned to `EcdsaSha256`. Annotations is verifier-only — admin is the sole token issuer for the suite. JWKS retrieval requires HTTPS. | `Auth/JwtExtensions.cs`. |
## Environment
| ID | Restriction | Evidence |
|----|-------------|----------|
| ENV-01 | Required env vars (fail-fast at startup via `ConfigurationResolver`): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`. Optional with defaults: `RABBITMQ_HOST`, `RABBITMQ_STREAM_PORT`, `RABBITMQ_PRODUCER_USER`, `RABBITMQ_PRODUCER_PASS`. | `Program.cs`, `Infrastructure/ConfigurationResolver.cs`, `Services/FailsafeProducer.cs`. |
| ENV-02 | Service listens on port `8080` HTTP, no TLS terminator inside the image. | `Dockerfile` `EXPOSE 8080`, `ASPNETCORE_URLS=http://+:8080`. |
| ENV-03 | Build stamps `AZAION_REVISION` from CI; `Program.cs` echoes it on startup. | `Dockerfile` `ARG AZAION_REVISION`, `Program.cs`. |
| ENV-04 | Image tag scheme is branch-driven: `${BRANCH}-arm`. No semver tags. | `.woodpecker/build-arm.yml`. |
| ENV-05 | Swagger UI is mounted unconditionally — present in production builds (ADR-005). | `Program.cs`. |
| ENV-06 | CORS is config-driven (`CorsConfig:AllowedOrigins` + opt-in `CorsConfig:AllowAnyOrigin`); `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when the allow-list is empty and `AllowAnyOrigin` is not set. ADR-006 retired. | `Program.cs`, `Infrastructure/CorsConfigurationValidator.cs`. |
| ENV-07 | Boot-time `DatabaseMigrator.MigrateAsync()` runs on startup — no separate migration step in the deploy pipeline (ADR-007). | `Program.cs`, `Database/DatabaseMigrator.cs`. |
## Operational
| ID | Restriction | Evidence |
|----|-------------|----------|
| OP-01 | SSE state is per-instance — no broker fan-out — so horizontal scaling is bounded today. | `Services/AnnotationEventService.cs` (in-process `Channel<>`). |
| OP-02 | Outbox drainer has no row-leasing — running multiple instances will double-publish until RB-09 deduplication contract is in place. | `Services/FailsafeProducer.cs`. |
| OP-03 | No automated test suite in repo; CI does build-and-push only. | `_docs/02_document/00_discovery.md`, `.woodpecker/build-arm.yml`. |
| OP-04 | No lint or formatter step in CI. | `.woodpecker/build-arm.yml`. |
| OP-05 | Dockerfile `HEALTHCHECK` calls `/health`; HTTP 200 expected by orchestrator. | `Dockerfile`. |
| OP-06 | The service must be the only writer of `annotations_queue_records` — the table is treated as a private outbox. | `Services/AnnotationService.cs`, `Services/FailsafeProducer.cs`. |
| OP-07 | DB connection string format is the Java/Hikari `jdbc:postgresql://…` style; `Helpers/PostgreSqlConnectionStringHelper` parses it. | `Helpers/PostgreSqlConnectionStringHelper.cs`. |
## Cross-cutting (suite-level, evidence in `suite/_docs/01_annotations.md`)
| ID | Restriction |
|----|-------------|
| SUITE-01 | The shared JWT secret family is cross-service; revoking it invalidates every service token. |
| SUITE-02 | Wire enums for `AnnotationStatus`, `MediaType`, `QueueOperation` are duplicated across services and must move in lock-step (or a single contract has to be published). |
| SUITE-03 | Stream consumers (admin worker, AI training) commit offsets independently — Annotations does not own retention semantics. |
-86
View File
@@ -1,86 +0,0 @@
# Azaion.Annotations — Security approach (retrospective)
> Inventory of the **security mechanisms actually present in code today** + the gaps that the autodev existing-code Step 14 (Security Audit) will close. Evidence anchored to source files. ADR references point to `_docs/02_document/architecture.md`.
## Authentication
- **Mechanism**: JWT Bearer with **ES256 asymmetric signing**, verified against admin's JWKS endpoint (`JWT_JWKS_URL`, default `https://admin.azaion.com/.well-known/jwks.json`).
- **Token validator code**: `src/Auth/JwtExtensions.cs` (verifier only — annotations does not mint tokens).
- **Validation parameters**: `ValidateIssuer = true`, `ValidateAudience = true`, `ValidateLifetime = true`, `ValidateIssuerSigningKey = true`, `ValidAlgorithms = [SecurityAlgorithms.EcdsaSha256]`, `RequireSignedTokens = true`, `RequireExpirationTime = true`, `ClockSkew = 30s`.
- **Anonymous endpoints**: only `GET /health` — every other endpoint requires authentication. The legacy `POST /auth/refresh` was removed; callers refresh against admin's `POST /token/refresh`.
- **Token storage**: stateless. Refresh tokens live in admin's DB (revocation is enforced by admin); annotations does not persist any token material.
## Authorization
- **Policies declared**: `ANN`, `DATASET`, `ADM` (in `src/Auth/JwtExtensions.cs`).
- **Policy → controller mapping** (verified):
- `ANN`: `AnnotationsController`, `MediaController` (annotation lifecycle + media upload — what humans on the annotation UI need).
- `DATASET`: `DatasetController` (dataset exploration).
- `ADM`: planned `[ADM]` writes on `/classes` (RB-06) and mutating routes on `/settings/*`.
- Mixed `[Authorize]` (any authenticated): the read endpoints under `/settings/*`.
- **Per-action overrides**: writes inside `/settings/*` typically require `ADM`; reads accept any authenticated user.
### Known weakness
- No row-level / tenancy authorization is implemented. A user with policy `ANN` can read/mutate any annotation regardless of mission ownership. This is acceptable for the current single-tenant deployment but must be documented before any multi-tenant rollout.
## Secrets handling
- **Source**: env vars resolved at `Program.cs` boot via `ConfigurationResolver.ResolveRequiredOrThrow` (env var → `IConfiguration` → throw if missing).
- **Required in any environment** (no fallback — service refuses to start without them): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`.
- **Soft defaults**: `RABBITMQ_*` keep their development defaults; operators MUST still override for production.
- **No secret manager integration** in code today — secrets land via container env (suite-level orchestrator's responsibility).
- `JWT_SECRET` was removed: annotations holds no HMAC material and no longer issues tokens.
## Transport
- Inside the container: HTTP only (`ASPNETCORE_URLS=http://+:8080`).
- TLS termination: outside the container (suite-level reverse proxy / orchestrator).
- No HSTS or HTTPS-redirect middleware in `Program.cs`.
## CORS
- Configuration: config-driven allow-list via `CorsConfig:AllowedOrigins` (`Program.cs` + `Infrastructure/CorsConfigurationValidator.cs`).
- `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when the allow-list is empty and `CorsConfig:AllowAnyOrigin` is not explicitly set; a `LogWarning` is emitted in lower environments when running with the permissive fallback so the drift is visible in logs.
## Input validation & sanitization
- **REST DTO binding**: ASP.NET model binding only — no `FluentValidation` or custom validator visible in code.
- **File upload validation**: no MIME / extension whitelist visible at the `MediaController` layer. Step 14 should confirm whether the upstream pipeline restricts upload format or whether the service itself needs to.
- **SQL**: all access via Linq2DB / parameterised queries — no raw string concatenation found.
- **YOLO label file write**: trusts caller-provided detections (class id, coordinates) — clamping / range checks would be a Step 14 candidate.
## Rate limiting / DoS
- **None present** — there is no rate-limiting middleware (`AddRateLimiter`, `UseRateLimiter`) registered in `Program.cs`.
- **Implicit limits**: SSE channel is unbounded; outbox table is unbounded; RabbitMQ stream is bounded by retention config (suite-level).
- Step 14 candidate: per-IP rate limit on `POST /annotations` and `POST /media`, since they accept image bytes and write to disk.
## Auditing & logging
- Console logger configured in `Program.cs` (default ASP.NET Core logging).
- Authentication failures: rely on `JwtBearer` middleware default 401s — not explicitly logged with extra detail.
- No audit log of mutations is written today; the `annotations_queue_records` outbox + downstream stream IS the de-facto audit trail (post RB-01 + RB-09).
## Observability boundary
- `/health` is the only pre-auth endpoint that reveals state. It returns 200/non-200 only — no version or DB info — so it is safe to expose to load balancers without auth.
- Swagger UI is mounted in all environments (ADR-005). It exposes the full controller surface but no secrets. Step 14 should consider gating it behind `ADM` or environment-conditional registration.
## Error response surface
- All errors returned via `Middleware/ErrorHandlingMiddleware` use the suite-standard envelope (`_docs/02_document/common-helpers/01_http-error-envelope.md`).
- Stack traces are not echoed back to the client in prod (verify the `IsDevelopment()` branch in the middleware during Step 14).
## Summary table — known security gaps to address in Step 14
| ID | Area | Gap | Status |
|----|------|-----|--------|
| SEC-01 | Auth | JWT issuer/audience not validated | **Closed**`ValidateIssuer`/`ValidateAudience` enforced; `JWT_ISSUER` and `JWT_AUDIENCE` are required env vars. |
| SEC-02 | Secrets | Dev fallback for `JWT_SECRET` in source | **Closed**`JWT_SECRET` removed; remaining required vars fail fast on startup via `ConfigurationResolver`. |
| SEC-03 | CORS | `AllowAnyOrigin` default | **Closed** — config-driven allow-list; `CorsConfigurationValidator` blocks empty list in `Production`. |
| SEC-04 | Surface | Swagger UI exposed in prod | Open — gate behind `ADM` or `Development` only. |
| SEC-05 | Upload | No MIME / extension whitelist on `/media` | Open — validate at controller before disk write. |
| SEC-06 | DoS | No rate limiting on hot write endpoints | Open — per-IP / per-user limiter on `POST /annotations`, `POST /media`. |
| SEC-07 | Tenancy | No row-level authorization | Open — document constraint; add mission-scoped check before multi-tenant rollout. |
| SEC-08 | Audit | No structured audit log | Open — use post-RB-01 lifecycle stream as the audit substrate; add structured fields. |
-175
View File
@@ -1,175 +0,0 @@
# Azaion.Annotations — Solution (retrospective)
> Retrospective view, derived from `_docs/02_document/`. Mirrors the artifact the `research` skill produces, but synthesized from verified code rather than user interview. Read alongside `_docs/02_document/architecture.md` (which carries the confirmed Architecture Vision and the ADR list) and the agreed Refactor Backlog (RB-01..RB-09).
## 1. Product solution description
`Azaion.Annotations` is the suite-internal HTTP + streaming service that owns the **annotation lifecycle**: ingest a video frame (or pre-existing media), record the YOLO detections produced by the upstream detection pipeline, expose CRUD over those annotations, and broadcast every lifecycle change to (a) the Annotator UI in real time via SSE and (b) downstream durable consumers (admin sync worker, AI training pipeline) via a transactional-outbox + RabbitMQ Stream pipeline. It also serves the dataset exploration surface, the media upload pipeline, and the system-metadata catalog (settings + detection classes).
Single .NET 10 binary, single Postgres state-of-record, content-addressed filesystem cache, ARM64 container deployed by branch via Woodpecker CI.
```mermaid
flowchart LR
subgraph clients [Clients]
UI[Annotator UI]
DSE[Dataset Explorer UI]
DET[Detections service]
ADM[Admin sync worker]
AI[AI training]
end
subgraph svc [Azaion.Annotations]
REST[01 Annotations REST]
RT[02 Realtime and sync]
MEDIA[03 Media]
DS[04 Dataset]
SET[05 Settings and metadata]
PLAT[06 Platform]
end
subgraph store [Stores]
DB[(PostgreSQL)]
FS[(Filesystem /data)]
STREAM[(RabbitMQ Stream)]
end
UI -- "REST + SSE" --> REST
UI -- "REST + SSE" --> RT
DSE -- "REST DATASET" --> DS
DET -- "POST + auth refresh" --> REST
DET -- "POST" --> MEDIA
REST --> RT
REST --> PLAT
RT --> PLAT
MEDIA --> PLAT
DS --> PLAT
SET --> PLAT
PLAT --> DB
PLAT --> FS
RT --> STREAM
STREAM --> ADM
STREAM --> AI
```
## 2. Architecture (as implemented)
The implemented architecture per component, with the agreed near-term direction (Refactor Backlog RB-01..RB-09) flagged in **Limitations** and **Requirements** rows.
### 2.1 — `06 Platform` (foundation)
| Field | Value |
|-------|-------|
| Solution | Shared kernel: `AppDataConnection` (Linq2DB), `DatabaseMigrator` (idempotent boot-time DDL), JWT verifier (`JwtExtensions.AddJwtAuth` — ES256 over admin's JWKS, no local minting), `Infrastructure/ConfigurationResolver` (fail-fast required-config resolution), `Infrastructure/CorsConfigurationValidator` (env-aware safety check), `PathResolver`, `ErrorHandlingMiddleware`, composition root (`Program.cs`). |
| Tools | .NET 10, ASP.NET Core, Linq2DB, Npgsql, JwtBearer (verifier-only), `Microsoft.IdentityModel.Protocols.OpenIdConnect` for JWKS resolution, Swashbuckle. |
| Advantages | Single composition root; idempotent migrator removes a separate migration tool from the deployment story (ADR-007); error envelope is uniform across all controllers; identity is fully out-sourced to admin (no HMAC secret to leak, no token-issuance code path to attack). |
| Limitations | Swagger UI mounted in all environments (ADR-005); JWKS retrieval requires HTTPS — test harnesses need a TLS-terminating sidecar or test-only relaxation. (ADR-002 / ADR-006 retired by the auth + CORS refactor.) |
| Requirements | `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL` are required at startup (fail-fast); `CorsConfig:AllowedOrigins` (or explicit `AllowAnyOrigin=true`) required in `Production`; `directory_settings` row reachable; Postgres 13+ behavior assumed by `ON CONFLICT` and `IF NOT EXISTS` clauses. |
| Security | JWT bearer with policies `ANN`, `DATASET`, `ADM`. `[AllowAnonymous]` only on `/health`; refresh is admin's responsibility. |
| Cost | Negligible — pure in-process plumbing. |
| Fit | Platform mandate is satisfied (single state-of-record, single secret family, single error envelope). Hardening items belong to the Security Audit step. |
### 2.2 — `02 Annotations realtime & sync`
| Field | Value |
|-------|-------|
| Solution | In-process SSE channel (`AnnotationEventService`, unbounded `Channel<AnnotationEventDto>`) + transactional outbox (`annotations_queue_records`) drained by `FailsafeProducer` (`IHostedService`) into the `azaion-annotations` RabbitMQ stream as MessagePack-gzip frames. |
| Tools | `System.Threading.Channels`, `RabbitMQ.Stream.Client`, `MessagePack`, `System.IO.Compression` (gzip). |
| Advantages | Sub-millisecond fan-out for UI (channel) without standing up a broker for the inner loop; durability for external consumers via the outbox even when RabbitMQ is unreachable; producer + drainer co-located removes a deployment unit. |
| Limitations | Per-instance SSE state — no cross-pod fan-out; no leasing on outbox rows (multi-instance can double-publish); empty `catch { }` at `FailsafeProducer.cs:138` swallows IOException on image read — RB-05. |
| Requirements | RabbitMQ Stream reachable on `RABBITMQ_HOST:RABBITMQ_STREAM_PORT`; `pathResolver` must resolve `images_dir/{id}.jpg` for `Created` operations; world-B mutation paths still TODO (RB-01). |
| Security | RabbitMQ stream auth via `RABBITMQ_PRODUCER_USER` / `_PASS`. SSE inherits `[Authorize(Policy = "ANN")]`. |
| Cost | Channel = O(1) memory per pending message; outbox row + drained delete = ~1 round-trip per lifecycle event. Stream send is gzip-batched. |
| Fit | Strong fit for current scale; horizontal-scale constraints surface at >1 instance and need to be designed before that point. |
### 2.3 — `01 Annotations REST`
| Field | Value |
|-------|-------|
| Solution | `AnnotationsController` (REST + image/thumbnail file routes) → `AnnotationService` → DB + filesystem; lifecycle producer for SSE + outbox. |
| Tools | ASP.NET Core controllers, Linq2DB, `System.IO.Hashing.XxHash64` (today; `XxHash3.Hash128` per RB-04), `System.IO.File`. |
| Advantages | Content-addressed annotation id deduplicates re-uploads; YOLO label written deterministically next to the image; SSE event carries the full detection payload so UIs can render without an extra round-trip. |
| Limitations | Today only `Create` publishes / enqueues — Update / UpdateStatus / Delete are silent (RB-01); not transactional across FS + DB + outbox (RB-03); sampled `XxHash64` collision domain is small (RB-04); thumbnails not generated inline. |
| Requirements | World-B publish + enqueue per RB-01; business-transaction wrapper per RB-03; switch to `XxHash3.Hash128` per RB-04; rename `FlightId``MissionId` per RB-07. |
| Security | `[Authorize(Policy = "ANN")]` on the controller; user identity derived from JWT `NameIdentifier`. |
| Cost | Per-create: 1 image write + (optional) 1 image copy + 3 DB INSERTs (media optional, annotation, detections via BulkCopy) + 1 label write + 1 SSE channel write + 1 outbox INSERT. |
| Fit | Solid current shape; the four RB items above bring it in line with the agreed direction. |
### 2.4 — `03 Media`
| Field | Value |
|-------|-------|
| Solution | `MediaController` (single + batch upload, list, file download, delete) → `MediaService` → DB + filesystem under media dir. |
| Tools | ASP.NET Core multipart binding (`IFormFileCollection`), Linq2DB, `System.IO.File`. |
| Advantages | Batch path takes a single waypoint id + multiple files in one request — avoids N round-trips for bulk video frame uploads. |
| Limitations | No format whitelist enforcement is visible at the controller layer (verify during Step 14 Security Audit); no per-tenant quota enforcement. |
| Requirements | `videos_dir` / `images_dir` writable; `MediaType` correctly set per upload. |
| Security | `[Authorize(Policy = "ANN")]`. User from JWT `NameIdentifier`. |
| Cost | One disk write per file + one DB INSERT per row. |
| Fit | Adequate for the current upload volumes. |
### 2.5 — `04 Dataset`
| Field | Value |
|-------|-------|
| Solution | Read-heavy `/dataset` surface (filtered queries, class distribution, single + bulk status updates). |
| Tools | Linq2DB queries against `annotations × media × detection`. |
| Advantages | Bulk status update collapses N row updates into a single `UPDATE … WHERE id IN (…)` — atomic at the SQL level. |
| Limitations | Tight coupling to the annotation domain via shared `AppDataConnection` (RB-08); writes are silent (no SSE / outbox today — fixed by RB-01); reads do not yet filter soft-deleted annotations (will need to once RB-01 lands). |
| Requirements | Decouple writes per RB-08 (route through `AnnotationService`); honor soft-delete filter on read paths once status `Deleted=40` becomes a soft-delete marker. |
| Security | `[Authorize(Policy = "DATASET")]`. |
| Cost | Read paths perform LINQ `EXISTS` subqueries (`db.Detections.Any(...)`) — acceptable for current data volume; revisit during Step 15 Performance Test if dataset grows substantially. |
| Fit | Fits the Dataset Explorer UI; the coupling fix will improve maintainability without changing user-visible behavior. |
### 2.6 — `05 Settings & metadata`
| Field | Value |
|-------|-------|
| Solution | CRUD endpoints for system / directory / camera / user settings under `/settings`; read-only `/classes` for the detection class catalog (becoming admin-managed per RB-06). |
| Tools | Linq2DB, ASP.NET Core controllers. |
| Advantages | Directory cache reset is wired (verified — `SettingsService.cs:71, 85`); single-row settings model keeps the surface simple. |
| Limitations | `system_settings.silent_detection` is a debug remnant scheduled for removal (RB-02); detection classes are migrator-only today, no admin write path (RB-06); `Smoke` and `Plane` share color `#000080` — fixed as part of RB-06. |
| Requirements | Add `[ADM]` CRUD on `/classes` + read-through cache (RB-06); drop `silent_detection` (RB-02). |
| Security | Mixed `[Authorize]` reads / `[ADM]` writes. |
| Cost | One Postgres row family per concern; cache reset is O(1). |
| Fit | Good fit; the two RB items above complete the surface. |
## 3. Testing strategy
**Current state (verified)**: there is **no automated test project** in this workspace (`00_discovery.md`). CI runs only the build + image push (`.woodpecker/build-arm.yml`) — no test step, no lint step. There is no Postman / Bruno collection in-repo either.
**Implication for the autodev existing-code flow**: Step 3 (Test Spec) and Step 6 (Implement Tests) of Phase A produce the missing test surface. The shape required is:
- **Functional / integration tests** — happy-path and error-path coverage for every controller endpoint listed in `system-flows.md` (F1F8), exercised against a real Postgres + RabbitMQ stack (test-environment parity is a `coderule.mdc` mandate).
- **Lifecycle-observability tests** (post-RB-01) — every mutation path emits an SSE event AND inserts the expected outbox row with the right `QueueOperation`.
- **Soft-delete contract tests** (post-RB-01) — `DELETE /annotations/{id}` flips status to `Deleted (40)`, leaves the row, and relocates files to `deleted_dir`.
- **Stream consumer dedupe tests** (post-RB-09) — outbox messages carry `(annotationId, operation, dateTime)` and a synthetic dedupe consumer collapses a deliberately re-published message.
- **Hash collision regression** (post-RB-04) — same image bytes still hash to the same 32-char hex id; two distinct images do not collide on the sampled `XxHash3.Hash128` domain at scale.
- **Auth boundary tests** — unauthenticated, wrong-policy, expired-token, and refresh-flow scenarios for every policy (`ANN`, `DATASET`, `ADM`, `[Authorize]`).
- **Migrator idempotence** — boot, boot, boot — schema converges to the same shape; seed rows respect `ON CONFLICT DO NOTHING`.
- **Path resolver invariants**`PUT /settings/directories` triggers `Reset()` and subsequent path lookups reflect the change.
Non-functional ones to layer on once the functional surface is green:
- **Throughput / latency** for `POST /annotations` with image bytes — service must handle the suite's current detections-pipeline cadence without queue backpressure surfacing as 5xx.
- **SSE longevity** — single connection survives 30+ minutes idle without buffer growth.
- **Outbox drain throughput**`FailsafeProducer` keeps queue depth ~constant under steady-state lifecycle traffic.
## 4. References
| Source | Relevance |
|--------|-----------|
| `src/Program.cs` | Composition root: services, JWT, CORS, Swagger, migrator, middleware, `/health`. |
| `src/Database/DatabaseMigrator.cs` | Authoritative DB schema + seeded rows. |
| `src/Services/AnnotationService.cs` | F1 lifecycle producer; the only producer call site for SSE + outbox. |
| `src/Services/FailsafeProducer.cs` | Outbox drainer + `EnqueueAsync` static helper; contains the empty-catch RB-05 finding. |
| `src/Services/SettingsService.cs:71,85` | `pathResolver.Reset()` invariant (verified). |
| `src/Dockerfile` | Multi-arch build; `EXPOSE 8080`; `AZAION_REVISION` stamp. |
| `.woodpecker/build-arm.yml` | CI: branch-driven `${BRANCH}-arm` tags; OCI labels; ARM64 only. |
| `_docs/02_document/architecture.md` | Architecture Vision + 13 ADRs + 9-item Refactor Backlog. |
| `_docs/02_document/system-flows.md` | F1F8 traces with verified sequences. |
| `_docs/02_document/data_model.md` | ERD, tables, columns, seed data, migration semantics. |
| `_docs/02_document/glossary.md` | Project-specific terminology, with code → suite term alignment. |
| `_docs/02_document/04_verification_log.md` | Step 4 corrections + stakeholder resolutions. |
| `suite/_docs/01_annotations.md` | Suite-level product/integration narrative; canonical for `Mission`, wire enums, REST contract. |
-208
View File
@@ -1,208 +0,0 @@
# Codebase discovery — Azaion.Annotations
## Canonical product documentation (external)
Suite-level API and integration reference (maintained with the monorepo):
`/Users/obezdienie001/dev/azaion/suite/_docs/01_annotations.md`
(Relative from this repo: `../_docs/01_annotations.md`.)
This `_docs/02_document/` run is **bottom-up from code**; keep `01_annotations.md` aligned when HTTP contracts or integration behavior change.
---
## Directory tree (source)
```
annotations/
├── README.md
├── src/
│ ├── Azaion.Annotations.csproj
│ ├── Dockerfile
│ ├── GlobalUsings.cs
│ ├── Program.cs
│ ├── Auth/
│ ├── Controllers/
│ ├── Database/
│ │ ├── AppDataConnection.cs
│ │ ├── DatabaseMigrator.cs
│ │ └── Entities/
│ ├── DTOs/
│ ├── Enums/
│ ├── Middleware/
│ └── Services/
└── _docs/
```
Ignored per scan policy: `bin/`, `obj/`, `.git`, `node_modules`, `__pycache__`.
---
## Tech stack
| Area | Choice |
|------|--------|
| Language | C# / .NET (`net10.0` in csproj) |
| Host | ASP.NET Core minimal hosting + controllers |
| ORM / DB | Linq2DB + Npgsql → PostgreSQL |
| Auth | JWT Bearer (`Microsoft.AspNetCore.Authentication.JwtBearer`) |
| Messaging | RabbitMQ Streams (`RabbitMQ.Stream.Client`), MessagePack |
| API docs | Swashbuckle (Swagger / OpenAPI) |
| Hashing | `System.IO.Hashing` (annotation id / media paths) |
---
## Package manifest
- `src/Azaion.Annotations.csproj` — single web project.
---
## Config and operations
| Artifact | Role |
|----------|------|
| `src/Dockerfile` | Container build |
| `Program.cs` | `DATABASE_URL`, `JWT_ISSUER` / `JWT_AUDIENCE` / `JWT_JWKS_URL`, `CorsConfig:*`, RabbitMQ env vars (`RABBITMQ_*`), migrator on startup. All required vars are resolved through `ConfigurationResolver` (fail-fast). |
| `.vscode/launch.json` | Local debugging (if present) |
No `.github/workflows` in this repository (CI may live in suite/monorepo).
---
## Entry points
- **`Program.cs`** — service registration, JWT, CORS, Swagger, `DatabaseMigrator.Migrate`, middleware pipeline, `MapControllers()`, `/health`.
---
## Tests
No `*.Tests.csproj` or `tests/` tree in this workspace — **no automated test project** discovered.
---
## Existing documentation in repo
- Root `README.md` — points to suite `01_annotations.md`.
- `src/README.md` — short service blurb + link to root README.
---
## Module boundaries (revised — aligned with `01_annotations.md`)
The suite file is organized around **annotation lifecycle**, **media**, **settings/camera**, **SSE**, **RabbitMQ sync**, and **auth** (JWT refresh). The codebase splits the same concerns across controllers/services; **dataset** and **detection classes** are additional HTTP surfaces referenced from suite `09_dataset_explorer.md` / UI.
| # | Module (doc file) | Primary code | Suite `01_annotations.md` anchor |
|---|-------------------|--------------|-----------------------------------|
| 1 | `wire-enums.md` | `src/Enums/*` | “Wire format”, enum tables |
| 2 | `database-layer.md` | `src/Database/*` | Annotation identity, tables, `SilentDetection` / `GenerateAnnotatedImage` columns |
| 3 | `common-infrastructure.md` | `PathResolver`, `ErrorHandlingMiddleware`, `PaginatedResponse`, `ErrorResponse`, `GlobalUsings.cs` | File paths for image/label/thumb/results; error JSON shape |
| 4 | `auth-identity.md` | `JwtExtensions` (verifier-only over admin's JWKS) | JWT forward only — refresh is admin's responsibility (annotations no longer hosts `/auth/refresh`) |
| 5 | `media-service.md` | `MediaService`, `MediaController`, media DTOs | §710 POST/GET/DELETE media, batch upload |
| 6 | `annotations-service.md` | `AnnotationService`, `AnnotationsController` (REST + static files, not SSE) | §16 annotations CRUD/query |
| 7 | `dataset-service.md` | `DatasetService`, `DatasetController`, dataset DTOs | Cross-ref §3 note (DATASET); `09_dataset_explorer.md` |
| 8 | `settings-metadata-service.md` | `SettingsService`, `SettingsController`, `ClassesController`, settings DTOs | §1112 camera; directories/system/user settings; GET `/classes` |
| 9 | `sse-realtime.md` | `AnnotationEventService`, SSE action on `AnnotationsController` | §SSE `GET /annotations/events`, `AnnotationEvent` |
| 10 | `rabbitmq-stream-sync.md` | `FailsafeProducer`, `RabbitMqConfig`, `DTOs/QueueMessages.cs`, queue entity | §Annotation Sync, Failsafe, Stream |
| 11 | `composition-program.md` | `Program.cs` | Wiring, env defaults, startup migrate |
**DTOs** (`src/DTOs/`) are documented **inside the module that owns the HTTP contract**, with cross-links (no separate monolithic “DTOs” module).
See `modules/README.md` for the same index and file naming.
---
## Module dependency graph (revised)
```mermaid
flowchart BT
WE[wire-enums]
DB[database-layer]
CI[common-infrastructure]
AUTH[auth-identity]
MEDIA[media-service]
ANN[annotations-service]
DS[dataset-service]
SET[settings-metadata-service]
SSE[sse-realtime]
RMQ[rabbitmq-stream-sync]
PRG[composition-program]
WE --> DB
DB --> CI
DB --> MEDIA
DB --> ANN
DB --> DS
DB --> SET
CI --> MEDIA
CI --> ANN
AUTH --> MEDIA
AUTH --> ANN
AUTH --> DS
AUTH --> SET
MEDIA --> ANN
ANN --> SSE
ANN --> RMQ
SET --> CI
SSE --> ANN
RMQ --> ANN
RMQ --> MEDIA
RMQ --> DB
MEDIA --> PRG
ANN --> PRG
DS --> PRG
SET --> PRG
SSE --> PRG
RMQ --> PRG
AUTH --> PRG
CI --> PRG
```
Edges are “depends on for types, DB, paths, or events” (approximate). `ClassesController` reads DB directly — captured under **settings-metadata-service** for doc cohesion (small surface).
---
## Topological order (document skill Step 1)
1. `wire-enums`
2. `database-layer`
3. `common-infrastructure`
4. `auth-identity`
5. `media-service`
6. `annotations-service`
7. `dataset-service`
8. `settings-metadata-service`
9. `sse-realtime`
10. `rabbitmq-stream-sync`
11. `composition-program`
---
## Notes for downstream steps
- **Component assembly (Step 2):** expect components such as “Annotations + sync”, “Media”, “Dataset”, “Settings”, “Platform (auth+db+infra)” — refine with user confirmation (BLOCKING gate).
- **RabbitMQ:** `RabbitMqConfig` class lives in `FailsafeProducer.cs`; document in `rabbitmq-stream-sync` module.
---
## Suite spec cross-check (`suite/_docs/01_annotations.md`)
Canonical product/API narrative for this service. Use it when writing module and component docs.
| Topic in suite doc | Adds context for this repo |
|--------------------|------------------------------|
| Annotation identity (hash id, image + YOLO label, `Time` / `CreatedDate`) | `annotations-service` + `database-layer` + `common-infrastructure` (`PathResolver`) |
| Wire enums as integers | `wire-enums` module |
| REST §16 vs §710 | `annotations-service` vs `media-service` |
| Settings §1112, directories | `settings-metadata-service` |
| SSE | `sse-realtime` |
| Failsafe + RabbitMQ Stream | `rabbitmq-stream-sync` |
| Dataset note (DATASET permission) | `dataset-service` |
**Drifts spotted (suite vs current code)** — reconcile in suite or in code as you prefer:
1. **POST /annotations user id:** Suite lists `UserId` on request body; code uses JWT `NameIdentifier` (`annotations-service`).
2. **GET /annotations filter:** Suite lists `missionId`; code has `FlightId` and a partial filter — see `annotations-service` module.
Module docs (`modules/*.md`) carry contract detail per slice; this section stays the cross-file index.
-151
View File
@@ -1,151 +0,0 @@
# Step 4 — Verification Log
Verification pass over `_docs/02_document/` against `src/` source.
## Scope
Documents verified:
- `architecture.md`
- `system-flows.md`
- `data_model.md`
- `deployment/{containerization,ci_cd_pipeline,environment_strategy,observability}.md`
- `diagrams/flows/{flow_annotation_create,flow_sse_subscription,flow_failsafe_drain}.md`
- (sanity re-check only) `module-layout.md`, `components/*/description.md`, `modules/*.md`
## Method
For each generated artifact:
1. Extracted code-entity references (controllers, services, methods, DTOs, env vars, table/column names, route paths).
2. Cross-referenced each against the actual source (`src/Program.cs`, `src/Controllers/*`, `src/Services/*`, `src/Database/*`, `src/Enums/*`, `src/DTOs/*`, `.woodpecker/build-arm.yml`, `src/Dockerfile`).
3. Re-traced each system flow's mermaid sequence against the corresponding service/controller code.
4. Listed corrections, applied them inline to the affected files, and recorded them below.
## Counts
| Item | Verified | Corrected | Open question |
|------|----------|-----------|----------------|
| Controllers + their routes | 6 | 0 | 0 |
| Services + their public methods | 8 | 0 | 0 |
| DB tables / columns | 9 / ~60 | 0 | 5 (lazy upsert / `media.duration` / class catalog mutability / id collision / outbox JSON shape) |
| Enums | 7 | 0 | 0 |
| Env vars | 8 | 0 | 0 |
| Flows | 8 | 4 (F1, F7, F8, dependencies table) | 6 (consolidated below) |
| ADRs | 7 | 1 (ADR-004 hash details) | 0 |
Module-level coverage: **11 / 11 modules** documented; **6 / 6 components** assembled.
## Corrections applied inline
### `architecture.md`
1. **Internal communication table**: tightened to reflect that SSE publish + outbox enqueue happen **only on `CreateAnnotation`**; outbox enqueue is gated by `system_settings.silent_detection`. Added explicit row noting `DatasetService` writes are silent on SSE/outbox today.
2. **ADR-004 (annotation id hash)**: replaced "hash of bytes" with the actual `ComputeHash` strategy — `XxHash64` over a deterministic sample (length prefix + head/middle/tail 1 KB for inputs > 3072 bytes; full bytes otherwise). Documented collision implication.
3. **Open Architectural Risks**: rewrote with verified findings — silent Update/Delete/dataset paths, `silent_detection` semantics, F1 non-atomicity, static `EnqueueAsync` vs project rule.
4. **Section 4 "Data flow summary"**: split into Create-only / Update-and-friends / read paths, removed the inaccurate claim that thumbnails are produced inline by Create.
### `system-flows.md`
1. **F1 sequence + data flow + error scenarios**: replaced with the verified ordering — image file → optional media row → annotation → detections (`BulkCopyAsync`) → label file → SSE publish → conditional outbox enqueue. Removed thumbnail write from the Create path.
2. **F7 ("Reset call missed" risk)**: removed — verified that `SettingsService` calls `pathResolver.Reset()` at lines 71 and 85 of `Services/SettingsService.cs`. Replaced with a "Verified" note.
3. **F8 (Dataset bulk status)**: rewrote — `DatasetService.UpdateStatus` and `BulkUpdateStatus` issue direct `UPDATE annotations SET status` statements only. **They do NOT publish SSE and do NOT enqueue the outbox.** Updated routes (`PATCH /dataset/{id}/status`, `POST /dataset/bulk-status`) and error scenarios accordingly.
4. **Flow Dependencies table**: corrected F1 row (gating + Create-only), F3 row (only F1 Create publishes), F8 row (no SSE / no outbox today).
### `diagrams/flows/flow_annotation_create.md`
- Replaced sequence + flowchart to match the verified F1 ordering (image first, optional media, label, SSE, conditional outbox); thumbnail removed.
- Added note that Update/UpdateStatus/Delete are silent today.
### `diagrams/flows/flow_sse_subscription.md`, `flow_failsafe_drain.md`
- No structural corrections needed; spot-checked sequence vs `AnnotationsController.Events`, `AnnotationEventService`, `FailsafeProducer.EnqueueAsync`. Notes already capture multi-drainer dedupe and channel-unbounded back-pressure concerns.
### `data_model.md`
- No structural corrections; verified every column name and default against `Database/DatabaseMigrator.cs` and `Database/Entities/*.cs`. Spot-quirk (`detection_classes` ids 9 + 10 share `#000080`) is pre-existing and noted.
### `deployment/*`
- No structural corrections; verified `.woodpecker/build-arm.yml` step-by-step, `Dockerfile` two-stage build, `Program.cs` env-var fallbacks.
## Confirmed entities (sample — full list traced during the pass)
Controllers and routes (file:line where attributes were inspected):
- `AnnotationsController``Controllers/AnnotationsController.cs:1080``[Route("annotations")]`, `[Authorize(Policy = "ANN")]`, all listed routes match.
- `MediaController``Controllers/MediaController.cs:1055``[Route("media")]`, `[Authorize(Policy = "ANN")]`, routes: `POST`, `POST /batch`, `GET`, `GET /{id}/file`, `DELETE /{id}`.
- `DatasetController``Controllers/DatasetController.cs:941``[Route("dataset")]`, `[Authorize(Policy = "DATASET")]`, routes: `GET`, `GET /{annotationId}`, `PATCH /{annotationId}/status`, `POST /bulk-status`, `GET /class-distribution`.
- `SettingsController``Controllers/SettingsController.cs:1066``[Route("settings")]`, segments `system`, `directories`, `camera`, `user` each with GET + PUT.
- `ClassesController``Controllers/ClassesController.cs:913``[Route("classes")]`, `[Authorize]`, single `[HttpGet]`.
- `AuthController`**removed** in the auth refactor; annotations no longer mints or refreshes tokens. `JwtExtensions.AddJwtAuth` (verifier-only, ES256 over admin's JWKS) is the sole auth wiring in `Program.cs`.
Services:
- `AnnotationService.CreateAnnotation` (`Services/AnnotationService.cs:13104`) — verified sequence used to rewrite F1.
- `AnnotationService.UpdateAnnotation` / `UpdateStatus` / `DeleteAnnotation` — verified that none publish SSE or enqueue outbox.
- `DatasetService.UpdateStatus` / `BulkUpdateStatus` (`Services/DatasetService.cs:7594`) — verified silent on SSE / outbox.
- `SettingsService` — verified `pathResolver.Reset()` calls at lines 71, 85.
- `FailsafeProducer.EnqueueAsync` — confirmed as the public outbox-write helper, called by `AnnotationService.CreateAnnotation` only.
Tables / migrator (`Database/DatabaseMigrator.cs`):
- All 9 tables referenced in `data_model.md` exist with the columns and defaults as documented; idempotent `CREATE TABLE IF NOT EXISTS` + `ALTER TABLE … IF NOT EXISTS`; `detection_classes` seed of 19 rows with `ON CONFLICT DO NOTHING`.
Env vars (`Program.cs`):
- Required (fail-fast via `ConfigurationResolver.ResolveRequiredOrThrow`): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`.
- Optional with defaults: `RABBITMQ_HOST`, `RABBITMQ_STREAM_PORT`, `RABBITMQ_PRODUCER_USER`, `RABBITMQ_PRODUCER_PASS`, `RABBITMQ_STREAM_NAME`.
- CORS: `CorsConfig:AllowedOrigins` (string array) + `CorsConfig:AllowAnyOrigin` (bool); `CorsConfigurationValidator.EnsureSafeForEnvironment` blocks startup in `Production` when origins are empty and `AllowAnyOrigin` is not explicitly set.
CI (`.woodpecker/build-arm.yml`):
- `event: [push, manual]`, `branch: [dev, stage, main]`, `platform: arm64`, secret refs, `${BRANCH}-arm` tag, OCI image labels — all verified.
## Stakeholder resolutions (closed 2026-05-14)
The six open questions surfaced by this pass were resolved with the maintainer. Authoritative wording lives in `architecture.md` (ADR-004, ADR-008..ADR-011 + Refactor Backlog RB-01..RB-06). Quick map:
| Question | Resolution | Tracked |
|----------|------------|---------|
| Are silent Update/Delete/dataset-status changes intentional? | No — World B is the design; the drainer (`FailsafeProducer.cs:108123`) was already plumbed for `Validated` + `Deleted` ops, the producer side was never wired in the new HTTP backend (legacy WPF UI did this directly). Wire all mutations to publish + enqueue. | ADR-009 / RB-01 |
| `silent_detection` semantics? | Remove the flag entirely — superseded by the suite e2e harness. | ADR-010 / RB-02 |
| F1 atomicity (FS / DB / outbox)? | Adopt a business-transaction wrapper (transactional outbox); FS writes go post-commit. | ADR-008 / RB-03 |
| `XxHash64` over sample collision risk? | Switch to `XxHash3.Hash128` over the same sample (file-size-independent — videos can be 35 GB). | ADR-004 / RB-04 |
| `FailsafeProducer.EnqueueAsync` static + DB I/O? | Accept as-is; documented `coderule.mdc` deviation. | (no refactor) |
| `detection_classes` static or admin-managed? | Admin-managed with read-through cache (`PathResolver`-style `Reset()`). | ADR-011 / RB-06 |
### Additional finding while verifying #1
- `FailsafeProducer.cs:138` has an empty `catch { }` that swallows `IOException` on image read and emits a stream message with `image = null`. Direct `coderule.mdc` violation ("never suppress errors silently"). Operationally invisible failure mode. Tracked as RB-05 (architecture doc).
## Step 4.5 follow-on resolutions (closed 2026-05-14)
Confirmed alongside the Step 4.5 condensed-view approval:
| Question | Resolution | Tracked |
|----------|------------|---------|
| Suite vs code: `Flight` (code) vs `mission` (suite spec) | Rename code → `Mission*`; suite stays canonical | ADR-012 / RB-07 |
| Stream consumer dedupe contract owner | This service owns it; dedupe by `(annotationId, operation, dateTime)` baked into the wire message | ADR-013 / RB-09 |
| Hard-delete vs soft-delete | Soft-delete: status → `Deleted (40)`, files relocated to a new `deleted_dir` | ADR-009 (folded in) / RB-01 |
| Tight coupling 04 Dataset ↔ 01 Annotations REST | Decouple — dataset writes flow through `AnnotationService` via a public domain interface | RB-08 |
## Remaining gaps and uncertainties (carried into Step 6 problem extraction)
1. **`media.duration` format**: TEXT NOT NULL is permissive; format is unspecified.
2. **Lazy-upsert semantics** for `system_settings` / `directory_settings` / `camera_settings` — confirm services initialize defaults vs rely on user-driven inserts.
3. **`UserId` body field vs JWT subject** drift — reconcile in suite spec or in code.
4. **No automated tests in repo**: addressed by autodev Phase A Steps 37.
## Completeness score
- 11 / 11 modules documented (`modules/*.md`).
- 6 / 6 components assembled (`components/*/description.md`).
- 1 / 1 module-layout file (`module-layout.md`).
- 1 / 1 architecture file (`architecture.md`).
- 1 / 1 system-flows file (`system-flows.md`) covering 8 flows.
- 1 / 1 data-model file (`data_model.md`) covering 9 tables.
- 4 / 4 deployment files (`deployment/*.md`).
- 3 flow diagrams (F1, F3, F4) in `diagrams/flows/`.
**Score: 100% of modules + components covered.** Remaining open items are behavioral questions, not coverage gaps.
-183
View File
@@ -1,183 +0,0 @@
# Azaion.Annotations — Documentation Report
## Executive Summary
Reverse-engineered the `Azaion.Annotations` codebase bottom-up — 11 module docs → 6 component specs → 1 system architecture + 8 verified flows + ER diagram + deployment + glossary, then synthesized a retrospective `solution.md` and a 5-file problem extraction. Verification surfaced 8 behavioral discrepancies between code and the suite-level `01_annotations.md` narrative; all 8 were resolved with stakeholder decisions, captured as 13 ADRs and 9 Refactor Backlog items (RB-01..RB-09) inside `architecture.md`.
## Problem Statement
`Azaion.Annotations` is the suite's annotation lifecycle service. It is the single owner of the `annotations` table, the YOLO label files on disk, and the lifecycle event stream. Three independent consumers (Annotator UI, AI training pipeline, admin sync worker) need the same data shaped differently — push for humans (SSE), durable-pull for machines (RabbitMQ Stream) — and this service is the one place that reconciles those needs. See `_docs/00_problem/problem.md`.
## Architecture Overview
Single .NET 10 ASP.NET Core service, single PostgreSQL state-of-record, content-addressed filesystem cache, in-process SSE channel + transactional outbox drained to RabbitMQ Stream by a hosted background service. JWT bearer with three policies (`ANN`, `DATASET`, `ADM`). Idempotent boot-time DDL migrator removes a separate migration deploy step.
13 ADRs captured the choices; 9 Refactor Backlog items capture the agreed-upon next moves. Full detail: `_docs/02_document/architecture.md`.
**Technology stack**: .NET 10 + ASP.NET Core + Linq2DB + Npgsql + JwtBearer + RabbitMQ.Stream.Client + MessagePack + xxHash3 (per RB-04) on PostgreSQL 13+.
**Deployment**: ARM64 multi-arch Docker image; branch-driven Woodpecker CI emits `${BRANCH}-arm` tags; orchestrator-managed at the suite level.
## Component Summary
| # | Component | Purpose | Dependencies | Epic |
|---|-----------|---------|--------------|------|
| 01 | annotations-rest | Annotation CRUD + image/thumbnail file routes; YOLO label write; lifecycle producer | 02, 06 | TBD (Phase B) |
| 02 | annotations-realtime-sync | In-process SSE channel + transactional outbox + `FailsafeProducer` to RabbitMQ Stream | 06 | TBD (Phase B) |
| 03 | media | Multipart media upload (single + batch), file download, soft delete | 06 | TBD (Phase B) |
| 04 | dataset | Dataset exploration: filters, class distribution, bulk status writes | 06 (today couples 01 — RB-08) | TBD (Phase B) |
| 05 | settings-metadata | System / directory / camera / user settings + detection class catalog | 06 | TBD (Phase B) |
| 06 | platform | Composition root, JWT, error envelope, path resolver, DB migrator | — | TBD (Phase B) |
**Implementation order** (logical layer dependency, not "to-build" — the codebase already exists):
1. `06 platform` is the foundation; every other component imports from it.
2. `02 annotations-realtime-sync` is the lifecycle substrate `01` and (post RB-01) `04` feed into.
3. `01 annotations-rest`, `03 media`, `05 settings-metadata` sit on top of `06` directly.
4. `04 dataset` reads the storage `01` writes; today via direct DB coupling, post RB-08 via `AnnotationService`.
**Refactor sequencing** is what Phase B will plan (Steps 8 onward); the Refactor Backlog already orders the items by impact:
1. RB-01 (lifecycle observability across mutations) — unblocks RB-09 stream contract and most Step 14 audit work.
2. RB-03 (transactional outbox wrapper) — required before RB-01 is testable.
3. RB-04 (xxHash3.Hash128) — small, isolated, can run parallel.
4. RB-02 (drop `silent_detection`) — small cleanup, after RB-01.
5. RB-08 (decouple `04 dataset` writes) — unblocks soft-delete read filtering.
6. RB-07 (`Flight*``Mission*` rename) — high-touch, needs coordination with suite consumers.
7. RB-06 (admin-managed detection classes) — feature, can run parallel.
8. RB-05 (replace `catch { }` in `FailsafeProducer.cs:138`) — trivial, anytime.
9. RB-09 (stream dedupe contract `(annotation_id, operation, date_time)`) — depends on RB-01.
## System Flows
| Flow | Description | Key Components |
|------|-------------|---------------|
| F1 | Annotation create — content-address image, persist, write label, fan-out (SSE + outbox) | 01, 02, 06 |
| F2 | Annotation listing / detail | 01, 06 |
| F3 | Real-time SSE subscription per mission | 01, 02, 06 |
| F4 | Failsafe outbox drain (background loop) → RabbitMQ Stream | 02, 06 |
| F5 | Media upload (single + batch) | 03, 06 |
| F6 | Auth: login + refresh token rotation | 06 |
| F7 | Directory settings change → `pathResolver.Reset()` invariant | 05, 06 |
| F8 | Dataset bulk status update | 04, 06 |
Full sequence diagrams: `_docs/02_document/system-flows.md` and `_docs/02_document/diagrams/flows/`.
## Risk Summary
Risks here are the operational / behavioral risks captured during verification + Step 14 candidates from `security_approach.md`. They live in `architecture.md` (Risks + Refactor Backlog) and will be mirrored into a formal risk register during Phase B Step 12.
| Level | Count | Key Risks |
|-------|-------|-----------|
| Critical | 0 | — |
| High | 3 | (1) silent mutation paths break downstream consumers (RB-01); (2) outbox not transactional with FS+DB write (RB-03); (3) outbox has no row-leasing → multi-instance double-publish (OP-02 / blocked by RB-09 contract). (Former SEC-01 — JWT issuer/audience not validated — closed by the auth refactor.) |
| Medium | 3 | xxHash64 collision tolerance (RB-04); `silent_detection` ambiguity (RB-02); Swagger in prod (SEC-04); `04 dataset` direct-DB coupling (RB-08). (Former SEC-03 — CORS wide-open — closed by `CorsConfigurationValidator`.) |
| Low | 6 | `Flight` vs `Mission` naming drift (RB-07); empty `catch{}` (RB-05); `detection_classes` not admin-CRUD (RB-06); upload MIME whitelist (SEC-05); rate limiting (SEC-06); audit log substrate (SEC-08). |
**Iterations completed**: 1 verification pass with stakeholder review.
**All Critical/High risks mitigated**: No — High items are tracked as RB-01, RB-03, and OP-02 (multi-instance constraint, time-boxed by current single-instance deployment). SEC-01 / SEC-02 / SEC-03 (the original auth + CORS gaps) were closed by the auth + CORS refactor between Steps 1 and 4. Remaining mitigations are scheduled, not executed.
## Test Coverage
The repo currently has **zero automated tests** (`_docs/02_document/00_discovery.md`) and CI runs only build-and-push (`.woodpecker/build-arm.yml`). Test coverage is planned, not measured.
| Component | Integration | Performance | Security | Acceptance | AC Coverage |
|-----------|-------------|-------------|----------|------------|-------------|
| 01 annotations-rest | 0 / TBD (Step 3) | 0 / TBD (Step 15) | 0 / TBD (Step 14) | 0 / 8 ACs | 0 / 8 |
| 02 realtime-sync | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 4 ACs | 0 / 4 |
| 03 media | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 2 ACs | 0 / 2 |
| 04 dataset | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 2 ACs | 0 / 2 |
| 05 settings-metadata | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 3 ACs | 0 / 3 |
| 06 platform | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 5 ACs | 0 / 5 |
**Overall acceptance criteria coverage**: 0 / 24 functional ACs + 0 / 5 non-functional ACs (0%).
The autodev existing-code Phase A Steps 3 (Test Spec) and 6 (Implement Tests) own filling this matrix.
## Epic Roadmap
The current invocation completed **Phase A Step 1 (Document)** of the autodev existing-code flow. No tracker epics have been opened yet. The work that follows in Phase A produces the test surface and the security/perf baselines:
| Order | Phase A Step | Output | Effort | Dependencies |
|-------|--------------|--------|--------|-------------|
| 1 | Step 2 — Documentation Quality Audit | gap log | S | this report |
| 2 | Step 3 — Test Spec | per-component `tests.md` (functional + integration shape) | M | step 2 |
| 3 | Step 4 — Risk Mitigations | `risk_mitigations.md` | S | step 2 |
| 4 | Step 5 — Solution Extraction (already done — see `_docs/01_solution/solution.md`) | — | — | — |
| 5 | Step 6 — Implement Tests | actual test project + green CI step | L | step 3 |
| 6 | Step 7 — Test Audit | coverage report against AC matrix | S | step 6 |
Phase B (Feature Cycle) then runs per feature/refactor. The 9 Refactor Backlog items become the first batch of Phase B epics; sizing per the user's Jira complexity rules will be 25 points each, except RB-07 (rename across DTOs/controllers/consumers — likely 5).
**Total estimated effort**: not committed. Phase A Steps 27 are scoped against this report; Phase B sizes per epic.
## Key Decisions Made
These are the 13 ADRs from `architecture.md`. Eight of them came from the verification stakeholder review.
| # | Decision | Rationale | Alternatives rejected |
|---|----------|-----------|----------------------|
| ADR-001 | In-process SSE channel for UI fan-out, separate transactional outbox for durable consumers | Sub-ms UI latency without standing up a broker for the inner loop | Single broker for both (UI latency hit); Postgres LISTEN/NOTIFY for UI (delivery semantics insufficient) |
| ADR-002 (RETIRED) | Originally: symmetric HS256 JWT, no issuer/audience validation. Now: ES256 verifier-only over admin's JWKS, with `iss` / `aud` / `exp` / `alg` all enforced. | Identity is centralised in admin; annotations holds no signing material | The original symmetric scheme it replaced |
| ADR-003 | Linq2DB + idempotent SQL DDL migrator (no EF, no DbUp/FluentMigrator) | Lighter dependency surface; one less deploy step (ADR-007) | EF Core migrations (heavier); FluentMigrator (separate runner) |
| ADR-004 | Annotation id = `XxHash3.Hash128` over a sampled image-bytes window | 128-bit space tolerates the suite's annotation volume; sampled keeps large-frame ingest cheap | Full SHA-256 (CPU); xxHash64 (collision space too small — RB-04 was the upgrade) |
| ADR-005 | Swagger UI mounted unconditionally | Internal-only deployment; aids debugging | Gating by env (deferred to SEC-04) |
| ADR-006 (RETIRED) | Originally: CORS `AllowAny*`. Now: config-driven allow-list (`CorsConfig:AllowedOrigins` + opt-in `AllowAnyOrigin`) gated by `CorsConfigurationValidator` per environment. | Production cannot start without an explicit origin policy | The original wide-open default |
| ADR-007 | DDL applied at boot, not in CI | Single deploy step; matches container-immutable model | Separate migration job (deploy complexity) |
| ADR-008 | Business-transaction wrapper (transactional outbox) for annotation lifecycle | Atomicity across DB + outbox; FS write tolerated as best-effort with cleanup | DTC across FS + DB + RabbitMQ (heavyweight, not portable) |
| ADR-009 | Every mutation path emits SSE + enqueues outbox row | One observability contract for humans + machines | SSE-only (durability gap for AI/admin worker); outbox-only (UI latency) |
| ADR-010 | Remove `silent_detection` flag | Behavior is contradictory once ADR-009 holds | Keep flag and gate on it (forces every consumer to interpret it) |
| ADR-011 | Detection class catalog becomes admin-managed (CRUD + cache) | Catalog evolves with deployments; migrator-only is a deploy-time-only escape hatch | Static catalog (RB-06 supersedes) |
| ADR-012 | Canonical term is `Mission`; `Flight*` symbols renamed | Single suite-level vocabulary | Keep `Flight` in this service (drift cost grows over time) |
| ADR-013 | On-the-wire dedupe key: `(annotation_id, operation, date_time)` | Lets every downstream consumer dedupe re-deliveries safely | Per-consumer offset trust (fragile under outbox replay) |
## Open Questions
All 6 verification-pass questions were resolved during stakeholder review. Genuinely-open follow-ups now:
| # | Question | Impact | Assigned To |
|---|----------|--------|-------------|
| 1 | Are P50/P95/P99 latency / throughput targets contracted anywhere in the suite? | Bounds NFR ACs (`AC-N-*`) and Step 15 perf-test shape. | Suite ops / product |
| 2 | What is the upload format whitelist `/media` should enforce? | Bounds SEC-05 fix scope. | Detections-pipeline owner |
| 3 | RPO/RTO contract for `images_dir` and `deleted_dir`? | Bounds the soft-delete restore story (post RB-01). | Suite ops |
| 4 | Stream retention window for `azaion-annotations`? | Bounds the consumer replay window the AI pipeline depends on. | Suite ops |
| 5 | Is multi-tenancy on the roadmap within the doc horizon? | Decides whether SEC-07 is a Step 14 must-fix or a deferred gap. | Product |
## Artifact Index
| File | Description |
|------|-------------|
| `_docs/02_document/architecture.md` | Architecture vision, 13 ADRs, refactor backlog, NFRs |
| `_docs/02_document/system-flows.md` | F1F8 verified flow narratives |
| `_docs/02_document/data_model.md` | ERD + per-table contract reproduced from `DatabaseMigrator.cs` |
| `_docs/02_document/glossary.md` | 36 canonical terms (suite + project + code-level) |
| `_docs/02_document/module-layout.md` | Module → component mapping |
| `_docs/02_document/04_verification_log.md` | Verification pass corrections + stakeholder resolutions |
| `_docs/02_document/components/01_annotations-rest/description.md` | Component 01 spec |
| `_docs/02_document/components/02_annotations-realtime-sync/description.md` | Component 02 spec |
| `_docs/02_document/components/03_media/description.md` | Component 03 spec |
| `_docs/02_document/components/04_dataset/description.md` | Component 04 spec |
| `_docs/02_document/components/05_settings-metadata/description.md` | Component 05 spec |
| `_docs/02_document/components/06_platform/description.md` | Component 06 spec |
| `_docs/02_document/modules/*.md` | 11 module-level deep-dives |
| `_docs/02_document/diagrams/components.md` | Component diagram (Mermaid) |
| `_docs/02_document/diagrams/flows/flow_annotation_create.md` | F1 sequence (verified) |
| `_docs/02_document/diagrams/flows/flow_sse_subscription.md` | F3 sequence |
| `_docs/02_document/diagrams/flows/flow_failsafe_drain.md` | F4 sequence |
| `_docs/02_document/deployment/containerization.md` | Dockerfile-derived deployment notes |
| `_docs/02_document/deployment/ci_cd_pipeline.md` | Woodpecker pipeline-derived notes |
| `_docs/02_document/deployment/environment_strategy.md` | Env-var contract + ASPNETCORE_ENVIRONMENT use |
| `_docs/02_document/deployment/observability.md` | Logging + `/health` + outbox depth gap |
| `_docs/02_document/common-helpers/01_http-error-envelope.md` | Suite error envelope contract |
| `_docs/01_solution/solution.md` | Retrospective per-component solution table |
| `_docs/00_problem/problem.md` | Retrospective problem statement |
| `_docs/00_problem/restrictions.md` | HW / SW / ENV / OP / suite-level restrictions |
| `_docs/00_problem/acceptance_criteria.md` | 24 functional + 5 non-functional ACs |
| `_docs/00_problem/input_data/data_parameters.md` | REST DTOs + env vars + seed data + wire format |
| `_docs/00_problem/security_approach.md` | Auth/AuthZ/secrets posture + 8 SEC-XX gaps |
## Cross-references
- Suite-level integration narrative: `suite/_docs/01_annotations.md`
- Repo-config (monorepo discovery): `_docs/_repo-config.yaml`
- Autodev state: `_docs/_autodev_state.md`
- Document skill internal state: `_docs/02_document/state.json`
-403
View File
@@ -1,403 +0,0 @@
# Azaion.Annotations — Architecture
> **Source of truth for service-internal architecture.** Suite-level integration narrative lives in `../../../suite/_docs/01_annotations.md`. This file documents what the code in `src/` actually implements, derived bottom-up from module and component docs.
## Architecture Vision
**Status**: confirmed-by-user 2026-05-14.
Azaion.Annotations is a single .NET 10 ASP.NET Core service in the Azaion suite that owns the authoritative HTTP + streaming surface for annotation lifecycle, media upload, dataset exploration, and system metadata. State of record is PostgreSQL (Linq2DB + Npgsql) with an idempotent boot-time migrator. Real-time fan-out is in-process SSE; durable cross-service export is a transactional-outbox + RabbitMQ Stream pipeline producing MessagePack frames consumed by the admin sync worker and the AI training pipeline. The runtime is one container per node, ARM64-first via Woodpecker CI, with branch-driven image tags (`dev` | `stage` | `main`).
### Components & responsibilities
- **06 Platform** — shared kernel: DB, enums, JWT, error middleware, paths, composition root.
- **02 Annotations realtime & sync** — SSE channel + RabbitMQ Stream failsafe drainer.
- **01 Annotations REST** — annotation CRUD + image/thumbnail file routes; the lifecycle producer.
- **03 Media** — upload (single + batch), list, download, delete.
- **04 Dataset** — read-heavy `/dataset` surface + `DATASET`-policy status writes (planned to route through `01 Annotations REST` per RB-08).
- **05 Settings & metadata** — system / directory / camera / user settings + `/classes` catalog (becoming admin-managed per RB-06).
### Major data flows
- **F1 — Annotation create**: bytes → image file → DB rows → label file → SSE → outbox; will be wrapped in a business transaction (ADR-008).
- **F3 — SSE subscription**: UI long-poll on `/annotations/events`.
- **F4 — Outbox drain**: `FailsafeProducer` pumps queue rows to the RabbitMQ stream `azaion-annotations`.
- **F2 / F5 / F6 / F7 / F8** — read paths, media uploads, auth refresh, directory cache reset, dataset bulk status.
### Principles / non-negotiables
- **Wire enums are integer-stable** (suite contract). [inferred-from: `modules/wire-enums.md`, `suite/_docs/01_annotations.md`]
- **Annotation id is content-addressed** via a sampled image-bytes hash; remains file-size-independent (videos to ~5 GB). [inferred-from: `AnnotationService.ComputeHash`, ADR-004]
- **PostgreSQL is the state of record**; the filesystem is a content-addressed cache. [inferred-from: `data_model.md`, `system-flows.md` F1]
- **The transactional outbox is the durability boundary**; SSE is best-effort. [inferred-from: ADR-003 / ADR-008]
- **Lifecycle observability is World B**: every mutation publishes SSE and enqueues the outbox. [inferred-from: `FailsafeProducer` drainer plumbing for `Validated`/`Deleted`; maintainer resolution 2026-05-14 → ADR-009 / RB-01]
- **Soft-delete with file relocation**: `DeleteAnnotation` flips status to `AnnotationStatus.Deleted = 40` and moves files to a deleted-files directory rather than removing rows. [inferred-from: maintainer resolution 2026-05-14 → ADR-009 / RB-01]
- **Stream consumer dedupe contract is owned by this service**: outbox messages must carry enough metadata for downstream consumers to dedupe on `(annotationId, operation, dateTime)`. [inferred-from: maintainer resolution 2026-05-14 → ADR-013 / RB-09]
- **Mission is the canonical domain term**: code currently uses `FlightId`; the suite spec uses `missionId`. Code aligns to suite (rename, not the other way). [inferred-from: `00_discovery.md` drift list; maintainer resolution 2026-05-14 → ADR-012 / RB-07]
- **Dataset writes flow through the annotation domain service**: `04 Dataset` does not edit `annotations` rows directly. [inferred-from: `module-layout.md` Verification Needed §1; maintainer resolution 2026-05-14 → RB-08]
- **DB-driven runtime config**: directory roots and detection classes change at runtime via `ADM` endpoints, not redeploy. [inferred-from: `PathResolver.Reset`, ADR-011]
### Open questions / drift signals (residual)
- `UserId` body field vs JWT `NameIdentifier` (suite spec lists `UserId` on `POST /annotations`; code uses JWT subject). Reconcile in suite or code.
- The exact dedupe key shape for downstream consumers — `(annotationId, operation, dateTime)` is the working assumption per RB-09; suite consumer doc must be updated to match.
---
## 1. System Context
**Problem being solved**: Provide the canonical HTTP + streaming API for **annotation lifecycle** (create / update / status / delete / list / files), **media** (upload, list, download), **dataset exploration** (`DATASET` policy reads + bulk status writes), and **system metadata** (settings + detection class catalog), with **real-time SSE** push to UI consumers and **failsafe** export to RabbitMQ Stream consumers (admin sync, AI training).
**System boundaries**:
- **Inside**: a single ASP.NET Core process (`Azaion.Annotations.dll`), its embedded migrator, in-memory SSE channel, in-process `BackgroundService` outbox drain, and the on-disk image / label / thumbnail / results layout under `directory_settings`.
- **Outside**: PostgreSQL (state of record), RabbitMQ Streams (durable annotation export), the on-disk media/data filesystem (mounted), and every authenticated HTTP / SSE consumer (UIs, detections service, admin sync worker, AI training).
**External systems**:
| System | Integration Type | Direction | Purpose |
|--------|------------------|-----------|---------|
| PostgreSQL | DB (Linq2DB / Npgsql) | Both | State of record (annotations, media, queue, settings, classes) |
| RabbitMQ Streams | Stream client (`RabbitMQ.Stream.Client`) | Outbound | Durable export of annotation lifecycle (`azaion-annotations` stream) |
| Filesystem (mounted) | File I/O | Both | Annotation images, YOLO label `.txt`, thumbnails, results, GPS routes/sat |
| Annotator UI / Dataset Explorer UI | REST + SSE | Inbound | User flows (suite `01_annotations.md`, `09_dataset_explorer.md`) |
| Detections service (suite `detections`) | REST | Inbound | POST annotations after model inference; long-running tokens are refreshed against admin (annotations no longer mints tokens) |
| Admin sync worker / AI training | RabbitMQ Streams | Outbound | Consume `azaion-annotations` stream offsets (suite `Annotation Sync`) |
## 2. Technology Stack
| Layer | Technology | Version | Rationale |
|-------|------------|---------|-----------|
| Language | C# | `net10.0` (`src/Azaion.Annotations.csproj`) | Single language across suite .NET services |
| Framework | ASP.NET Core (minimal hosting + controllers) | net10.0 | Built-in JWT, CORS, Swagger, hosted services |
| ORM / DB driver | Linq2DB + Npgsql | per `csproj` | Linq2DB used for `ITable<>` repositories; Npgsql under the hood |
| Database | PostgreSQL | not pinned in code (URL-driven) | Suite-wide datastore |
| Auth | JWT Bearer (`Microsoft.AspNetCore.Authentication.JwtBearer`) — verifier-only, ES256 over admin's JWKS | net10.0 | Issuer/audience/lifetime/signature all validated; admin is the sole issuer (see Section 7) |
| Messaging | RabbitMQ Streams (`RabbitMQ.Stream.Client`) + MessagePack | per `csproj` | Durable, replayable annotation export |
| API docs | Swashbuckle (Swagger / Swagger UI) | per `csproj` | Always mounted (see ADR-005) |
| Hashing | `System.IO.Hashing` | net10.0 stdlib | Annotation id derived from image bytes hash |
| Hosting | `WebApplication` + `IHostedService` | net10.0 | `FailsafeProducer` runs in-process |
| Container | `mcr.microsoft.com/dotnet/aspnet:10.0` | linux/arm64 + linux/amd64 | Multi-arch image, ARM-first per Woodpecker |
| CI | Woodpecker CI (`.woodpecker/build-arm.yml`) | n/a | Branch-based image tag (`${BRANCH}-arm`) |
**Key constraints (evidenced in code/config)**:
- `DATABASE_URL` is **required** at startup — `ConfigurationResolver.ResolveRequiredOrThrow` throws if not set. The string is auto-converted from `postgresql://user:pass@host:port/db` URI form to Linq2DB's `Host=…;Username=…` form by `Program.ConvertPostgresUrl`.
- JWT verification is **required** at startup — `JWT_ISSUER`, `JWT_AUDIENCE`, and `JWT_JWKS_URL` are all resolved by `ConfigurationResolver.ResolveRequiredOrThrow`. There is no insecure fallback. The JWKS URL is fetched with `HttpDocumentRetriever`, whose `RequireHttps` flag is gated on `ASPNETCORE_ENVIRONMENT`: HTTPS is required for any value other than `E2ETest` (Development, Staging, Production, and unset all enforce HTTPS); only `E2ETest` relaxes the flag to support the in-cluster mock issuer documented in `tests/environment.md`. The relaxation is gated in source (`src/Auth/JwtExtensions.cs`), not in config.
- Default directory roots are `/data/{videos,images,labels,results,thumbnails,gps_sat,gps_route}` (migrator `directory_settings` defaults) → operator must mount or override at the DB level via `PUT /settings/directories`.
- CORS is **environment-gated**: `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when `CorsConfig:AllowedOrigins` is empty unless `CorsConfig:AllowAnyOrigin=true` is set explicitly. ADR-006 was retired together with the wide-open default.
## 3. Deployment Model
**Environments** (evidenced from CI branches): `dev`, `stage`, `main` → image tag `${CI_COMMIT_BRANCH}-arm` pushed to a private registry resolved from `REGISTRY_HOST` secret.
**Infrastructure**:
- Single .NET service container; container exposes port `8080`.
- Multi-arch build supported in the Dockerfile (`--platform=$BUILDPLATFORM`, `$TARGETARCH`); the ARM Woodpecker pipeline currently only emits `arm64`.
- Scaling is **vertical-only** as written: SSE uses an in-process `Channel<AnnotationEventDto>`, and the `FailsafeProducer` outbox drainer is a per-instance `BackgroundService` — see "Open Architectural Risks".
**Environment-specific configuration** (defaults vs production):
| Config | Source | Development default | Production behavior |
|--------|--------|---------------------|---------------------|
| `DATABASE_URL` | env or `Database:Url` config key | none — fail-fast on missing (`ConfigurationResolver`) | MUST set |
| `JWT_ISSUER` | env or `Jwt:Issuer` config key | none — fail-fast | MUST set (matches admin's issuer) |
| `JWT_AUDIENCE` | env or `Jwt:Audience` config key | none — fail-fast | MUST set (matches admin's audience for this service) |
| `JWT_JWKS_URL` | env or `Jwt:JwksUrl` config key | none — fail-fast; HTTPS required | MUST set to admin's JWKS endpoint |
| `RABBITMQ_HOST` / `RABBITMQ_STREAM_PORT` | env | `127.0.0.1` / `5552` | Override per environment |
| `RABBITMQ_PRODUCER_USER` / `_PASS` | env | `azaion_producer` / `producer_pass` | Override |
| `RABBITMQ_STREAM_NAME` | env | `azaion-annotations` | Usually kept (suite contract) |
| `CorsConfig:AllowedOrigins` | `IConfiguration` (string array) | empty | MUST set (or set `AllowAnyOrigin=true` explicitly) — `CorsConfigurationValidator` refuses to start in Production otherwise |
| `CorsConfig:AllowAnyOrigin` | `IConfiguration` (bool) | false | Explicit opt-in for permissive policy |
| Directory roots (`/data/...`) | DB `directory_settings` | hard-coded SQL defaults | Tune via `PUT /settings/directories` (calls `PathResolver.Reset`) |
| Swagger UI | `Program.cs` | mounted | **Also mounted in prod** (ADR-005) |
| `AZAION_REVISION` | Dockerfile build arg `CI_COMMIT_SHA` | `unknown` | Stamped per-image |
## 4. Data Model Overview
> Detailed ERD, indexes, and migration semantics live in `data_model.md`. This section is the cross-component summary.
**Core entities** (owned by `06_platform`; consumed by feature components):
| Entity | Description | Owned by component |
|--------|-------------|---------------------|
| `media` | Uploaded image/video reference (waypoint-scoped) | `03_media` (writes) / `01_annotations-rest` (reads) |
| `annotations` | Annotation row keyed by image-bytes hash, soft-versioned by `created_date`, `time` (BIGINT ticks) | `01_annotations-rest` |
| `detection` | YOLO bounding boxes (`center_x/y, width, height`, class, affiliation, combat readiness) per annotation | `01_annotations-rest` |
| `annotations_queue_records` | Outbox for failsafe stream sync (`operation`, `annotation_ids` JSON array) | `02_annotations-realtime-sync` (writer) / `01_annotations-rest` (writer side) |
| `system_settings` | Singleton-ish org settings + `generate_annotated_image`, `silent_detection` toggles | `05_settings-metadata` |
| `directory_settings` | Filesystem roots consumed by `PathResolver` | `05_settings-metadata` |
| `detection_classes` | Seeded class catalog for UI label/color (ids 018, names + Cyrillic short names + hex colors) | `05_settings-metadata` (read-only `ClassesController`) |
| `user_settings` | Per-user UI prefs (panel widths, selected flight) | `05_settings-metadata` |
| `camera_settings` | Calibration (altitude, focal length, sensor width) | `05_settings-metadata` |
**Key relationships**:
- `annotations.media_id``media.id` (FK).
- `detection.annotation_id``annotations.id` (FK; cascades on annotation update logic in service layer, not DB).
- `annotations_queue_records.annotation_ids` is a **JSON array of TEXT ids** (no FK); single-row outbox entry can reference multiple annotations (bulk).
**Data flow summary**:
- **Inbound write (Create)***today*: HTTP body → `AnnotationService.CreateAnnotation` → image bytes to `images_dir/{id}.jpg`, optional `media` row insert, `annotations` + `detection` rows, YOLO label to `labels_dir/{id}.txt`, SSE publish, then (if `silent_detection != true`) outbox row → drained by `FailsafeProducer` → MessagePack frame on RabbitMQ stream. **Thumbnails are not produced by this flow** — they are read-only via `PhysicalFile` and presumed populated out-of-band.
- **Inbound write (Update / UpdateStatus / Delete annotations, dataset PATCH / bulk-status)***today*: DB-only, silent. *Target* (RB-01): every mutation publishes SSE and enqueues the outbox with the appropriate `QueueOperation` (`Created`, `Validated`, or `Deleted`).
- **Lifecycle ordering***target* (RB-03): all DB writes plus the outbox row commit inside a single business transaction; FS writes (image / label / future thumbnail generation) and SSE publish are post-commit, with the outbox row as the durable promise.
- **Inbound read**: HTTP query → DB joins (`annotations × detection × media`) → JSON list (`PaginatedResponse<AnnotationListItem>`); image/thumbnail served as `PhysicalFile`.
## 5. Integration Points
### Internal communication (in-process)
| From | To | Protocol | Pattern | Notes |
|------|----|----------|---------|-------|
| `01_annotations-rest` (`AnnotationService`) | `02_annotations-realtime-sync` (`AnnotationEventService`) | C# call | Fire-and-forget publish to `Channel<>` | **Today**: only on Create. **Target (RB-01)**: every mutation publishes (Create, Update, UpdateStatus, Delete) |
| `01_annotations-rest` (`AnnotationService`) | `02_annotations-realtime-sync` (`annotations_queue_records` table) | DB INSERT via `FailsafeProducer.EnqueueAsync` (static helper) | Outbox | **Today**: Create only, gated by `silent_detection`. **Target (RB-01 + RB-02)**: every mutation enqueues with the appropriate `QueueOperation`; gating flag removed |
| `02_annotations-realtime-sync` (`FailsafeProducer`) | `06_platform` (`AppDataConnection`, `PathResolver`) | C# call | Read-then-delete | Drainer is **already plumbed** for `Created`, `Validated`, and `Deleted` operations (see `FailsafeProducer.cs:108123`) |
| `04_dataset` (`DatasetService.UpdateStatus` / `BulkUpdateStatus`) | `01_annotations-rest` (`AnnotationEventService`) + outbox | shared DB + cross-component call | Direct write today; lifecycle publish + enqueue per RB-01 | Bulk path enqueues a single `Validated` outbox record carrying all ids |
| `05_settings-metadata` (directory PUT) | `06_platform` (`PathResolver.Reset`) | C# call | Cache invalidation | Required after directory change |
### External integrations
| External system | Protocol | Auth | Rate limits | Failure mode |
|-----------------|----------|------|-------------|--------------|
| PostgreSQL | TCP / Linq2DB / Npgsql | Conn string | n/a | Surfaced as 500 via `ErrorHandlingMiddleware` |
| RabbitMQ Stream `azaion-annotations` | Stream protocol (5552) | Stream user/pass (`azaion_producer` default) | Stream-level | `FailsafeProducer` retries; rows stay in `annotations_queue_records` until drained |
| Filesystem (`/data/...`) | POSIX | OS perms | n/a | `IOException` → 500; missing image on GET → 404 |
| HTTP clients (UIs, detections, admin) | REST + SSE | JWT Bearer (`ANN`, `DATASET`, `ADM`) | n/a | `401` if invalid; `403` if missing claim |
## 6. Non-Functional Requirements
> Pulled only from code-level evidence — config defaults, validators, health checks, idempotent migrator. Anything not evidenced is left blank rather than guessed.
| Requirement | Target | Measurement | Priority | Source |
|-------------|--------|-------------|----------|--------|
| Liveness | 200 OK on `GET /health` | route in `Program.cs` | High | `Program.cs` |
| Idempotent startup | DB schema applies cleanly on every boot | `DatabaseMigrator.Migrate` uses `CREATE TABLE IF NOT EXISTS` + `ALTER TABLE … IF NOT EXISTS` and `INSERT … ON CONFLICT DO NOTHING` | High | `Database/DatabaseMigrator.cs` |
| Recovery: queue durability | Annotation lifecycle events are not lost across pod restarts | DB-backed outbox (`annotations_queue_records`) drained by `FailsafeProducer` | High | `Services/FailsafeProducer.cs` |
| Auth lifetime / clock skew | per `JwtExtensions.AddJwtAuth` config | `auth-identity` module | Medium | `Auth/JwtExtensions.cs` |
| Pagination defaults | `PaginatedResponse<T>` total/page/pageSize | applied in list endpoints | Medium | `DTOs/PaginatedResponse.cs` |
| Thumbnail dimensions | `240×135` with `10` border (defaults) | `system_settings.thumbnail_*` | Low | migrator defaults |
| Throughput / latency / availability targets | **not evidenced in code** | — | — | open question, see `00_problem` extraction (Step 6) |
## 7. Security Architecture
**Authentication**: JWT Bearer; **ES256 signature** verified against admin's JWKS endpoint (`JWT_JWKS_URL`, default `https://admin.azaion.com/.well-known/jwks.json`). `ValidateIssuer`, `ValidateAudience`, `RequireSignedTokens`, and `RequireExpirationTime` are all enforced; algorithms are pinned to `EcdsaSha256` to block HS256-confusion forgeries. Admin is the sole token issuer for the suite — annotations no longer holds an HMAC secret and no longer mints tokens (`TokenService` and `POST /auth/refresh` were removed; callers refresh against admin).
**Authorization** (per-endpoint policy claims, all evidenced in controllers):
- `ANN``AnnotationsController`, `MediaController`.
- `DATASET``DatasetController` (status writes including bulk).
- `ADM` — mutating routes on `SettingsController`.
- `[Authorize]` (any authenticated user) — read endpoints on settings, `ClassesController`.
- `[AllowAnonymous]``/health`.
**User identity**: server resolves user from JWT `NameIdentifier` (e.g., `AnnotationsController.Create` parses `User.FindFirstValue(ClaimTypes.NameIdentifier)``Guid`). Suite spec sometimes lists `UserId` in body — drift recorded in `00_discovery.md`.
**Data protection**:
- **At rest**: nothing in-code — relies on the underlying Postgres deployment + filesystem.
- **In transit**: terminated outside the container; service speaks plain HTTP on `:8080`.
- **Secrets**: env-driven (`DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`, `RABBITMQ_*`). `DATABASE_URL` and the three JWT vars now fail-fast on startup if unset (no insecure default). ADR-002 was retired together with `JWT_SECRET`.
- **CORS**: config-driven allow-list (`CorsConfig:AllowedOrigins`); `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` with an empty list unless `CorsConfig:AllowAnyOrigin=true` is explicitly set. ADR-006 was retired together with the wide-open default.
**Audit logging**: not evidenced beyond ASP.NET Core defaults — open gap; flag in retro/security audit.
**Input validation**: surfaces through model binding + `ErrorHandlingMiddleware` mapping (`400 / 404 / 409 / 500`); detailed validators per DTO live in `DTOs/Requests/` (component specs to confirm during Step 4 verification).
## 8. Key Architectural Decisions (inferred from code)
These ADRs document choices the codebase already evidences. They are descriptive, not prescriptive — call them out so downstream skills can challenge them deliberately.
### ADR-001: In-process SSE via `Channel<T>`
**Context**: Real-time annotation activity must reach the Annotator UI within 100ms of a write.
**Decision**: Use a singleton `AnnotationEventService` exposing an unbounded `Channel<AnnotationEventDto>` and serve subscribers from `AnnotationsController.Events` over `text/event-stream`.
**Alternatives considered (implicitly rejected)**:
1. Broker-backed pub/sub (Redis / RabbitMQ exchange) — rejected because it adds a dependency for what is already a single-process workload, and the failsafe queue covers durable export needs.
2. Server-side polling — rejected because it cannot meet sub-second latency cheaply.
**Consequences**: SSE state is **per-instance only**. Horizontal scaling requires a broker fanout layer or sticky sessions on the LB.
### ADR-002 (RETIRED): Symmetric JWT, no issuer/audience validation
**Status**: superseded — annotations is now a JWKS verifier of admin-signed ES256 tokens. `AddJwtAuth(IConfiguration)` pins `ValidAlgorithms = [SecurityAlgorithms.EcdsaSha256]`, enforces `ValidateIssuer`/`ValidateAudience`/`RequireSignedTokens`/`RequireExpirationTime`, and resolves keys through `ConfigurationManager<JsonWebKeySet>` against `JWT_JWKS_URL`. `JWT_SECRET` was removed along with the local refresh path; admin is the sole issuer for the suite. The original ADR is preserved here for historical context only.
### ADR-003: Failsafe outbox + RabbitMQ Stream (not direct publish)
**Context**: Annotation lifecycle must reach external consumers (admin sync, AI training) durably even when RabbitMQ is unavailable at the moment of the write.
**Decision**: Every mutation writes a row to `annotations_queue_records`; the in-process `FailsafeProducer` (`IHostedService`) drains this table and publishes MessagePack frames on the `azaion-annotations` stream, deleting rows after success.
**Alternatives considered**:
1. Direct publish in the request path — rejected because RabbitMQ unavailability would either drop events (`fire-and-forget`) or fail user-visible writes (sync publish).
2. Transactional outbox via Debezium / CDC — heavier, deferred.
**Consequences**: One outbox-drainer per service instance. Multiple instances drain concurrently → safe because the deletion is keyed on `id` and re-reads of disk bytes are idempotent, **but** ordering across consumers is not guaranteed.
### ADR-004: Annotation id from a sampled `XxHash3.Hash128` of image bytes
**Context**: Annotation rows must be deduplicated when the same image is re-uploaded (e.g., re-runs of the detection pipeline). The system also serves video media up to **35 GB**, so hashing must remain **constant-time with respect to file size** to keep create-path latency stable under load.
**Decision** (resolved 2026-05-14): Hash a deterministic **fixed-size sample** with `XxHash3.Hash128` (128-bit output, 32-char lower-case hex). Sample composition is unchanged from the current implementation:
- For inputs **≤ 3072 bytes**: `[length(8 bytes)] + [full bytes]`.
- For inputs **> 3072 bytes**: `[length(8 bytes)] + [first 1024] + [middle 1024 starting at len/2 512] + [last 1024]`.
When `MediaId` is provided instead of bytes, the annotation id is reused from the referenced media row.
**Why this combination**:
- **Sampling preserves file-size independence.** Reading a 5 GB video front-to-back just to derive an id is unacceptable on the hot path.
- **`XxHash3.Hash128` over the same sample** keeps the hashing itself O(1) in file size while moving the collision space from 2^64 to 2^128. Distinct large images that happen to share `(length, head 1 KB, middle 1 KB, tail 1 KB)` still collide deterministically — but the practical collision probability among such samples is now negligible at any realistic volume.
**Migration consequences**:
- The annotation `id` column is `TEXT PRIMARY KEY`; switching from 16-char (`XxHash64`) to 32-char (`XxHash3.Hash128`) hex requires no schema change.
- Existing rows keep their 16-char ids; new rows get 32-char ids. Re-create of an image whose original id was generated under `XxHash64` will produce a **different** new id under `XxHash3.Hash128` — i.e., re-creates after the upgrade no longer collide with their pre-upgrade row. Acceptable (and expected): old ids are stable, the deduplication property is preserved going forward, and the upgrade is irreversible by design.
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-04).
### ADR-005: Swagger UI mounted in all environments
**Context**: Internal debugging / partner integration friction.
**Decision**: `app.UseSwagger()` and `app.UseSwaggerUI()` are unconditional in `Program.cs`.
**Consequences**: Schema is publicly readable wherever the service is reachable. If the perimeter is not closed, this leaks endpoint surface — treat as a security finding for production-internet exposure.
### ADR-006 (RETIRED): Wide-open CORS
**Status**: superseded — the default policy now reads `CorsConfig:AllowedOrigins` (string array) and `CorsConfig:AllowAnyOrigin` (boolean opt-in). `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when origins are empty and `AllowAnyOrigin` is not explicitly set; a `LogWarning` is emitted in non-production when running with the permissive default. The original ADR is preserved here for historical context only.
### ADR-007: Embedded SQL migrator (not EF migrations / Flyway)
**Context**: Suite values single-binary deploys; the team prefers idempotent boot-time DDL over a separate migration tool.
**Decision**: `DatabaseMigrator.Migrate` runs a single multi-statement script via Linq2DB on every startup. Schema evolution is additive (`ALTER … ADD COLUMN IF NOT EXISTS`).
**Consequences**: Backwards-only, no down migrations. Renames or destructive changes need an explicit out-of-band script. Drift detection requires diffing live DB against `Database/DatabaseMigrator.cs`.
### ADR-008: Annotation lifecycle wrapped in a business transaction (planned)
**Context**: `CreateAnnotation` today touches the filesystem, three DB tables, an in-memory channel, and an outbox row, with no atomicity. World B (lifecycle is observable — see ADR-009) widens this surface to Update / Delete / status-change paths. A naive DB transaction does not wrap the FS writes; we want a single conceptual transactional boundary for the lifecycle, not just for the DB rows.
**Decision** (resolved 2026-05-14, to-be-implemented): introduce a **business-transaction wrapper** for annotation lifecycle operations. Concretely the chosen pattern is the **transactional outbox**:
1. Write all relevant DB rows (annotation / detection / annotations_queue_records) inside a single `db.BeginTransaction` scope.
2. Commit. The outbox row is the durable promise that the post-commit work is owed.
3. **Post-commit**, perform side effects: write image / label / thumbnail files, publish SSE event. These steps are idempotent on retry; the outbox row stays until the drainer succeeds.
4. The drainer (`FailsafeProducer`) is unchanged in role — it consumes the outbox.
**Implications**:
- FS write order shifts: today image is first, before any DB row; after the refactor, DB rows + outbox commit first, then FS writes execute (with the outbox row as the recovery anchor).
- A new abstraction (e.g., `AnnotationLifecycleTransaction` or a thin extension on `AppDataConnection`) is the right place to centralize this. Implementation deferred to RB-03.
**Alternatives considered**:
1. Pure DB transaction wrapping current order — rejected: doesn't cover FS, leaves orphan-file risk.
2. Saga / compensation steps with explicit rollback handlers — rejected: overkill for the linear lifecycle here.
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-03).
### ADR-009: Lifecycle observability — World B (planned)
**Context**: Today only `CreateAnnotation` publishes SSE and enqueues the outbox. Update / UpdateStatus / Delete (annotations) and UpdateStatus / BulkUpdateStatus (dataset) are silent. The `QueueOperation` enum already declares `Validated` and `Deleted`, and `FailsafeProducer.cs:108123` has a dedicated drainer branch for both — strong evidence that the design always intended every lifecycle change to be observable. The producer side simply was never wired (the prior WPF codebase blended UI + backend; lifecycle calls likely came from the UI directly, which the new HTTP backend has not replicated).
**Decision** (resolved 2026-05-14, to-be-implemented): every annotation mutation publishes SSE and enqueues the outbox.
Mapping (initial; sub-questions to be resolved at implementation time):
| Mutation | SSE | Outbox `QueueOperation` |
|----------|-----|--------------------------|
| `AnnotationService.CreateAnnotation` | yes (today) | `Created` (today) |
| `AnnotationService.UpdateAnnotation` (replace detections, status → `Edited`) | yes | open: re-enqueue as `Created` (richer payload) **or** add `QueueOperation.Updated` + corresponding drainer branch |
| `AnnotationService.UpdateStatus` (status → `Validated (30)` or `Deleted (40)`) | yes | `Validated` |
| `AnnotationService.UpdateStatus` (other transitions) | yes | open: skip outbox, or always enqueue `Validated`? |
| `AnnotationService.DeleteAnnotation` | yes | `Deleted`**soft-delete**: status flips to `AnnotationStatus.Deleted = 40`, the row stays, image / label / thumbnail files relocate to a `deleted_dir` (new `directory_settings` column added by RB-01) |
| `DatasetService.UpdateStatus` / `BulkUpdateStatus` | yes (per-id for bulk) | `Validated` (single record covers the whole bulk via `AnnotationIds`) |
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-01).
### ADR-010: Remove `system_settings.silent_detection`
**Context**: `silent_detection` was a debug-time switch to keep the RabbitMQ stream clean while a developer iterated locally. Now that the suite has e2e tests with isolated queues (per `_docs/_repo-config.yaml` suite-e2e), the in-product flag is dead code — debug isolation belongs in the test harness, not in `system_settings`.
**Decision** (resolved 2026-05-14, to-be-implemented):
- Remove the gating block in `AnnotationService.CreateAnnotation:100102` (always enqueue).
- Drop `silent_detection` from `system_settings` (column, entity, migrator `CREATE TABLE`, migrator `ALTER` line, any DTO references).
- Remove the field from `UpdateSystemSettingsRequest` if present.
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-02). Schema column removal is a destructive change explicitly authorized by the maintainer.
### ADR-012: Rename `Flight``Mission` to align with suite canonical (planned)
**Context**: The suite product spec (`suite/_docs/01_annotations.md`) calls the domain concept `mission` / `missionId`. The code uses `Flight` / `FlightId` (table `media.waypoint_id` + DTO `FlightId` filter). This drift was flagged in `00_discovery.md`.
**Decision** (resolved 2026-05-14, to-be-implemented): align code to the suite. `Flight*``Mission*` rename across DTOs, controllers, services, and the relevant query-parameter names. The `media.waypoint_id` column stays (it is the underlying physical identifier; mission is the logical grouping concept above it).
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-07). Schema column changes are scoped to renames in DTOs and code only — no DB column rename is required for this ADR.
### ADR-013: Stream consumer dedupe contract is owned by this service (planned)
**Context**: The failsafe outbox + RabbitMQ Stream pipeline can produce duplicate stream entries when (a) the drainer retries after a partial publish or (b) two service instances both pick up the same outbox row before either deletes it. Today there is no documented dedupe contract; consumers (admin sync, AI training) silently accept whatever they get.
**Decision** (resolved 2026-05-14, to-be-implemented): publish a documented dedupe contract owned by this service. Working shape: consumers MUST dedupe by `(annotationId, operation, dateTime)`. The outbox row's `DateTime` (already populated by `EnqueueAsync`) becomes part of the on-the-wire stream message, alongside the `annotationId` and `operation` already in `AnnotationQueueMessage` / `AnnotationBulkQueueMessage`.
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-09).
### ADR-011: Detection class catalog is admin-managed with in-memory cache (planned)
**Context**: `detection_classes` is currently seeded by the migrator (19 rows) and read-only via `GET /classes`. Operators have no way to add or correct classes (e.g., the `Smoke`/`Plane` color clash on `#000080`) without a code change and redeploy.
**Decision** (resolved 2026-05-14, to-be-implemented):
- `ClassesController` exposes `POST /classes`, `PUT /classes/{id}`, `DELETE /classes/{id}` under `[Authorize(Policy = "ADM")]`. `GET /classes` stays `[Authorize]`.
- Reads go through a new `DetectionClassCache` (DI singleton) modeled on `PathResolver`: lazy-load on first read, `Reset()` after any write.
- Migrator-seeded rows remain as the bootstrap state; admin writes overwrite them per id.
**Status**: agreed. Implementation lives in the Refactor Backlog (RB-06). Adds a new feature surface; must land before any UI change relying on dynamic class management.
## Resolved Architectural Decisions (Step 4 verification)
The following items were surfaced during verification and resolved with the maintainer on 2026-05-14. Each one either becomes an ADR above or maps to a refactor backlog entry below.
| # | Concern | Resolution | Tracked as |
|---|---------|------------|------------|
| 1 | Update / Delete / dataset-status changes are silent on SSE + outbox | Treat as gap; lifecycle is observable (World B) — every mutation publishes + enqueues | ADR-009 / RB-01 |
| 2 | `system_settings.silent_detection` semantics | Remove the flag; e2e harness covers debug isolation now | ADR-010 / RB-02 |
| 3 | F1 not transactional across FS + DB + outbox | Wrap lifecycle in a business-transaction (transactional outbox); FS writes happen post-commit | ADR-008 / RB-03 |
| 4 | `XxHash64` over sampled bytes — collision risk | Switch to `XxHash3.Hash128` over the same sample (file-size-independent + 128-bit space) | ADR-004 / RB-04 |
| 5 | `FailsafeProducer.EnqueueAsync` static method does DB I/O — violates `coderule.mdc` | Accept as-is; documented deviation from rule | (no refactor) |
| 6 | `detection_classes` schema-mutable but no controller writes | Admin-managed CRUD with read-through cache (modeled on `PathResolver`) | ADR-011 / RB-06 |
| 7 | `Flight` (code) vs `mission` (suite spec) drift | Rename code → `Mission*`; suite spec stays canonical | ADR-012 / RB-07 |
| 8 | Dataset writes coupled directly to annotation rows via shared `AppDataConnection` | Route dataset writes through `AnnotationService` (via a public domain interface) | RB-08 |
| 9 | Stream consumer dedupe contract owner | This service owns it; dedupe by `(annotationId, operation, dateTime)` baked into the wire message | ADR-013 / RB-09 |
| 10 | Hard-delete vs soft-delete on `DeleteAnnotation` | Soft-delete: status → `Deleted (40)`, files moved to a `deleted_dir` | ADR-009 (folded in) / RB-01 |
## Remaining Open Architectural Risks
These are residual risks that still need attention from later autodev steps (Test Spec, Refactor, Security Audit). Items previously listed here that have been resolved as of 2026-05-14 (Flight/mission drift, dataset coupling, hard-vs-soft delete, JWT issuer/audience validation, CORS environment gating, dev secret fallback) moved to the Resolved Architectural Decisions table above and the Refactor Backlog below.
1. **Horizontal scaling**: SSE channel is per-instance (singleton `AnnotationEventService`); the failsafe outbox uses no leasing/locking. Two pods will independently drain rows, with deletion keyed on `id`; under high concurrency the same row can be picked by both before either deletes — duplicate stream entries possible. Consumers must dedupe per ADR-013. (Touched by RB-03 / RB-09 indirectly but not solved by them.)
2. **Swagger exposure** in production: see ADR-005. Belongs to Step 14 (Security Audit). (CORS exposure was resolved by `CorsConfigurationValidator`; ADR-006 retired.)
3. **`UserId` body field vs JWT `NameIdentifier`** drift (suite spec lists `UserId` on `POST /annotations`; code uses JWT subject). Reconcile in the suite spec.
4. **No automated tests**: addressed by autodev Phase A Steps 37 (Test Spec → Implement Tests → Run Tests).
5. **`FailsafeProducer.cs:138` swallows `IOException` on image read silently** (`catch { }`). Direct `coderule.mdc` violation. Symptom in product: a missing or unreadable image yields a stream message with `image = null` and no log/metric — the gap is invisible to operators. Track on Refactor Backlog (RB-05).
6. ~~**JWKS HTTPS-only retrieval blocks containerised test harnesses**~~**RESOLVED 2026-05-14** by Step 4 (Code Testability Revision). `JwtExtensions.AddJwtAuth` now gates `HttpDocumentRetriever.RequireHttps` on `ASPNETCORE_ENVIRONMENT == "E2ETest"` (case-insensitive). Production / Staging / Development / unset all retain HTTPS-required behavior; only the `E2ETest` value relaxes the flag. Verified via the smoke harness in `_docs/04_refactoring/01-testability-refactoring/verification.md`. See `_docs/04_refactoring/01-testability-refactoring/testability_changes_summary.md` (item C01) for the full change log.
## Refactor Backlog
These items are the implementation work for the resolved decisions above. They are **not** part of Step 4 (Verification) corrections — they will be picked up by the autodev existing-code flow at Step 8 (Refactor) and/or new feature tasks in Phase B.
| ID | Scope | Source ADR / Risk | Notes |
|----|-------|--------------------|-------|
| RB-01 | Wire lifecycle publish + outbox enqueue across Update / UpdateStatus / Delete (annotations + dataset). Includes the soft-delete behavior: `DeleteAnnotation` flips `AnnotationStatus → Deleted (40)`, leaves the row, and moves image / label / thumbnail files to a new `deleted_dir` (added to `directory_settings`). Read paths must filter `Status = Deleted (40)` from default lists. | ADR-009 | Open sub-questions: (a) `UpdateAnnotation` mapping — re-enqueue as `Created` or add `QueueOperation.Updated` + drainer branch; (b) which non-Validated/Deleted status transitions enqueue at all |
| RB-02 | Remove `silent_detection` (schema column, entity field, gating logic, DTOs) | ADR-010 | Destructive schema change explicitly authorized |
| RB-03 | Introduce business-transaction wrapper (transactional outbox) for annotation lifecycle | ADR-008 | Reorders FS writes to post-commit; covers all mutation paths |
| RB-04 | Switch annotation id hashing to `XxHash3.Hash128` over the same sampled buffer | ADR-004 | Existing 16-char ids stay; new ids are 32-char hex |
| RB-05 | Replace `catch { }` at `FailsafeProducer.cs:138` with logged failure path; surface as a metric | Open Risk §6 | Downstream consumer should know an image-less message means a real disk error |
| RB-06 | Admin-managed `detection_classes` (CRUD endpoints `[ADM]`, in-memory cache with `Reset()`) | ADR-011 | Migrator seed remains as bootstrap; admin overrides per id; fix `Smoke`/`Plane` color collision while at it |
| RB-07 | Rename `Flight*``Mission*` across DTOs, controllers, services, and query-parameter names. `media.waypoint_id` column is unchanged (it's the physical id; mission is the logical concept). | ADR-012 | Code-only rename to align with suite spec; suite stays canonical |
| RB-08 | Decouple `04 Dataset` writes from direct `annotations` row mutations — route status writes through a public `AnnotationService` interface. Reads can stay direct for now (read coupling is lower-risk than write coupling). | Open Risks (former §4) | Likely introduces an `IAnnotationLifecycle` (or similar) interface owned by `01 Annotations REST` that `04 Dataset` consumes via DI |
| RB-09 | Bake `(annotationId, operation, dateTime)` into the on-the-wire stream message; document the dedupe contract in `suite/_docs/01_annotations.md`. | ADR-013 | Coordinate the suite-doc update with admin sync + AI training maintainers |
## References
- Suite product spec: `../../../suite/_docs/01_annotations.md` (REST contracts, SSE, Annotation Sync, camera, classes).
- Suite dataset narrative: `../../../suite/_docs/09_dataset_explorer.md`.
- Component specs: `components/01..06_*/description.md`.
- Module docs: `modules/*.md`.
- File ownership (downstream skills): `module-layout.md`.
- Component diagram: `diagrams/components.md`.
- Per-flow diagrams: `diagrams/flows/`.
@@ -1,105 +0,0 @@
# Architecture Compliance Baseline — Azaion.Annotations
**Mode**: code-review baseline (Phase 1 + Phase 7 only)
**Scope**: full codebase under `src/` (57 C# files)
**Date**: 2026-05-14
**Verdict**: PASS_WITH_WARNINGS
This is the **one-time architecture baseline** for the existing-code flow's Step 2. Future per-batch code-review runs partition findings against this baseline (carried over / resolved / newly introduced) per `.cursor/skills/code-review/SKILL.md` → "Baseline delta".
## Inputs (verified loaded)
- `_docs/02_document/architecture.md` — layering rules, ADRs, refactor backlog
- `_docs/02_document/module-layout.md` — per-component file ownership, Public API, Allowed Dependencies table
- `_docs/02_document/components/*/description.md` — six component specs
- `_docs/00_problem/restrictions.md` — operational constraints
- `_docs/01_solution/solution.md` — solution overview
## Method
Per Phase 7 of the code-review skill:
1. Mapped all 57 C# files to one of six logical components per `module-layout.md`.
2. Parsed every `using Azaion.Annotations.*` directive and every constructor-injection / static reference between domain types (`AnnotationService`, `AnnotationEventService`, `FailsafeProducer`, `MediaService`, `DatasetService`, `SettingsService`, `PathResolver`, `AppDataConnection`). Note: `TokenService` and `AuthController` were removed in the auth refactor and are no longer part of the reference graph.
3. Resolved each cross-file reference against the Allowed Dependencies table (Layer 1 → Layer 2 → Layer 3) and the per-component Public API list.
4. Applied the five Phase 7 checks: layer direction, Public API respect, cyclic dependencies, duplicate symbols, cross-cutting concerns.
## Findings summary
| Severity | Count | Categories |
|----------|-------|------------|
| Critical | 0 | — |
| High | 0 | — |
| Medium | 1 | Architecture |
| Low | 2 | Architecture, Maintainability |
**No new High or Critical findings.** Per the existing-code flow Step 2 auto-chain rule, this allows direct progression to Step 3 (Test Spec).
## Findings detail
### F1 — `04 dataset` writes directly to the annotation domain (Medium / Architecture)
- **Location**: `src/Services/DatasetService.cs:75-94` (`UpdateStatus`, `BulkUpdateStatus`).
- **Description**: `DatasetService` mutates `db.Annotations.Set(a => a.Status, …)` directly. The `annotations` row is part of `01 annotations-rest`'s domain (per `module-layout.md` → DTO ownership table and component spec). Today this is technically allowed because the only cross-component reference is `AppDataConnection` (a `06_platform` foundation type), but it duplicates ownership of the annotation lifecycle: there are now two paths that mutate `annotations.status``AnnotationService.UpdateStatus` (in 01) and `DatasetService.UpdateStatus` / `BulkUpdateStatus` (in 04) — and only the former is wired (post RB-01) into the lifecycle observability contract (SSE + outbox).
- **Architecture vision impact**: ADR-009 + RB-01 require every mutation to emit lifecycle events. As long as 04 has its own DB write path, that contract cannot be enforced from one place — RB-08 fixes this by routing 04's status writes through `AnnotationService`.
- **Suggestion**: Track via RB-08; no inline action required at this baseline. Confirms the refactor backlog item is well-grounded.
- **Module-layout reference**: section "Allowed Dependencies → Rules" — "today: only via shared AppDataConnection in same assembly — acceptable but treat as tight coupling; prefer domain services for new code."
### F2 — `ClassesController` bypasses the service layer (Low / Architecture)
- **Location**: `src/Controllers/ClassesController.cs:11`.
- **Description**: `ClassesController` injects `AppDataConnection` directly and queries the `detection_classes` table inline. Every other controller in the codebase (`AnnotationsController`, `MediaController`, `DatasetController`, `SettingsController`) routes through a service. Allowed by the layering rules (06 is a foundation), but inconsistent with the project convention.
- **Architecture vision impact**: RB-06 introduces admin-managed CRUD for detection classes plus an in-memory cache with `Reset()`. That work will naturally land a `DetectionClassService` (or extend `SettingsService` to own this surface), retiring the direct-DB pattern.
- **Suggestion**: Defer to RB-06; do not address inline.
### F3 — `FailsafeProducer.EnqueueAsync` is a static method that performs DB I/O (Low / Maintainability)
- **Location**: `src/Services/FailsafeProducer.cs:195` (the static helper) and the call site `src/Services/AnnotationService.cs:102`.
- **Description**: `FailsafeProducer.EnqueueAsync(AppDataConnection db, string annotationId, QueueOperation operation)` is a static method on the same type as the hosted-service producer, called from `AnnotationService` to insert a row into `annotations_queue_records`. `coderule.mdc` discourages static methods that touch resources; the user has explicitly accepted this as technical debt during the verification stakeholder review (no RB item — keep as-is).
- **Architecture impact**: The Public API of `02 annotations-realtime-sync` includes both the running `FailsafeProducer` *and* this static helper; that is documented in `module-layout.md` Component 02. So there is no boundary violation — it is a deliberate API choice the user has confirmed.
- **Suggestion**: No action. Recorded for future reviewers so the pattern is not flagged again as a finding.
## Phase 7 checklist (full traversal)
| Check | Result | Notes |
|-------|--------|-------|
| 1. Layer direction (Layer 3 → Layer 2 → Layer 1 only) | PASS | All Layer 3 components import only `06` (and `01` additionally `02`'s `AnnotationEventService` + `FailsafeProducer.EnqueueAsync`). No reverse imports detected. |
| 2. Public API respect | PASS | Every cross-component constructor injection or static call targets a type listed in `module-layout.md` → Public API. No internal-file imports across components. |
| 3. No cyclic module dependencies | PASS | DI graph: `06 ← {01, 02, 03, 04, 05}`, `02 ← 01`. No cycles. |
| 4. Duplicate symbols across components | PASS (with F1) | No class/function name collisions. F1 is a *logical* duplication of write authority over `annotations.status`, captured separately. |
| 5. Cross-cutting concerns not locally re-implemented | PASS | Logging via `ILogger<T>`; auth in `06_platform/Auth`; error envelope in `06_platform/Middleware/ErrorHandlingMiddleware`; config / env reading concentrated in `Program.cs`. No per-component re-implementations. |
## Static reference graph (verified)
```mermaid
graph LR
P06[06 platform]
P02[02 realtime sync]
P01[01 annotations rest]
P03[03 media]
P04[04 dataset]
P05[05 settings metadata]
P02 --> P06
P01 --> P06
P01 --> P02
P03 --> P06
P04 --> P06
P05 --> P06
```
Edges represent constructor-injection or static-call dependencies. `Program.cs` (composition root, in 06) is excluded — it touches everything by definition.
## Cross-references
- ADR-008 (transactional outbox) and RB-01, RB-03, RB-08 in `_docs/02_document/architecture.md` cover the lifecycle / coupling concerns.
- `module-layout.md` → "Allowed Dependencies" table is the source of truth for layer membership.
- `_docs/02_document/04_verification_log.md` documents the stakeholder acceptance of `FailsafeProducer.EnqueueAsync` as tech debt (F3).
## Auto-chain decision
Per `.cursor/skills/autodev/flows/existing-code.md` → Step 2 auto-chain rule:
> If the baseline report contains High or Critical Architecture findings: append to Step 4 testability inputs OR surface to user. If clean (no High/Critical): auto-chain directly to Step 3.
This baseline contains **0 Critical, 0 High, 1 Medium, 2 Low** Architecture findings. → **Auto-chain to Step 3 (Test Spec)** without user gate.
@@ -1,7 +0,0 @@
# Shared: HTTP error envelope
**Used by:** All controllers (via pipeline).
**Implementation:** `ErrorHandlingMiddleware` — lives under **06 Platform** for ownership; feature components rely on it without duplication.
See `modules/common-infrastructure.md` and `components/06_platform/description.md`.
@@ -1,45 +0,0 @@
# Annotations (REST & files)
## 1. High-Level Overview
**Purpose:** HTTP API for annotation CRUD, status, listing, and serving **annotation image / thumbnail** bytes — the surface described in `suite/_docs/01_annotations.md` §16 (excluding the SSE stream).
**Architectural pattern:** Layered API — controller → application service → database + filesystem.
**Upstream dependencies:** Platform (DB, auth, paths), Media (annotation create may reference existing `MediaId`).
**Downstream consumers:** Annotator UI, Detections pipeline (HTTP POST annotations), Dataset (read-only overlap on entities).
## 2. Internal interfaces
Primary application API: `AnnotationService` — create/update/status/delete/query/get-by-id; uses `PathResolver`, `AppDataConnection`, hashing, label files, thumbnails, and triggers **real-time publish** (calls into `AnnotationEventService`) and **queue enqueue** (via failsafe path — see component 02).
Controller: `AnnotationsController`**REST and file routes**; the `GET …/events` SSE action is **specified and operated** from the **Annotations realtime & sync** component (same source file, split responsibility for docs).
### Representative HTTP
| Area | Routes (policy `ANN`) |
|------|------------------------|
| CRUD | `POST/PUT/PATCH/DELETE/GET` under `/annotations` |
| Files | `GET /annotations/{id}/thumbnail`, `GET /annotations/{id}/image` |
## 3. External API specification
See `01_annotations.md` §16; confirm drift notes in `00_discovery.md` (JWT user id, `FlightId` vs suite `missionId`).
## 4. Data access patterns
Heavy use of `annotations`, `detection`, `media` joins for list filters; writes cascade detections on update.
## 5. Implementation notes
- Annotation id from image hash when bytes provided; else copy from existing media path.
- `TimeSpan` persisted as ticks on `Annotation`.
## 6. Dependency graph (relative to other components)
**Imports from:** Platform, Media (logical). **Consumed by:** Dataset (read), UI, external Detections service.
## 7. Modules included
`annotations-service` (module doc); **shared file** `AnnotationsController.cs` with component 02 for SSE action only.
@@ -1,40 +0,0 @@
# Annotations (realtime & stream sync)
## 1. High-Level Overview
**Purpose:** **SSE** push for annotation changes and **RabbitMQ Stream** failsafe export — `01_annotations.md` sections *SSE Communication* and *Annotation Sync* / *Failsafe* / *RabbitMQ Stream*.
**Architectural pattern:** Event channel + background outbox producer.
**Upstream dependencies:** Platform (DB, config, paths), Annotations REST (domain mutations enqueue/publish).
**Downstream consumers:** Browser UI (SSE); Admin sync worker; AI Training consumer (external).
## 2. Internal interfaces
- `AnnotationEventService` — in-process `Channel<AnnotationEventDto>`; `PublishAsync` / `Reader`.
- `FailsafeProducer` + `RabbitMqConfig` — stream client, MessagePack payloads, drains `annotations_queue_records`. `RABBITMQ_HOST` accepts both literal IPv4/IPv6 (used as-is via `IPAddress.TryParse`) and DNS hostnames (resolved through `Dns.GetHostAddressesAsync` — required for Docker/Kubernetes service-name connectivity).
- **HTTP:** `AnnotationsController.Events``text/event-stream` subscription (same controller file as REST component; **doc ownership** here for SSE).
## 3. External API / integration
| Surface | Notes |
|---------|--------|
| `GET /annotations/events` | SSE; see suite SSE section |
| RabbitMQ stream `azaion-annotations` | Env `RABBITMQ_*` from `Program` |
## 4. Data access patterns
Queue table buffering; stream send on connectivity; image bytes in create messages per suite.
## 5. Caveats
MessagePack key stability; stream consumer offsets independent per consumer type.
## 6. Dependency graph
**Imports from:** Platform, annotations domain (via service calls / shared types). **Consumed by:** External infrastructure.
## 7. Modules included
`sse-realtime`, `rabbitmq-stream-sync`.
@@ -1,29 +0,0 @@
# Media
## 1. High-Level Overview
**Purpose:** Upload, batch-upload, list, delete, and download **media** files for missions/waypoints — `01_annotations.md` §710 and *Media Browsing*.
**Architectural pattern:** Service + controller; filesystem + DB.
**Upstream dependencies:** Platform (auth, DB, paths for storage roots).
**Downstream consumers:** Annotator UI; Annotations REST (references `MediaId`).
## 2. Internal interfaces
`MediaService`, `MediaController` — JSON create, **multipart** batch with `waypointId`, paged list, file download, delete.
## 3. External API
| Policy | Base |
|--------|------|
| `ANN` | `/media` |
## 4. Data access
`media` table; files on disk under configured video/image roots.
## 5. Modules included
`media-service`.
@@ -1,25 +0,0 @@
# Dataset Explorer (API)
## 1. High-Level Overview
**Purpose:** Backend for **Dataset Explorer** — grid, detail, status PATCH, bulk status — `DATASET` policy; cross-ref `suite/_docs/09_dataset_explorer.md`.
**Architectural pattern:** Read-heavy + controlled writes on annotation status.
**Upstream dependencies:** Platform, shared annotation entities/status with Annotations REST.
**Downstream consumers:** Dataset Explorer UI.
## 2. Internal interfaces
`DatasetService`, `DatasetController``/dataset` routes.
## 3. External API
| Policy | Base |
|--------|------|
| `DATASET` | `/dataset` |
## 4. Modules included
`dataset-service`.
@@ -1,23 +0,0 @@
# Settings & metadata
## 1. High-Level Overview
**Purpose:** System, directory, camera, and per-user UI settings; **detection class catalog** for labels/colors — `01_annotations.md` §1112 (camera) plus settings/directory narratives; `GET /classes` for annotator.
**Architectural pattern:** CRUD settings services + thin metadata read controller.
**Upstream dependencies:** Platform (DB, auth — `ADM` on mutating settings).
**Downstream consumers:** All UIs; `PathResolver` after directory updates (reset).
## 2. Internal interfaces
`SettingsService`, `SettingsController`, `ClassesController`.
## 3. External API
Mixed `[Authorize]` and policy `ADM` for writes under `/settings`; `GET /classes` authenticated read.
## 4. Modules included
`settings-metadata-service`.
@@ -1,27 +0,0 @@
# Platform foundation
## 1. High-Level Overview
**Purpose:** **Wire enums**, **PostgreSQL schema/mapping**, **cross-cutting HTTP error handling**, **path resolution**, **JWT policies + token refresh**, and **application bootstrap** — no standalone product feature; enables all other components.
**Architectural pattern:** Shared kernel / infrastructure.
**Upstream dependencies:** None (root).
**Downstream consumers:** All feature components.
## 2. Internal interfaces
- `src/Enums/*`
- `src/Database/*` (`AppDataConnection`, `DatabaseMigrator`, entities)
- `ErrorHandlingMiddleware`, `PathResolver`, `PaginatedResponse`, `ErrorResponse`, `GlobalUsings.cs`
- `JwtExtensions` (JWKS verifier; `HttpDocumentRetriever.RequireHttps` is gated on `ASPNETCORE_ENVIRONMENT` — HTTPS-required for every value except `E2ETest`), `ConfigurationResolver`, `CorsConfigurationValidator`
- `Program.cs`
## 3. External API
`/health` (AllowAnonymous); Swagger in development configuration. Token refresh is no longer hosted here — callers refresh against admin's `POST /token/refresh`.
## 4. Modules included
`wire-enums`, `database-layer`, `common-infrastructure`, `auth-identity`, `composition-program`.
-254
View File
@@ -1,254 +0,0 @@
# Azaion.Annotations — Data Model
> Source-of-truth: `src/Database/DatabaseMigrator.cs` and `src/Database/Entities/*.cs`. Every column name and type below is reproduced from migrator SQL.
## Schema overview
```mermaid
erDiagram
media ||--o{ annotations : "media_id"
annotations ||--o{ detection : "annotation_id"
annotations_queue_records }o..o{ annotations : "annotation_ids JSON (no FK)"
detection_classes ||..o{ detection : "class_num (logical, no FK)"
media {
TEXT id PK
TEXT name
TEXT path
INTEGER media_type "MediaType enum"
INTEGER media_status "MediaStatus enum"
UUID waypoint_id
UUID user_id
TEXT duration "added later (ALTER)"
}
annotations {
TEXT id PK "image-bytes hash (ADR-004)"
TEXT media_id FK
BIGINT time "ticks of TimeSpan"
TIMESTAMP created_date
UUID user_id
INTEGER source "AnnotationSource enum"
INTEGER status "AnnotationStatus enum"
BOOLEAN is_split "added via ALTER"
TEXT split_tile "added via ALTER"
}
detection {
UUID id PK
REAL center_x
REAL center_y
REAL width
REAL height
INTEGER class_num
TEXT label
TEXT description
REAL confidence
INTEGER affiliation "AffiliationEnum"
INTEGER combat_readiness "CombatReadiness"
TEXT annotation_id FK
}
annotations_queue_records {
UUID id PK
TIMESTAMP date_time
INTEGER operation "QueueOperation enum"
TEXT annotation_ids "JSON array of TEXT ids"
}
system_settings {
UUID id PK
TEXT name
TEXT military_unit
INTEGER default_camera_width
NUMERIC default_camera_fov
INTEGER thumbnail_width "default 240"
INTEGER thumbnail_height "default 135"
INTEGER thumbnail_border "default 10"
BOOLEAN generate_annotated_image "default false"
BOOLEAN silent_detection "default false"
}
directory_settings {
UUID id PK
TEXT videos_dir "default /data/videos"
TEXT images_dir "default /data/images"
TEXT labels_dir "default /data/labels"
TEXT results_dir "default /data/results"
TEXT thumbnails_dir "default /data/thumbnails"
TEXT gps_sat_dir "default /data/gps_sat"
TEXT gps_route_dir "default /data/gps_route"
}
detection_classes {
SERIAL id PK
TEXT name
TEXT short_name
TEXT color "hex e.g. #FF0000"
INTEGER max_size_m
INTEGER photo_mode
}
user_settings {
UUID id PK
UUID user_id "UNIQUE (ix_user_settings_user_id)"
UUID selected_flight_id
NUMERIC annotations_left_panel_width
NUMERIC annotations_right_panel_width
NUMERIC dataset_left_panel_width
NUMERIC dataset_right_panel_width
}
camera_settings {
UUID id PK
NUMERIC altitude "default 100"
NUMERIC focal_length "default 50"
NUMERIC sensor_width "default 36"
}
```
> Mermaid `erDiagram` does not represent JSON-array references; the dotted line for `annotations_queue_records ↔ annotations` is logical only — there is **no FK** in the schema.
## Tables
### `media`
Owned writes: `03_media`. Reads: `01_annotations-rest`, `04_dataset`.
| Column | Type | Notes |
|--------|------|-------|
| `id` | TEXT PK | Application-generated |
| `name` | TEXT NOT NULL | |
| `path` | TEXT NOT NULL | Filesystem path under media dir |
| `media_type` | INTEGER NOT NULL DEFAULT 0 | `MediaType` enum (numeric wire — see `wire-enums.md`) |
| `media_status` | INTEGER NOT NULL DEFAULT 0 | `MediaStatus` enum |
| `waypoint_id` | UUID | Indexed `ix_media_waypoint_id` |
| `user_id` | UUID NOT NULL | |
| `duration` | TEXT | Added via `ALTER`; nullable |
### `annotations`
Owned writes: `01_annotations-rest`. Status writes: `04_dataset` (bulk + single PATCH).
| Column | Type | Notes |
|--------|------|-------|
| `id` | TEXT PK | **Hash of image bytes** (ADR-004); collision implication noted |
| `media_id` | TEXT NOT NULL FK → `media.id` | |
| `time` | BIGINT NOT NULL DEFAULT 0 | Ticks of `TimeSpan` (suite spec stores `time` as ticks) |
| `created_date` | TIMESTAMP NOT NULL DEFAULT NOW() | Indexed `ix_annotations_created_date` |
| `user_id` | UUID NOT NULL | Indexed `ix_annotations_user_id` |
| `source` | INTEGER NOT NULL DEFAULT 0 | `AnnotationSource` enum (AI=0, Manual=1) |
| `status` | INTEGER NOT NULL DEFAULT 0 | `AnnotationStatus` enum (Created=10, Edited=20, …) |
| `is_split` | BOOLEAN NOT NULL DEFAULT false | Added via `ALTER`; tile-splitting flag |
| `split_tile` | TEXT | Tile id reference |
Indexes: `ix_annotations_media_id`, `ix_annotations_created_date`, `ix_annotations_user_id`.
### `detection`
| Column | Type | Notes |
|--------|------|-------|
| `id` | UUID PK | |
| `center_x`, `center_y`, `width`, `height` | REAL NOT NULL | YOLO-normalized box |
| `class_num` | INTEGER NOT NULL | Logical reference to `detection_classes.id` |
| `label` | TEXT NOT NULL DEFAULT '' | |
| `description` | TEXT | |
| `confidence` | REAL NOT NULL DEFAULT 0 | |
| `affiliation` | INTEGER NOT NULL DEFAULT 0 | `AffiliationEnum` |
| `combat_readiness` | INTEGER NOT NULL DEFAULT 0 | `CombatReadiness` enum |
| `annotation_id` | TEXT NOT NULL FK → `annotations.id` | Indexed `ix_detection_annotation_id` |
### `annotations_queue_records` (failsafe outbox)
| Column | Type | Notes |
|--------|------|-------|
| `id` | UUID PK | |
| `date_time` | TIMESTAMP NOT NULL DEFAULT NOW() | |
| `operation` | INTEGER NOT NULL DEFAULT 0 | `QueueOperation` enum |
| `annotation_ids` | TEXT NOT NULL DEFAULT '[]' | JSON array of annotation ids — single or bulk |
No FK to `annotations` — by design, since rows can survive an annotation deletion if export is in flight.
### `system_settings`
Singleton-ish (one row in practice). Includes:
- `generate_annotated_image` (BOOLEAN) — emits a baked-in annotated image alongside YOLO label when true (suite spec).
- `silent_detection` (BOOLEAN) — suppresses SSE / sync for detection events.
- `thumbnail_*` — defaults 240×135 with 10 border.
### `directory_settings`
Roots consumed by `PathResolver` (`06_platform`). Defaults: `/data/{videos,images,labels,results,thumbnails,gps_sat,gps_route}`. Updates require `PathResolver.Reset` (Flow F7 invariant).
### `detection_classes`
Seeded with 19 rows (ids 018) on first run via `INSERT ... ON CONFLICT (id) DO NOTHING`. Names + Cyrillic short names + hex colors + `max_size_m` + `photo_mode`.
| id | name | short_name | color | max_size_m |
|----|------|------------|-------|-------------|
| 0 | ArmorVehicle | Броня | `#FF0000` | 7 |
| 1 | Truck | Вантаж. | `#00FF00` | 8 |
| 2 | Vehicle | Машина | `#0000FF` | 7 |
| 3 | Artillery | Арта | `#FFFF00` | 14 |
| 4 | Shadow | Тінь | `#FF00FF` | 9 |
| 5 | Trenches | Окопи | `#00FFFF` | 10 |
| 6 | MilitaryMan | Військов | `#188021` | 2 |
| 7 | TyreTracks | Накати | `#800000` | 5 |
| 8 | AdditionArmoredTank | Танк.захист | `#008000` | 7 |
| 9 | Smoke | Дим | `#000080` | 8 |
| 10 | Plane | Літак | `#000080` | 12 |
| 11 | Moto | Мото | `#808000` | 3 |
| 12 | CamouflageNet | Сітка | `#800080` | 14 |
| 13 | CamouflageBranches | Гілки | `#2f4f4f` | 8 |
| 14 | Roof | Дах | `#1e90ff` | 15 |
| 15 | Building | Будівля | `#ffb6c1` | 20 |
| 16 | Caponier | Капонір | `#ffb6c1` | 10 |
| 17 | Ammo | БК | `#33658a` | 2 |
| 18 | Protect.Struct | Зуби.драк | `#969647` | 2 |
Note: ids 9 and 10 (`Smoke`, `Plane`) share `#000080` — a pre-existing data quirk, not a bug introduced by this skill.
### `user_settings`
Per-user UI prefs. Unique index on `user_id` (`ix_user_settings_user_id`). Carries selected flight + four panel widths (annotator left/right, dataset left/right).
### `camera_settings`
Calibration triple `(altitude, focal_length, sensor_width)` with defaults `(100, 50, 36)`.
## Migration strategy
- **Tool**: hand-rolled embedded SQL in `DatabaseMigrator.Migrate`, executed at every startup via Linq2DB.
- **Safety**: every statement is idempotent — `CREATE TABLE IF NOT EXISTS`, `ALTER TABLE … ADD COLUMN IF NOT EXISTS`, seed `INSERT … ON CONFLICT DO NOTHING`.
- **Direction**: forward-only. No down migrations or `DROP` operations; renames or destructive changes require an out-of-band migration.
- **Drift**: the only authoritative schema definition is `Database/DatabaseMigrator.cs`. Live DBs should be diffed against it on cadence; suite-level monitoring is out of scope here.
## Seed data observations
Only `detection_classes` has seeded data; all other tables start empty. `system_settings`, `directory_settings`, and `camera_settings` are inserted **lazily** by their respective services on first read/write — confirm exact upsert semantics in Step 4 verification.
## Backward compatibility
- Wire enums are **integer-stable** (suite contract). Renaming an enum case does not break wire compatibility because numeric values are the contract.
- Annotation id format is the hash of image bytes — changing the hashing algorithm would invalidate cross-build references; treat as a contract.
- MessagePack key order in `DTOs/QueueMessages.cs` is the export contract for RabbitMQ stream consumers — changing it breaks downstream.
## Cross-component data ownership
| Component | Writes | Reads |
|-----------|--------|-------|
| `01_annotations-rest` | `annotations`, `detection`, files on disk, `annotations_queue_records` (Created/Updated/Deleted) | `media` |
| `02_annotations-realtime-sync` | drains `annotations_queue_records` | `annotations`, `detection`, file bytes |
| `03_media` | `media`, files on disk | — |
| `04_dataset` | `annotations.status` (single + bulk) → also writes `annotations_queue_records`, publishes SSE | `annotations`, `detection`, `media` |
| `05_settings-metadata` | all `*_settings` tables | `detection_classes` (read-through for UI) |
| `06_platform` | none (pure infra) | `directory_settings` (via `PathResolver`) |
## Open data-model questions (Step 4 verification)
1. **`annotations.id` collisions**: behavior under same-bytes re-upload (insert vs noop vs error) is implicit — confirm in `AnnotationService`.
2. **`annotations_queue_records.annotation_ids` shape**: confirm consistent JSON formatting (escaped strings vs raw) across `Created`, `Updated`, `StatusChanged`, `Deleted`, bulk variants.
3. **`detection_classes` mutability**: schema permits inserts via `ALTER`/seed, but no controller exposes writes today — confirm whether class catalog is intended to be DB-managed or static.
4. **`media.duration`**: nullable TEXT — confirm format (`hh:mm:ss` vs ISO 8601 vs ticks).
5. **Lazy upsert** of `system_settings` / `directory_settings` / `camera_settings` first-row creation — confirm services initialize defaults vs rely on user-driven inserts.
@@ -1,72 +0,0 @@
# CI / CD Pipeline
Source of truth: `.woodpecker/build-arm.yml`.
## Engine
Woodpecker CI. No GitHub Actions / GitLab CI / Azure Pipelines configured in this repo — `.github/workflows/` is absent (`00_discovery.md`). Suite-wide CI may layer on top of this; that lives outside the workspace.
## Trigger
```yaml
when:
event: [push, manual]
branch: [dev, stage, main]
```
- Builds run on push to **`dev`**, **`stage`**, or **`main`**, plus manual triggers.
- Other branches do **not** build images.
## Runner constraint
```yaml
labels:
platform: arm64
```
Pipeline pins to ARM64 runners. The Dockerfile is multi-arch capable but this pipeline only builds `arm64`.
## Steps (single step `build-push`)
1. Login to private registry using secrets `registry_host`, `registry_user`, `registry_token`.
2. Compute `TAG=${CI_COMMIT_BRANCH}-arm` and `BUILD_DATE` (`date -u +%Y-%m-%dT%H:%M:%SZ`).
3. `docker build -f src/Dockerfile` with build args + OCI labels:
- `--build-arg CI_COMMIT_SHA=$CI_COMMIT_SHA`
- `--label org.opencontainers.image.revision=$CI_COMMIT_SHA`
- `--label org.opencontainers.image.created=$BUILD_DATE`
- `--label org.opencontainers.image.source=$CI_REPO_URL`
- tag: `$REGISTRY_HOST/azaion/annotations:$TAG`
4. `docker push` of that tag.
5. Mounts `/var/run/docker.sock` into the build container (Docker-out-of-Docker pattern).
## Image tagging
Per branch:
| Branch | Image tag |
|--------|-----------|
| `dev` | `dev-arm` |
| `stage` | `stage-arm` |
| `main` | `main-arm` |
Tags are **mutable** — every push to a branch overwrites the prior image at that tag. No immutable revision-tagged images are produced today (`main-arm-${SHA}` is not pushed). Adding immutable tags would simplify rollback and trace-back from a running image to a commit.
## Secrets
| Secret | Purpose |
|--------|---------|
| `registry_host` | Registry hostname (also used in pushed image FQN) |
| `registry_user` | Registry login user |
| `registry_token` | Registry login token (used via `--password-stdin`) |
Secrets are referenced via `from_secret:` and never echoed.
## What CI does NOT do today
- No tests run (no test project exists in repo per `00_discovery.md`).
- No linters / format checks (`dotnet format`).
- No `amd64` image.
- No scan (Trivy / Grype) on the produced image.
- No automated rollback on failed deploy (deploy itself is out of pipeline scope).
These are gaps to track when the test project is added in autodev Phase A Step 6.
@@ -1,53 +0,0 @@
# Containerization
Source of truth: `src/Dockerfile`.
## Build
Two-stage build:
1. **build stage**`mcr.microsoft.com/dotnet/sdk:10.0`, `--platform=$BUILDPLATFORM`. Reads `$TARGETARCH` and runs `dotnet publish -c Release -o /app --os linux --arch $arch` (mapping `amd64 → x64`, otherwise `$TARGETARCH`).
2. **runtime stage**`mcr.microsoft.com/dotnet/aspnet:10.0`. Copies the published output, exposes port `8080`, sets `ENTRYPOINT ["dotnet", "Azaion.Annotations.dll"]`.
## Build arguments
| Arg | Default | Purpose |
|-----|---------|---------|
| `BUILDPLATFORM` | provided by Buildx | Multi-arch host platform |
| `TARGETARCH` | provided by Buildx | Output arch (`amd64` / `arm64`) |
| `CI_COMMIT_SHA` | `unknown` | Stamped into `AZAION_REVISION` env at runtime |
## Runtime
| Aspect | Value |
|--------|-------|
| Base image | `mcr.microsoft.com/dotnet/aspnet:10.0` |
| Working dir | `/app` |
| Exposed port | `8080` (HTTP) |
| Entry point | `dotnet Azaion.Annotations.dll` |
| Runtime env stamped at build | `AZAION_REVISION = $CI_COMMIT_SHA` |
## Multi-arch
Dockerfile is multi-arch capable via Buildx. The current Woodpecker pipeline emits **`arm64` only** (label `platform: arm64`, tag `${BRANCH}-arm`). Producing `amd64` requires an additional pipeline (or extending the existing one to a matrix).
## Image size & caching
- Layers: SDK install → `COPY . .` → publish → runtime copy. The final layer is the published `/app` directory only — no SDK in runtime image.
- Cache hit on `COPY . .` is wide (entire `src/`); finer caching (e.g., `COPY *.csproj` first, then `dotnet restore`, then sources) is **not configured** — improvement candidate.
## Image labels
Set in CI (`.woodpecker/build-arm.yml`), not in the Dockerfile:
- `org.opencontainers.image.revision = $CI_COMMIT_SHA`
- `org.opencontainers.image.created = $BUILD_DATE`
- `org.opencontainers.image.source = $CI_REPO_URL`
These follow the OCI standard so the registry surfaces them correctly.
## Open items
- Add `amd64` build target if non-ARM hosts are required.
- Consider non-root user inside the runtime image (none configured today).
- Consider `dotnet restore` cache layer split for faster CI builds.
@@ -1,75 +0,0 @@
# Environment Strategy
Source of truth: `src/Program.cs` + `src/Database/DatabaseMigrator.cs` + `.woodpecker/build-arm.yml`.
## Environments
Branch-driven from CI:
| Branch | Image tag | Intended environment |
|--------|-----------|----------------------|
| `dev` | `dev-arm` | Development (shared) |
| `stage` | `stage-arm` | Pre-production |
| `main` | `main-arm` | Production |
The service binary is identical across environments — all variation is **runtime configuration via env vars** (no per-environment build flags).
## Configuration sources (priority order, per `Program.cs`)
1. `Environment.GetEnvironmentVariable("KEY")`.
2. ASP.NET Core `IConfiguration` (`builder.Configuration["KEY"]`) — covers `appsettings.json`, command-line args, etc.
3. **No hard-coded fallback for security-sensitive values.** `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`, and (in `Production`) a non-empty `CorsConfig:AllowedOrigins` are required; missing values cause startup to fail fast via `ConfigurationResolver.ResolveRequiredOrThrow` / `CorsConfigurationValidator.EnsureSafeForEnvironment`.
## Required environment variables
| Variable | Purpose | Default | Production action |
|----------|---------|---------|---------------------|
| `DATABASE_URL` | Postgres connection (URL or LinqToDB conn string) | — (required, fail-fast) | **MUST set** |
| `JWT_ISSUER` | Expected `iss` claim; must match admin's `JwtConfig:Issuer` | — (required, fail-fast) | **MUST set** |
| `JWT_AUDIENCE` | Expected `aud` claim; must match admin's `JwtConfig:Audience` | — (required, fail-fast) | **MUST set** |
| `JWT_JWKS_URL` | Admin's JWKS endpoint (HTTPS) | — (required, fail-fast) | `https://admin.azaion.com/.well-known/jwks.json` |
| `CorsConfig__AllowedOrigins__0` | First allowed CORS origin (array via `__N` indices) | — | **MUST set** (or `CorsConfig__AllowAnyOrigin=true`) in Production |
| `CorsConfig__AllowAnyOrigin` | Opt-in to permissive CORS (non-production only) | `false` | Leave `false` in Production |
| `RABBITMQ_HOST` | Stream host | `127.0.0.1` | Override |
| `RABBITMQ_STREAM_PORT` | Stream port | `5552` | Override if non-default |
| `RABBITMQ_PRODUCER_USER` | Stream user | `azaion_producer` | Override |
| `RABBITMQ_PRODUCER_PASS` | Stream password | `producer_pass` | Override |
| `RABBITMQ_STREAM_NAME` | Stream name | `azaion-annotations` | Usually keep (suite contract) |
`JWT_SECRET` was removed in this cycle — annotations no longer mints HS256 tokens; admin is the sole token issuer (ES256).
## URL format conversion
`Program.cs` accepts `DATABASE_URL` either as a Linq2DB connection string or as a `postgresql://user:pass@host:port/db` URL. The `ConvertPostgresUrl` helper rewrites the URL form into LinqToDB conn-string form. This means operators can use either ENV-style URLs (kubectl/Postgres operator output) or `Host=...` directly.
## DB-driven configuration
Several runtime concerns are stored in **database tables**, not env:
- **Filesystem roots**`directory_settings` (defaults `/data/...`). Updated via `PUT /settings/directories`; **must trigger** `PathResolver.Reset` for the change to take effect (Flow F7).
- **System settings**`system_settings` (`generate_annotated_image`, `silent_detection`, thumbnail dimensions).
- **User settings**`user_settings` (per UI session prefs).
Operators changing filesystem layout in production need an `ADM` JWT and the right cluster connectivity, **not** a redeploy.
## Filesystem mounts
The container expects `/data/` (or whatever `directory_settings` points at) to be a **writable persistent mount**:
- `/data/images` — annotation full images
- `/data/labels` — YOLO `.txt` files
- `/data/thumbnails` — thumbnails
- `/data/results` — annotated images (when `generate_annotated_image=true`)
- `/data/videos` — media uploads
- `/data/gps_sat`, `/data/gps_route` — GPS overlays
Without these mounts, every annotation-create / media-upload flow returns 500 from `ErrorHandlingMiddleware` (FS write fails).
## Config drift between environments
Today, environment-specific config is held wherever the deployment platform places env vars (Helm values / Kustomize overlays / Compose files in `_infra/`). This repo intentionally does not commit per-environment values; the only environment-aware file in-repo is `.woodpecker/build-arm.yml`.
## Open items
- No `appsettings.Production.json` — all env-specific config is operator-supplied.
- `Swagger UI` is mounted in all environments (ADR-005); production exposure must be controlled at the perimeter.
@@ -1,55 +0,0 @@
# Observability
Source of truth: `src/Program.cs` (no dedicated logging config files exist in repo).
## Health check
```csharp
app.MapGet("/health", () => Results.Ok(new { status = "healthy" }));
```
- Path: `GET /health`
- Auth: none (`MapGet` bypasses controller-level `[Authorize]`).
- Response: `200 { "status": "healthy" }`
- **Liveness only**: this endpoint does not probe the DB, RabbitMQ, or filesystem. A pod can return healthy while the failsafe outbox is unable to publish or while DB connectivity is broken.
## API documentation
- `app.UseSwagger()` and `app.UseSwaggerUI()` mounted unconditionally (ADR-005).
- Endpoints: `/swagger/v1/swagger.json` (OpenAPI), `/swagger/index.html` (UI).
- No version pinning of the OpenAPI document (Swashbuckle defaults).
## Logging
- Default ASP.NET Core console logger. No `appsettings.json` overrides in repo.
- No structured logger (Serilog / NLog) configured.
- No correlation id middleware in repo (`X-Request-Id` not propagated).
## Metrics
None configured today. Possible additions:
- `prometheus-net` exporter on `/metrics`.
- ASP.NET Core `MetricsCollector` (built-in HTTP / runtime counters).
## Traces
None configured. OpenTelemetry SDK is not referenced in `csproj`.
## Image revision stamp
The runtime container has `AZAION_REVISION = $CI_COMMIT_SHA` set as an env var (Dockerfile + Woodpecker pipeline). This makes "what's running?" diagnosable from inside the container with `printenv AZAION_REVISION` or by surfacing it in a future `/info` endpoint.
## Error visibility to clients
`ErrorHandlingMiddleware` maps exceptions to JSON `{ statusCode, message }` with HTTP 400 / 404 / 409 / 500. Internal exception details are not leaked beyond the `message` string (confirm during Step 4 verification — make sure 500 paths do not echo stack traces).
## Open observability items
- **Readiness vs liveness split**: today there is one endpoint that does not check dependencies. A `GET /ready` that pings DB and (optionally) RabbitMQ would unblock proper rolling-update gates.
- **Structured logs** with request id correlation across HTTP + outbox drain + SSE.
- **Outbox depth metric** (`COUNT(*)` on `annotations_queue_records`) — surfaces stuck-failsafe scenarios early.
- **SSE subscriber count metric** — visibility into connected UIs.
- **Stream publish lag** — time from outbox row insertion to RabbitMQ publish.
- **Failure injection / chaos hooks** — none today.
These are candidates for the deploy-and-retro phase of autodev (Steps 1417 once the project enters Phase B).
-44
View File
@@ -1,44 +0,0 @@
# Component diagram (Azaion.Annotations)
Derived from the **six-component** breakdown (user choice **B**: Annotations REST split from realtime + RabbitMQ sync).
```mermaid
flowchart LR
subgraph platform [06 Platform]
DB[(PostgreSQL)]
AUTH[JWT / refresh]
PATH[Paths + errors]
end
subgraph media [03 Media]
MAPI["/media"]
end
subgraph annRest [01 Annotations REST]
ARAPI["/annotations REST + files"]
end
subgraph annRT [02 Annotations realtime and sync]
SSE["SSE /events"]
RMQ[RabbitMQ stream]
end
subgraph dataset [04 Dataset]
DAPI["/dataset DATASET"]
end
subgraph settings [05 Settings and metadata]
SAPI["/settings /classes"]
end
platform --> media
platform --> annRest
platform --> annRT
platform --> dataset
platform --> settings
media --> annRest
annRest --> annRT
annRest --> dataset
```
**Shared source file:** `AnnotationsController.cs` is **split by concern** between **01** (REST + static files) and **02** (SSE `Events` action).
@@ -1,83 +0,0 @@
# Flow F1 — Annotation Create
Cross-reference: `system-flows.md` → Flow F1.
## Sequence (verified against `Services/AnnotationService.cs`)
```mermaid
sequenceDiagram
autonumber
participant Caller as Detections / UI
participant Ctrl as AnnotationsController (01)
participant Svc as AnnotationService (01)
participant Path as PathResolver (06)
participant DB as PostgreSQL (06)
participant FS as Filesystem
participant Evt as AnnotationEventService (02)
participant Q as annotations_queue_records (DB / 02)
Caller->>Ctrl: POST /annotations (CreateAnnotationRequest, JWT ANN)
Ctrl->>Svc: CreateAnnotation(request, userIdFromJwt)
alt request.Image bytes provided
Svc->>Svc: ComputeHash (XxHash64 over sampled bytes) -> id
Svc->>FS: write {id}.jpg under images_dir
Svc->>DB: SELECT media WHERE id = :id
opt media row missing
Svc->>DB: INSERT media (Image, MediaStatus.New, ...)
end
else MediaId provided
Svc->>DB: SELECT media WHERE id = :MediaId (404 if missing)
opt source media file exists & target image missing
Svc->>FS: copy media.Path -> images_dir/{id}.jpg
end
end
Svc->>DB: INSERT annotations
Svc->>DB: BulkCopy detection rows
Svc->>FS: write {id}.txt (YOLO label) under labels_dir
Svc->>Evt: PublishAsync(AnnotationEventDto)
Svc->>DB: SELECT system_settings (FirstOrDefault)
alt SilentDetection != true
Svc->>Q: FailsafeProducer.EnqueueAsync(db, id, QueueOperation.Created)
end
Svc-->>Ctrl: Annotation
Ctrl-->>Caller: 201 Created (Location: /annotations/{id})
```
## Flowchart
```mermaid
flowchart TD
start([POST /annotations]) --> auth{JWT valid + ANN claim?}
auth -->|no| rej401([401 / 403])
auth -->|yes| input{bytes or MediaId?}
input -->|neither| arg([400 ArgumentException])
input -->|bytes| hash[ComputeHash sampled XxHash64 -> id]
input -->|MediaId| lookupMedia[SELECT media WHERE id = MediaId]
lookupMedia -->|missing| nf404([404 KeyNotFound])
lookupMedia -->|exists| copyImg[copy media.Path to images dir if missing]
hash --> writeImg[write {id}.jpg]
writeImg --> mediaRow[INSERT media if absent]
mediaRow --> writeDb
copyImg --> writeDb[INSERT annotations + BulkCopy detections]
writeDb --> writeLabel[write {id}.txt YOLO label]
writeLabel --> sse[PublishAsync SSE event]
sse --> readSettings[SELECT system_settings]
readSettings --> silentChk{SilentDetection?}
silentChk -->|yes| ok([201 Created])
silentChk -->|no| outbox[FailsafeProducer.EnqueueAsync Created]
outbox --> ok
writeImg -->|IOException| err500([500 via ErrorHandlingMiddleware])
writeDb -->|DB error| err500
writeLabel -->|IOException| err500
outbox -->|DB error| err500
```
## Notes
- Image hashing is `XxHash64` over a **sampled** input (length prefix + head/middle/tail 1KB) for inputs > 3072 bytes. See ADR-004 in `architecture.md` for collision implications.
- The implementation is **not transactional across FS + DB + outbox**. Partial failure can leave orphan files or unsent outbox rows. Captured in `system-flows.md` → Open Behavioral Questions §4.
- `Update`, `UpdateStatus`, `DeleteAnnotation` paths do **NOT** publish SSE or enqueue outbox today. Captured in `system-flows.md` → Open Behavioral Questions §1.
- Outbox row is consumed asynchronously by Flow F4 (`FailsafeProducer`).
@@ -1,52 +0,0 @@
# Flow F4 — Failsafe Outbox Drain → RabbitMQ Stream
Cross-reference: `system-flows.md` → Flow F4.
## Sequence
```mermaid
sequenceDiagram
autonumber
participant FP as FailsafeProducer (02, IHostedService)
participant DB
participant Path as PathResolver (06)
participant FS as Filesystem
participant RMQ as RabbitMQ Stream "azaion-annotations"
loop while host running
FP->>DB: SELECT FROM annotations_queue_records
DB-->>FP: pending rows (operation, annotation_ids JSON)
loop per row
alt operation = Created
FP->>Path: GetImagePath(annotationId)
FP->>FS: read bytes
end
FP->>FP: serialize MessagePack (Annotation*QueueMessage)
FP->>RMQ: publish stream entry
alt publish ok
FP->>DB: DELETE annotations_queue_records WHERE id = :id
else stream unavailable
FP->>FP: log + backoff
end
end
end
```
## State
```mermaid
stateDiagram-v2
[*] --> Idle
Idle --> Draining: queue rows present
Draining --> Publishing: row picked
Publishing --> Acked: stream publish ok
Acked --> Idle: row deleted
Publishing --> Backoff: stream unavailable
Backoff --> Idle: backoff elapsed
```
## Notes
- See ADR-003 in `architecture.md` for rationale.
- Multi-instance drain: no leasing in DB → duplicate stream entries possible. Suite consumer contract should dedupe.
- Bulk message (`AnnotationBulkQueueMessage`) carries multiple annotation ids; `Created` semantics on bulk are out of scope here — confirm during Step 4 verification.
@@ -1,43 +0,0 @@
# Flow F3 — Real-time SSE Subscription
Cross-reference: `system-flows.md` → Flow F3.
## Sequence
```mermaid
sequenceDiagram
autonumber
participant UI
participant Ctrl as AnnotationsController.Events (component 02 doc-ownership)
participant Evt as AnnotationEventService (02)
participant ProducerF1 as Flow F1 (annotation create)
participant ProducerF8 as Flow F8 (dataset bulk status)
UI->>Ctrl: GET /annotations/events (Accept: text/event-stream, JWT ANN)
Ctrl->>Ctrl: set Content-Type: text/event-stream, no-cache
Ctrl->>Evt: ReadAllAsync(cancellationToken)
par event sources
ProducerF1->>Evt: PublishAsync(eventDto)
ProducerF8->>Evt: PublishAsync(eventDto)
end
Evt-->>Ctrl: yield AnnotationEventDto
Ctrl-->>UI: data: {json}\n\n
UI--xCtrl: client disconnects
Ctrl->>Ctrl: cancellation token fires; loop exits
```
## State
```mermaid
stateDiagram-v2
[*] --> Subscribing
Subscribing --> Streaming: header sent + reader attached
Streaming --> Streaming: PublishAsync -> data frame
Streaming --> Closed: client cancel / process restart
Closed --> [*]
```
## Notes
- Channel is **unbounded**: a slow client cannot back-pressure the producer. If a client stalls indefinitely, memory growth is bounded by per-publisher cancellation tokens at the controller level. Step 4 verification candidate.
- Cross-pod fan-out is **not provided** — each pod has its own channel. Sticky sessions or a broker-backed bus required for horizontal scale.
-73
View File
@@ -1,73 +0,0 @@
# Glossary
**Status**: confirmed-by-user 2026-05-14.
System-wide terminology for `Azaion.Annotations`. Generic CS / industry terms (HTTP, JWT mechanics, REST, etc.) are excluded — only project-specific or domain-specific terms are listed. Each entry cites the doc or source file that establishes it.
---
**Annotation** — Hash-keyed record carrying detections, status, source, user, and time, attached to a media row. Central object of the service. *source: `data_model.md`, `modules/annotations-service.md`.*
**Annotation event** — SSE payload (`AnnotationEventDto`) describing a lifecycle change broadcast to UI subscribers. *source: `modules/sse-realtime.md`, `DTOs/AnnotationEventDto.cs`.*
**AnnotationSource** — Wire enum: `AI = 0`, `Manual = 1`. *source: `Enums/AnnotationSource.cs`.*
**AnnotationStatus** — Wire enum: `None = 0`, `Created = 10`, `Edited = 20`, `Validated = 30`, `Deleted = 40`. Soft-delete uses value 40 (per ADR-009). *source: `Enums/AnnotationStatus.cs`.*
**Annotator UI** — Operator-facing client of `01 Annotations REST` + SSE. Active editing surface. *source: `components/01_annotations-rest/description.md`.*
**Bulk status** — Multi-id status update via `POST /dataset/bulk-status` carrying `BulkStatusRequest { AnnotationIds, Status }`. *source: `Controllers/DatasetController.cs:34`.*
**Business transaction** — The lifecycle-level transactional boundary planned per ADR-008: DB rows + outbox commit atomically; FS writes and SSE publish run post-commit using the outbox row as the durable promise. *source: `architecture.md` ADR-008.*
**Camera settings** — Per-camera calibration (`altitude`, `focal_length`, `sensor_width`) used by detection geometry. *source: `data_model.md`, `Database/Entities/CameraSettings.cs`.*
**Combat readiness** — Wire enum on a detection (`CombatReadiness`). *source: `Enums/CombatReadiness.cs`, `modules/wire-enums.md`.*
**Dataset Explorer** — Read-heavy UI exposed under `/dataset` (policy `DATASET`). *source: `components/04_dataset/description.md`, suite `09_dataset_explorer.md`.*
**Detection** — Bounding box (`center_x/y, width, height`) + class number + label + affiliation + combat readiness, child of an annotation. *source: `data_model.md`, `Database/Entities/Detection.cs`.*
**Detection class** — Row in `detection_classes` (id, name, short_name, color, max_size_m, photo_mode). 19 rows seeded by the migrator; becoming admin-managed per RB-06. *source: `data_model.md`, `Database/DatabaseMigrator.cs`.*
**Directory settings** — DB-driven filesystem roots (`videos_dir`, `images_dir`, `labels_dir`, `thumbnails_dir`, `results_dir`, `gps_sat_dir`, `gps_route_dir`). Consumed via `PathResolver`. RB-01 will add `deleted_dir` for soft-delete relocation. *source: `data_model.md`, `Database/DatabaseMigrator.cs`, `modules/common-infrastructure.md`.*
**Failsafe outbox** — `annotations_queue_records` table; the durable bridge between local writes and the RabbitMQ stream. Drained by `FailsafeProducer`. *source: `architecture.md` ADR-003, `modules/rabbitmq-stream-sync.md`.*
**Flight** — *Deprecated synonym for Mission.* The codebase currently uses `FlightId` (DTOs and service queries) but will rename to `MissionId` per RB-07 to align with the suite spec. *source: `00_discovery.md` drift list, ADR-012.*
**JWT policies** — Authorization claims `ANN`, `DATASET`, `ADM` checked by `[Authorize(Policy = ...)]` on controllers. *source: `modules/auth-identity.md`, `Auth/JwtExtensions.cs`.*
**Media** — Uploaded image / video reference, waypoint-scoped, written via `MediaController`. *source: `data_model.md`, `components/03_media/description.md`.*
**MessagePack** — Wire encoding for outbox messages on the RabbitMQ stream (`AnnotationQueueMessage`, `AnnotationBulkQueueMessage`). Gzip-compressed at the producer. *source: `modules/rabbitmq-stream-sync.md`, `Services/FailsafeProducer.cs`.*
**Mission** — *Canonical domain term* per the suite spec — the logical grouping that the codebase currently calls "Flight" and that physically backs onto `media.waypoint_id`. The code → suite alignment is RB-07 / ADR-012; the suite remains canonical. *source: `suite/_docs/01_annotations.md`, `00_discovery.md`.*
**PathResolver** — DI singleton that lazy-loads filesystem roots from `directory_settings` and exposes per-annotation paths (image / label / thumbnail / result). Calls `Reset()` after directory updates. *source: `modules/common-infrastructure.md`, `Services/PathResolver.cs`.*
**QueueOperation** — Outbox enum: `Created = 0`, `Validated = 1`, `Deleted = 2`. RB-01 may add `Updated` for `UpdateAnnotation` semantics. *source: `Enums/QueueOperation.cs`.*
**RabbitMQ Stream `azaion-annotations`** — Durable export channel consumed by the admin sync worker and the AI training pipeline. Default port `5552`. *source: `architecture.md` ADR-003, `Program.cs:43`.*
**Refresh token** — Long-lived credential issued and rotated by the **admin** service. Annotations is a verifier only — it neither mints nor refreshes tokens. Long-running callers (e.g. the detections service) refresh against admin's `POST /token/refresh` and pass the resulting ES256 access token to annotations. *source: `modules/auth-identity.md`.*
**Silent detection** — *Deprecated.* Boolean flag on `system_settings` that gated outbox enqueue during development debugging. Scheduled for removal per ADR-010 / RB-02 — the suite e2e harness covers this need now. *source: `architecture.md` ADR-010.*
**Soft-delete** — `DeleteAnnotation` semantics agreed on 2026-05-14: status flips to `AnnotationStatus.Deleted = 40`, the annotation row stays, and image / label / thumbnail files relocate to `deleted_dir`. RB-01 implements this; today's code is hard-delete. *source: `architecture.md` ADR-009 / RB-01.*
**SSE (Server-Sent Events)** — `text/event-stream` channel on `GET /annotations/events` carrying `AnnotationEventDto` payloads. In-process, per-instance; no cross-pod fan-out. *source: `modules/sse-realtime.md`, `Controllers/AnnotationsController.cs`.*
**System settings** — Singleton-ish service-config row (`thumbnail_*`, `generate_annotated_image`, etc.). *source: `data_model.md`.*
**Thumbnail** — Per-annotation small image at `thumbnails_dir/{id}.jpg`. **Not produced by `CreateAnnotation`** — read-only via `PhysicalFile`; populated out-of-band today. *source: `system-flows.md` Flow F1, F2.*
**Transactional outbox** — Pattern adopted in ADR-008: a queue table populated inside a DB transaction, drained asynchronously by a background worker (`FailsafeProducer`), used to bridge local commits to a remote stream durably. *source: `architecture.md` ADR-003, ADR-008.*
**User settings** — Per-user UI prefs (selected flight / mission, panel widths). Unique on `user_id`. *source: `data_model.md`, `Database/Entities/UserSettings.cs`.*
**Waypoint** — UUID associated with media uploads, used for mission-scoped grouping. Physical foreign key under the logical "Mission" concept. *source: `Database/Entities/Media.cs`.*
**World B** — Internal label for the agreed lifecycle-observability stance: every annotation mutation publishes SSE and enqueues the outbox, not just `Create`. *source: `architecture.md` ADR-009.*
**YOLO label** — Plain-text format used in `{id}.txt` files: one detection per line, fields `class cx cy w h` (normalized box). *source: `Services/AnnotationService.cs:243249`, `modules/annotations-service.md`.*
-178
View File
@@ -1,178 +0,0 @@
# Module Layout
**Status**: derived-from-code
**Language**: csharp
**Layout Convention**: single-assembly (`Azaion.Annotations`), vertical slices expressed as **logical components** under flat `src/` folders (`Controllers/`, `Services/`, `DTOs/`).
**Root**: `src/`
**Last Updated**: 2026-05-14
## Verification Needed
1. **No per-component physical root** — all components share `src/Controllers/`, `src/Services/`, `src/DTOs/`. File ownership below is **exclusive for implementation planning**; merges touching the same file need explicit batch ordering.
2. **`AnnotationsController.cs` split (user-approved six-component model)** — **REST + static files** belong to **01 Annotations REST**; **SSE** (`Events`) belongs to **02 Annotations realtime & sync**. For `/implement`, treat **`src/Controllers/AnnotationsController.cs` as owned by 01**; tasks that **only** change SSE must still edit this file — flag as **cross-component** (01 + 02) or split into partial classes in a future refactor.
3. **`src/DTOs/`** — no subfolders; each file has a **primary owner** in [Shared: DTO files](#shared-dto-files-primary-ownership) to resolve FORBIDDEN vs OWNED during tasks.
4. **`FailsafeProducer.cs`** contains `RabbitMqConfig` and `FailsafeProducer` — fully owned by **02** (even though `Program.cs` registers the config as singleton).
---
## Layout Rules (adapted for this repo)
1. Components map to **logical** slices from `_docs/02_document/components/*/description.md`, not to separate top-level directories.
2. **Foundation** (`06_platform`) owns schema, enums, auth registration helpers, middleware, path resolution, and **composition** (`Program.cs`, csproj, Dockerfile).
3. **Feature** components own listed **service + controller** files; they **read** foundation public APIs and **shared DTO** types per the DTO table.
4. Tests are **not present** in-repo; future test project should follow `tests/Azaion.Annotations.Tests/` (conventional) — not owned by feature slices until created.
---
## Per-Component Mapping
### Component: `01_annotations-rest`
- **Epic**: (assign per change; layout is structural)
- **Directory (primary)**: `src/Services/` (partial), `src/Controllers/` (partial)
- **Public API** (types other components may reference through DI / same assembly):
- `Azaion.Annotations.Services.AnnotationService`
- `Azaion.Annotations.Controllers.AnnotationsController` (REST + image/thumbnail actions only — see Verification)
- **Internal**: private methods inside owned types; do not reach into other components services from new code without updating this layout.
- **Owns (exclusive write scope)**:
- `src/Services/AnnotationService.cs`
- `src/Controllers/AnnotationsController.cs`**primary file owner** (REST, `GetThumbnail`, `GetImage`; coordinate with 02 for `Events`)
- **Imports from**: `06_platform` (Database, Enums, DTOs used here, PathResolver, Middleware indirectly), `02_annotations-realtime-sync` (`AnnotationEventService` for publish)
- **Consumed by**: `04_dataset` (read paths share DB entities; no direct `AnnotationService` reference required), external callers
### Component: `02_annotations-realtime-sync`
- **Epic**: (assign per change)
- **Directory (primary)**: `src/Services/` (partial)
- **Public API**:
- `Azaion.Annotations.Services.AnnotationEventService`
- `Azaion.Annotations.Services.RabbitMqConfig`
- `Azaion.Annotations.Services.FailsafeProducer`
- `FailsafeProducer.EnqueueAsync` (static helper on same type as producer)
- **Owns**:
- `src/Services/FailsafeProducer.cs` (includes `RabbitMqConfig` + `FailsafeProducer`)
- `src/Services/AnnotationEventService.cs`
- **SSE contract**: `AnnotationsController.Events` (same `.cs` as 01 — see Verification)
- **Imports from**: `06_platform` (Database, Entities, PathResolver, Enums, `DTOs/QueueMessages.cs`), `DTOs/AnnotationEventDto.cs` (see Shared)
- **Consumed by**: `01_annotations-rest` (publishes events), external RabbitMQ consumers
### Component: `03_media`
- **Epic**: (assign per change)
- **Public API**: `MediaService`, `MediaController`
- **Owns**:
- `src/Services/MediaService.cs`
- `src/Controllers/MediaController.cs`
- **Imports from**: `06_platform`
- **Consumed by**: `01_annotations-rest` (domain: media rows referenced by annotations), UI
### Component: `04_dataset`
- **Epic**: (assign per change)
- **Public API**: `DatasetService`, `DatasetController`
- **Owns**:
- `src/Services/DatasetService.cs`
- `src/Controllers/DatasetController.cs`
- **Imports from**: `06_platform`
- **Consumed by**: Dataset Explorer UI
### Component: `05_settings-metadata`
- **Epic**: (assign per change)
- **Public API**: `SettingsService`, `SettingsController`, `ClassesController`
- **Owns**:
- `src/Services/SettingsService.cs`
- `src/Controllers/SettingsController.cs`
- `src/Controllers/ClassesController.cs`
- **Imports from**: `06_platform`
- **Consumed by**: UI; `PathResolver` / directory settings (via DB) interact with **06** cache reset when dirs change
### Component: `06_platform`
- **Epic**: (assign per change)
- **Public API** (representative; all `public` types in these areas are the integration surface):
- `Azaion.Annotations.Database.AppDataConnection`, `DatabaseMigrator`, `Azaion.Annotations.Database.Entities.*`
- `Azaion.Annotations.Enums.*`
- `Azaion.Annotations.Middleware.ErrorHandlingMiddleware`
- `Azaion.Annotations.Auth.JwtExtensions`
- `Azaion.Annotations.Infrastructure.ConfigurationResolver`, `CorsConfigurationValidator`
- `Azaion.Annotations.Services.PathResolver`
- `Program` (implicit entry), `src/Program.cs`
- **Owns**:
- `src/Enums/**`
- `src/Database/**`
- `src/Middleware/**`
- `src/Auth/**`
- `src/Infrastructure/**`
- `src/Services/PathResolver.cs`
- `src/GlobalUsings.cs`
- `src/Program.cs`
- `src/Azaion.Annotations.csproj`
- `src/Dockerfile`
- **Imports from**: (none — foundation)
- **Consumed by**: all other components
---
## Shared: DTO files (primary ownership)
| File under `src/DTOs/` | Primary component | Notes |
|------------------------|-------------------|--------|
| `PaginatedResponse.cs` | `06_platform` | Generic list wrapper — cross-cutting |
| `ErrorResponse.cs` | `06_platform` | Shared error shape |
| `CreateAnnotationRequest.cs` | `01_annotations-rest` | |
| `UpdateAnnotationRequest.cs` | `01_annotations-rest` | |
| `UpdateStatusRequest.cs` | `01_annotations-rest` / `04_dataset` | **Shared type****01** owns file edits; `04_dataset` uses for PATCH |
| `GetAnnotationsQuery.cs` | `01_annotations-rest` | |
| `AnnotationListItem.cs` | `01_annotations-rest` | |
| `DetectionDto.cs` | `01_annotations-rest` | |
| `AnnotationEventDto.cs` | `02_annotations-realtime-sync` | SSE payload |
| `QueueMessages.cs` | `02_annotations-realtime-sync` | MessagePack stream payloads |
| `CreateMediaRequest.cs` | `03_media` | |
| `GetMediaQuery.cs` | `03_media` | |
| `MediaListItem.cs` | `03_media` | |
| `GetDatasetQuery.cs` | `04_dataset` | |
| `DatasetItem.cs` | `04_dataset` | |
| `ClassDistributionItem.cs` | `04_dataset` | |
| `BulkStatusRequest.cs` | `04_dataset` | |
| `UpdateSystemSettingsRequest.cs` | `05_settings-metadata` | |
| `UpdateDirectoriesRequest.cs` | `05_settings-metadata` | |
| `UpdateCameraSettingsRequest.cs` | `05_settings-metadata` | |
| `UpdateUserSettingsRequest.cs` | `05_settings-metadata` | |
---
## Shared / Cross-Cutting (non-DTO)
### `common-helpers/01_http-error-envelope.md`
- **Purpose**: Documents middleware as cross-cutting (see `_docs/02_document/common-helpers/`).
- **Owned by**: tasks touching **06_platform** (`ErrorHandlingMiddleware`).
- **Consumed by**: all HTTP components.
---
## Allowed Dependencies (layering)
Higher layers may depend on lower; **not** the reverse. Same-layer components should not introduce compile-time cycles (current codebase: none detected).
| Layer | Components | May import from (namespaces / types from) |
|-------|------------|---------------------------------------------|
| 1 — Foundation | `06_platform` | *(none)* |
| 2 — Realtime infra | `02_annotations-realtime-sync` | `06_platform` |
| 3 — Application features | `01_annotations-rest`, `03_media`, `04_dataset`, `05_settings-metadata` | `06_platform`, and **`01` additionally `02`** (`AnnotationEventService`) |
**Rules**
- `03`, `04`, `05`**must not** reference `AnnotationService` / `MediaService` across features without an explicit API (today: only via shared `AppDataConnection` in same assembly — acceptable but treat as **tight coupling**; prefer domain services for new code).
- `02`**must not** reference `01` service types (no reverse dependency today).
Violations are **Architecture** findings for code-review Phase 7.
---
## Layout Conventions (reference)
| Language | Root | This repo |
|----------|------|-----------|
| C# (.NET) | `src/` | Single web project; vertical slices = **logical** component rows above + DTO table |
-19
View File
@@ -1,19 +0,0 @@
# Module documentation index
Modules follow **`suite/_docs/01_annotations.md`**: annotations vs media, SSE, auth/JWT refresh, DB, RabbitMQ sync, plus **dataset** (DATASET) and **settings / detection classes** as implemented in this repo.
| Order | File | Scope |
|------:|------|--------|
| 1 | [wire-enums.md](./wire-enums.md) | `src/Enums/*` |
| 2 | [database-layer.md](./database-layer.md) | `src/Database/*` |
| 3 | [common-infrastructure.md](./common-infrastructure.md) | `PathResolver`, `ErrorHandlingMiddleware`, shared small types |
| 4 | [auth-identity.md](./auth-identity.md) | `JwtExtensions` (JWKS verifier), `ConfigurationResolver`, `CorsConfigurationValidator` |
| 5 | [media-service.md](./media-service.md) | `MediaService`, `MediaController`, media DTOs |
| 6 | [annotations-service.md](./annotations-service.md) | `AnnotationService`, `AnnotationsController` (REST + files) |
| 7 | [dataset-service.md](./dataset-service.md) | `DatasetService`, `DatasetController`, dataset DTOs |
| 8 | [settings-metadata-service.md](./settings-metadata-service.md) | `SettingsService`, `SettingsController`, `ClassesController`, settings DTOs |
| 9 | [sse-realtime.md](./sse-realtime.md) | `AnnotationEventService`, SSE endpoint |
| 10 | [rabbitmq-stream-sync.md](./rabbitmq-stream-sync.md) | `FailsafeProducer`, `RabbitMqConfig`, `QueueMessages` |
| 11 | [composition-program.md](./composition-program.md) | `Program.cs` |
`src/DTOs/` types are described in the module that exposes them on the wire.
@@ -1,30 +0,0 @@
# Module: Annotations service
## Purpose
Core **annotation CRUD**, listing, static image/thumbnail delivery, and coordination with **media** and **files on disk**. Maps to **`01_annotations.md` §16** (not SSE — see `sse-realtime.md`).
## Code
- `AnnotationService` — create/update/status/delete/query/get one; uses `PathResolver`, hashing, label/thumbnail generation, queue handoff to failsafe path as implemented.
- `AnnotationsController``[Route("annotations")]`, `[Authorize(Policy = "ANN")]` except where noted.
- REST: `POST`, `PUT/{id}`, `PATCH/{id}/status`, `DELETE/{id}`, `GET`, `GET/{id}`.
- Files: `GET/{id}/thumbnail`, `GET/{id}/image`.
- **SSE** `GET/events` documented in `sse-realtime.md` (same controller type).
## DTOs (this module)
- `CreateAnnotationRequest`, `UpdateAnnotationRequest`, `UpdateStatusRequest`, `GetAnnotationsQuery`, `AnnotationListItem`, `DetectionDto` (annotation payloads).
## Dependencies
Database, `PathResolver`, optional integration with queue/SSE services.
## Suite vs code (maintain in suite or code)
- **UserId:** suite pseudo-code shows `UserId` on create; **implementation** uses JWT subject (`AnnotationsController`).
- **GET filter:** suite `missionId` vs code `FlightId` + filter behavior — track as open alignment.
## Suite doc
§16; annotation identity at top of `01_annotations.md`.
@@ -1,40 +0,0 @@
# Module: Auth & identity
## Purpose
JWT validation for API policies. Tokens are minted exclusively by the admin service (ES256-signed); annotations is a **verifier only**.
## Components
### `JwtExtensions` (`Auth/JwtExtensions.cs`)
- `AddJwtAuth(IConfiguration)` — pulls `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL` via `ConfigurationResolver.ResolveRequiredOrThrow` (fail-fast at startup if any is missing).
- `TokenValidationParameters` mirrors admin's verifier contract:
- `ValidateIssuer = true` / `ValidateAudience = true` / `ValidateLifetime = true`.
- `ValidAlgorithms = [SecurityAlgorithms.EcdsaSha256]` — pinned so an HS256-forgery using the public key as the HMAC secret cannot pass.
- `RequireSignedTokens = true`, `RequireExpirationTime = true`.
- `ClockSkew = 30s`.
- Signing keys are fetched from admin's `/.well-known/jwks.json` via a `ConfigurationManager<JsonWebKeySet>` backed by a minimal `IConfigurationRetriever<JsonWebKeySet>` (admin exposes JWKS but not the full OIDC discovery document). The manager honours admin's `Cache-Control: public, max-age=3600` and refreshes on the default schedule. During key rotation both `kid`s are present in JWKS so in-flight tokens still verify.
- Policies: `ANN`, `DATASET`, `ADM` — each requires claim `permissions` with that code (matches suite "Required permission: ANN" and Dataset Explorer `DATASET`).
## Dependencies
Configuration (all required, no defaults):
- `JWT_ISSUER` (alt `Jwt:Issuer`) — must match admin's `JwtConfig:Issuer`.
- `JWT_AUDIENCE` (alt `Jwt:Audience`) — must match admin's `JwtConfig:Audience`.
- `JWT_JWKS_URL` (alt `Jwt:JwksUrl`) — `https://admin.azaion.com/.well-known/jwks.json` in production.
## Consumers
All `[Authorize]` controllers.
## Removed in this cycle
- `Services/TokenService.cs` (HS256 minting of access tokens from refresh tokens) — deleted; refresh is now the admin service's responsibility (`POST /token/refresh`).
- `Controllers/AuthController.cs` and the `POST /auth/refresh` endpoint — deleted along with `TokenService`. Detections (and any other client) must call admin's refresh endpoint and pass the returned access token to annotations.
- `JWT_SECRET` env var — no longer read.
## Suite doc
`01_annotations.md` §Annotation Sync (verifier role); suite `10_auth.md` for full auth story (admin = issuer, satellite-provider / annotations / flights / ui = verifiers).
@@ -1,38 +0,0 @@
# Module: Common infrastructure
## Purpose
Cross-cutting **filesystem layout** (annotation images, labels, thumbnails, results), **global error JSON**, and trivial shared API types.
## Components
### `PathResolver` (`Services/PathResolver.cs`)
- Lazy-loads paths from `directory_settings` via `AppDataConnection`.
- Methods: `GetImagePath`, `GetLabelPath`, `GetThumbnailPath`, `GetResultPath`, `GetMediaDir` — paths under configured dirs with `{annotationId}.jpg` / `.txt` patterns.
- `Reset()` clears cache after directory settings change.
### `ErrorHandlingMiddleware` (`Middleware/`)
Maps exceptions to `{ statusCode, message }` JSON (400/404/409/500). Aligns HTTP outcomes with `01_annotations.md` status tables.
### Shared DTOs (`DTOs/`)
- `PaginatedResponse<T>` — list + `totalCount` / `page` / `pageSize` (annotations list, media list, dataset).
- `ErrorResponse` — available for explicit error contracts where used.
### `GlobalUsings.cs`
Project-wide usings only.
## Dependencies
- `PathResolver``AppDataConnection`, `DirectorySettings` entity.
## Consumers
All services and controllers that touch disk or return paged lists.
## Suite doc
File cleanup on DELETE annotation (`GetImgPath` / label / thumb) in `01_annotations.md` §4.
@@ -1,21 +0,0 @@
# Module: Composition (`Program.cs`)
## Purpose
Single **composition root**: configuration, PostgreSQL `AppDataConnection`, service registrations, **JWT**, **CORS**, Swagger, **migrator** on startup, **middleware** order, `MapControllers`, `/health`, `WebApplication.Run`.
## Notable wiring
- `DATABASE_URL` (required, no fallback — startup fails fast via `ConfigurationResolver.ResolveRequiredOrThrow`) → Npgsql connection string helper.
- `JWT_ISSUER` / `JWT_AUDIENCE` / `JWT_JWKS_URL` for `AddJwtAuth` (all required; resolved by `ConfigurationResolver`). The validator pulls public ES256 keys from admin's JWKS endpoint; this service no longer holds an HMAC secret.
- `CorsConfig:AllowedOrigins` / `CorsConfig:AllowAnyOrigin` for the default CORS policy; `CorsConfigurationValidator` refuses to start with a permissive policy in `Production`.
- `RabbitMqConfig` from env + `AddHostedService<FailsafeProducer>()`.
- Scoped services: `AnnotationService`, `MediaService`, `DatasetService`, `SettingsService`, `PathResolver`; singletons: `AnnotationEventService`, `RabbitMqConfig`.
## Dependencies
All modules; documented last after slices are understood.
## Suite doc
Operational/env story complements `01_annotations.md` deployment sections in suite architecture docs.
@@ -1,30 +0,0 @@
# Module: Database layer (`src/Database`)
## Purpose
PostgreSQL schema and Linq2DB mapping for annotations, media, detections, queue buffer, settings, and `detection_classes`. Underpins every HTTP module in `01_annotations.md`.
## Public interface
- `AppDataConnection``ITable<>` for all mapped entities.
- `DatabaseMigrator.Migrate` — embedded SQL: `CREATE TABLE IF NOT EXISTS` / `ALTER … IF NOT EXISTS`, seed detection classes.
## Entities (summary)
- `Annotation`, `Media`, `Detection` — core annotation + YOLO row model (`time` stored as BIGINT ticks).
- `AnnotationsQueueRecord` — failsafe outbox (`operation`, `annotation_ids`).
- `SystemSettings` — includes `GenerateAnnotatedImage`, `SilentDetection` (suite §Annotated Image / Silent Detection).
- `DirectorySettings``/data/...` roots consumed by `PathResolver`.
- `DetectionClass`, `UserSettings`, `CameraSettings`.
## Dependencies
Wire enums on columns.
## Consumers
All services and `ClassesController`.
## Suite doc
Annotation identity and ER-level behavior; cross-check `00_database_schema.md` in suite when entities evolve.
@@ -1,22 +0,0 @@
# Module: Dataset service
## Purpose
**Dataset Explorer** backend: paginated grid, detail, status updates, bulk status — **`[Authorize(Policy = "DATASET")]`** per suite note on PATCH status (`01_annotations.md` §3 points to `09_dataset_explorer.md`).
## Code
- `DatasetService` — queries tuned for dataset views; may reuse annotation entities.
- `DatasetController``[Route("dataset")]`.
## DTOs (this module)
- `GetDatasetQuery`, `DatasetItem`, `ClassDistributionItem`, `BulkStatusRequest` — and shared `UpdateStatusRequest` where used for PATCH.
## Dependencies
Database, same status enums as annotator.
## Suite doc
Primary behavioral spec: `suite/_docs/09_dataset_explorer.md`; permission cross-ref in `01_annotations.md` §3.
@@ -1,27 +0,0 @@
# Module: Media service
## Purpose
HTTP surface and domain logic for **§710** in `01_annotations.md`: create media, batch upload, list, delete, and download raw media file.
## Code
- `MediaService` — persistence + disk writes, batch from `IFormFileCollection`.
- `MediaController``[Route("media")]`, `[Authorize(Policy = "ANN")]`.
- `POST /media`, `POST /media/batch` (form: `waypointId` + files), `GET /media`, `GET /media/{id}/file`, delete route as implemented.
## DTOs (this module)
- `CreateMediaRequest`, `GetMediaQuery`, `MediaListItem` — plus any media-specific shapes used only here.
## Dependencies
`AppDataConnection`, `PathResolver` (media dir), JWT user id from claims.
## Consumers
Annotator UI / React upload flows described in suite §Media Browsing.
## Suite doc
`01_annotations.md` §710; accepted formats table in same doc.
@@ -1,22 +0,0 @@
# Module: RabbitMQ stream sync (failsafe)
## Purpose
**Annotation Sync** outbox and **RabbitMQ Stream** producer — `01_annotations.md` §Annotation Sync, Failsafe Queue, RabbitMQ Stream.
## Code
- `RabbitMqConfig` + `FailsafeProducer` (`Services/FailsafeProducer.cs`) — `BackgroundService`; builds `StreamSystem`, drains `annotations_queue_records`, serializes **MessagePack** payloads (`AnnotationQueueMessage`, `AnnotationBulkQueueMessage` in `DTOs/QueueMessages.cs`), gzip as implemented.
- Entity `AnnotationsQueueRecord` — see `database-layer.md`.
## Dependencies
`AppDataConnection`, `PathResolver` (for image bytes on create), env-driven `RABBITMQ_*` from `Program`.
## Consumers (downstream, external)
Admin `AnnotationSyncWorker`, AI Training consumer — described in suite doc.
## Suite doc
Full sync topology and stream semantics in `01_annotations.md`; keep MessagePack key layout stable.

Some files were not shown because too many files have changed in this diff Show More