mirror of
https://github.com/azaion/loader.git
synced 2026-06-21 06:41:08 +00:00
Oleksandr Bezdieniezhnykh
40 lines
3.1 KiB
Markdown
40 lines
3.1 KiB
Markdown
# Step 3: Blackbox Test Task Decomposition (tests-only mode only)
|
|
|
|
**Role**: Professional Quality Assurance Engineer
|
|
**Goal**: Decompose blackbox test specs into atomic, implementable task specs.
|
|
**Constraints**: Behavioral specs only — describe what, not how. No test code.
|
|
|
|
## Numbering
|
|
|
|
- In tests-only mode: start from 02 (01 is the test infrastructure bootstrap from Step 1t).
|
|
|
|
## Steps
|
|
|
|
1. Read all test specs from `DOCUMENT_DIR/tests/` (`blackbox-tests.md`, `performance-tests.md`, `resilience-tests.md`, `security-tests.md`, `resource-limit-tests.md`)
|
|
2. Group related test scenarios into atomic tasks (e.g., one task per test category or per component under test)
|
|
3. Each task should reference the specific test scenarios it implements and the environment/test-data specs
|
|
4. Add a **System Under Test Boundary** section to every e2e/blackbox test task:
|
|
- The test must drive the product through public runtime boundaries and compare actual outputs to `_docs/00_problem/input_data/expected_results/results_report.md` and any referenced machine-readable expected-result files.
|
|
- Stubs are allowed only for external systems outside the product boundary: flight controller/SITL, QGC observer, satellite-provider/Suite service, physical Jetson hardware, physical camera, licensed public datasets, and network services.
|
|
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are not allowed for internal product modules that the scenario is meant to validate, such as VIO, safety/anchor wrapper, satellite retrieval, anchor verification, tile manager, MAVLink output adapter, or FDR.
|
|
- If an internal module is not implemented, the test must fail/block as missing product implementation; it must not pass by replacing that module with a test stub.
|
|
5. Dependencies:
|
|
- In tests-only mode: blackbox test tasks depend on the test infrastructure bootstrap task (Step 1t)
|
|
6. Write each task spec using `templates/task.md`
|
|
7. Estimate complexity per task (1, 2, 3, 5 points); no task should exceed 5 points — split if it does
|
|
8. Note task dependencies (referencing tracker IDs of already-created dependency tasks)
|
|
9. **Immediately after writing each task file**: create a work item ticket under the "Blackbox Tests" epic, write the work item ticket ID and Epic ID back into the task header, then rename the file from `todo/[##]_[short_name].md` to `todo/[TRACKER-ID]_[short_name].md`.
|
|
|
|
## Self-verification
|
|
|
|
- [ ] Every scenario from `tests/blackbox-tests.md` is covered by a task
|
|
- [ ] Every scenario from `tests/performance-tests.md`, `tests/resilience-tests.md`, `tests/security-tests.md`, and `tests/resource-limit-tests.md` is covered by a task
|
|
- [ ] No task exceeds 5 complexity points
|
|
- [ ] Dependencies correctly reference the test infrastructure task
|
|
- [ ] Every task has a work item ticket linked to the "Blackbox Tests" epic
|
|
- [ ] Every e2e/blackbox task forbids internal product stubs/fakes and requires comparison against expected-results artifacts
|
|
|
|
## Save action
|
|
|
|
Write each `todo/[##]_[short_name].md` (temporary numeric name), create work item ticket inline, then rename to `todo/[TRACKER-ID]_[short_name].md`.
|