mirror of
https://github.com/azaion/annotations.git
synced 2026-06-21 23:31:07 +00:00
188 lines
6.8 KiB
Markdown
188 lines
6.8 KiB
Markdown
## Research Engine — Analysis Phase (Steps 4–8)
|
||
|
||
### Step 4: Build Comparison/Analysis Framework
|
||
|
||
Based on the question type, select fixed analysis dimensions. **For dimension lists** (General, Concept Comparison, Decision Support): Read `references/comparison-frameworks.md`
|
||
|
||
**Save action**:
|
||
Write to `03_comparison_framework.md`:
|
||
```markdown
|
||
# Comparison Framework
|
||
|
||
## Selected Framework Type
|
||
[Concept Comparison / Decision Support / ...]
|
||
|
||
## Selected Dimensions
|
||
1. [Dimension 1]
|
||
2. [Dimension 2]
|
||
...
|
||
|
||
## Initial Population
|
||
| Dimension | X | Y | Factual Basis |
|
||
|-----------|---|---|---------------|
|
||
| [Dimension 1] | [description] | [description] | Fact #1, #3 |
|
||
| ... | | | |
|
||
```
|
||
|
||
**Required exact-fit dimensions for component/tool decisions**:
|
||
When the output selects or recommends a component, tool, library, service, architecture pattern, or algorithm, the framework MUST include these dimensions unless explicitly not applicable:
|
||
- Option family (`Simple baseline`, `Established production`, `Open-source`, `Commercial/vendor`, `Current SOTA`, `Adjacent-domain`, `No-build/defer`, `Known bad`)
|
||
- Required inputs/outputs and ownership boundaries
|
||
- Operating context and lifecycle fit
|
||
- Non-functional envelope fit
|
||
- Implementation assumptions and hard disqualifiers
|
||
- Evidence quality and source tier
|
||
- Selection status (`Selected`, `Rejected`, `Experimental only`, `Needs user decision`)
|
||
|
||
For each component area, include multiple candidates in the initial population. Do not present only the preferred option unless the investigation found no realistic alternatives; if so, state the searches that proved the narrow landscape.
|
||
|
||
---
|
||
|
||
### Step 5: Reference Point Baseline Alignment
|
||
|
||
Ensure all compared parties have clear, consistent definitions:
|
||
|
||
**Checklist**:
|
||
- [ ] Is the reference point's definition stable/widely accepted?
|
||
- [ ] Does it need verification, or can domain common knowledge be used?
|
||
- [ ] Does the reader's understanding of the reference point match mine?
|
||
- [ ] Are there ambiguities that need to be clarified first?
|
||
|
||
---
|
||
|
||
### Step 6: Fact-to-Conclusion Reasoning Chain
|
||
|
||
Explicitly write out the "fact → comparison → conclusion" reasoning process:
|
||
|
||
```markdown
|
||
## Reasoning Process
|
||
|
||
### Regarding [Dimension Name]
|
||
|
||
1. **Fact confirmation**: According to [source], X's mechanism is...
|
||
2. **Compare with reference**: While Y's mechanism is...
|
||
3. **Conclusion**: Therefore, the difference between X and Y on this dimension is...
|
||
```
|
||
|
||
**Key discipline**:
|
||
- Conclusions come from mechanism comparison, not "gut feelings"
|
||
- Every conclusion must be traceable to specific facts
|
||
- Uncertain conclusions must be annotated
|
||
|
||
**Save action**:
|
||
Write to `04_reasoning_chain.md`:
|
||
```markdown
|
||
# Reasoning Chain
|
||
|
||
## Dimension 1: [Dimension Name]
|
||
|
||
### Fact Confirmation
|
||
According to [Fact #X], X's mechanism is...
|
||
|
||
### Reference Comparison
|
||
While Y's mechanism is... (Source: [Fact #Y])
|
||
|
||
### Conclusion
|
||
Therefore, the difference between X and Y on this dimension is...
|
||
|
||
### Confidence
|
||
✅/⚠️/❓ + rationale
|
||
|
||
---
|
||
## Dimension 2: [Dimension Name]
|
||
...
|
||
```
|
||
|
||
---
|
||
|
||
### Step 7: Use-Case Validation (Sanity Check)
|
||
|
||
Validate conclusions against a typical scenario:
|
||
|
||
**Validation questions**:
|
||
- Based on my conclusions, how should this scenario be handled?
|
||
- Is that actually the case?
|
||
- Are there counterexamples that need to be addressed?
|
||
|
||
**Review checklist**:
|
||
- [ ] Are draft conclusions consistent with Step 3 fact cards?
|
||
- [ ] Are there any important dimensions missed?
|
||
- [ ] Is there any over-extrapolation?
|
||
- [ ] Are conclusions actionable/verifiable?
|
||
- [ ] Does every selected component/tool/pattern match the Project Constraint Matrix?
|
||
- [ ] Are mismatches marked as disqualifiers instead of hidden as generic "limitations"?
|
||
|
||
**Save action**:
|
||
Write to `05_validation_log.md`:
|
||
```markdown
|
||
# Validation Log
|
||
|
||
## Validation Scenario
|
||
[Scenario description]
|
||
|
||
## Expected Based on Conclusions
|
||
If using X: [expected behavior]
|
||
If using Y: [expected behavior]
|
||
|
||
## Actual Validation Results
|
||
[actual situation]
|
||
|
||
## Counterexamples
|
||
[yes/no, describe if yes]
|
||
|
||
## Review Checklist
|
||
- [x] Draft conclusions consistent with fact cards
|
||
- [x] No important dimensions missed
|
||
- [x] No over-extrapolation
|
||
- [ ] Issue found: [if any]
|
||
|
||
## Conclusions Requiring Revision
|
||
[if any]
|
||
```
|
||
|
||
---
|
||
|
||
### Step 7.5: Component Applicability Gate (BLOCKING)
|
||
|
||
Before finalizing the solution draft, build an exact-fit matrix for every component/tool/library/service/pattern/algorithm that is selected, recommended, rejected, or treated as a fallback.
|
||
|
||
```markdown
|
||
# Component Fit Matrix
|
||
|
||
| Component Area | Candidate | Option Family | Intended Role | Project Constraints Checked | Evidence | Mismatches / Disqualifiers | Status | Decision Rationale |
|
||
|----------------|-----------|---------------|---------------|-----------------------------|----------|----------------------------|--------|--------------------|
|
||
| [area] | [name] | [family] | [role] | [constraints] | [Fact # / Source #] | [none / list] | Selected / Rejected / Experimental only / Needs user decision | [why] |
|
||
```
|
||
|
||
Rules:
|
||
- `Selected` is allowed only when the candidate's documented implementation assumptions match the project's explicit constraints and acceptance criteria.
|
||
- `Experimental only` is required when a candidate might work but lacks proof for the exact operating context.
|
||
- `Rejected` is required when documented assumptions conflict with project constraints.
|
||
- `Needs user decision` is required when a mismatch changes scope, cost, safety, product behavior, or acceptance criteria.
|
||
- Each component area must include at least one selected or fallback-safe option, plus the most credible rejected/experimental alternatives discovered during web research.
|
||
- A component area with only one candidate is incomplete unless `00_question_decomposition.md` documents the broader searches and why they yielded no realistic alternatives.
|
||
- A candidate may not appear as the lead solution in Step 8 unless this gate marks it `Selected`.
|
||
|
||
**Save action**: Write `06_component_fit_matrix.md`.
|
||
|
||
**BLOCKING**: If any lead candidate is `Experimental only`, `Rejected`, or `Needs user decision`, do not silently proceed. Ask the user or choose a different selected candidate.
|
||
|
||
---
|
||
|
||
### Step 8: Deliverable Formatting
|
||
|
||
Make the output **readable, traceable, and actionable**.
|
||
|
||
**Save action**:
|
||
Integrate all intermediate artifacts. Write to `OUTPUT_DIR/solution_draft##.md` using the appropriate output template based on active mode:
|
||
- Mode A: `templates/solution_draft_mode_a.md`
|
||
- Mode B: `templates/solution_draft_mode_b.md`
|
||
|
||
Sources to integrate:
|
||
- Extract background from `00_question_decomposition.md`
|
||
- Reference key facts from `02_fact_cards.md`
|
||
- Organize conclusions from `04_reasoning_chain.md`
|
||
- Generate references from `01_source_registry.md`
|
||
- Supplement with use cases from `05_validation_log.md`
|
||
- For Mode A: include AC assessment from `00_ac_assessment.md`
|