## Research Engine — Analysis Phase (Steps 4–8) ### Step 4: Build Comparison/Analysis Framework Based on the question type, select fixed analysis dimensions. **For dimension lists** (General, Concept Comparison, Decision Support): Read `references/comparison-frameworks.md` **Save action**: Write to `03_comparison_framework.md`: ```markdown # Comparison Framework ## Selected Framework Type [Concept Comparison / Decision Support / ...] ## Selected Dimensions 1. [Dimension 1] 2. [Dimension 2] ... ## Initial Population | Dimension | X | Y | Factual Basis | |-----------|---|---|---------------| | [Dimension 1] | [description] | [description] | Fact #1, #3 | | ... | | | | ``` **Required exact-fit dimensions for component/tool decisions**: When the output selects or recommends a component, tool, library, service, architecture pattern, or algorithm, the framework MUST include these dimensions unless explicitly not applicable: - Option family (`Simple baseline`, `Established production`, `Open-source`, `Commercial/vendor`, `Current SOTA`, `Adjacent-domain`, `No-build/defer`, `Known bad`) - Required inputs/outputs and ownership boundaries - Operating context and lifecycle fit - Non-functional envelope fit - Implementation assumptions and hard disqualifiers - Evidence quality and source tier - Selection status (`Selected`, `Rejected`, `Experimental only`, `Needs user decision`) For each component area, include multiple candidates in the initial population. Do not present only the preferred option unless the investigation found no realistic alternatives; if so, state the searches that proved the narrow landscape. --- ### Step 5: Reference Point Baseline Alignment Ensure all compared parties have clear, consistent definitions: **Checklist**: - [ ] Is the reference point's definition stable/widely accepted? - [ ] Does it need verification, or can domain common knowledge be used? - [ ] Does the reader's understanding of the reference point match mine? - [ ] Are there ambiguities that need to be clarified first? --- ### Step 6: Fact-to-Conclusion Reasoning Chain Explicitly write out the "fact → comparison → conclusion" reasoning process: ```markdown ## Reasoning Process ### Regarding [Dimension Name] 1. **Fact confirmation**: According to [source], X's mechanism is... 2. **Compare with reference**: While Y's mechanism is... 3. **Conclusion**: Therefore, the difference between X and Y on this dimension is... ``` **Key discipline**: - Conclusions come from mechanism comparison, not "gut feelings" - Every conclusion must be traceable to specific facts - Uncertain conclusions must be annotated **Save action**: Write to `04_reasoning_chain.md`: ```markdown # Reasoning Chain ## Dimension 1: [Dimension Name] ### Fact Confirmation According to [Fact #X], X's mechanism is... ### Reference Comparison While Y's mechanism is... (Source: [Fact #Y]) ### Conclusion Therefore, the difference between X and Y on this dimension is... ### Confidence ✅/⚠️/❓ + rationale --- ## Dimension 2: [Dimension Name] ... ``` --- ### Step 7: Use-Case Validation (Sanity Check) Validate conclusions against a typical scenario: **Validation questions**: - Based on my conclusions, how should this scenario be handled? - Is that actually the case? - Are there counterexamples that need to be addressed? **Review checklist**: - [ ] Are draft conclusions consistent with Step 3 fact cards? - [ ] Are there any important dimensions missed? - [ ] Is there any over-extrapolation? - [ ] Are conclusions actionable/verifiable? - [ ] Does every selected component/tool/pattern match the Project Constraint Matrix? - [ ] Are mismatches marked as disqualifiers instead of hidden as generic "limitations"? **Save action**: Write to `05_validation_log.md`: ```markdown # Validation Log ## Validation Scenario [Scenario description] ## Expected Based on Conclusions If using X: [expected behavior] If using Y: [expected behavior] ## Actual Validation Results [actual situation] ## Counterexamples [yes/no, describe if yes] ## Review Checklist - [x] Draft conclusions consistent with fact cards - [x] No important dimensions missed - [x] No over-extrapolation - [ ] Issue found: [if any] ## Conclusions Requiring Revision [if any] ``` --- ### Step 7.5: Component Applicability Gate (BLOCKING) **Applicability**: this gate applies only when the run is classified as **Technical-component selection** in the SKILL's Research Output Class section. For non-technical research (concept comparison, market/policy investigation, root-cause analysis without tooling, knowledge organization), skip this entire step and proceed to Step 8 — there are no components to gate. State the skip once in `05_validation_log.md`: `Step 7.5 (Component Applicability Gate): not applicable — Non-technical investigation`. For mixed runs (some component areas technical, some not), apply this gate only to the technical component areas; the non-technical ones do not produce 7.5 rows. Before finalizing the solution draft, build an exact-fit matrix for every component/tool/library/service/pattern/algorithm that is selected, recommended, rejected, or treated as a fallback. Free-form prose in a "Project Constraints Checked" column is **not sufficient** — mismatches hide inside rationale text. The matrix must be structured per restriction and per acceptance criterion. #### 7.5.1 Top-level Component Fit Matrix ```markdown # Component Fit Matrix | Component Area | Candidate | Pinned Mode/Config | Option Family | Intended Role | API Capability Evidence | Mismatches / Disqualifiers | Status | Decision Rationale | |----------------|-----------|--------------------|---------------|---------------|-------------------------|----------------------------|--------|--------------------| | [area] | [name] | [exact mode/config the project will use, copied verbatim from the MVE block in Step 2] | [family] | [role] | MVE: [link to MVE block in `02_fact_cards.md` / `02_fact_cards/` or `02_mve_evidence.md`]; docs: [Source #] | [none / list] | Selected / Rejected / Experimental only / Needs user decision | [why] | ``` The new **Pinned Mode/Config** column is mandatory. A row without a pinned mode is incomplete. The new **API Capability Evidence** column links to the Minimum Viable Example saved during Step 2's API Capability Verification — without an MVE link the candidate cannot be `Selected`. #### 7.5.2 Restrictions × Candidate-Modes Sub-Matrix (MANDATORY) For each lead candidate row in the top-level matrix, append a structured cross-check that walks every numbered line of `restrictions.md` and `acceptance_criteria.md` against the candidate's **pinned mode/config**. ```markdown ## Sub-Matrix — in | Restriction / AC | Candidate-mode behavior | Result | Evidence | |------------------|-------------------------|--------|----------| | R1: | | ✅ Pass / ❌ Fail / ❓ Verify / N/A | [Fact # / Source # / MVE link] | | R2: ... | ... | ... | ... | | ... | ... | ... | ... | | AC-1.1: | | ✅ / ❌ / ❓ / N/A | [Fact # / Source # / MVE link] | | AC-1.2: ... | ... | ... | ... | | ... | ... | ... | ... | ``` Cell semantics: - ✅ **Pass** — the candidate's pinned mode satisfies this line, with cited official-doc or MVE evidence. - ❌ **Fail** — the candidate's pinned mode contradicts this line, with cited evidence. Even one ❌ disqualifies the candidate from `Selected` status. - ❓ **Verify** — no evidence yet either way; further investigation required (loops back to Step 2 / Step 3.5). A row left ❓ at the end of analysis blocks the candidate. - **N/A** — the line is irrelevant to this component area (state why in one phrase). A candidate row may not be marked `Selected` while any cell is ❌ or ❓. #### 7.5.3 Decision Rules - `Selected` is allowed only when (a) the top-level row has an MVE link, (b) the sub-matrix has zero ❌, (c) the sub-matrix has zero ❓, and (d) the candidate's documented implementation assumptions match the project's explicit constraints and acceptance criteria. - `Experimental only` is required when a candidate might work but lacks proof for the exact operating context (e.g., MVE exists for a similar configuration but not the exact one). - `Rejected` is required when documented assumptions conflict with project constraints (any sub-matrix row is ❌ with cited evidence). - `Needs user decision` is required when a mismatch changes scope, cost, safety, product behavior, or acceptance criteria — and the user has not yet been consulted. - Each component area must include at least one selected or fallback-safe option, plus the most credible rejected/experimental alternatives discovered during web research. - A component area with only one candidate is incomplete unless `00_question_decomposition.md` documents the broader searches and why they yielded no realistic alternatives. - A candidate may not appear as the lead solution in Step 8 unless this gate marks it `Selected`. - "Validation gate required" footnotes are not equivalent to `Selected`. If the validation gate concerns API capability (does the mode produce the required output?), that is a Step-2 / Step-7.5 question and must be resolved here, not deferred to runtime. Only validation gates concerning *runtime quality* (e.g., "does this VO converge on this terrain class?") may be carried forward as `Selected with runtime gate`. **Save action**: Write `06_component_fit_matrix.md` (or, when split, the equivalent files under `06_component_fit_matrix/` — typically `00_summary.md` for the top-level matrix plus per-component sub-matrix files) containing both 7.5.1 (top-level) and 7.5.2 (per-candidate sub-matrices). **BLOCKING**: If any lead candidate has ❌, ❓, `Experimental only`, `Rejected`, or `Needs user decision` status, do not silently proceed. Ask the user or choose a different selected candidate. --- ### Step 8: Deliverable Formatting Make the output **readable, traceable, and actionable**. **Save action**: Integrate all intermediate artifacts. Write to `OUTPUT_DIR/solution_draft##.md` using the appropriate output template based on active mode: - Mode A: `templates/solution_draft_mode_a.md` - Mode B: `templates/solution_draft_mode_b.md` Sources to integrate: - Extract background from `00_question_decomposition.md` - Reference key facts from `02_fact_cards.md` (or files under `02_fact_cards/` if split) - Organize conclusions from `04_reasoning_chain.md` - Generate references from `01_source_registry.md` (or files under `01_source_registry/` if split) - Supplement with use cases from `05_validation_log.md` - For Mode A: include AC assessment from `00_ac_assessment.md`