mirror of
https://github.com/azaion/admin.git
synced 2026-06-21 08:21:10 +00:00
7.7 KiB
7.7 KiB
Quality Checklists — Reference
General Quality
- All core conclusions have L1/L2 tier factual support
- No use of vague words like "possibly", "probably" without annotating uncertainty
- Comparison dimensions are complete with no key differences missed
- At least one real use case validates conclusions
- References are complete with accessible links
- Every citation can be directly verified by the user (source verifiability)
- Structure hierarchy is clear; executives can quickly locate information
Decomposition Completeness
- Domain discovery search executed: searched "key factors when [problem domain]" before starting research
- Completeness probes applied: every probe from
references/comparison-frameworks.mdchecked against sub-questions - No uncovered areas remain: all gaps filled with sub-questions or justified as not applicable
Internet Search Depth
- Every sub-question was searched with at least 3-5 different query variants
- At least 3 perspectives from the Perspective Rotation were applied and searched
- Search saturation reached: last searches stopped producing new substantive information
- Adjacent fields and analogous problems were searched, not just direct matches
- Contrarian viewpoints were actively sought ("why not X", "X criticism", "X failure")
- Practitioner experience was searched (production use, real-world results, lessons learned)
- Iterative deepening completed: follow-up questions from initial findings were searched
- No sub-question relies solely on training data without web verification
Component Option Breadth
00_question_decomposition.mdcontains a Component Option Search Plan- Every component area was searched across simple baseline, established production, open-source, commercial/vendor, current SOTA, adjacent-domain, no-build/defer, and known-bad options where applicable
- Every component area has at least 3 realistic candidates, or a documented explanation of why broad searches found fewer
- Each lead candidate has official/source-of-truth evidence plus independent validation when available
- Each component area includes at least one baseline/fallback option and at least one rejected or experimental option when possible
- Alternative names, synonyms, and neighboring-domain terms were searched before declaring the option landscape complete
- Licensing, runtime, platform, maintenance, and unsupported-scenario searches were performed for every lead, fallback, and rejected candidate
Mode A Specific
- Phase 1 completed: AC assessment was presented to and confirmed by user
- AC assessment consistent: Solution draft respects the (possibly adjusted) acceptance criteria and restrictions
- Competitor analysis included: Existing solutions were researched
- All components have comparison tables: Each component lists alternatives with tools, advantages, limitations, security, cost
- Component options are broad: component tables include baseline, production, open-source, commercial/vendor, SOTA/research, adjacent-domain, defer/no-build, and disqualified options where applicable
- Tools/libraries verified: Suggested tools actually exist and work as described
- Component fit matrix completed:
06_component_fit_matrix.mdexists and every selected component/tool/pattern is markedSelected - No field-adjacent substitution: no selected candidate is chosen only because it solves a similar class of problem while failing the project's explicit constraints
- Testing strategy covers AC: Tests map to acceptance criteria
- Tech stack documented (if Phase 3 ran):
tech_stack.mdhas evaluation tables, risk assessment, and learning requirements - Security analysis documented (if Phase 4 ran):
security_analysis.mdhas threat model and per-component controls
Mode B Specific
- Findings table complete: All identified weak points documented with solutions
- Weak point categories covered: Functional, security, and performance assessed
- New draft is self-contained: Written as if from scratch, no "updated" markers
- Performance column included: Mode B comparison tables include performance characteristics
- Previous draft issues addressed: Every finding in the table is resolved in the new draft
- Existing selected components were challenged against a broad alternative landscape before being kept
- Existing component fit audited: every old and new component/tool/pattern was checked against
restrictions.md,acceptance_criteria.md, and the Project Constraint Matrix - Rejected/experimental candidates are not lead recommendations unless the user explicitly accepted the risk
Timeliness Check (High-Sensitivity Domain BLOCKING)
When the research topic has Critical or High sensitivity level:
- Timeliness sensitivity assessment completed:
00_question_decomposition.mdcontains a timeliness assessment section - Source timeliness annotated: Every source has publication date, timeliness status, version info
- No outdated sources used as factual evidence (Critical: within 6 months; High: within 1 year)
- Version numbers explicitly annotated for all technical products/APIs/SDKs
- Official sources prioritized: Core conclusions have support from official documentation/blogs
- Cross-validation completed: Key technical information confirmed from at least 2 independent sources
- Download page directly verified: Platform support info comes from real-time extraction of official download pages
- Protocol/feature names searched: Searched for product-supported protocol names (MCP, ACP, etc.)
- GitHub Issues mined: Reviewed product's GitHub Issues popular discussions
- Community hotspots identified: Identified and recorded feature points users care most about
Target Audience Consistency Check (BLOCKING)
- Research boundary clearly defined:
00_question_decomposition.mdhas clear population/geography/timeframe/level boundaries - Every source has target audience annotated in
01_source_registry.md - Mismatched sources properly handled (excluded, annotated, or marked reference-only)
- No audience confusion in fact cards: Every fact has target audience consistent with research boundary
- No audience confusion in the report: Policies/research/data cited have consistent target audiences
Source Verifiability
- All cited links are publicly accessible (annotate
[login required]if not) - Citations include exact section/page/timestamp for long documents
- Cited facts have corresponding statements in the original text (no over-interpretation)
- Source publication/update dates annotated; technical docs include version numbers
- Unverifiable information annotated
[limited source]and not sole support for core conclusions
Exact-Fit Validation (BLOCKING)
- Project Constraint Matrix extracted from problem context before component selection
- Component fit matrix includes
Component AreaandOption Familycolumns - Every selected component/tool/library/service/pattern/algorithm has evidence for required inputs/outputs and integration boundaries
- Every selected candidate has evidence for the operating context and lifecycle assumptions it must support
- Every selected candidate has evidence for non-functional targets that are binding for the project
- Known unsupported scenarios and failure reports were searched for every selected candidate
- Mismatches are recorded as disqualifiers, not softened into generic limitations
- Any candidate with unproven fit is marked
Experimental onlyor escalated for user decision - Any candidate with documented constraint conflict is marked
Rejected