mirror of
https://github.com/azaion/ai-training.git
synced 2026-04-22 11:36:36 +00:00
Refactor constants management to use Pydantic BaseModel for configuration
- Replaced module-level path variables in constants.py with a structured Pydantic Config class. - Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure. - Fixed bugs related to image processing and model saving. - Enhanced test infrastructure to accommodate the new configuration approach. This refactor improves code maintainability and clarity by centralizing configuration management.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,40 @@
|
||||
# Phase 0: Context & Baseline
|
||||
|
||||
**Role**: Software engineer preparing for refactoring
|
||||
**Goal**: Collect refactoring goals and capture baseline metrics
|
||||
**Constraints**: Measurement only — no code changes
|
||||
|
||||
## 0a. Collect Goals
|
||||
|
||||
If PROBLEM_DIR files do not yet exist, help the user create them:
|
||||
|
||||
1. `problem.md` — what the system currently does, what changes are needed, pain points
|
||||
2. `acceptance_criteria.md` — success criteria for the refactoring
|
||||
3. `security_approach.md` — security requirements (if applicable)
|
||||
|
||||
Store in PROBLEM_DIR.
|
||||
|
||||
## 0b. Capture Baseline
|
||||
|
||||
1. Read problem description and acceptance criteria
|
||||
2. Measure current system metrics using project-appropriate tools:
|
||||
|
||||
| Metric Category | What to Capture |
|
||||
|----------------|-----------------|
|
||||
| **Coverage** | Overall, unit, blackbox, critical paths |
|
||||
| **Complexity** | Cyclomatic complexity (avg + top 5 functions), LOC, tech debt ratio |
|
||||
| **Code Smells** | Total, critical, major |
|
||||
| **Performance** | Response times (P50/P95/P99), CPU/memory, throughput |
|
||||
| **Dependencies** | Total count, outdated, security vulnerabilities |
|
||||
| **Build** | Build time, test execution time, deployment time |
|
||||
|
||||
3. Create functionality inventory: all features/endpoints with status and coverage
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] All metric categories measured (or noted as N/A with reason)
|
||||
- [ ] Functionality inventory is complete
|
||||
- [ ] Measurements are reproducible
|
||||
|
||||
**Save action**: Write `REFACTOR_DIR/baseline_metrics.md`
|
||||
|
||||
**BLOCKING**: Present baseline summary to user. Do NOT proceed until user confirms.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Phase 1: Discovery
|
||||
|
||||
**Role**: Principal software architect
|
||||
**Goal**: Generate documentation from existing code and form solution description
|
||||
**Constraints**: Document what exists, not what should be. No code changes.
|
||||
|
||||
**Skip condition** (Targeted mode): If `COMPONENTS_DIR` and `SOLUTION_DIR` already contain documentation for the target area, skip to Phase 2. Ask user to confirm skip.
|
||||
|
||||
## 1a. Document Components
|
||||
|
||||
For each component in the codebase:
|
||||
|
||||
1. Analyze project structure, directories, files
|
||||
2. Go file by file, analyze each method
|
||||
3. Analyze connections between components
|
||||
|
||||
Write per component to `REFACTOR_DIR/discovery/components/[##]_[name].md`:
|
||||
- Purpose and architectural patterns
|
||||
- Mermaid diagrams for logic flows
|
||||
- API reference table (name, description, input, output)
|
||||
- Implementation details: algorithmic complexity, state management, dependencies
|
||||
- Caveats, edge cases, known limitations
|
||||
|
||||
## 1b. Synthesize Solution & Flows
|
||||
|
||||
1. Review all generated component documentation
|
||||
2. Synthesize into a cohesive solution description
|
||||
3. Create flow diagrams showing component interactions
|
||||
|
||||
Write:
|
||||
- `REFACTOR_DIR/discovery/solution.md` — product description, component overview, interaction diagram
|
||||
- `REFACTOR_DIR/discovery/system_flows.md` — Mermaid flowcharts per major use case
|
||||
|
||||
Also copy to project standard locations if in project mode:
|
||||
- `SOLUTION_DIR/solution.md`
|
||||
- `DOCUMENT_DIR/system_flows.md`
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] Every component in the codebase is documented
|
||||
- [ ] Solution description covers all components
|
||||
- [ ] Flow diagrams cover all major use cases
|
||||
- [ ] Mermaid diagrams are syntactically correct
|
||||
|
||||
**Save action**: Write discovery artifacts
|
||||
|
||||
**BLOCKING**: Present discovery summary to user. Do NOT proceed until user confirms documentation accuracy.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Phase 2: Analysis
|
||||
|
||||
**Role**: Researcher and software architect
|
||||
**Goal**: Research improvements and produce a refactoring roadmap
|
||||
**Constraints**: Analysis only — no code changes
|
||||
|
||||
## 2a. Deep Research
|
||||
|
||||
1. Analyze current implementation patterns
|
||||
2. Research modern approaches for similar systems
|
||||
3. Identify what could be done differently
|
||||
4. Suggest improvements based on state-of-the-art practices
|
||||
|
||||
Write `REFACTOR_DIR/analysis/research_findings.md`:
|
||||
- Current state analysis: patterns used, strengths, weaknesses
|
||||
- Alternative approaches per component: current vs alternative, pros/cons, migration effort
|
||||
- Prioritized recommendations: quick wins + strategic improvements
|
||||
|
||||
## 2b. Solution Assessment
|
||||
|
||||
1. Assess current implementation against acceptance criteria
|
||||
2. Identify weak points in codebase, map to specific code areas
|
||||
3. Perform gap analysis: acceptance criteria vs current state
|
||||
4. Prioritize changes by impact and effort
|
||||
|
||||
Write `REFACTOR_DIR/analysis/refactoring_roadmap.md`:
|
||||
- Weak points assessment: location, description, impact, proposed solution
|
||||
- Gap analysis: what's missing, what needs improvement
|
||||
- Phased roadmap: Phase 1 (critical fixes), Phase 2 (major improvements), Phase 3 (enhancements)
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] All acceptance criteria are addressed in gap analysis
|
||||
- [ ] Recommendations are grounded in actual code, not abstract
|
||||
- [ ] Roadmap phases are prioritized by impact
|
||||
- [ ] Quick wins are identified separately
|
||||
|
||||
**Save action**: Write analysis artifacts
|
||||
|
||||
**BLOCKING**: Present refactoring roadmap to user. Do NOT proceed until user confirms.
|
||||
|
||||
**Quick Assessment mode stops here.** Present final summary and write `FINAL_report.md` with phases 0-2 content.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Phase 3: Safety Net
|
||||
|
||||
**Role**: QA engineer and developer
|
||||
**Goal**: Design and implement tests that capture current behavior before refactoring
|
||||
**Constraints**: Tests must all pass on the current codebase before proceeding
|
||||
|
||||
## 3a. Design Test Specs
|
||||
|
||||
Coverage requirements (must meet before refactoring — see `.cursor/rules/cursor-meta.mdc` Quality Thresholds):
|
||||
- Minimum overall coverage: 75%
|
||||
- Critical path coverage: 90%
|
||||
- All public APIs must have blackbox tests
|
||||
- All error handling paths must be tested
|
||||
|
||||
For each critical area, write test specs to `REFACTOR_DIR/test_specs/[##]_[test_name].md`:
|
||||
- Blackbox tests: summary, current behavior, input data, expected result, max expected time
|
||||
- Acceptance tests: summary, preconditions, steps with expected results
|
||||
- Coverage analysis: current %, target %, uncovered critical paths
|
||||
|
||||
## 3b. Implement Tests
|
||||
|
||||
1. Set up test environment and infrastructure if not exists
|
||||
2. Implement each test from specs
|
||||
3. Run tests, verify all pass on current codebase
|
||||
4. Document any discovered issues
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] Coverage requirements met (75% overall, 90% critical paths)
|
||||
- [ ] All tests pass on current codebase
|
||||
- [ ] All public APIs have blackbox tests
|
||||
- [ ] Test data fixtures are configured
|
||||
|
||||
**Save action**: Write test specs; implemented tests go into the project's test folder
|
||||
|
||||
**GATE (BLOCKING)**: ALL tests must pass before proceeding to Phase 4. If tests fail, fix the tests (not the code) or ask user for guidance. Do NOT proceed to Phase 4 with failing tests.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Phase 4: Execution
|
||||
|
||||
**Role**: Software architect and developer
|
||||
**Goal**: Analyze coupling and execute decoupling changes
|
||||
**Constraints**: Small incremental changes; tests must stay green after every change
|
||||
|
||||
## 4a. Analyze Coupling
|
||||
|
||||
1. Analyze coupling between components/modules
|
||||
2. Map dependencies (direct and transitive)
|
||||
3. Identify circular dependencies
|
||||
4. Form decoupling strategy
|
||||
|
||||
Write `REFACTOR_DIR/coupling_analysis.md`:
|
||||
- Dependency graph (Mermaid)
|
||||
- Coupling metrics per component
|
||||
- Problem areas: components involved, coupling type, severity, impact
|
||||
- Decoupling strategy: priority order, proposed interfaces/abstractions, effort estimates
|
||||
|
||||
**BLOCKING**: Present coupling analysis to user. Do NOT proceed until user confirms strategy.
|
||||
|
||||
## 4b. Execute Decoupling
|
||||
|
||||
For each change in the decoupling strategy:
|
||||
|
||||
1. Implement the change
|
||||
2. Run blackbox tests
|
||||
3. Fix any failures
|
||||
4. Commit with descriptive message
|
||||
|
||||
Address code smells encountered: long methods, large classes, duplicate code, dead code, magic numbers.
|
||||
|
||||
Write `REFACTOR_DIR/execution_log.md`:
|
||||
- Change description, files affected, test status per change
|
||||
- Before/after metrics comparison against baseline
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] All tests still pass after execution
|
||||
- [ ] No circular dependencies remain (or reduced per plan)
|
||||
- [ ] Code smells addressed
|
||||
- [ ] Metrics improved compared to baseline
|
||||
|
||||
**Save action**: Write execution artifacts
|
||||
|
||||
**BLOCKING**: Present execution summary to user. Do NOT proceed until user confirms.
|
||||
@@ -0,0 +1,51 @@
|
||||
# Phase 5: Hardening (Optional, Parallel Tracks)
|
||||
|
||||
**Role**: Varies per track
|
||||
**Goal**: Address technical debt, performance, and security
|
||||
**Constraints**: Each track is optional; user picks which to run
|
||||
|
||||
Present the three tracks and let user choose which to execute:
|
||||
|
||||
## Track A: Technical Debt
|
||||
|
||||
**Role**: Technical debt analyst
|
||||
|
||||
1. Identify and categorize debt items: design, code, test, documentation
|
||||
2. Assess each: location, description, impact, effort, interest (cost of not fixing)
|
||||
3. Prioritize: quick wins → strategic debt → tolerable debt
|
||||
4. Create actionable plan with prevention measures
|
||||
|
||||
Write `REFACTOR_DIR/hardening/technical_debt.md`
|
||||
|
||||
## Track B: Performance Optimization
|
||||
|
||||
**Role**: Performance engineer
|
||||
|
||||
1. Profile current performance, identify bottlenecks
|
||||
2. For each bottleneck: location, symptom, root cause, impact
|
||||
3. Propose optimizations with expected improvement and risk
|
||||
4. Implement one at a time, benchmark after each change
|
||||
5. Verify tests still pass
|
||||
|
||||
Write `REFACTOR_DIR/hardening/performance.md` with before/after benchmarks
|
||||
|
||||
## Track C: Security Review
|
||||
|
||||
**Role**: Security engineer
|
||||
|
||||
1. Review code against OWASP Top 10
|
||||
2. Verify security requirements from `security_approach.md` are met
|
||||
3. Check: authentication, authorization, input validation, output encoding, encryption, logging
|
||||
|
||||
Write `REFACTOR_DIR/hardening/security.md`:
|
||||
- Vulnerability assessment: location, type, severity, exploit scenario, fix
|
||||
- Security controls review
|
||||
- Compliance check against `security_approach.md`
|
||||
- Recommendations: critical fixes, improvements, hardening
|
||||
|
||||
**Self-verification** (per track):
|
||||
- [ ] All findings are grounded in actual code
|
||||
- [ ] Recommendations are actionable with effort estimates
|
||||
- [ ] All tests still pass after any changes
|
||||
|
||||
**Save action**: Write hardening artifacts
|
||||
@@ -0,0 +1,51 @@
|
||||
# Phase 6: Test Synchronization
|
||||
|
||||
**Role**: QA engineer and developer
|
||||
**Goal**: Reconcile the test suite with the refactored codebase — remove obsolete tests, update broken tests, add tests for new code
|
||||
**Constraints**: All tests must pass at the end of this phase. Do not change production code here — only tests.
|
||||
|
||||
## 6a. Identify Obsolete Tests
|
||||
|
||||
1. Compare the pre-refactoring codebase structure (from Phase 0 inventory) with the current state
|
||||
2. Find tests that reference removed functions, classes, modules, or endpoints
|
||||
3. Find tests that duplicate coverage due to merged/consolidated code
|
||||
4. Decide per test: **delete** (functionality removed) or **merge** (duplicates)
|
||||
|
||||
Write `REFACTOR_DIR/test_sync/obsolete_tests.md`:
|
||||
- Test file, test name, reason (target removed / target merged / duplicate coverage), action taken (deleted / merged into)
|
||||
|
||||
## 6b. Update Existing Tests
|
||||
|
||||
1. Run the full test suite — collect failures and errors
|
||||
2. For each failing test, determine the cause:
|
||||
- Renamed/moved function or module → update import paths and references
|
||||
- Changed function signature → update call sites and assertions
|
||||
- Changed behavior (intentional per refactoring plan) → update expected values
|
||||
- Changed data structures → update fixtures and assertions
|
||||
3. Fix each test, re-run to confirm it passes
|
||||
|
||||
Write `REFACTOR_DIR/test_sync/updated_tests.md`:
|
||||
- Test file, test name, change type (import path / signature / assertion / fixture), description of update
|
||||
|
||||
## 6c. Add New Tests
|
||||
|
||||
1. Identify new code introduced during Phases 4–5 that lacks test coverage:
|
||||
- New public functions, classes, or modules
|
||||
- New interfaces or abstractions introduced during decoupling
|
||||
- New error handling paths
|
||||
2. Write tests following the same patterns and conventions as the existing test suite
|
||||
3. Ensure coverage targets from Phase 3 are maintained or improved
|
||||
|
||||
Write `REFACTOR_DIR/test_sync/new_tests.md`:
|
||||
- Test file, test name, target function/module, coverage type (unit / integration / blackbox)
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] All obsolete tests removed or merged
|
||||
- [ ] All pre-existing tests pass after updates
|
||||
- [ ] New code from Phases 4–5 has test coverage
|
||||
- [ ] Overall coverage meets or exceeds Phase 3 baseline (75% overall, 90% critical paths)
|
||||
- [ ] No tests reference removed or renamed code
|
||||
|
||||
**Save action**: Write test_sync artifacts; implemented tests go into the project's test folder
|
||||
|
||||
**GATE (BLOCKING)**: ALL tests must pass before proceeding to Phase 7. If tests fail, fix the tests or ask user for guidance.
|
||||
@@ -0,0 +1,51 @@
|
||||
# Phase 7: Final Verification
|
||||
|
||||
**Role**: QA engineer
|
||||
**Goal**: Run all tests end-to-end, compare final metrics against baseline, and confirm the refactoring succeeded
|
||||
**Constraints**: No code changes. If failures are found, go back to the appropriate phase (4/5/6) to fix before retrying.
|
||||
|
||||
## 7a. Run Full Test Suite
|
||||
|
||||
1. Run unit tests, integration tests, and blackbox tests
|
||||
2. Run acceptance tests derived from `acceptance_criteria.md`
|
||||
3. Record pass/fail counts and any failures
|
||||
|
||||
If any test fails:
|
||||
- Determine whether the failure is a test issue (→ return to Phase 6) or a code issue (→ return to Phase 4/5)
|
||||
- Do NOT proceed until all tests pass
|
||||
|
||||
## 7b. Capture Final Metrics
|
||||
|
||||
Re-measure all metrics from Phase 0 baseline using the same tools:
|
||||
|
||||
| Metric Category | What to Capture |
|
||||
|----------------|-----------------|
|
||||
| **Coverage** | Overall, unit, blackbox, critical paths |
|
||||
| **Complexity** | Cyclomatic complexity (avg + top 5 functions), LOC, tech debt ratio |
|
||||
| **Code Smells** | Total, critical, major |
|
||||
| **Performance** | Response times (P50/P95/P99), CPU/memory, throughput |
|
||||
| **Dependencies** | Total count, outdated, security vulnerabilities |
|
||||
| **Build** | Build time, test execution time, deployment time |
|
||||
|
||||
## 7c. Compare Against Baseline
|
||||
|
||||
1. Read `REFACTOR_DIR/baseline_metrics.md`
|
||||
2. Produce a side-by-side comparison: baseline vs final for every metric
|
||||
3. Flag any regressions (metrics that got worse)
|
||||
4. Verify acceptance criteria are met
|
||||
|
||||
Write `REFACTOR_DIR/verification_report.md`:
|
||||
- Test results summary: total, passed, failed, skipped
|
||||
- Metric comparison table: metric, baseline value, final value, delta, status (improved / unchanged / regressed)
|
||||
- Acceptance criteria checklist: criterion, status (met / not met), evidence
|
||||
- Regressions (if any): metric, severity, explanation
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] All tests pass (zero failures)
|
||||
- [ ] All acceptance criteria are met
|
||||
- [ ] No critical metric regressions
|
||||
- [ ] Metrics are captured with the same tools/methodology as Phase 0
|
||||
|
||||
**Save action**: Write `REFACTOR_DIR/verification_report.md`
|
||||
|
||||
**GATE (BLOCKING)**: All tests must pass and no critical regressions. Present verification report to user. Do NOT proceed to Phase 8 until user confirms.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Phase 8: Documentation Update
|
||||
|
||||
**Role**: Technical writer
|
||||
**Goal**: Update existing `_docs/` artifacts to reflect all changes made during refactoring
|
||||
**Constraints**: Documentation only — no code changes. Only update docs that are affected by refactoring changes.
|
||||
|
||||
**Skip condition**: If no `_docs/02_document/` directory exists (standalone mode), skip this phase entirely.
|
||||
|
||||
## 8a. Identify Affected Documentation
|
||||
|
||||
1. Review `REFACTOR_DIR/execution_log.md` to list all files changed during Phase 4
|
||||
2. Review any hardening changes from Phase 5
|
||||
3. Review test changes from Phase 6
|
||||
4. Map changed files to their corresponding module docs in `_docs/02_document/modules/`
|
||||
5. Map changed modules to their parent component docs in `_docs/02_document/components/`
|
||||
6. Determine if system-level docs need updates (`architecture.md`, `system-flows.md`, `data_model.md`)
|
||||
7. Determine if test documentation needs updates (`_docs/02_document/tests/`)
|
||||
|
||||
## 8b. Update Module Documentation
|
||||
|
||||
For each module doc affected by refactoring changes:
|
||||
1. Re-read the current source file
|
||||
2. Update the module doc to reflect new/changed interfaces, dependencies, internal logic
|
||||
3. Remove documentation for deleted code; add documentation for new code
|
||||
|
||||
## 8c. Update Component Documentation
|
||||
|
||||
For each component doc affected:
|
||||
1. Re-read the updated module docs within the component
|
||||
2. Update inter-module interfaces, dependency graphs, caveats
|
||||
3. Update the component relationship diagram if component boundaries changed
|
||||
|
||||
## 8d. Update System-Level Documentation
|
||||
|
||||
If structural changes were made (new modules, removed modules, changed interfaces):
|
||||
1. Update `_docs/02_document/architecture.md` if architecture changed
|
||||
2. Update `_docs/02_document/system-flows.md` if flow sequences changed
|
||||
3. Update `_docs/02_document/diagrams/components.md` if component relationships changed
|
||||
|
||||
**Self-verification**:
|
||||
- [ ] Every changed source file has an up-to-date module doc
|
||||
- [ ] Component docs reflect the refactored structure
|
||||
- [ ] No stale references to removed code in any doc
|
||||
- [ ] Dependency graphs in docs match actual imports
|
||||
|
||||
**Save action**: Updated docs written in-place to `_docs/02_document/`
|
||||
Reference in New Issue
Block a user