azaion/satellite-provider
1
0
Fork 0
mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 12:41:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
745f4840e6ca53b0b38d1cc51f17012ad734411f
satellite-provider/_docs/02_document/module-layout.md
T
Oleksandr Bezdieniezhnykh 745f4840e6
ci/woodpecker/push/01-test Pipeline was successful
Details
ci/woodpecker/push/02-build-push Pipeline was successful
Details
[AZ-493] Cycle 3 batch 3: integration test DB-reset hook 
AZ-493 (2 SP): replace the cycle-2 wallclock-seeded _coordinateCounter
workaround with a proper Postgres state-reset hook that runs at
integration test runner startup, eliminating the per-source-unique-index
collision risk that the persistent docker-compose Postgres volume
introduced post-AZ-484.

The reset is split into two surfaces:

* SatelliteProvider.TestSupport.IntegrationTestResetGuard - pure
  static class, I/O-free, unit-tested. Two independent guards: (a)
  ASPNETCORE_ENVIRONMENT must equal "Testing", (b) DB_CONNECTION_STRING
  Host must be in the allowed-host list (postgres, localhost, 127.0.0.1).
  Failure of either guard surfaces a structured operator-friendly
  InvalidOperationException.
* SatelliteProvider.IntegrationTests.IntegrationTestDatabaseReset -
  instance class owning the Npgsql side effects. Calls the guard then
  runs TRUNCATE TABLE route_regions, route_points, routes, regions,
  tiles RESTART IDENTITY CASCADE inside a single Npgsql transaction.

Spec-vs-reality: the task spec prescribed "DB name contains _test" as
Guard 2; the actual compose file uses Database=satelliteprovider and
DB rename is gated on user confirmation per coderule.mdc. Substituted
a Host allowlist as the equivalent guard (intent identical: reject
remote / production hosts). Recorded as Low/Spec-Gap in the review.

Program.cs adds --keep-state CLI flag and INTEGRATION_KEEP_STATE env
var (1/true) opt-outs so a developer can inspect leftover state when
debugging. Startup banner shows which path executed.
docker-compose.tests.yml gets ASPNETCORE_ENVIRONMENT=Testing +
passthrough for INTEGRATION_KEEP_STATE. scripts/run-tests.sh wires the
--keep-state flag through to compose.

UavUploadTests._coordinateCounter wallclock seed is retained as
defense-in-depth (per the task spec's implementer choice). The reset
is the primary isolation path; the seed is the belt-and-suspenders
fallback for --keep-state runs.

8 new unit tests in SatelliteProvider.Tests/TestSupport/
IntegrationTestResetGuardTests.cs cover Production/Staging/missing-env
throw, allowed-host case-insensitivity, disallowed-host rejection
with representative prod hostnames, and the AllowedHosts contract.

tests_integration.md gains a Reliability section that documents the
hook, the two guards, the truncate order, and the three opt-out forms.
module-layout.md TestSupport entry extended with the new pure guard
and the explicit "Npgsql stays in IntegrationTests" boundary.

Test-suite gate (AC-6) deferred to Step 16 Final Test Run per implement
skill convention. Per-batch review verdict: PASS_WITH_WARNINGS with 1
Low (spec-vs-reality on Guard 2, non-blocking).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 01:38:42 +03:00

196 lines
17 KiB
Markdown
Raw Blame History

# Module Layout
**Status**: derived-from-code
**Language**: csharp
**Layout Convention**: custom (per-component .csproj per logical component)
**Root**: ./
**Last Updated**: 2026-05-11 (cycle 2 — AZ-487 JWT validation baseline + AZ-488 UAV tile upload added; supersedes prior post-AZ-350 update)
## Layout Rules
1. Each component owns ONE top-level project directory (`.csproj` boundary). The previous shared `SatelliteProvider.Services` project was split into three per-component csprojs in epic AZ-309.
2. Shared code lives under `SatelliteProvider.Common/` — the foundation layer.
3. Cross-cutting concerns (DTOs, interfaces, configs, geo-math, common exceptions) all reside in Common.
4. Public API surface per component = `public` types in the namespace root. Everything marked `internal` or private is internal.
5. Tests live in separate projects: `SatelliteProvider.Tests/` (unit) and `SatelliteProvider.IntegrationTests/` (integration).
6. DI registration per component lives in a `<Component>ServiceCollectionExtensions.cs` adjacent to the component's classes (e.g. `TileDownloaderServiceCollectionExtensions.AddTileDownloader()`).
## Documentation Layout (canonical — AZ-495)
Each Layer-3 service component (`Common`, `DataAccess`, `TileDownloader`, `RegionProcessing`, `RouteManagement`) owns one description file under `_docs/02_document/components/0N_<name>/description.md`. The numeric prefix (`01_common` ... `05_route_management`) matches the architectural-layer order — not the alphabetical order.
**The WebApi component (`SatelliteProvider.Api`) intentionally does NOT have a `components/*` folder.** Its documentation lives in `_docs/02_document/modules/api_program.md`. The rationale is that WebApi is the orchestrator / entry-point at Layer 4 rather than a Layer-3 service component — its concerns are minimal-API endpoint mapping, DI composition, and middleware chain composition, all of which are documented at module-level alongside the other process-level concerns (`tests_unit.md`, `tests_integration.md`, `migrations.md`). Splitting WebApi documentation into a component-stub plus a module file would create two sources of truth.
When authoring or reading a task that touches WebApi, use `_docs/02_document/modules/api_program.md` as the documentation anchor. Task-spec templates and the `new-task` / `decompose` skills point at this path; the `components/06_web_api/` folder is intentionally absent and MUST NOT be created.
The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low / Style) finding because task specs referenced the non-existent `components/01_web_api/description.md` path. AZ-495 settles this convention; the finding should not recur.
## Per-Component Mapping
### Component: Common
- **Directory**: `SatelliteProvider.Common/`
- **Public API**:
- `SatelliteProvider.Common/Configs/MapConfig.cs`
- `SatelliteProvider.Common/Configs/StorageConfig.cs`
- `SatelliteProvider.Common/Configs/ProcessingConfig.cs`
- `SatelliteProvider.Common/Configs/DatabaseConfig.cs`
- `SatelliteProvider.Common/Configs/UavQualityConfig.cs` (added by AZ-488; UAV quality-gate + request-envelope knobs)
- `SatelliteProvider.Common/DTO/*.cs` (all DTOs; AZ-488 added `UavTileMetadata`, `UavTileBatchMetadataPayload`, `UavTileBatchUploadResponse`, `UavTileUploadResultItem`, `UavTileUploadStatus`, `UavTileRejectReasons` — placed in Common to keep `TileDownloader` from depending on the API layer)
- `SatelliteProvider.Common/Enums/RegionStatus.cs`
- `SatelliteProvider.Common/Enums/RoutePointType.cs`
- `SatelliteProvider.Common/Enums/TileSource.cs` (added by AZ-484; backed by the `tile-storage` v1.0.0 contract)
- `SatelliteProvider.Common/Enums/TileSourceConverter.cs` (added by AZ-484; converts `TileSource` enum to/from the snake_case wire string used by `TileEntity.Source`)
- `SatelliteProvider.Common/Exceptions/RateLimitException.cs`
- `SatelliteProvider.Common/Interfaces/*.cs` (all service interfaces)
- `SatelliteProvider.Common/Utils/GeoUtils.cs`
- **Internal**: (none — all types are public, shared across components)
- **Owns**: `SatelliteProvider.Common/**`
- **Imports from**: (none)
- **Consumed by**: DataAccess, TileDownloader, RegionProcessing, RouteManagement, WebApi
### Component: DataAccess
- **Directory**: `SatelliteProvider.DataAccess/`
- **Public API**:
- `SatelliteProvider.DataAccess/Models/TileEntity.cs`
- `SatelliteProvider.DataAccess/Models/RegionEntity.cs`
- `SatelliteProvider.DataAccess/Models/RouteEntity.cs`
- `SatelliteProvider.DataAccess/Models/RoutePointEntity.cs`
- `SatelliteProvider.DataAccess/Repositories/ITileRepository.cs`
- `SatelliteProvider.DataAccess/Repositories/IRegionRepository.cs`
- `SatelliteProvider.DataAccess/Repositories/IRouteRepository.cs`
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs`
- `SatelliteProvider.DataAccess/Repositories/RegionRepository.cs`
- `SatelliteProvider.DataAccess/Repositories/RouteRepository.cs`
- `SatelliteProvider.DataAccess/DatabaseMigrator.cs`
- **Internal**: (none — all repository types are public for DI registration)
- **Owns**: `SatelliteProvider.DataAccess/**`
- **ProjectReferences**: `SatelliteProvider.Common`
- **Imports from**: `SatelliteProvider.Common.Enums` (6 sites: `RegionRepository`, `IRegionRepository`, `Models/RegionEntity`, `Models/RoutePointEntity`, `TypeHandlers/EnumStringTypeHandler`, `Models/TileEntity` — references `TileSourceConverter.GoogleMapsWireValue` const for the AZ-484 default value); `SatelliteProvider.Common.Configs` (`MapConfig.DefaultTileSizePixels` in `TileRepository`); `SatelliteProvider.Common.Utils` (`GeoUtils.EarthEquatorialCircumferenceMeters`, `GeoUtils.MetersPerDegreeLatitude` in `TileRepository`).
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement, WebApi
### Component: TileDownloader
- **Directory**: `SatelliteProvider.Services.TileDownloader/`
- **csproj**: `SatelliteProvider.Services.TileDownloader/SatelliteProvider.Services.TileDownloader.csproj`
- **Public API**:
- `SatelliteProvider.Services.TileDownloader/GoogleMapsDownloaderV2.cs` (implements `ISatelliteDownloader`)
- `SatelliteProvider.Services.TileDownloader/TileService.cs` (implements `ITileService`)
- `SatelliteProvider.Services.TileDownloader/UavTileQualityGate.cs` + `IUavTileQualityGate` (added by AZ-488; 5-rule synchronous validator over `ReadOnlyMemory<byte>` JPEGs, uses `SixLabors.ImageSharp` 3.1.11 + `TimeProvider`)
- `SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs` + `IUavTileUploadHandler` (added by AZ-488; orchestrates batch validation → file-first persistence → `ITileRepository.InsertAsync` UPSERT; owns the UAV `./tiles/uav/{z}/{x}/{y}.jpg` path layout)
- `SatelliteProvider.Services.TileDownloader/TileDownloaderServiceCollectionExtensions.cs` (DI: `AddTileDownloader()` — also registers the AZ-488 quality gate and upload handler as singletons)
- **Internal**: (none)
- **Owns**: `SatelliteProvider.Services.TileDownloader/**`
- **ProjectReferences**: `SatelliteProvider.Common`, `SatelliteProvider.DataAccess`
- **PackageReferences (added by AZ-488)**: `SixLabors.ImageSharp` 3.1.11 (image identify / `L8` decode / downsample for the variance heuristic).
- **Imports from**: Common, DataAccess
- **Consumed by**: RegionProcessing (via `ITileService` from Common; no direct ProjectReference), WebApi
### Component: RegionProcessing
- **Directory**: `SatelliteProvider.Services.RegionProcessing/`
- **csproj**: `SatelliteProvider.Services.RegionProcessing/SatelliteProvider.Services.RegionProcessing.csproj`
- **Public API**:
- `SatelliteProvider.Services.RegionProcessing/RegionService.cs` (implements `IRegionService`)
- `SatelliteProvider.Services.RegionProcessing/RegionProcessingService.cs` (background hosted service)
- `SatelliteProvider.Services.RegionProcessing/RegionRequestQueue.cs` (implements `IRegionRequestQueue`)
- `SatelliteProvider.Services.RegionProcessing/RegionProcessingServiceCollectionExtensions.cs` (DI: `AddRegionProcessing()`)
- **Internal**: (none)
- **Owns**: `SatelliteProvider.Services.RegionProcessing/**`
- **ProjectReferences**: `SatelliteProvider.Common`, `SatelliteProvider.DataAccess`
- **Imports from**: Common, DataAccess (uses `ITileService` from Common — no compile-time dependency on TileDownloader)
- **Consumed by**: RouteManagement (via `IRegionService` and `IRegionRequestQueue` from Common; no direct ProjectReference), WebApi
### Component: RouteManagement
- **Directory**: `SatelliteProvider.Services.RouteManagement/`
- **csproj**: `SatelliteProvider.Services.RouteManagement/SatelliteProvider.Services.RouteManagement.csproj`
- **Public API**:
- `SatelliteProvider.Services.RouteManagement/RouteService.cs` (implements `IRouteService`)
- `SatelliteProvider.Services.RouteManagement/RouteProcessingService.cs` (background hosted service)
- `SatelliteProvider.Services.RouteManagement/RouteManagementServiceCollectionExtensions.cs` (DI: `AddRouteManagement()`)
- **Internal**: (none)
- **Owns**: `SatelliteProvider.Services.RouteManagement/**`
- **ProjectReferences**: `SatelliteProvider.Common`, `SatelliteProvider.DataAccess`
- **Imports from**: Common, DataAccess (uses `IRegionService` / `IRegionRequestQueue` from Common — no compile-time dependency on RegionProcessing)
- **Consumed by**: WebApi
### Component: WebApi
- **Directory**: `SatelliteProvider.Api/`
- **Public API**:
- `SatelliteProvider.Api/Program.cs` (minimal API endpoints, DI setup, middleware chain — `UseAuthentication` + `UseAuthorization` added in AZ-487; `/api/satellite/upload` rewired in AZ-488)
- `SatelliteProvider.Api/Authentication/AuthenticationServiceCollectionExtensions.cs` (added by AZ-487; `AddSatelliteJwt(IConfiguration)` registers `JwtBearer` with the suite-wide HS256 contract from `suite/_docs/10_auth.md`; validates `JWT_SECRET` ≥ 32 bytes at startup)
- `SatelliteProvider.Api/Authentication/PermissionsRequirement.cs` + `PermissionsAuthorizationHandler` + `SatellitePermissions` (added by AZ-488; custom requirement that accepts a `permissions` claim shaped as either a single string or a JSON array; powers the `UavUploadPolicy` requiring the `GPS` permission)
- `SatelliteProvider.Api/DTOs/UavTileBatchUploadRequest.cs` (added by AZ-488; multipart form binding envelope — kept in WebApi because it depends on `IFormFileCollection` + `[FromForm]`, both API-layer types)
- **Internal**: (none)
- **Owns**: `SatelliteProvider.Api/**`
- **PackageReferences (added by AZ-487, bumped by AZ-496)**: `Microsoft.AspNetCore.Authentication.JwtBearer` 8.0.25 (pinned to the same minor patch as `Microsoft.AspNetCore.OpenApi` 8.0.25; AZ-496 bumped both packages from 8.0.21 → 8.0.25 to close cycle-1 D1 + cycle-2 D3 supply-chain findings).
- **Imports from**: Common (incl. AZ-488 UAV DTOs + `UavQualityConfig`), DataAccess, TileDownloader (incl. AZ-488 `IUavTileUploadHandler`), RegionProcessing, RouteManagement
- **Consumed by**: (none — top-level entry point)
## Shared / Cross-Cutting
### Common/Configs
- **Directory**: `SatelliteProvider.Common/Configs/`
- **Purpose**: Strongly-typed configuration POCOs bound via `IOptions<T>`
- **Consumed by**: all components
### Common/DTO
- **Directory**: `SatelliteProvider.Common/DTO/`
- **Purpose**: Data transfer objects shared across layers (request/response models, value types)
- **Consumed by**: all components
### Common/Interfaces
- **Directory**: `SatelliteProvider.Common/Interfaces/`
- **Purpose**: Service contracts enabling DI and testability
- **Consumed by**: all components (services implement, API and consumers depend on)
### Common/Utils
- **Directory**: `SatelliteProvider.Common/Utils/`
- **Purpose**: Stateless geospatial utility functions (coordinate math, distance, bearing)
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement
### Common/Enums
- **Directory**: `SatelliteProvider.Common/Enums/`
- **Purpose**: Domain enums shared across layers (`RegionStatus`, `RoutePointType`, `TileSource`) plus their explicit wire-value converters when persistence requires snake_case strings (`TileSourceConverter`). Converter classes belong here — not in DataAccess — because they encode a domain-level vocabulary that must be visible to every component.
- **Consumed by**: DataAccess (entity defaults, type handler registration), TileDownloader (sets `TileEntity.Source` via `TileSourceConverter.ToWireValue`), Tests
- **Important constraint**: Dapper's `SqlMapper.TypeHandler<TEnum>` is bypassed for enum reads (Dapper issue #259 — see `_docs/LESSONS.md` L-001). For any new enum that must round-trip through a database column, prefer the `string`-on-entity + `Enum`-at-API-boundary pattern with a converter class in this folder. Do NOT register a `TypeHandler<TEnum>` and assume it will be honored on reads.
### TestSupport (added by AZ-491; extended by AZ-493)
- **Directory**: `SatelliteProvider.TestSupport/`
- **csproj**: `SatelliteProvider.TestSupport/SatelliteProvider.TestSupport.csproj` (class library, no test framework)
- **Purpose**: Canonical home for cross-project test utilities. Currently holds `JwtTokenFactory` (HS256 token minting + signature tampering) and `IntegrationTestResetGuard` (pure-string guard for the integration-test DB-reset hook). Replaces the cycle-2 duplicate that lived in both `SatelliteProvider.Tests/TestUtilities/JwtTokenFactory.cs` and `SatelliteProvider.IntegrationTests/JwtTestHelpers.cs` and required parallel fixes. Future additions: shared image-fixture factories, shared deterministic clocks / test-data builders that need to be visible to both unit and integration projects.
- **Public API**:
- `SatelliteProvider.TestSupport/JwtTokenFactory.cs` (`Create`, `CreateExpired`, `TamperSignature`) — added by AZ-491.
- `SatelliteProvider.TestSupport/IntegrationTestResetGuard.cs` (`EnsureGuardPassesOrThrow`, `AllowedHosts`, `EnvironmentEnvVar`, `TestingEnvironment`) — added by AZ-493. Pure static class — no I/O, no DB calls. Consumed by `SatelliteProvider.IntegrationTests/IntegrationTestDatabaseReset.cs` (instance class that owns the Npgsql side effects) and unit-tested in `SatelliteProvider.Tests/TestSupport/IntegrationTestResetGuardTests.cs`.
- **PackageReferences**: `Microsoft.IdentityModel.Tokens` 7.0.3, `System.IdentityModel.Tokens.Jwt` 7.0.3 (matches the integration tests' pre-AZ-491 explicit reference). The AZ-493 guard introduced no new package dependencies — it is pure string comparison over the BCL.
- **Consumed by**: `SatelliteProvider.Tests`, `SatelliteProvider.IntegrationTests` (both via `ProjectReference`).
- **Not consumed by**: production projects (`Api`, `Common`, `DataAccess`, `Services.*`). The TestSupport library is test-only by design; production code must NOT depend on it.
- **Runner-side concerns NOT in TestSupport**: `SatelliteProvider.IntegrationTests/JwtTestHelpers.cs` retains `ResolveSecretOrThrow`, `AttachDefaultAuthorization`, and the `DefaultSubject = "integration-tests"` constant — these are runner-specific (env-var reads, `HttpClient` mutation, runner-identity subject) and intentionally not consolidated. `SatelliteProvider.IntegrationTests/IntegrationTestDatabaseReset.cs` (AZ-493) holds the Npgsql side effects of the reset — it sits in the integration-tests project (not TestSupport) so the Npgsql dependency doesn't leak into unit tests.
## Allowed Dependencies (layering)
| Layer | Components | May import from (compile-time ProjectReferences) |
|-------|------------|--------------------------------------------------|
| 4. API / Entry | WebApi | Common, DataAccess, TileDownloader, RegionProcessing, RouteManagement |
| 3. Application | TileDownloader, RegionProcessing, RouteManagement | Common, DataAccess only — siblings communicate through interfaces in Common, never through direct ProjectReferences |
| 1. Foundation | Common (leaf-most), DataAccess | Common: (none); DataAccess: Common only — Common MUST NOT import from DataAccess |
**Key constraint enforced by the AZ-309 split**: the three Layer-3 components are compile-time siblings. Any cross-sibling call (e.g. `RegionProcessing` invoking tile download) MUST go through an interface defined in `SatelliteProvider.Common.Interfaces` and resolved via DI — adding a `ProjectReference` between siblings is now structurally impossible without re-introducing the coupling the refactor removed.
## Verification
- **No detected cycles**: The dependency graph is a clean DAG.
- **No cross-sibling ProjectReferences**: TileDownloader, RegionProcessing, and RouteManagement each reference only Common + DataAccess. Verified by inspecting all three csproj files.
- **DataAccess layer placement**: DataAccess sits at Layer 1 (Foundation) alongside Common because it is consumed uniformly by all service components. It is one half-step above Common because it depends on Common for shared enums and a small number of constants/configs.
- **DataAccess→Common ProjectReference**: confirmed present in `SatelliteProvider.DataAccess.csproj` line 18 and used by 7 source sites (5 enum imports, 1 `MapConfig.DefaultTileSizePixels` site, 1 `GeoUtils.*` site). The earlier compliance baseline F5 entry that claimed "DataAccess has no Common dependency" was inaccurate — both `module-layout.md` and `architecture_compliance_baseline.md` were corrected during the 03-code-quality-refactoring run (2026-05-11). The actual constraint that holds is one-way: `Common` MUST NOT import from `DataAccess`.
Reference in New Issue View Git Blame Copy Permalink