documentation

AGENTS.md

@@ -0,0 +1,291 @@
+# Satellite Provider - Agent Documentation
+
+## System Overview
+
+This is a .NET 8.0 ASP.NET Web API service that downloads, stores, and manages satellite imagery tiles from Google Maps. The service supports region-based tile requests, route planning with intermediate points, and geofencing capabilities.
+
+## Tech Stack
+
+- **.NET 8.0** with ASP.NET Core Web API
+- **PostgreSQL** database (via Docker)
+- **Dapper** for data access (ORM)
+- **DbUp** for database migrations
+- **Serilog** for logging
+- **Swagger/OpenAPI** for API documentation
+- **XUnit** and **Moq** for testing
+- **ImageSharp** for image processing
+- **Docker** for containerization
+
+## Architecture
+
+### Project Structure
+
+```
+SatelliteProvider.Api/           # Web API layer (Program.cs)
+SatelliteProvider.Common/        # DTOs, interfaces, configs, utils
+SatelliteProvider.DataAccess/    # Database entities, repositories, migrations
+SatelliteProvider.Services/      # Business logic implementations
+SatelliteProvider.Tests/         # Unit tests
+SatelliteProvider.IntegrationTests/ # Integration tests (console app)
+```
+
+### Key Components
+
+1. **API Layer** (`Program.cs`)
+   - Minimal API endpoints
+   - Dependency injection configuration
+   - Swagger documentation
+   - Database migration on startup
+
+2. **Services**
+   - `TileService`: Downloads and stores individual tiles
+   - `RegionService`: Manages region requests and processes tiles for regions
+   - `RouteService`: Creates routes with intermediate points and manages route regions
+   - `GoogleMapsDownloaderV2`: Handles actual tile downloads from Google Maps
+   - `RegionProcessingService`: Background hosted service for processing region requests
+   - `RouteProcessingService`: Background hosted service for processing route map requests
+
+3. **Data Access**
+   - Repositories: `TileRepository`, `RegionRepository`, `RouteRepository`
+   - Entities: `TileEntity`, `RegionEntity`, `RouteEntity`, `RoutePointEntity`
+   - Migrations in `SatelliteProvider.DataAccess/Migrations/` (numbered SQL files)
+
+4. **Async Processing**
+   - Queue-based processing via `IRegionRequestQueue`
+   - Background services process requests asynchronously
+   - Status tracking via database
+
+## Database Schema
+
+### Tables
+
+**tiles**
+- Stores downloaded satellite tiles
+- Fields: id, zoom_level, latitude, longitude, tile_size_meters, tile_size_pixels, image_type, maps_version, version, file_path, created_at, updated_at
+- Composite index on (latitude, longitude, tile_size_meters, zoom_level)
+
+**regions**
+- Stores region requests and their processing status
+- Fields: id, latitude, longitude, size_meters, zoom_level, status, stitch_tiles, stitched_image_path, csv_file_path, summary_file_path, tiles_downloaded, tiles_reused, created_at, updated_at
+- Status values: "pending", "processing", "completed", "failed"
+
+**routes**
+- Stores route definitions
+- Fields: id, name, description, region_size_meters, zoom_level, total_distance_meters, total_points, request_maps, create_tiles_zip, tiles_zip_path, geofence_polygons, created_at, updated_at
+
+**route_points**
+- Stores all points (original and interpolated) for routes
+- Fields: id, route_id, sequence_number, latitude, longitude, point_type, segment_index, distance_from_previous, created_at
+- Point types: "original", "intermediate"
+
+**route_regions**
+- Junction table linking routes to regions for map tile requests
+- Fields: route_id, region_id, inside_geofence, created_at
+
+### Migration Strategy
+- SQL-based migrations managed by DbUp
+- Embedded resources in assembly
+- Sequential numbered files (001_, 002_, etc.)
+- Runs automatically on application startup
+
+## API Endpoints
+
+### Tile Management
+- `GET /api/satellite/tiles/latlon` - Download single tile by lat/lon and zoom level
+- `GET /api/satellite/tiles/mgrs` - Get tiles by MGRS coordinates (stub)
+
+### Region Management
+- `POST /api/satellite/request` - Request tiles for a region (async processing)
+- `GET /api/satellite/region/{id}` - Get region status and file paths
+
+### Route Management
+- `POST /api/satellite/route` - Create route with intermediate points
+- `GET /api/satellite/route/{id}` - Get route information
+
+### Image Upload
+- `POST /api/satellite/upload` - Upload image with metadata (stub)
+
+## Key Patterns and Conventions
+
+### Naming
+- Database: snake_case (tiles, route_points)
+- C# Code: PascalCase for classes, camelCase for parameters
+- Entities suffix: `Entity` (TileEntity, RegionEntity)
+- Interfaces prefix: `I` (ITileService, IRegionService)
+
+### Configuration
+Configuration sections in appsettings.json:
+- `MapConfig`: Google Maps API settings
+- `StorageConfig`: File storage paths (tiles, ready directories)
+- `ProcessingConfig`: Concurrency and processing limits
+- `ConnectionStrings`: Database connection
+
+### File Storage
+- Tiles stored in: `./tiles/{zoomLevel}/{x}/{y}.jpg`
+- CSV outputs: `./ready/region_{id}_ready.csv`
+- Summary files: `./ready/region_{id}_summary.txt`
+- Stitched images: `./ready/region_{id}_stitched.jpg`
+- Route zip files: `./ready/route_{id}_tiles.zip`
+
+### Async Processing Flow
+1. Client posts request to API
+2. Request queued in `IRegionRequestQueue`
+3. Background service picks up request
+4. Downloads/processes tiles
+5. Updates database status
+6. Creates output files
+
+### Geofencing
+- Routes can define multiple polygon geofences
+- Each geofence has an array of lat/lon points
+- Route regions are marked if they fall inside/outside geofences
+- Used to filter which regions to process for map tiles
+
+### Route Processing
+- Routes broken into segments between original points
+- Intermediate points calculated every ~200 meters
+- Each point becomes center of a region request
+- Distance calculations use Haversine formula
+
+## Development Workflow
+
+### Running Locally
+
+**Start service and dependencies:**
+```bash
+docker-compose up --build
+```
+
+**Run tests:**
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.tests.yml up --build --abort-on-container-exit
+```
+
+### Important Notes
+
+1. **DO NOT run `dotnet build` or `dotnet restore` via terminal tools** - these commands hang. Ask user to run manually.
+
+2. **Test verification:** Primary method is via Docker Compose with tests.
+
+3. **File system structure:** Service expects mounted volumes for:
+   - `./tiles` - tile storage
+   - `./ready` - output files
+   - `./logs` - application logs
+
+4. **Database:** Auto-creates schema on startup via migrations.
+
+### Configuration Values
+
+Development defaults:
+- PostgreSQL: localhost:5432, user/pass: postgres/postgres
+- API: http://localhost:5100
+- Max zoom level: 20
+- Default zoom level: 18
+- Queue capacity: 1000
+- Max concurrent downloads: 4
+- Max concurrent regions: 20
+
+## Testing
+
+### Integration Tests
+- Console application in `SatelliteProvider.IntegrationTests`
+- Tests run in Docker container
+- Tests cover:
+  - Tile downloads
+  - Region requests and processing
+  - Route creation and point interpolation
+  - Geofencing logic
+  - Extended routes with map requests
+
+### Test Helpers
+`RouteTestHelpers.cs` provides utilities:
+- Waiting for region completion
+- Creating test geofence polygons
+- Common test data builders
+
+## Common Tasks
+
+### Adding New Migration
+1. Create numbered SQL file in `SatelliteProvider.DataAccess/Migrations/`
+2. Set Build Action to `EmbeddedResource`
+3. Follow naming: `NNN_DescriptiveName.sql`
+4. Migration runs automatically on next startup
+
+### Adding New API Endpoint
+1. Add endpoint mapping in `Program.cs`
+2. Add handler method (async preferred)
+3. Define request/response DTOs
+4. Add Swagger documentation with `.WithOpenApi()`
+5. Update interfaces if adding service methods
+
+### Adding New Configuration
+1. Add config class in `SatelliteProvider.Common/Configs/`
+2. Register in `Program.cs` with `builder.Services.Configure<T>()`
+3. Add section to `appsettings.json` and `appsettings.Development.json`
+4. Inject via `IOptions<T>` in services
+
+## Code Style Guidelines
+
+- No comments in code (self-documenting code preferred)
+- No excessive logging (only exceptions unless specifically needed)
+- Simple, concise solutions preferred
+- Minimal code changes related to task
+- Avoid renaming database objects without confirmation
+- Check for duplicate code before adding new functionality
+- Consider environment differences (dev/prod)
+
+## Dependencies and Versions
+
+Key packages (all .NET 8.0):
+- Microsoft.AspNetCore.OpenApi 8.0.21
+- Swashbuckle.AspNetCore 6.6.2
+- Serilog.AspNetCore 8.0.3
+- SixLabors.ImageSharp 3.1.11
+- Newtonsoft.Json 13.0.4
+- Dapper (check DataAccess csproj)
+- Npgsql (check DataAccess csproj)
+- DbUp (check DataAccess csproj)
+
+## Outstanding TODOs
+
+From goal.md:
+1. ✅ Add geo fences (2 regions) - IMPLEMENTED
+2. ✅ Add parameter to zip resulting tiles, 50 mb max - IMPLEMENTED
+3. ✅ Implement API to download tile - lat, lon and zoom level - IMPLEMENTED
+4. ⚠️ Implement parallel tiles fetching from google maps - PARTIAL (session token reuse implemented)
+
+## Docker Structure
+
+**docker-compose.yml:**
+- PostgreSQL database service
+- SatelliteProvider.Api service
+- Shared network
+- Volume mounts for tiles, ready, logs
+
+**docker-compose.tests.yml:**
+- Integration tests service
+- Depends on api and db services
+- Runs and exits after completion
+
+## Troubleshooting
+
+**Database connection issues:**
+- Check PostgreSQL container is running
+- Verify connection string in appsettings
+- Check network connectivity between containers
+
+**Tile download failures:**
+- Check Google Maps API key in MapConfig
+- Verify internet connectivity
+- Check rate limiting settings
+
+**Test failures:**
+- Ensure all services are up before tests run
+- Check logs in `./logs/` directory
+- Verify database migrations completed
+
+**Queue processing stuck:**
+- Check RegionProcessingService and RouteProcessingService are running
+- Verify queue capacity not exceeded
+- Check database connection for status updates
+