satellite-provider/AGENTS.md

# Satellite Provider - Agent Documentation

## System Overview

This is a .NET 8.0 ASP.NET Web API service that downloads, stores, and manages satellite imagery tiles from Google Maps. The service supports region-based tile requests, route planning with intermediate points, and geofencing capabilities.

## Tech Stack

- **.NET 8.0** with ASP.NET Core Web API
- **PostgreSQL** database (via Docker)
- **Dapper** for data access (ORM)
- **DbUp** for database migrations
- **Serilog** for logging
- **Swagger/OpenAPI** for API documentation
- **XUnit** and **Moq** for testing
- **ImageSharp** for image processing
- **Docker** for containerization

## Architecture

### Project Structure

```
SatelliteProvider.Api/           # Web API layer (Program.cs)
SatelliteProvider.Common/        # DTOs, interfaces, configs, utils
SatelliteProvider.DataAccess/    # Database entities, repositories, migrations
SatelliteProvider.Services/      # Business logic implementations
SatelliteProvider.Tests/         # Unit tests
SatelliteProvider.IntegrationTests/ # Integration tests (console app)
```

### Key Components

1. **API Layer** (`Program.cs`)
   - Minimal API endpoints
   - Dependency injection configuration
   - Swagger documentation
   - Database migration on startup

2. **Services**
   - `TileService`: Downloads and stores individual tiles
   - `RegionService`: Manages region requests and processes tiles for regions
   - `RouteService`: Creates routes with intermediate points and manages route regions
   - `GoogleMapsDownloaderV2`: Handles actual tile downloads from Google Maps
   - `RegionProcessingService`: Background hosted service for processing region requests
   - `RouteProcessingService`: Background hosted service for processing route map requests

3. **Data Access**
   - Repositories: `TileRepository`, `RegionRepository`, `RouteRepository`
   - Entities: `TileEntity`, `RegionEntity`, `RouteEntity`, `RoutePointEntity`
   - Migrations in `SatelliteProvider.DataAccess/Migrations/` (numbered SQL files)

4. **Async Processing**
   - Queue-based processing via `IRegionRequestQueue`
   - Background services process requests asynchronously
   - Status tracking via database

## Database Schema

### Tables

**tiles**
- Stores downloaded satellite tiles
- Fields: id, zoom_level, latitude, longitude, tile_size_meters, tile_size_pixels, image_type, maps_version, version, file_path, created_at, updated_at
- Composite index on (latitude, longitude, tile_size_meters, zoom_level)

**regions**
- Stores region requests and their processing status
- Fields: id, latitude, longitude, size_meters, zoom_level, status, stitch_tiles, stitched_image_path, csv_file_path, summary_file_path, tiles_downloaded, tiles_reused, created_at, updated_at
- Status values: "pending", "processing", "completed", "failed"

**routes**
- Stores route definitions
- Fields: id, name, description, region_size_meters, zoom_level, total_distance_meters, total_points, request_maps, create_tiles_zip, tiles_zip_path, geofence_polygons, created_at, updated_at

**route_points**
- Stores all points (original and interpolated) for routes
- Fields: id, route_id, sequence_number, latitude, longitude, point_type, segment_index, distance_from_previous, created_at
- Point types: "original", "intermediate"

**route_regions**
- Junction table linking routes to regions for map tile requests
- Fields: route_id, region_id, inside_geofence, created_at

### Migration Strategy
- SQL-based migrations managed by DbUp
- Embedded resources in assembly
- Sequential numbered files (001_, 002_, etc.)
- Runs automatically on application startup

## API Endpoints

### Tile Management
- `GET /api/satellite/tiles/latlon` - Download single tile by lat/lon and zoom level
- `GET /api/satellite/tiles/mgrs` - Get tiles by MGRS coordinates (stub)

### Region Management
- `POST /api/satellite/request` - Request tiles for a region (async processing)
- `GET /api/satellite/region/{id}` - Get region status and file paths

### Route Management
- `POST /api/satellite/route` - Create route with intermediate points
- `GET /api/satellite/route/{id}` - Get route information

### Image Upload
- `POST /api/satellite/upload` - Upload image with metadata (stub)

## Key Patterns and Conventions

### Naming
- Database: snake_case (tiles, route_points)
- C# Code: PascalCase for classes, camelCase for parameters
- Entities suffix: `Entity` (TileEntity, RegionEntity)
- Interfaces prefix: `I` (ITileService, IRegionService)

### Configuration
Configuration sections in appsettings.json:
- `MapConfig`: Google Maps API settings
- `StorageConfig`: File storage paths (tiles, ready directories)
- `ProcessingConfig`: Concurrency and processing limits
- `ConnectionStrings`: Database connection

### File Storage
- Tiles stored in: `./tiles/{zoomLevel}/{x}/{y}.jpg`
- CSV outputs: `./ready/region_{id}_ready.csv`
- Summary files: `./ready/region_{id}_summary.txt`
- Stitched images: `./ready/region_{id}_stitched.jpg`
- Route zip files: `./ready/route_{id}_tiles.zip`

### Async Processing Flow
1. Client posts request to API
2. Request queued in `IRegionRequestQueue`
3. Background service picks up request
4. Downloads/processes tiles
5. Updates database status
6. Creates output files

### Geofencing
- Routes can define multiple polygon geofences
- Each geofence has an array of lat/lon points
- Route regions are marked if they fall inside/outside geofences
- Used to filter which regions to process for map tiles

### Route Processing
- Routes broken into segments between original points
- Intermediate points calculated every ~200 meters
- Each point becomes center of a region request
- Distance calculations use Haversine formula

## Development Workflow

### Running Locally

**Start service and dependencies:**
```bash
docker-compose up --build
```

**Run tests:**
```bash
docker-compose -f docker-compose.yml -f docker-compose.tests.yml up --build --abort-on-container-exit
```

### Important Notes

1. **DO NOT run `dotnet build` or `dotnet restore` via terminal tools** - these commands hang. Ask user to run manually.

2. **Test verification:** Primary method is via Docker Compose with tests.

3. **File system structure:** Service expects mounted volumes for:
   - `./tiles` - tile storage
   - `./ready` - output files
   - `./logs` - application logs

4. **Database:** Auto-creates schema on startup via migrations.

### Configuration Values

Development defaults:
- PostgreSQL: localhost:5432, user/pass: postgres/postgres
- API: http://localhost:5100
- Max zoom level: 20
- Default zoom level: 18
- Queue capacity: 1000
- Max concurrent downloads: 4
- Max concurrent regions: 20

## Testing

### Integration Tests
- Console application in `SatelliteProvider.IntegrationTests`
- Tests run in Docker container
- Tests cover:
  - Tile downloads
  - Region requests and processing
  - Route creation and point interpolation
  - Geofencing logic
  - Extended routes with map requests

### Test Helpers
`RouteTestHelpers.cs` provides utilities:
- Waiting for region completion
- Creating test geofence polygons
- Common test data builders

## Common Tasks

### Adding New Migration
1. Create numbered SQL file in `SatelliteProvider.DataAccess/Migrations/`
2. Set Build Action to `EmbeddedResource`
3. Follow naming: `NNN_DescriptiveName.sql`
4. Migration runs automatically on next startup

### Adding New API Endpoint
1. Add endpoint mapping in `Program.cs`
2. Add handler method (async preferred)
3. Define request/response DTOs
4. Add Swagger documentation with `.WithOpenApi()`
5. Update interfaces if adding service methods

### Adding New Configuration
1. Add config class in `SatelliteProvider.Common/Configs/`
2. Register in `Program.cs` with `builder.Services.Configure<T>()`
3. Add section to `appsettings.json` and `appsettings.Development.json`
4. Inject via `IOptions<T>` in services

## Code Style Guidelines

- No comments in code (self-documenting code preferred)
- No excessive logging (only exceptions unless specifically needed)
- Simple, concise solutions preferred
- Minimal code changes related to task
- Avoid renaming database objects without confirmation
- Check for duplicate code before adding new functionality
- Consider environment differences (dev/prod)

## Dependencies and Versions

Key packages (all .NET 8.0):
- Microsoft.AspNetCore.OpenApi 8.0.21
- Swashbuckle.AspNetCore 6.6.2
- Serilog.AspNetCore 8.0.3
- SixLabors.ImageSharp 3.1.11
- Newtonsoft.Json 13.0.4
- Dapper (check DataAccess csproj)
- Npgsql (check DataAccess csproj)
- DbUp (check DataAccess csproj)

## Outstanding TODOs

From goal.md:
1. ✅ Add geo fences (2 regions) - IMPLEMENTED
2. ✅ Add parameter to zip resulting tiles, 50 mb max - IMPLEMENTED
3. ✅ Implement API to download tile - lat, lon and zoom level - IMPLEMENTED
4. ⚠️ Implement parallel tiles fetching from google maps - PARTIAL (session token reuse implemented)

## Docker Structure

**docker-compose.yml:**
- PostgreSQL database service
- SatelliteProvider.Api service
- Shared network
- Volume mounts for tiles, ready, logs

**docker-compose.tests.yml:**
- Integration tests service
- Depends on api and db services
- Runs and exits after completion

## Troubleshooting

**Database connection issues:**
- Check PostgreSQL container is running
- Verify connection string in appsettings
- Check network connectivity between containers

**Tile download failures:**
- Check Google Maps API key in MapConfig
- Verify internet connectivity
- Check rate limiting settings

**Test failures:**
- Ensure all services are up before tests run
- Check logs in `./logs/` directory
- Verify database migrations completed

**Queue processing stuck:**
- Check RegionProcessingService and RouteProcessingService are running
- Verify queue capacity not exceeded
- Check database connection for status updates