satellite-provider/AGENTS.md

Satellite Provider - Agent Documentation

System Overview

This is a .NET 8.0 ASP.NET Web API service that downloads, stores, and manages satellite imagery tiles from Google Maps. The service supports region-based tile requests, route planning with intermediate points, and geofencing capabilities.

Tech Stack

• .NET 8.0 with ASP.NET Core Web API
• PostgreSQL database (via Docker)
• Dapper for data access (ORM)
• DbUp for database migrations
• Serilog for logging
• Swagger/OpenAPI for API documentation
• XUnit and Moq for testing
• ImageSharp for image processing
• Docker for containerization

Architecture

Project Structure

SatelliteProvider.Api/           # Web API layer (Program.cs)
SatelliteProvider.Common/        # DTOs, interfaces, configs, utils
SatelliteProvider.DataAccess/    # Database entities, repositories, migrations
SatelliteProvider.Services/      # Business logic implementations
SatelliteProvider.Tests/         # Unit tests
SatelliteProvider.IntegrationTests/ # Integration tests (console app)

Key Components

1. API Layer (Program.cs)

   • Minimal API endpoints
   • Dependency injection configuration
   • Swagger documentation
   • Database migration on startup

2. Services

   • TileService: Downloads and stores individual tiles
   • RegionService: Manages region requests and processes tiles for regions
   • RouteService: Creates routes with intermediate points and manages route regions
   • GoogleMapsDownloaderV2: Handles actual tile downloads from Google Maps
   • RegionProcessingService: Background hosted service for processing region requests
   • RouteProcessingService: Background hosted service for processing route map requests

3. Data Access

   • Repositories: TileRepository, RegionRepository, RouteRepository
   • Entities: TileEntity, RegionEntity, RouteEntity, RoutePointEntity
   • Migrations in SatelliteProvider.DataAccess/Migrations/ (numbered SQL files)

4. Async Processing

   • Queue-based processing via IRegionRequestQueue
   • Background services process requests asynchronously
   • Status tracking via database

Database Schema

Tables

tiles

• Stores downloaded satellite tiles
• Fields: id, zoom_level, latitude, longitude, tile_size_meters, tile_size_pixels, image_type, maps_version, version, file_path, created_at, updated_at
• Composite index on (latitude, longitude, tile_size_meters, zoom_level)

regions

• Stores region requests and their processing status
• Fields: id, latitude, longitude, size_meters, zoom_level, status, stitch_tiles, stitched_image_path, csv_file_path, summary_file_path, tiles_downloaded, tiles_reused, created_at, updated_at
• Status values: "pending", "processing", "completed", "failed"

routes

• Stores route definitions
• Fields: id, name, description, region_size_meters, zoom_level, total_distance_meters, total_points, request_maps, create_tiles_zip, tiles_zip_path, geofence_polygons, created_at, updated_at

route_points

• Stores all points (original and interpolated) for routes
• Fields: id, route_id, sequence_number, latitude, longitude, point_type, segment_index, distance_from_previous, created_at
• Point types: "original", "intermediate"

route_regions

• Junction table linking routes to regions for map tile requests
• Fields: route_id, region_id, inside_geofence, created_at

Migration Strategy

• SQL-based migrations managed by DbUp
• Embedded resources in assembly
• Sequential numbered files (001_, 002_, etc.)
• Runs automatically on application startup

API Endpoints

Tile Management

• GET /api/satellite/tiles/latlon - Download single tile by lat/lon and zoom level
• GET /api/satellite/tiles/mgrs - Get tiles by MGRS coordinates (stub)

Region Management

• POST /api/satellite/request - Request tiles for a region (async processing)
• GET /api/satellite/region/{id} - Get region status and file paths

Route Management

• POST /api/satellite/route - Create route with intermediate points
• GET /api/satellite/route/{id} - Get route information

Image Upload

• POST /api/satellite/upload - Upload image with metadata (stub)

Key Patterns and Conventions

Naming

• Database: snake_case (tiles, route_points)
• C# Code: PascalCase for classes, camelCase for parameters
• Entities suffix: Entity (TileEntity, RegionEntity)
• Interfaces prefix: I (ITileService, IRegionService)

Configuration

Configuration sections in appsettings.json:

• MapConfig: Google Maps API settings
• StorageConfig: File storage paths (tiles, ready directories)
• ProcessingConfig: Concurrency and processing limits
• ConnectionStrings: Database connection

File Storage

• Tiles stored in: ./tiles/{zoomLevel}/{x}/{y}.jpg
• CSV outputs: ./ready/region_{id}_ready.csv
• Summary files: ./ready/region_{id}_summary.txt
• Stitched images: ./ready/region_{id}_stitched.jpg
• Route zip files: ./ready/route_{id}_tiles.zip

Async Processing Flow

1. Client posts request to API
2. Request queued in IRegionRequestQueue
3. Background service picks up request
4. Downloads/processes tiles
5. Updates database status
6. Creates output files

Geofencing

• Routes can define multiple polygon geofences
• Each geofence has an array of lat/lon points
• Route regions are marked if they fall inside/outside geofences
• Used to filter which regions to process for map tiles

Route Processing

• Routes broken into segments between original points
• Intermediate points calculated every ~200 meters
• Each point becomes center of a region request
• Distance calculations use Haversine formula

Development Workflow

Running Locally

Start service and dependencies:

docker-compose up --build

Run tests:

docker-compose -f docker-compose.yml -f docker-compose.tests.yml up --build --abort-on-container-exit

Important Notes

1. DO NOT run dotnet build or dotnet restore via terminal tools - these commands hang. Ask user to run manually.

2. Test verification: Primary method is via Docker Compose with tests.

3. File system structure: Service expects mounted volumes for:

   • ./tiles - tile storage
   • ./ready - output files
   • ./logs - application logs

4. Database: Auto-creates schema on startup via migrations.

Configuration Values

Development defaults:

• PostgreSQL: localhost:5432, user/pass: postgres/postgres
• API: http://localhost:5100
• Max zoom level: 20
• Default zoom level: 18
• Queue capacity: 1000
• Max concurrent downloads: 4
• Max concurrent regions: 20

Testing

Integration Tests

• Console application in SatelliteProvider.IntegrationTests
• Tests run in Docker container
• Tests cover:
  • Tile downloads
  • Region requests and processing
  • Route creation and point interpolation
  • Geofencing logic
  • Extended routes with map requests

Test Helpers

RouteTestHelpers.cs provides utilities:

• Waiting for region completion
• Creating test geofence polygons
• Common test data builders

Common Tasks

Adding New Migration

1. Create numbered SQL file in SatelliteProvider.DataAccess/Migrations/
2. Set Build Action to EmbeddedResource
3. Follow naming: NNN_DescriptiveName.sql
4. Migration runs automatically on next startup

Adding New API Endpoint

1. Add endpoint mapping in Program.cs
2. Add handler method (async preferred)
3. Define request/response DTOs
4. Add Swagger documentation with .WithOpenApi()
5. Update interfaces if adding service methods

Adding New Configuration

1. Add config class in SatelliteProvider.Common/Configs/
2. Register in Program.cs with builder.Services.Configure<T>()
3. Add section to appsettings.json and appsettings.Development.json
4. Inject via IOptions<T> in services

Code Style Guidelines

• No comments in code (self-documenting code preferred)
• No excessive logging (only exceptions unless specifically needed)
• Simple, concise solutions preferred
• Minimal code changes related to task
• Avoid renaming database objects without confirmation
• Check for duplicate code before adding new functionality
• Consider environment differences (dev/prod)

Dependencies and Versions

Key packages (all .NET 8.0):

• Microsoft.AspNetCore.OpenApi 8.0.21
• Swashbuckle.AspNetCore 6.6.2
• Serilog.AspNetCore 8.0.3
• SixLabors.ImageSharp 3.1.11
• Newtonsoft.Json 13.0.4
• Dapper (check DataAccess csproj)
• Npgsql (check DataAccess csproj)
• DbUp (check DataAccess csproj)

Outstanding TODOs

From goal.md:

1. ✅ Add geo fences (2 regions) - IMPLEMENTED
2. ✅ Add parameter to zip resulting tiles, 50 mb max - IMPLEMENTED
3. ✅ Implement API to download tile - lat, lon and zoom level - IMPLEMENTED
4. ⚠️ Implement parallel tiles fetching from google maps - PARTIAL (session token reuse implemented)

Docker Structure

docker-compose.yml:

• PostgreSQL database service
• SatelliteProvider.Api service
• Shared network
• Volume mounts for tiles, ready, logs

docker-compose.tests.yml:

• Integration tests service
• Depends on api and db services
• Runs and exits after completion

Troubleshooting

Database connection issues:

• Check PostgreSQL container is running
• Verify connection string in appsettings
• Check network connectivity between containers

Tile download failures:

• Check Google Maps API key in MapConfig
• Verify internet connectivity
• Check rate limiting settings

Test failures:

• Ensure all services are up before tests run
• Check logs in ./logs/ directory
• Verify database migrations completed

Queue processing stuck:

• Check RegionProcessingService and RouteProcessingService are running
• Verify queue capacity not exceeded
• Check database connection for status updates