satellite-provider/README.md

# Satellite Provider API

A .NET 8.0 microservice for downloading, managing, and serving satellite imagery tiles from Google Maps. Supports region-based tile requests, route planning with automatic intermediate point generation, and geofencing capabilities.

## Features

- **Tile Management**: Download and cache satellite imagery tiles from Google Maps
- **Region Requests**: Request tiles for specific geographic areas (100m - 10km squares)
- **Route Planning**: Create routes with automatic intermediate point interpolation (~200m intervals)
- **Geofencing**: Define polygon boundaries to filter regions inside/outside zones
- **Image Stitching**: Combine tiles into single images for regions
- **Tile Packaging**: Create ZIP archives of route tiles (max 50MB)
- **Async Processing**: Queue-based background processing for large requests
- **Caching**: Reuse previously downloaded tiles to minimize redundant downloads
- **REST API**: OpenAPI/Swagger documented endpoints

## Prerequisites

- Docker and Docker Compose
- .NET 8.0 SDK (for local development)
- PostgreSQL (provided via Docker)

## Quick Start

### Run the Service

```bash
docker-compose up --build
```

The API will be available at `http://localhost:5100`

Swagger documentation: `http://localhost:5100/swagger`

### Run with Tests

```bash
docker-compose -f docker-compose.yml -f docker-compose.tests.yml up --build --abort-on-container-exit
```

This command:
- Builds and starts all services
- Runs integration tests
- Exits when tests complete

## Architecture

The service follows a layered architecture:

```
┌─────────────────────────────────────┐
│     API Layer (ASP.NET Core)        │  HTTP endpoints
├─────────────────────────────────────┤
│   Services (Business Logic)         │  Processing, validation
├─────────────────────────────────────┤
│   Data Access (Dapper + Repos)      │  Database operations
├─────────────────────────────────────┤
│   PostgreSQL Database               │  Persistence
└─────────────────────────────────────┘
```

### Projects

- **SatelliteProvider.Api**: Web API and endpoints
- **SatelliteProvider.Services**: Business logic and processing
- **SatelliteProvider.DataAccess**: Database access and migrations
- **SatelliteProvider.Common**: Shared DTOs, interfaces, utilities
- **SatelliteProvider.IntegrationTests**: Integration test suite
- **SatelliteProvider.Tests**: Unit tests

## API Endpoints

### Download Single Tile

```http
GET /api/satellite/tiles/latlon?Latitude={lat}&Longitude={lon}&ZoomLevel={zoom}
```

Downloads a single tile at specified coordinates and zoom level.

**Parameters:**
- `Latitude` (double): Center latitude
- `Longitude` (double): Center longitude  
- `ZoomLevel` (int): Zoom level (higher = more detail, max ~20)

**Response:**
```json
{
  "id": "uuid",
  "zoomLevel": 18,
  "latitude": 37.7749,
  "longitude": -122.4194,
  "tileSizeMeters": 38.2,
  "tileSizePixels": 256,
  "imageType": "jpg",
  "mapsVersion": "downloaded_2024-11-20",
  "version": 2024,
  "filePath": "18/158/90.jpg",
  "createdAt": "2024-11-20T10:30:00Z",
  "updatedAt": "2024-11-20T10:30:00Z"
}
```

### Request Region Tiles

```http
POST /api/satellite/request
Content-Type: application/json

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "latitude": 37.7749,
  "longitude": -122.4194,
  "sizeMeters": 500,
  "zoomLevel": 18,
  "stitchTiles": false
}
```

Requests tiles for a square region. Processing happens asynchronously.

**Parameters:**
- `id` (guid): Unique identifier for the region
- `latitude` (double): Center latitude
- `longitude` (double): Center longitude
- `sizeMeters` (double): Square side length (100-10000 meters)
- `zoomLevel` (int): Zoom level (default: 18)
- `stitchTiles` (bool): Create single stitched image (default: false)

**Response:**
```json
{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "status": "pending",
  "csvFilePath": null,
  "summaryFilePath": null,
  "tilesDownloaded": 0,
  "tilesReused": 0,
  "createdAt": "2024-11-20T10:30:00Z",
  "updatedAt": "2024-11-20T10:30:00Z"
}
```

### Get Region Status

```http
GET /api/satellite/region/{id}
```

Check processing status and get file paths when complete.

**Status Values:**
- `pending`: Queued for processing
- `processing`: Currently downloading tiles
- `completed`: All tiles ready, files created
- `failed`: Processing failed

**Response (completed):**
```json
{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "status": "completed",
  "csvFilePath": "./ready/region_3fa85f64-5717-4562-b3fc-2c963f66afa6_ready.csv",
  "summaryFilePath": "./ready/region_3fa85f64-5717-4562-b3fc-2c963f66afa6_summary.txt",
  "tilesDownloaded": 12,
  "tilesReused": 4,
  "createdAt": "2024-11-20T10:30:00Z",
  "updatedAt": "2024-11-20T10:32:15Z"
}
```

### Create Route

```http
POST /api/satellite/route
Content-Type: application/json

{
  "id": "7fa85f64-5717-4562-b3fc-2c963f66afa6",
  "name": "City Tour Route",
  "description": "Downtown to waterfront",
  "regionSizeMeters": 200,
  "zoomLevel": 18,
  "points": [
    { "latitude": 37.7749, "longitude": -122.4194, "name": "Start" },
    { "latitude": 37.7849, "longitude": -122.4094, "name": "Middle" },
    { "latitude": 37.7949, "longitude": -122.3994, "name": "End" }
  ],
  "geofences": {
    "polygons": [
      {
        "name": "Restricted Area",
        "points": [
          { "latitude": 37.775, "longitude": -122.420 },
          { "latitude": 37.776, "longitude": -122.420 },
          { "latitude": 37.776, "longitude": -122.418 },
          { "latitude": 37.775, "longitude": -122.418 }
        ]
      }
    ],
    "direction": "inside"
  },
  "requestMaps": true,
  "createTilesZip": true
}
```

Creates a route with intermediate points calculated automatically.

**Parameters:**
- `id` (guid): Route identifier
- `name` (string): Route name
- `description` (string): Optional description
- `regionSizeMeters` (double): Size of region at each point
- `zoomLevel` (int): Zoom level for tiles
- `points` (array): Original route points
- `geofences` (object): Optional polygon boundaries
  - `polygons`: Array of polygon definitions
  - `direction`: "inside" or "outside" (filter regions)
- `requestMaps` (bool): Automatically request tiles for route regions
- `createTilesZip` (bool): Create ZIP file of all tiles

**Response:**
```json
{
  "id": "7fa85f64-5717-4562-b3fc-2c963f66afa6",
  "name": "City Tour Route",
  "description": "Downtown to waterfront",
  "regionSizeMeters": 200,
  "zoomLevel": 18,
  "totalDistanceMeters": 2486.5,
  "totalPoints": 15,
  "points": [
    {
      "sequenceNumber": 0,
      "latitude": 37.7749,
      "longitude": -122.4194,
      "pointType": "original",
      "segmentIndex": 0,
      "distanceFromPrevious": 0
    },
    {
      "sequenceNumber": 1,
      "latitude": 37.7765,
      "longitude": -122.4177,
      "pointType": "intermediate",
      "segmentIndex": 0,
      "distanceFromPrevious": 198.3
    }
  ],
  "regions": [
    "region-id-1",
    "region-id-2"
  ],
  "tilesZipPath": "./ready/route_7fa85f64-5717-4562-b3fc-2c963f66afa6_tiles.zip",
  "createdAt": "2024-11-20T10:30:00Z"
}
```

### Get Route

```http
GET /api/satellite/route/{id}
```

Retrieve route details including all interpolated points.

## Configuration

Configuration is managed through `appsettings.json`:

```json
{
  "ConnectionStrings": {
    "DefaultConnection": "Host=localhost;Database=satelliteprovider;Username=postgres;Password=postgres"
  },
  "MapConfig": {
    "Service": "GoogleMaps",
    "ApiKey": "your-google-maps-api-key"
  },
  "StorageConfig": {
    "TilesDirectory": "./tiles",
    "ReadyDirectory": "./ready"
  },
  "ProcessingConfig": {
    "MaxConcurrentDownloads": 4,
    "MaxConcurrentRegions": 20,
    "DefaultZoomLevel": 20,
    "QueueCapacity": 1000,
    "DelayBetweenRequestsMs": 50,
    "SessionTokenReuseCount": 100
  }
}
```

### Environment Variables (Docker)

Override settings via environment variables in `docker-compose.yml`:

```yaml
environment:
  - ConnectionStrings__DefaultConnection=Host=db;Database=satelliteprovider;Username=postgres;Password=postgres
  - MapConfig__ApiKey=your-api-key-here
```

## File Storage

### Tiles
Downloaded tiles are stored in:
```
./tiles/{zoomLevel}/{x}/{y}.jpg
```

### Output Files

**CSV File** (`region_{id}_ready.csv`):
Comma-separated list of tiles covering the region, ordered top-left to bottom-right:
```csv
latitude,longitude,filepath
37.7750,-122.4195,18/158/90.jpg
37.7750,-122.4193,18/158/91.jpg
```

**Summary File** (`region_{id}_summary.txt`):
Processing statistics:
```
Region: 3fa85f64-5717-4562-b3fc-2c963f66afa6
Size: 500m x 500m
Center: 37.7749, -122.4194
Zoom Level: 18
Tiles Downloaded: 12
Tiles Reused: 4
Total Tiles: 16
Processing Time: 3.5s
```

**Stitched Image** (`region_{id}_stitched.jpg`):
Single combined image when `stitchTiles: true`

**Route ZIP** (`route_{id}_tiles.zip`):
ZIP archive of all tiles for route when `createTilesZip: true` (max 50MB)

## Database Schema

### Tables

**tiles**: Cached satellite tiles
- Unique per (latitude, longitude, tile_size_meters, zoom_level)
- Tracks maps version for cache invalidation

**regions**: Region requests and processing status
- Status tracking: pending → processing → completed/failed
- Links to output files when complete

**routes**: Route definitions
- Stores original configuration
- Links to calculated points and regions

**route_points**: All points (original + interpolated)
- Sequenced in route order
- Type: "original" or "intermediate"
- Includes distance calculations

**route_regions**: Junction table
- Links routes to regions for tile requests
- Tracks geofence status (inside/outside)

## Development

### Building Locally

```bash
cd SatelliteProvider.Api
dotnet build
dotnet run
```

### Running Tests

**Unit Tests:**
```bash
cd SatelliteProvider.Tests
dotnet test
```

**Integration Tests:**
```bash
docker-compose -f docker-compose.yml -f docker-compose.tests.yml up --build --abort-on-container-exit
```

### Database Migrations

Migrations run automatically on startup. SQL files are located in:
```
SatelliteProvider.DataAccess/Migrations/
```

To add a new migration:
1. Create `NNN_DescriptiveName.sql` (increment number)
2. Set Build Action to `EmbeddedResource`
3. Restart application

### Logging

Logs are written to:
- Console (stdout)
- `./logs/satellite-provider-YYYYMMDD.log`

Log level can be adjusted in `appsettings.json` under `Serilog:MinimumLevel`.

## Performance Considerations

### Tile Caching
- Tiles are cached indefinitely (no expiration)
- Reused across multiple region/route requests
- Reduces redundant downloads significantly

### Concurrent Processing
- Configurable concurrent downloads (default: 4)
- Configurable concurrent regions (default: 20)
- Queue capacity: 1000 requests

### Rate Limiting
- Delay between requests: 50ms default
- Session token reuse: 100 tiles per token
- Helps avoid Google Maps rate limits

### Geofencing
- Uses point-in-polygon algorithm
- Reduces unnecessary tile downloads
- Spatial index on polygon data

## Troubleshooting

### Service won't start
- Check Docker is running
- Verify ports 5100 and 5432 are available
- Check logs: `docker-compose logs api`

### Tiles not downloading
- Verify Google Maps API key is configured
- Check internet connectivity
- Review logs for HTTP errors
- Check rate limiting settings

### Region stuck in "processing"
- Check background service is running: `docker-compose logs api`
- Verify database connection
- Check queue capacity not exceeded
- Review logs for exceptions

### Tests failing
- Ensure all containers are healthy before tests run
- Check test container logs: `docker-compose logs tests`
- Verify database migrations completed
- Check network connectivity between containers

### Out of disk space
- Tiles accumulate in `./tiles/` directory
- Implement cleanup strategy for old tiles
- Monitor disk usage

## License

This project is proprietary software.

## Support

For issues and questions, please contact the development team.