satellite-provider/README.md

# Satellite Provider API

A RESTful API service for downloading, caching, and processing satellite imagery from Google Maps. The service provides endpoints for downloading individual tiles and processing entire regions with intelligent tile caching and versioning.

## Features

- **Satellite Tile Downloads**: Download individual satellite tiles from Google Maps at various zoom levels
- **Region Processing**: Request satellite imagery for entire geographic regions
- **Intelligent Tile Caching**: Automatic tile reuse with year-based versioning
- **Image Stitching**: Combine multiple tiles into seamless regional images
- **CSV Export**: Generate coordinate mappings for downloaded tiles
- **Summary Reports**: Detailed processing statistics and metadata

## Tech Stack

- **.NET 8.0** - ASP.NET Web API
- **PostgreSQL** - Primary data storage
- **Dapper** - Lightweight ORM for database access
- **DbUp** - Database migrations
- **Serilog** - Structured logging
- **ImageSharp** - Image processing and stitching
- **Docker** - Containerized deployment

## Tile Versioning Strategy

### Year-Based Tile Versions

The service implements a **year-based versioning strategy** for cached satellite tiles. This approach is based on the assumption that Google Maps satellite imagery updates infrequently, typically once per year or less.

#### How It Works

1. **Version Assignment**: When a tile is downloaded, it is assigned a version number equal to the current year (e.g., 2025).

2. **Unique Constraint**: Tiles are uniquely identified by the combination of:
   - Latitude (center point)
   - Longitude (center point)
   - Zoom level
   - Tile size in meters
   - **Version** (year)

3. **Cache Behavior**:
   - **Same Year**: If requesting a tile that was downloaded in the current year, the cached tile is reused
   - **New Year**: When the calendar year changes, new tiles are downloaded even for the same coordinates
   - **Historical Data**: Old tile versions remain in the database for historical reference

#### Example

```
Request in 2025:
- Downloads tile at (47.461747, 37.647063) zoom 18 → version = 2025
- Saves to database with version 2025

Second request in 2025:
- Finds tile at (47.461747, 37.647063) zoom 18 version 2025
- Reuses cached tile (no download)

Request in 2026:
- Looks for tile at (47.461747, 37.647063) zoom 18 version 2026
- Not found (only 2025 version exists)
- Downloads new tile → version = 2026
- Both 2025 and 2026 versions now exist in database
```

#### Benefits

- **Automatic Updates**: Ensures fresh satellite imagery each calendar year
- **Historical Archive**: Maintains previous years' imagery for comparison
- **Efficient Caching**: Maximizes cache hit rate within the same year
- **Simple Logic**: No complex timestamp comparisons or manual cache invalidation

#### Database Schema

```sql
CREATE TABLE tiles (
    id UUID PRIMARY KEY,
    zoom_level INT NOT NULL,
    latitude DOUBLE PRECISION NOT NULL,
    longitude DOUBLE PRECISION NOT NULL,
    tile_size_meters DOUBLE PRECISION NOT NULL,
    tile_size_pixels INT NOT NULL,
    image_type VARCHAR(10) NOT NULL,
    maps_version VARCHAR(50),
    version INT NOT NULL,  -- Year-based version
    file_path VARCHAR(500) NOT NULL,
    created_at TIMESTAMP NOT NULL,
    updated_at TIMESTAMP NOT NULL
);

CREATE UNIQUE INDEX idx_tiles_unique_location 
ON tiles(latitude, longitude, zoom_level, tile_size_meters, version);
```

## Architecture

### Project Structure

```
SatelliteProvider/
├── SatelliteProvider.Api/          # REST API endpoints
├── SatelliteProvider.Common/       # Shared DTOs and interfaces
├── SatelliteProvider.DataAccess/   # Database access and migrations
├── SatelliteProvider.Services/     # Business logic
├── SatelliteProvider.Tests/        # Unit tests
└── SatelliteProvider.IntegrationTests/  # Integration tests
```

### Key Components

- **GoogleMapsDownloaderV2**: Downloads satellite tiles from Google Maps API
- **TileService**: Manages tile caching, retrieval, and storage with version control
- **RegionService**: Processes region requests, stitches tiles, generates outputs
- **RegionRequestQueue**: Background processing queue for region requests
- **DatabaseMigrator**: Manages database schema migrations

## API Endpoints

### Download Single Tile

```http
POST /api/satellite/tiles/download
Content-Type: application/json

{
  "latitude": 47.461747,
  "longitude": 37.647063,
  "zoomLevel": 18
}
```

**Response:**
```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "zoomLevel": 18,
  "latitude": 47.462451,
  "longitude": 37.646027,
  "tileSizeMeters": 103.35,
  "tileSizePixels": 256,
  "imageType": "jpg",
  "mapsVersion": "downloaded_2025-10-29",
  "version": 2025,
  "filePath": "./tiles/tile_18_158485_91707_20251029103256.jpg",
  "createdAt": "2025-10-29T10:32:56Z",
  "updatedAt": "2025-10-29T10:32:56Z"
}
```

### Request Region Processing

```http
POST /api/satellite/request
Content-Type: application/json

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "latitude": 47.461747,
  "longitude": 37.647063,
  "sizeMeters": 200,
  "zoomLevel": 18
}
```

### Check Region Status

```http
GET /api/satellite/region/{regionId}
```

**Response:**
```json
{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "status": "completed",
  "csvFilePath": "./ready/region_{id}_ready.csv",
  "summaryFilePath": "./ready/region_{id}_summary.txt",
  "tilesDownloaded": 5,
  "tilesReused": 4,
  "createdAt": "2025-10-29T10:32:56Z",
  "updatedAt": "2025-10-29T10:32:57Z"
}
```

## Configuration

### Environment Variables

- `ASPNETCORE_ENVIRONMENT`: Development/Production
- `ASPNETCORE_URLS`: HTTP binding address (default: http://+:8080)
- `ConnectionStrings__DefaultConnection`: PostgreSQL connection string
- `MapConfig__ApiKey`: Google Maps API key

### appsettings.json

```json
{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft.AspNetCore": "Warning"
      }
    },
    "WriteTo": [
      { "Name": "Console" },
      {
        "Name": "File",
        "Args": {
          "path": "./logs/satellite-provider-.log",
          "rollingInterval": "Day"
        }
      }
    ]
  },
  "StorageConfig": {
    "TilesDirectory": "./tiles",
    "ReadyDirectory": "./ready"
  },
  "ProcessingConfig": {
    "MaxConcurrentDownloads": 4,
    "DefaultZoomLevel": 18,
    "QueueCapacity": 100
  }
}
```

## Running with Docker

### Prerequisites

- Docker and Docker Compose
- Google Maps API key with Tile API enabled

### Setup

1. Create `.env` file in project root:
```bash
GOOGLE_MAPS_API_KEY=your_api_key_here
```

2. Start services:
```bash
docker-compose up -d
```

3. View logs:
```bash
docker-compose logs -f api
```

4. Stop services:
```bash
docker-compose down
```

### Docker Compose Services

- **postgres**: PostgreSQL database server
- **api**: Satellite Provider API service

### Volume Mounts

- `./tiles`: Downloaded tile images
- `./ready`: Processed region outputs (CSV, stitched images, summaries)
- `./logs`: Application logs

## Development

### Local Development Setup

1. Install .NET 8.0 SDK
2. Install PostgreSQL 16
3. Set up database:
```bash
createdb satelliteprovider
```

4. Update connection string in `appsettings.Development.json`
5. Run migrations (automatic on startup)
6. Start API:
```bash
cd SatelliteProvider.Api
dotnet run
```

### Running Tests

```bash
dotnet test
```

### Integration Tests

```bash
docker-compose -f docker-compose.tests.yml up --build
```

## Output Files

### Region Processing Outputs

For each processed region, the service generates:

1. **CSV File** (`region_{id}_ready.csv`):
   - Tile coordinates and file paths
   - Ordered by latitude (descending) and longitude (ascending)

2. **Stitched Image** (`region_{id}_stitched.jpg`):
   - Combined satellite imagery for the entire region
   - Red crosshair marking the requested center point

3. **Summary Report** (`region_{id}_summary.txt`):
   - Region metadata and coordinates
   - Processing statistics (tiles downloaded, tiles reused, total tiles)
   - Processing time and timestamps
   - File paths for all generated outputs

### Example Summary Report

```
Region Processing Summary
========================
Region ID: 5d218e0d-92bc-483c-9e88-fd79200b84e6
Center: 47.461747, 37.647063
Size: 200 meters
Zoom Level: 18

Processing Statistics:
- Tiles Downloaded: 5
- Tiles Reused from Cache: 4
- Total Tiles: 9
- Processing Time: 0.50 seconds
- Started: 2025-10-29 10:32:56 UTC
- Completed: 2025-10-29 10:32:57 UTC

Files Created:
- CSV: region_5d218e0d-92bc-483c-9e88-fd79200b84e6_ready.csv
- Stitched Image: region_5d218e0d-92bc-483c-9e88-fd79200b84e6_stitched.jpg
- Summary: region_5d218e0d-92bc-483c-9e88-fd79200b84e6_summary.txt
```

## Zoom Levels

Supported zoom levels: **15, 16, 17, 18, 19**

- **Zoom 15**: ~2,600m per tile (larger areas, less detail)
- **Zoom 16**: ~1,300m per tile
- **Zoom 17**: ~650m per tile
- **Zoom 18**: ~103m per tile (recommended for most uses)
- **Zoom 19**: ~52m per tile (maximum detail)

## Performance Considerations

### Tile Caching

- First request for a region downloads all required tiles
- Subsequent requests (same year) reuse cached tiles
- Typical 3x3 region (9 tiles) at zoom 18: ~300ms first request, ~50ms cached

### Region Processing

- Processed asynchronously in background queue
- Status polling recommended at 1-second intervals
- Processing time depends on:
  - Number of tiles required
  - Cache hit rate
  - Network latency to Google Maps API
  - Image stitching complexity

## License

This project is proprietary software. All rights reserved.

## Support

For issues, questions, or feature requests, please contact the development team.