# C13 Mid-Flight Tile Snapshot Path + Filesystem Layout

**Task**: AZ-294_c13_mid_flight_tile_snapshot
**Name**: C13 Mid-Flight Tile Snapshot Path
**Description**: Implement the sidecar-file path that persists mid-flight orthorectified tile snapshots produced by C6 / C11 (per AC-8.4 / F4) onto the per-flight FDR tree, and emit the corresponding `kind="mid_flight_tile_snapshot"` `FdrRecord` carrying a pointer (`snapshot_path` + `captured_at`) — NOT the JPEG bytes — so the FdrRecord schema's "embedded binary blobs ≤ 4 KiB" invariant is preserved. The sidecar files live under `flight_root/<flight_id>/tiles/<tile_id>.jpg`. This task does NOT generate the tiles (C6 / C11 own that); it provides the FDR-side storage layout, the sidecar write helper, and the pointer-record emission path.
**Complexity**: 3 points
**Dependencies**: AZ-291_c13_writer_thread, AZ-272_fdr_record_schema, AZ-263_initial_structure, AZ-269_config_loader
**Component**: c13_fdr (epic AZ-248 / E-C13)
**Tracker**: AZ-294
**Epic**: AZ-248 (E-C13)

### Document Dependencies

- `_docs/02_document/contracts/shared_fdr_client/fdr_record_schema.md` — defines the `kind="mid_flight_tile_snapshot"` payload shape (`snapshot_path`, `captured_at`) AND the ≤ 4 KiB inline-blob invariant this task respects by emitting a pointer instead of bytes.
- `_docs/02_document/contracts/shared_config/composition_root_protocol.md` — Config block carrying the per-flight tile cap byte budget (`tile_snapshot_cap_bytes`, default ~50 MiB per `description.md` storage estimate).

## Problem

Mid-flight tile snapshots are generated by C6 / C11 (per F4 mid-flight tile gen) at sizes 50–200 KiB each, up to ~50 MB per 8 h flight. They cannot be inlined into FdrRecords (the schema invariant caps inline blobs at 4 KiB) and they cannot live in the segment file (segment files are append-only streams of FdrRecords; appending arbitrary JPEG bytes would break the record framing AZ-291 + AZ-272 jointly establish).

Without a sidecar path:
- Producers (C6 / C11) have no canonical filesystem location to write the JPEGs. Each component would invent its own, drifting on naming and breaking post-flight retrieval.
- The FdrRecord that ties the JPEG to a frame_id / tile_id / timestamp would either go missing (no record at all) or violate the schema invariant (inlining the JPEG bytes), poisoning the whole FDR.
- The per-flight tile cap (~50 MB per `description.md`) has no enforcement layer; a runaway tile producer could exhaust the same NVM the segment files compete for.

## Outcome

- A `MidFlightTileSnapshotSink(flight_root: Path, flight_id: UUID, fdr_client: FdrClient, config: TileSnapshotConfig)` class is the single sidecar write path. C6 / C11 producers call its `write_snapshot(tile_id: str, jpeg_bytes: bytes, captured_at: datetime, frame_id: int | None) -> Path` method; this task does NOT produce the JPEG itself.
- Sidecar files land at `flight_root/<flight_id>/tiles/<tile_id>.jpg`; the directory `tiles/` is created on first write (lazy creation — empty flights leave no `tiles/` directory).
- Per call, ONE `kind="mid_flight_tile_snapshot"` FdrRecord is enqueued via the shared FdrClient with `payload.snapshot_path = "tiles/<tile_id>.jpg"` (relative to `flight_root/<flight_id>/` so the FDR is portable) and `payload.captured_at = <ISO 8601>`. The JPEG bytes are NEVER inlined.
- The per-flight tile cap (`tile_snapshot_cap_bytes`, default 64 MiB to comfortably fit the ~50 MB worst case from `description.md`) is enforced via oldest-tile-dropped policy, mirroring the segment cap policy from AZ-293 but scoped to the `tiles/` subdirectory and emitted as a `kind="overrun"` record (NOT `segment_rollover` — that kind is reserved for segment-file drops). Each tile drop emits a record with `payload.producer_id="shared.fdr_client"` and `payload.dropped_count=1`.
- The sink is thread-safe for many producers (C6, C11 may call concurrently from different threads); the file write itself uses `atomicwrites` to avoid partial JPEGs on crash.

## Scope

### Included

- `MidFlightTileSnapshotSink` class as defined above.
- `write_snapshot(tile_id: str, jpeg_bytes: bytes, captured_at: datetime, frame_id: int | None = None) -> Path`:
  1. Validate `len(jpeg_bytes) <= jpeg_max_bytes` (default 256 KiB; rejects with `TileSnapshotTooLargeError` — not infinite-trust on producers).
  2. Validate `tile_id` matches `[a-zA-Z0-9_-]{1,128}` (rejects with `TileSnapshotInvalidIdError`).
  3. Compute the absolute sidecar path; create `flight_root/<flight_id>/tiles/` if missing (`os.makedirs(exist_ok=True)`).
  4. Write the JPEG via `atomicwrites` (temp file + `os.rename` after `fsync`).
  5. Enqueue the `kind="mid_flight_tile_snapshot"` FdrRecord with relative path + ISO timestamp + optional `frame_id`.
  6. Check the cap (sum of bytes under `tiles/`). If over cap, drop the oldest `tile_id`-by-`captured_at` and emit an overrun record.
  7. Return the absolute sidecar path to the caller (so the producer can log it if needed).
- `tile_snapshot_cap_bytes` config field (`composition_root_protocol`); default `64 * 1024**2` (64 MiB).
- `jpeg_max_bytes` config field; default `256 * 1024` (256 KiB; per `description.md` "50–200 KB each", 256 KiB gives a small safety margin while bounding adversarial growth).
- Thread-safe API: a single `threading.Lock` around the cap-check + drop sequence (the file write itself is `atomicwrites` so it is independently safe). The lock is held for ≤ 5 ms p99; `write_snapshot` is NOT a hot path (tiles are sparse — ~0.01–0.1 Hz typical).
- A diagnostic INFO log on each successful write (`kind="fdr.tile_snapshot_written"; tile_id; size_bytes`) and WARN on each cap-driven drop (`kind="fdr.tile_snapshot_dropped"; tile_id; size_bytes_freed; cap_bytes_after`).
- Recovery on existing `tiles/` directory: on construction, the sink scans `flight_root/<flight_id>/tiles/` for any pre-existing tiles (e.g. from a crashed and resumed flight via the same flight_id); the cap policy treats them as in-cap unless they push the directory over cap. No tiles are auto-deleted on construction; only on overflow.
- The sink does NOT interact with AZ-293's segment cap policy directly. The `tiles/` subdirectory is excluded from segment-cap accounting (per AZ-293 constraint "sidecar files outside the segment files are NOT counted toward the cap"); the tile cap is independent.

### Excluded

- Generating tile JPEGs (orthorectification, downsampling, encoding) — owned by F4 / C6 / C11 producers.
- The `kind="mid_flight_tile_snapshot"` payload schema — owned by AZ-272.
- Post-flight retrieval / upload of tile sidecars — owned by C12 post-landing upload trigger (out of scope this cycle).
- Failed-tile thumbnail rate limiter — owned by task #5 (this task is for SUCCESS-path tile snapshots from F4; failed-tile thumbnails are a separate, AC-8.5-governed forensic category).
- Per-segment file rotation, 64 GB cap on segments, header/footer accounting — owned by AZ-291 / AZ-292 / AZ-293.
- Compression of the JPEG (already JPEG; no further compression).
- Encryption / signing of the JPEG — out of scope this cycle.

## Acceptance Criteria

**AC-1: write_snapshot persists JPEG to canonical sidecar path**
Given a sink constructed for `flight_root=/tmp/fdr` and `flight_id=abc-123`, and a JPEG byte string of size 100 KiB
When `write_snapshot(tile_id="t_001", jpeg_bytes=<100 KiB>, captured_at=now)` is called
Then `/tmp/fdr/abc-123/tiles/t_001.jpg` exists on disk; its bytes equal the input JPEG byte-for-byte; the file is fully written (no temp file artifacts left)

**AC-2: Pointer record is enqueued, not inline bytes**
Given the same call as AC-1
When the consumer drains the FdrClient and parses the record
Then ONE record with `kind="mid_flight_tile_snapshot"` is observed; `payload.snapshot_path == "tiles/t_001.jpg"` (relative to flight directory); `payload.captured_at` is the ISO 8601 string of the input timestamp; `payload.frame_id` matches the input (or is absent if input was None); NO `payload.jpeg_bytes` field exists; the serialised record is < 1 KiB

**AC-3: Cap-driven drop emits overrun record + deletes oldest tile**
Given `tile_snapshot_cap_bytes=200 KiB`, three tiles already on disk: `t_old.jpg=100 KiB` (captured_at=t0), `t_mid.jpg=80 KiB` (t1), `t_new.jpg=100 KiB` (t2)
When `write_snapshot(tile_id="t_overflow", jpeg_bytes=<60 KiB>, captured_at=t3)` is called
Then `t_old.jpg` is deleted (oldest by `captured_at`); the new tile is persisted; ONE `kind="overrun"` record is enqueued with `payload.producer_id="shared.fdr_client"` and `payload.dropped_count=1`

**AC-4: TileSnapshotTooLargeError on oversized JPEG**
Given `jpeg_max_bytes=256 KiB` and an input of 300 KiB
When `write_snapshot` is called
Then `TileSnapshotTooLargeError` is raised before any file or record is written; the `tiles/` directory is unchanged; no FdrRecord lands on the FdrClient

**AC-5: TileSnapshotInvalidIdError on bad tile_id**
Given an input `tile_id="../../../etc/passwd"` (path traversal attempt)
When `write_snapshot` is called
Then `TileSnapshotInvalidIdError` is raised before any file write; the `tiles/` directory is unchanged

**AC-6: Concurrent writes are serialised correctly**
Given two threads each calling `write_snapshot` 100 times with distinct `tile_id`s under cap
When both threads run concurrently
Then all 200 sidecar files exist with byte-correct contents; 200 `mid_flight_tile_snapshot` records were enqueued (one per call); zero overruns; no partial files in `tiles/`

**AC-7: Existing tiles preserved on sink construction**
Given `/tmp/fdr/abc-123/tiles/` already contains 3 tile files from a prior process (totaling 150 KiB; cap is 200 KiB)
When the sink is constructed for the same `flight_id`
Then the existing tiles are NOT deleted on construction; the sink's internal cap accounting includes them; a subsequent `write_snapshot` of 60 KiB triggers a drop of the oldest existing tile

**AC-8: Atomic write — no partial JPEGs on crash**
Given a `write_snapshot` call that is interrupted (simulated kill) between the `atomicwrites` temp-file write and the rename
When the test re-inspects the `tiles/` directory
Then NO file exists at the canonical sidecar path with partial content; either the file is absent OR it is fully written and parseable

## Non-Functional Requirements

**Performance**
- `write_snapshot` returns within 50 ms p99 for a 200 KiB JPEG on Tier-2 NVM (dominated by `fsync` on rename; tiles are sparse so no batching needed).
- The cap-check sequence (lock acquire + scan tiles/ + drop if needed + lock release) p99 ≤ 5 ms when no drop is needed; p99 ≤ 50 ms when one drop is needed.
- Producer-perceived latency must NOT exceed 100 ms p99 in any scenario — F4 mid-flight tile generation is NOT a hot path but operators do see the result.

**Reliability**
- The sink's `write_snapshot` is at-most-once per call: a successful write is exactly one sidecar + exactly one FdrRecord; a failed write is zero sidecar (atomicwrites ensures this) + zero FdrRecord (the record is enqueued only after the sidecar write completes).
- The cap-driven drop is also at-most-once per overflow event: the overrun record is enqueued exactly once even under contention (the lock covers the drop + emission sequence).

## Unit Tests

| AC Ref | What to Test | Required Outcome |
|--------|-------------|-----------------|
| AC-1 | write_snapshot 100 KiB JPEG | sidecar file exists at canonical path with byte-correct content |
| AC-2 | Parse the enqueued record | kind=mid_flight_tile_snapshot; payload has snapshot_path + captured_at + frame_id; no jpeg_bytes; serialised < 1 KiB |
| AC-3 | Cap=200 KiB, 3 existing tiles + 60 KiB new | Oldest tile deleted; new tile present; one overrun record enqueued |
| AC-4 | 300 KiB JPEG with cap on size | TileSnapshotTooLargeError; no file or record written |
| AC-5 | tile_id with path traversal characters | TileSnapshotInvalidIdError; no file or record written |
| AC-6 | 2 threads × 100 calls each | All 200 sidecars present; 200 records enqueued; no partials |
| AC-7 | Pre-populate tiles/ then construct sink | Existing tiles untouched; cap accounting includes them |
| AC-8 | Kill mid-write | No partial file at canonical path; either complete or absent |
| NFR-perf-write | Microbench write_snapshot for 200 KiB | p99 ≤ 50 ms |
| NFR-perf-cap-check | Microbench cap-check no-drop path | p99 ≤ 5 ms |
| NFR-perf-cap-drop | Microbench cap-check drop path | p99 ≤ 50 ms |
| NFR-reliability-atomic | Inject failure between temp-write and rename | No half-written canonical file |

## Constraints

- The sidecar path is RELATIVE to `flight_root/<flight_id>/` in the FdrRecord (`payload.snapshot_path = "tiles/<tile_id>.jpg"`). This makes the FDR portable: the operator can copy the entire flight directory anywhere and the records still reference the right files.
- `tile_id` validation regex `^[a-zA-Z0-9_-]{1,128}$` is the contract; producers may use any naming scheme inside that envelope.
- The cap is `tile_snapshot_cap_bytes`, distinct from segment `flight_cap_bytes` in AZ-293. The two caps are independent — exceeding one does NOT trigger drops in the other domain.
- The shared FdrClient's `producer_id` for the records emitted by this sink is `"shared.fdr_client"` (the sink itself is shared infrastructure); the originating producer (C6 / C11) is reflected ONLY in the optional `frame_id` payload field, not in the outer envelope. Rationale: F4 tiles may be produced collaboratively across multiple components and the canonical attribution is the captured_at timestamp + tile_id.
- This task does NOT introduce any new dependency: `atomicwrites` is already pinned at AZ-263 / E-BOOT.

## Risks & Mitigation

**Risk 1: Producer (C6 / C11) flushes tiles faster than the cap can absorb**
- *Risk*: A pathological case where 1000 small tiles per second push the cap into constant churn.
- *Mitigation*: F4 tile generation is rate-limited at the producer side per the C6 / C11 specs (typical 0.01–0.1 Hz). The cap is sized at 64 MiB to comfortably hold the per-flight worst case. The cap-driven overrun record is the canonical signal if a producer misbehaves; AC-3 covers the policy.

**Risk 2: tile_id collisions across producers**
- *Risk*: C6 and C11 both pick `tile_id="x_42"`; the second call overwrites the first.
- *Mitigation*: `atomicwrites` uses temp files but the rename targets the canonical name — second call OVERWRITES the first. The `payload.snapshot_path` in the second record is identical to the first; the test operator sees ONE file at the path with the second JPEG and TWO records pointing to it. Documented as a limitation: producers MUST namespace their `tile_id`s (e.g. `c6_<area>_<index>`); the sink does NOT enforce uniqueness. Code-review Phase 7 (Architecture) catches collisions in `tile_id` schemes across components.

**Risk 3: A failed sidecar write leaves the FdrRecord pointing at a missing file**
- *Risk*: `atomicwrites` succeeds in the temp file but `os.rename` fails; we already enqueued the FdrRecord pointing at the canonical name.
- *Mitigation*: The order is FIRST sidecar write (must complete) THEN FdrRecord enqueue. AC-2 implicitly covers this — if the sidecar write raises, no record is enqueued. The implementation MUST NOT enqueue the record before `atomicwrites` returns.

**Risk 4: `os.scandir` of `tiles/` becomes slow with thousands of tiles**
- *Risk*: A 100 MiB cap with tiny tiles ends up with ~10k files in `tiles/`; scanning that directory on every write becomes the bottleneck.
- *Mitigation*: The sink caches the in-memory tile list (sorted by `captured_at`) and updates it on every write; `os.scandir` runs only once on construction (AC-7). Cache invalidation on a manually-deleted tile mirrors AZ-293's stale-list refresh.

## Runtime Completeness

- **Named capability**: per-flight mid-flight tile snapshot sidecar storage + pointer-record emission (architecture / E-C13 / AC-8.4 quality metadata, F4 mid-flight tile gen).
- **Production code that must exist**: real `atomicwrites`-based sidecar writer, real FdrRecord pointer emission, real cap-policy + overrun record on overflow.
- **Allowed external stubs**: tests MAY use `FakeFdrSink` (AZ-275) and a tmp `flight_root`; production wiring uses the real `FdrClient` from the composition root.
- **Unacceptable substitutes**: inlining JPEG bytes into FdrRecords ("for now we don't have a sidecar path"), unbounded tile growth without cap enforcement ("the segment cap will catch it" — it won't, AZ-293 explicitly excludes the `tiles/` subdirectory), or skipping `atomicwrites` ("crash-tolerance is a nice-to-have") — operators ARE going to crash-resume mid-flight on Jetson hardware.
