# Contract: fdr_record_schema

**Component**: shared_fdr_client (cross-cutting concern owned by E-CC-FDR-CLIENT / AZ-247)
**Producer task**: AZ-272 — `_docs/02_tasks/todo/AZ-272_fdr_record_schema.md`
**Consumer tasks**: every onboard component that emits FDR records (C1–C13), the C13 writer (AZ-248 / E-C13), post-flight tooling (E-C12 operator side), the FdrClient ring buffer (AZ-XX / E-CC-FDR-CLIENT next task), and `FakeFdrSink` (AZ-XX / E-CC-FDR-CLIENT fourth task)
**Version**: 1.3.0
**Status**: draft
**Last Updated**: 2026-05-12

## Purpose

Frozen, versioned wire format for every record written to the Flight Data Recorder. Every onboard producer (logs, VIO ticks, state ticks, tile matches, overruns, rollovers, failed-tile thumbnails, mid-flight tile snapshots, flight headers/footers) MUST round-trip through this schema, and the C13 writer + post-flight tooling MUST be the only readers. The schema enforces forward-compatibility so post-flight tooling pinned at version N keeps working when producers move to N+1.

## Shape

### Outer envelope (one of these per record on the wire)

```python
# Conceptual dataclass — actual implementation may emit via orjson- or msgpack-backed serialiser pinned at E-BOOT.
@frozen
class FdrRecord:
    schema_version: int    # MUST be >= 1; reader uses this to pick the right parser branch
    ts: str                # ISO 8601 UTC, microsecond precision, e.g. "2026-05-10T03:14:15.123456Z"
    producer_id: str       # non-empty; component slug from module-layout.md (e.g. "c2_vpr") or "shared.<name>" for cross-cutting producers
    kind: str              # one of the v1.0.0 kinds (closed enum below) OR an unknown future tag (preserved opaquely)
    payload: dict[str, Any]  # kind-specific shape; well-known shapes documented per kind below

    # Forward-compat bucket — populated by parser when the wire bytes carry fields the local schema does not know.
    # NEVER set by producers; producers leave it empty.
    extra: dict[str, Any] = field(default_factory=dict)
```

| Field | Type | Required | Description | Constraints |
|-------|------|----------|-------------|-------------|
| `schema_version` | integer | yes | Schema major.minor packed as integer (1 for 1.x, 2 for 2.x) | `>= 1` |
| `ts` | string (ISO 8601 UTC, µs) | yes | Emit timestamp | RFC 3339 with `Z` suffix |
| `producer_id` | string | yes | Origin producer slug | non-empty; matches a module-layout component slug or `shared.<name>` |
| `kind` | string | yes | Record category | dotted snake_case, max 64 chars; v1.0.0 closed enum below |
| `payload` | object | yes (may be `{}` for kinds whose payload is empty) | Kind-specific data | JSON-safe / msgpack-safe scalars, nested dicts/arrays, no binary blobs >4 KiB |
| `extra` | object | parser-only | Forward-compat bucket for unknown future fields | populated by parser; producers MUST leave empty |

### v1.0.0 closed enum of `kind` values

| `kind` | Producer | Payload shape (required keys) | Notes |
|--------|----------|-------------------------------|-------|
| `log` | every component (via E-CC-LOG bridge) | `{level, component, frame_id?, kind, msg, kv, exc?}` (matches `log_record_schema` v1.0.0) | Forwarded WARN/ERROR records (per AZ-267 fdr_log_bridge) |
| `vio.tick` | C1 | `{frame_id, R, t, P, last_anchor_age_ms, mre_px?, imu_bias_norm?}` | Per-frame VIO output |
| `state.tick` | C5 | `{frame_id, fused_pose, covariance_2x2, estimator_label}` | Smoothed fused-pose tick |
| `tile_match` | C2.5 / C3 | `{frame_id, tile_id, score, match_count, ransac_inliers}` | Tile-matching diagnostics |
| `overrun` | E-CC-FDR-CLIENT itself | `{producer_id, dropped_count}` (`dropped_count > 0`) | AC-NEW-3: never silent. Emitted by drop-oldest hook |
| `segment_rollover` | E-C13 (writer) | `{old_segment, new_segment, total_bytes_after}` | Emitted on segment rotation, including 64 GB-cap drops |
| `failed_tile_thumbnail` | C6 / C11 | `{frame_id, tile_id, jpeg_bytes_b64}` (≤ 0.1 Hz rate cap) | AC-8.5 forensic exception |
| `mid_flight_tile_snapshot` | C13 (snapshot path) | `{snapshot_path, captured_at, frame_id?}` | AC-8.4 mid-flight snapshot pointer (envelope `producer_id="shared.fdr_client"`); `frame_id` optional (AZ-294) |
| `flight_header` | C13 (writer) | `{flight_id, flight_started_at_iso, flight_started_at_monotonic_ns, config_snapshot, signing_key_rotation_event, manifest_content_hashes, build_info}` | Single record at flight open (envelope `producer_id="shared.fdr_client"`) |
| `flight_footer` | C13 (writer) | `{flight_id, flight_ended_at_iso, flight_ended_at_monotonic_ns, records_written, records_dropped_overrun, bytes_written, rollover_count, clean_shutdown}` | Single record at flight close (envelope `producer_id="shared.fdr_client"`) |
| `c6.write` | C6 (`PostgresFilesystemStore`) | `{tile_id, source, disk_bytes, content_sha256}` | v1.1.0 (AZ-305). Emitted on every successful `write_tile`. `tile_id` is the canonical UUIDv5 derived from `(zoom, x, y, source, flight_id)`; `source` is the `TileSource` enum value; `disk_bytes` is the JPEG payload length; `content_sha256` is the lowercase hex digest of the body. Envelope `producer_id="c6_tile_cache.store"`. |
| `c6.write_failed` | C6 (`PostgresFilesystemStore`) | `{tile_id, source, reason, error_class, message}` | v1.1.0 (AZ-305). Emitted on every failed `write_tile` path. `reason` ∈ `{content_hash_mismatch, freshness_reject, metadata_error, fs_error}`; `error_class` is the exception class name; `message` is the rewrapped exception's `str` (truncated to 512 chars to keep the record inline). Envelope `producer_id="c6_tile_cache.store"`. |
| `c6.freshness.rejected` | C6 (`FreshnessGate`) | `{tile_id, age_seconds, classification, rule_action, rule_max_age_seconds}` | v1.2.0 (AZ-307). Emitted on every active-conflict-stale reject. `tile_id` is the canonical UUIDv5; `age_seconds` is the integer-rounded `(now - capture_timestamp).total_seconds()` at decision time; `classification` is the `SectorClassification` enum value (always `"active_conflict"` for this kind in practice); `rule_action` is always `"reject"`; `rule_max_age_seconds` is the rule's threshold (e.g. `15552000` for the 6-month default). Envelope `producer_id="c6_tile_cache.freshness"`. |
| `c6.freshness.downgraded` | C6 (`FreshnessGate`) | `{tile_id, age_seconds, classification, rule_action, rule_max_age_seconds}` | v1.2.0 (AZ-307). Emitted on every stable-rear-stale downgrade (including the implicit-default path for tiles outside every loaded sector). Same payload shape as `c6.freshness.rejected` so reject/downgrade FDR traces are line-for-line comparable; `rule_action` is always `"downgrade"` and `classification` is always `"stable_rear"` for this kind. Envelope `producer_id="c6_tile_cache.freshness"`. |
| `c6.eviction_batch` | C6 (`CacheBudgetEnforcer`) | `{trigger_tile_id, freed_bytes, evicted_count, evicted_tile_ids}` | v1.3.0 (AZ-308). Emitted once per `reserve_headroom` call that actually evicted at least one tile (RESTRICT-SAT-2 enforcement). `trigger_tile_id` is the canonical UUIDv5 of the tile whose write triggered the sweep; `freed_bytes` is the integer total reclaimed; `evicted_count` is the FULL count of evictions in the batch regardless of payload caps; `evicted_tile_ids` is bounded to the first **5** evicted ids (the full list lives in the per-tile `c6.evicted` INFO logs). Envelope `producer_id="c6_tile_cache.budget"`. |

### Wire bytes

- `serialise(record: FdrRecord) -> bytes` returns a single self-delimited byte string (length-prefixed if msgpack, single-line UTF-8 if orjson — pinned at E-BOOT in `pyproject.toml`).
- `parse(buf: bytes) -> FdrRecord` is the inverse for a single record. Streaming parser (multi-record) is not part of this contract — C13 writer/reader own that.

## Invariants

- `schema_version >= 1` on every record; missing or non-integer values are rejected by `parse` with `FdrSchemaError`.
- `producer_id` is non-empty on every record. Anonymous records on the wire are a contract violation — `serialise` rejects them with `FdrSchemaError`.
- For `kind="overrun"`: `payload.producer_id` MUST equal the originating producer's slug, and `payload.dropped_count` MUST be `> 0`. (The OUTER envelope's `producer_id` is `"shared.fdr_client"` because the overrun record is emitted by the FdrClient itself, not by the producer whose enqueue overran.)
- Forward-compatible parser: a record at minor version N+1 carrying fields unknown at version N parses without exception; unknown payload fields land in `payload.extra`; unknown top-level fields land in record-level `extra`. Tooling MAY then choose to skip the record.
- Unknown future `kind` values do NOT raise — `parse` returns an `FdrRecord` with `kind` set to the raw string and `payload` set to whatever decoded; tooling MAY skip.
- Renaming a field, changing a field type, or removing a required field requires a major version bump (schema_version 2.x).
- Embedded binary blobs ≤ 4 KiB only. Bigger payloads (e.g. mid-flight tile JPEGs, ML inference inputs) MUST be referenced by sidecar path on disk; the contract test rejects oversized inline blobs.
- `serialise` and `parse` are pure: same input → byte-identical output (or deep-equal record).
- `FdrSchemaError` is the ONLY exception type either function raises on schema violation; library-specific exceptions (`orjson.JSONDecodeError`, `msgpack.UnpackException`, etc.) MUST be wrapped before crossing the public API.

## Non-Goals

- This contract does NOT define the lock-free SPSC ring buffer (`FdrClient`) — owned by the next task in E-CC-FDR-CLIENT.
- This contract does NOT define the writer thread, segment files, or 64 GB cap — owned by E-C13 (AZ-248).
- This contract does NOT define what triggers a record (per-component § 9 logging policies, VIO tick rate, etc. are owned by component epics).
- This contract does NOT define multi-record framing on disk — that is C13's segment file format, owned separately.

## Versioning Rules

- **Breaking changes** (field renamed/removed, type changed, ordering changed for length-prefixed wire format, library choice changed) → new major version (e.g. 2.0.0) + a deprecation pass through every consumer + a paired major bump on this contract.
- **Non-breaking additions** (new optional payload field appended, new `kind` value, new top-level optional field) → minor version bump. Forward-compat parser tolerates these by design.
- **Patch changes** (clarification, doc-only, no wire change) → patch bump.
- The contract test (`tests/contract/fdr_record_schema.py`) MUST be updated alongside any version bump.

## Test Cases

| Case | Input | Expected | Notes |
|------|-------|----------|-------|
| valid-roundtrip-log | `kind="log", payload={"level":"INFO","component":"c2_vpr","kind":"vpr.warmup","msg":"loaded","kv":{"model":"salad"}}` | `parse(serialise(r)) == r` | covers AC-1 |
| valid-roundtrip-overrun | `kind="overrun", producer_id="shared.fdr_client", payload={"producer_id":"c1_vio","dropped_count":42}` | round-trips; both producer_ids preserved | covers AC-1 + AC-5 |
| forward-compat-future-field | wire bytes carry `payload.new_field="x"` (hypothetical v1.1) parsed at v1.0 | record parses; `payload.extra["new_field"] == "x"` | covers AC-2 |
| forward-compat-unknown-kind | `kind="future.kind", payload={"foo":1}` | record parses opaquely; no exception | covers AC-3 |
| invalid-missing-version | bytes missing `schema_version` field | `FdrSchemaError`; message names `schema_version` | covers AC-4 |
| invalid-overrun-missing-dropped-count | `kind="overrun", payload={"producer_id":"c1_vio"}` | `FdrSchemaError`; message names `dropped_count` | covers AC-5 |
| invalid-overrun-zero-dropped-count | `kind="overrun", payload={"producer_id":"c1_vio","dropped_count":0}` | `FdrSchemaError`; message names `dropped_count` | covers AC-5 (must be `> 0`) |
| invalid-empty-producer-id | `producer_id=""` on serialise | `FdrSchemaError`; message names `producer_id` | covers AC-6 |
| invalid-oversized-blob | `payload={"jpeg":<8 KiB bytes>}` | `FdrSchemaError`; message says "use sidecar path" | invariant: ≤ 4 KiB inline |
| pure-determinism | call `serialise(r)` twice | byte-identical outputs | NFR-reliability |

## Change Log

| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-10 | Initial contract derived from E-CC-FDR-CLIENT epic (AZ-247) | autodev decompose Step 2 |
| 1.1.0 | 2026-05-12 | Add `c6.write` and `c6.write_failed` kinds emitted by C6 `PostgresFilesystemStore` (AZ-305). Non-breaking; v1.0 parsers see the records as unknown kinds and route them through the forward-compat opaque path. | AZ-305 implement |
| 1.2.0 | 2026-05-12 | Add `c6.freshness.rejected` and `c6.freshness.downgraded` kinds emitted by the C6 `FreshnessGate` (AZ-307). Non-breaking; v1.1 parsers see the records as unknown kinds and route them through the forward-compat opaque path. | AZ-307 implement |
| 1.3.0 | 2026-05-12 | Add `c6.eviction_batch` kind emitted by the C6 `CacheBudgetEnforcer` (AZ-308). Non-breaking; v1.2 parsers see the record as an unknown kind and route it through the forward-compat opaque path. | AZ-308 implement |
