# C13 Failed-Tile Thumbnail Rate Limiter + AC-8.5 Forbidden-Kind Enforcement

**Task**: AZ-295_c13_thumbnail_rate_limiter
**Name**: C13 AC-8.5 Forbidden-Kind + Thumbnail Rate Cap
**Description**: Implement two paired record-policy gates required by AC-8.5 / C13-IT-03 / RESTRICT-UAV-4: (1) a synchronous producer-side validator that REFUSES `kind="raw_nav_frame"` (and any other AI-cam / nav-cam raw-frame kind) by raising `RawFrameWriteForbiddenError` BEFORE the record is enqueued, so the security violation is visible to the offending producer at the call site; (2) a writer-thread-side rate cap on `kind="failed_tile_thumbnail"` records (default ≤ 0.1 Hz per `description.md` § 7) that drops over-cap thumbnails with a WARN log + emits a `kind="overrun"` record carrying the dropped count, while letting in-cap thumbnails pass through to disk untouched. Together they enforce the only allowed raw-imagery-adjacent persistence path on the FDR.
**Complexity**: 3 points
**Dependencies**: AZ-291_c13_writer_thread, AZ-272_fdr_record_schema, AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module
**Component**: c13_fdr (epic AZ-248 / E-C13)
**Tracker**: AZ-295
**Epic**: AZ-248 (E-C13)

### Document Dependencies

- `_docs/02_document/contracts/shared_fdr_client/fdr_record_schema.md` — defines `kind="failed_tile_thumbnail"` payload (`{frame_id, tile_id, jpeg_bytes_b64}`) and the ≤ 4 KiB inline-blob invariant the cap respects.
- `_docs/02_document/contracts/shared_config/composition_root_protocol.md` — Config block carrying `forbidden_record_kinds` (default frozen set including `raw_nav_frame`, `raw_ai_cam_frame`) and `failed_tile_thumbnail_max_hz` (default 0.1).

## Problem

Per AC-8.5 + RESTRICT-UAV-4, the FDR is the ONLY persistence path for raw-imagery-adjacent data, and the ONLY allowed raw-imagery-adjacent kind is `failed_tile_thumbnail`, capped at ≤ 0.1 Hz. Without:

- A synchronous validator that rejects `kind="raw_nav_frame"` (and equivalents) at the producer's call site, a careless or compromised producer could enqueue a stream of raw frames; by the time the writer thread sees them and drops them, gigabytes of raw imagery have already been serialised onto the wire format and (worst case) onto a segment file. Even an "asynchronous reject + drop" model leaks the bytes through transient memory.
- A writer-side rate cap on `failed_tile_thumbnail`, a producer (C6 / C11) bug or thumbnail spam attack could push the inline-thumbnail throughput from the documented ≤ 0.1 Hz to many Hz, blowing past the inline-blob budget and burying real diagnostic records under thumbnail noise.

The two gates are intentionally asymmetric: forbidden-kind violation is a HARD security error visible to the caller (raw_nav_frame is never legitimate); over-cap thumbnails are a SOFT throughput control with WARN logging (over-eager producers are common; rate-limit and continue).

## Outcome

- A `RecordKindPolicy` object is the single source of truth for both gates. It exposes `enforce_or_raise(record: FdrRecord) -> None` (synchronous; raises `RawFrameWriteForbiddenError` for forbidden kinds; returns silently for everything else including `failed_tile_thumbnail`) and `gate_for_writer(record: FdrRecord) -> GateDecision` (returns `ENQUEUE` or `DROP` for thumbnail rate-cap purposes).
- Producers (C6 / C11 thumbnail emission paths; future producers) call `policy.enforce_or_raise(record)` immediately before `fdr_client.enqueue(record)`. The composition root injects the policy; producers do not construct it themselves.
- The writer thread (AZ-291) calls `policy.gate_for_writer(record)` immediately after dequeue. On `DROP`, the writer skips the append + emits a `kind="overrun"` record with `payload.producer_id="shared.fdr_client"` and `payload.dropped_count` aggregated across the cap window.
- The `failed_tile_thumbnail` rate cap uses a sliding-window counter (1-second windows summed over the last 10 seconds at 0.1 Hz default) so a producer that bursts 5 thumbnails in one second still gets averaged correctly across the window — instead of a tight token-bucket that would either reject every thumbnail after the burst or let through a steady-state too-fast trickle.
- The forbidden-kind set is config-driven (`forbidden_record_kinds`) but its DEFAULT MUST include `raw_nav_frame` and `raw_ai_cam_frame`. Removing those defaults requires a major-version Config bump and is a security-critical review item.

## Scope

### Included

- `RecordKindPolicy` dataclass / class with two methods: `enforce_or_raise(record)` and `gate_for_writer(record)`.
- Forbidden-kind enforcement:
  - `enforce_or_raise` raises `RawFrameWriteForbiddenError` if `record.kind` is in the configured forbidden set. The exception's message includes the offending kind and the producer slug from the record envelope so logs identify the source.
  - The forbidden set defaults to `frozenset({"raw_nav_frame", "raw_ai_cam_frame"})` and is configurable via `forbidden_record_kinds` on the Config; runtime additions are allowed (you can ADD kinds at runtime), but the `Config` validator REJECTS any preset that REMOVES a default kind from the set (unless an explicit `unsafe_remove_default_forbidden=True` flag is set, which is a security-review-required path documented as such; the flag does NOT exist in any standard preset).
- Failed-tile thumbnail rate cap:
  - `gate_for_writer` checks the kind. For non-thumbnail kinds, returns `ENQUEUE`.
  - For `kind="failed_tile_thumbnail"`, applies a sliding-window rate cap at `failed_tile_thumbnail_max_hz` (default 0.1 Hz). The window is (1 / max_hz) seconds; up to one record per window passes through.
  - On `DROP`, increments a running `thumbnail_dropped_count` counter and emits ONE `kind="overrun"` record per per-cap-window with `payload.dropped_count == accumulated_count_during_window` (coalesced; matches the AZ-274 overrun-coalescing semantics so post-flight tooling sees consistent overrun records regardless of whether the source is FdrClient queue overrun or thumbnail rate cap).
  - WARN log per drop window (`kind="fdr.thumbnail_rate_cap_exceeded"; producer_id; dropped_in_window`). Per-second rate cap on the WARN log itself (≤ 1 WARN/sec) so a thumbnail flood does not flood the operational log.
- Composition-root wiring: `make_record_kind_policy(config)` factory; the composition root constructs ONE policy instance and injects it into both (a) every producer's enqueue path and (b) the `FileFdrWriter`'s post-dequeue gate.
- `failed_tile_thumbnail_max_hz` config field (default 0.1; valid range > 0 .. 10.0); `0` is REJECTED at config validation (would silence thumbnails entirely; producers must declare intent explicitly via `disable_failed_tile_thumbnails=True` on a separate flag if they truly want to silence the kind — this requires a security-review-required preset, similar to forbidden-kind removal).

### Excluded

- Thumbnail GENERATION (orthorectification failure detection, JPEG encoding) — owned by C6 / C11 producers; this task only validates / rate-caps RECORDS already constructed.
- Mid-flight tile snapshot SUCCESS path (sidecar storage of orthorectified tiles) — owned by AZ-294 / task #4. Failed-tile thumbnails are a DIFFERENT kind with inline (≤ 4 KiB) JPEG bytes, NOT sidecar.
- The `kind="raw_nav_frame"` / `kind="failed_tile_thumbnail"` payload schemas — owned by AZ-272.
- Per-segment / per-flight cap policies — owned by AZ-291 / AZ-293.
- Producer-side rate limiting BEFORE thumbnails are constructed (e.g. C6's decision to attempt orthorectification at most every N frames) — that is per-producer concern; the C13 cap is a defense-in-depth global ceiling.
- Cryptographic signing of records — out of scope this cycle.

## Acceptance Criteria

**AC-1: enforce_or_raise rejects raw_nav_frame**
Given `RecordKindPolicy` constructed with default config, and an `FdrRecord(kind="raw_nav_frame", producer_id="c1_vio", payload={...})`
When the producer calls `enforce_or_raise(record)`
Then `RawFrameWriteForbiddenError` is raised; the message includes both `"raw_nav_frame"` and `"c1_vio"`; no record is enqueued (the producer's subsequent `fdr_client.enqueue` is never reached because the call site re-raises)

**AC-2: enforce_or_raise rejects raw_ai_cam_frame**
Given the default-configured policy
When `enforce_or_raise` is called with `kind="raw_ai_cam_frame"`
Then `RawFrameWriteForbiddenError` is raised (same as AC-1)

**AC-3: enforce_or_raise passes through failed_tile_thumbnail**
Given the default policy and `FdrRecord(kind="failed_tile_thumbnail", payload={frame_id: 1, tile_id: "x", jpeg_bytes_b64: "..."})`
When `enforce_or_raise` is called
Then the call returns silently; no exception is raised; the producer is free to enqueue

**AC-4: gate_for_writer admits in-cap thumbnails**
Given `failed_tile_thumbnail_max_hz=0.1` (one per 10 s window) and the writer is starting fresh
When `gate_for_writer(record)` is called once with a `failed_tile_thumbnail` record
Then the return value is `ENQUEUE`; the record proceeds to disk

**AC-5: gate_for_writer drops over-cap thumbnails + emits coalesced overrun record**
Given `failed_tile_thumbnail_max_hz=0.1` and 5 thumbnails arrive within a single 10-second window
When the writer calls `gate_for_writer` on each
Then the FIRST returns `ENQUEUE`; the next 4 return `DROP`; ONE `kind="overrun"` record is emitted at the end of the window with `payload.dropped_count==4` and `payload.producer_id=<originating producer>`; the WARN log fires at most once per second

**AC-6: Forbidden set REJECTS removal of defaults**
Given a Config preset that attempts to set `forbidden_record_kinds = frozenset()` (empty — removing all defaults)
When the Config is validated
Then a `ConfigValidationError` is raised naming the missing default kinds; the policy cannot be constructed from this config

**AC-7: Forbidden set ALLOWS additions**
Given a Config preset that sets `forbidden_record_kinds = frozenset({"raw_nav_frame", "raw_ai_cam_frame", "raw_thermal_frame"})`
When the policy is constructed
Then the policy rejects all three kinds via `enforce_or_raise`; the existing tests for the original two kinds still pass

**AC-8: Hz=0 is rejected at config validation**
Given a Config preset with `failed_tile_thumbnail_max_hz=0`
When the Config is validated
Then a `ConfigValidationError` is raised; the policy cannot be constructed

**AC-9: Sliding window resets — bursts spread across windows are admitted**
Given `failed_tile_thumbnail_max_hz=0.1` and one thumbnail at t=0, one at t=11s, one at t=22s
When `gate_for_writer` is called for each
Then ALL THREE return `ENQUEUE` (one per window); zero overrun records are emitted

**AC-10: Producer slug propagates to overrun.payload.producer_id under cap-driven drops**
Given thumbnails arriving under cap-driven drop conditions, with the originating producer being `c6_tile_cache`
When the overrun record is emitted
Then `payload.producer_id == "c6_tile_cache"` (matches the producer the original thumbnails came from, NOT `"shared.fdr_client"` for the payload — the OUTER envelope's producer_id is `"shared.fdr_client"` per the schema contract)

## Non-Functional Requirements

**Performance**
- `enforce_or_raise` p99 ≤ 1 µs (a single set membership check; no allocation).
- `gate_for_writer` p99 ≤ 5 µs on the in-cap path; p99 ≤ 10 µs on the cap-driven drop path (sliding-window counter update + overrun-record construction).
- Both methods are allocation-free on the steady-state in-cap path.

**Reliability**
- The forbidden-kind set is read once at policy construction and stored as an `frozenset` (immutable across the policy's lifetime). Runtime mutation via reflection is detected by code-review Phase 7 (architecture/security).
- The sliding-window counter is per-policy-instance, not global; a single policy serves the whole flight (one composition-root construction). Resetting between flights happens via a new policy instance at takeoff.
- The policy's WARN-log rate cap uses the same `kind="fdr.write_failure"` rate cap pattern from AZ-291 (≤ 1 WARN/sec) — implemented inside the policy, no shared rate-limit state with the writer thread.

## Unit Tests

| AC Ref | What to Test | Required Outcome |
|--------|-------------|-----------------|
| AC-1 | enforce_or_raise on `kind="raw_nav_frame"` | RawFrameWriteForbiddenError; message contains kind + producer_id |
| AC-2 | enforce_or_raise on `kind="raw_ai_cam_frame"` | RawFrameWriteForbiddenError |
| AC-3 | enforce_or_raise on `kind="failed_tile_thumbnail"` | Returns silently |
| AC-4 | gate_for_writer for first thumbnail in fresh window | Returns ENQUEUE |
| AC-5 | 5 thumbnails in one 10 s window | First ENQUEUE; next 4 DROP; one overrun record (dropped_count=4); ≤ 1 WARN |
| AC-6 | Empty `forbidden_record_kinds` config | ConfigValidationError |
| AC-7 | Adding `raw_thermal_frame` to forbidden set | All three kinds rejected; defaults still rejected |
| AC-8 | `failed_tile_thumbnail_max_hz=0` | ConfigValidationError |
| AC-9 | Thumbnails at t=0, t=11, t=22 with 10 s window | All three ENQUEUE; zero overrun records |
| AC-10 | Drop scenario with originating producer `c6_tile_cache` | overrun record's `payload.producer_id == "c6_tile_cache"` |
| NFR-perf-enforce | Microbench `enforce_or_raise` 10k iter | p99 ≤ 1 µs |
| NFR-perf-gate-allow | Microbench `gate_for_writer` in-cap | p99 ≤ 5 µs |
| NFR-perf-gate-drop | Microbench `gate_for_writer` over-cap | p99 ≤ 10 µs |
| NFR-reliability-immutable | Attempt to mutate `policy.forbidden_kinds` after construction | TypeError (frozenset) or AttributeError (no setter) |

## Constraints

- The forbidden-kind set is defense-in-depth, NOT the primary line of defense. Producers MUST not construct `raw_nav_frame` records in the first place (that is owned by their respective component specs); this gate catches regressions and malicious producers.
- The sliding-window counter MUST be O(1) update per call; an O(N) implementation that scans a list of timestamps is rejected at code-review Phase 7 (architecture).
- The cap and forbidden set apply globally across all producers within a flight, NOT per-producer. A single producer cannot consume the entire 0.1 Hz budget by exclusion of others — the budget is a global capacity for the FDR's inline thumbnail throughput. (Per-producer caps, if needed, are owned by individual component specs.)
- This task does NOT introduce new dependencies. Stdlib `time.monotonic_ns` + a fixed-size deque (or constant counter) suffice for the sliding window.

## Risks & Mitigation

**Risk 1: AC-7's "additions allowed" path is abused to add legitimate kinds (e.g. `state.tick`)**
- *Risk*: A misconfigured deployment adds `state.tick` to the forbidden set and silently breaks the entire FDR.
- *Mitigation*: Config validation cross-checks the forbidden set against the v1.0.0 schema's closed enum of legitimate kinds and REJECTS additions that are in the schema. The forbidden set is intended to be a SUBSET of "kinds that don't appear in v1.0.0 closed enum + raw-frame variants we explicitly want to ban". Documented in the Config validator + AC-6 tests.

**Risk 2: Producer-side enforce_or_raise wrapper not actually called**
- *Risk*: A future producer forgets to call `policy.enforce_or_raise` and goes straight to `fdr_client.enqueue` — bypassing the synchronous gate.
- *Mitigation*: A code-review Phase 2 (Spec Compliance) check requires every producer calling `fdr_client.enqueue` to also call `policy.enforce_or_raise` immediately before. The writer-side `gate_for_writer` is the defense-in-depth catch — even if a forbidden-kind record sneaks past the producer, the writer drops it and emits an `overrun` record (the security AC is "no raw frame on disk", not "no raw frame in producer memory"). Both gates exist precisely so producer-side bypasses become observable in logs.

**Risk 3: Sliding-window counter clock drift**
- *Risk*: `time.monotonic_ns` is per-process; if the process is suspended (Jetson power management) the window appears to compress.
- *Mitigation*: `monotonic_ns` does NOT advance during suspend (per CPython docs); on resume, the counter sees a single large gap. The sliding window adapts naturally — old samples drop out, and the next thumbnail is admitted. Documented; no special mitigation needed.

**Risk 4: WARN log rate cap interferes with debugging**
- *Risk*: An operator investigating a thumbnail flood sees only one WARN per second and misses the burst pattern.
- *Mitigation*: The OVERRUN RECORD emitted into the FDR carries the per-window `dropped_count`; that is the canonical record. The WARN log is operator convenience only. Documented in the policy's docstring.

## Runtime Completeness

- **Named capability**: AC-8.5 forbidden-kind synchronous enforcement + failed-tile thumbnail rate cap (architecture / E-C13 / AC-8.5 / RESTRICT-UAV-4 / C13-IT-03).
- **Production code that must exist**: real `RecordKindPolicy` with both methods, real composition-root wiring into producer paths AND the writer thread, real sliding-window counter, real overrun-record emission on drop.
- **Allowed external stubs**: tests MAY use `FakeFdrSink` (AZ-275); production wiring uses the real shared FdrClient.
- **Unacceptable substitutes**: writer-only enforcement without producer-side `enforce_or_raise` ("the writer will catch it" — too late, the bytes already crossed the wire format), config that allows removing `raw_nav_frame` from defaults silently ("operators know what they're doing"), token-bucket without coalescing ("we'll emit one overrun per drop") — all break C13-IT-03 + AC-NEW-3 + the fundamental AC-8.5 invariant that raw frames MUST NEVER touch durable storage.
