# Component — `vlm_client` (optional)

**Layer**: Perception (data plane in)
**Status**: forward-looking design (Rust); optional behind a feature flag and a runtime config flag

## 1. Purpose

Tier 3 of the perception pipeline. Asks a local NanoLLM/VILA1.5-3B process to confirm a zoom-in endpoint POI using one bounded ROI crop and a short prompt. Returns a structured `VlmAssessment`. The free-form VLM text is **not** a downstream API contract — only the validated structured output is.

VLM is optional; the system MUST function correctly when VLM is disabled or absent.

## 2. Inputs

| Input | Source | Cadence | Notes |
|---|---|---|---|
| Zoom-in ROI crop + prompt | `scan_controller` | per zoom-in endpoint hold | One bounded crop, short prompt, short answer. |
| `vlm_enabled` runtime flag | startup config | once at start (re-readable on SIGHUP if implemented) | Gates whether `scan_controller` calls this component at all. |
| IPC socket path | startup config | once | Unix-domain socket to the NanoLLM process. |

## 3. Outputs

| Output | Consumer | Shape |
|---|---|---|
| `VlmAssessment` | `scan_controller` | `{ label, confidence, status: ok \| inconclusive \| timeout \| schema_invalid \| ipc_error \| disabled, source_roi_id, latency_ms, model_version }` |
| Health metric | health aggregator | `enabled`, `vlm_latency_p50/p99`, `errors_by_kind`, `peer_cred_check_pass_rate`. |

## 4. Key Responsibilities

- Validate the ROI payload (size, format) **before** sending it across the IPC channel.
- Maintain the Unix-domain-socket connection to the NanoLLM process; perform a peer-credential check on connect (where supported by the platform).
- Send one bounded ROI + short prompt; await one short response within ≤5 s.
- Validate the response against the `VlmAssessment` schema; on schema-invalid, return `status: schema_invalid` to `scan_controller` and surface to health.
- Return `status: disabled` when the runtime flag is `false`; `scan_controller` treats this identically to "VLM not present" and proceeds with Tier 2 evidence alone.
- Capture `model_version` (whatever the NanoLLM process reports for its loaded weights) on every assessment for forensic correlation; log the version on change.

## 5. Internal State

- IPC socket handle and peer-credential cache.
- In-flight request map (request id → caller).

State is in-process only.

## 6. Failure Modes

| Failure | Detection | Behaviour |
|---|---|---|
| VLM process not reachable | connect / send error | Return `status: ipc_error`; bounded-backoff reconnect; health → yellow then red. |
| Peer-cred check fails | platform API | Hard-fail the connect; do not retry without operator intervention; health → red. |
| Response timeout (>5 s) | wall-clock | Return `status: timeout`; do not block `scan_controller` past the budget. |
| Schema-invalid response | response parser | Return `status: schema_invalid`; log the raw response (size-capped) for offline analysis. |
| ROI payload too large | pre-send size check | Return `status: schema_invalid` synchronously; never send. |
| Optional component absent at build time | feature flag off at compile | `scan_controller` depends only on the `VlmAssessment` provider trait; the default impl returns `status: disabled`. The binary builds and runs identically without `vlm_client`. |

## 7. Dependencies

**In-process**: `scan_controller`.

**External**: NanoLLM / VILA1.5-3B local process. IPC over Unix-domain socket. No network egress.

## 8. Non-Functional Targets

| Concern | Target |
|---|---|
| Per-ROI latency | ≤5 s p99 |
| Memory budget | within the 6 GB shared budget after Tier 1 + Tier 2 |
| Cloud egress | **none** (hard rule) |
| Failure mode | fail-closed — never surface a POI with VLM evidence on a degraded VLM call |

## 9. Optionality Model

Two complementary mechanisms; the implementation chooses one or both:

1. **Runtime flag (`vlm_enabled`)** gated by the benchmark-gate result. When `false`, `scan_controller` skips VLM confirmation; the zoom-in hold proceeds with Tier 2 evidence alone.
2. **Build-time feature module.** `vlm_client` is a separate Cargo feature; the binary builds, links, and runs identically when the feature is off. `scan_controller` depends on a `VlmAssessmentProvider` trait whose default impl returns `status: disabled`.

Both must yield the same observable behaviour: the system functions correctly with VLM absent, only losing the zoom-in confirmation step.

## 10. References

- `architecture.md §5 Architectural Principles` (no cloud egress, fail-closed), `§7.6 Local VLM confirmation`.
- `system-flows.md §F3 VLM confirmation` (with explicit fail-closed and disabled branches).
- `data_model.md §VlmAssessment`.