Files
autopilot/_docs/02_document/module-layout.md
T
Oleksandr Bezdieniezhnykh 0854d3be1c [AZ-659] [AZ-660] [AZ-661] Implement frame publisher + gRPC detection client
AZ-659: FramePublisher with per-consumer drop accounting (Arc<Bytes>
zero-copy fan-out). Adds ConsumerId enum, PublisherStats, FrameReceiver
wrapper, and publisher integration tests (AC-1, AC-2, AC-3).

AZ-660: Bi-directional tonic gRPC stream to ../detections. Reconnect
with bounded exponential backoff (1 s → 30 s cap). Drop-oldest
in-flight budgeting (max_concurrent_in_flight = 2). ai_locked frame
skipping. Integration tests against fixture in-process server
(AC-1: happy path 30 fps/10 s, AC-2: reconnect, AC-3: budget drops,
AC-4: ai_locked skipping).

AZ-661: Schema validation (hard SchemaMismatch error on version
mismatch), model_version latch with ModelVersionChanged events,
sliding-window p99 latency tracker with Tier1Degraded/Tier1Recovered
transitions. Integration tests (AC-1, AC-2, AC-3).

Also: update module-layout.md for frame_ingest and detection_client
to reflect the streaming API shape; code review report batch_18.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 18:23:56 +03:00

24 KiB
Raw Blame History

Module Layout

Language: rust Layout Convention: crates-workspace Root: crates/ Last Updated: 2026-05-19

Layout Rules

  1. Each component owns ONE top-level directory under crates/. The directory name matches the component name in _docs/02_document/components/.
  2. Shared code lives in a single crates/shared/ crate. Cross-cutting concerns are modules inside it (shared/models/, shared/config/, shared/error/, shared/health/, shared/observability/, shared/clock/, shared/contracts/). Other crates re-export from shared::; they MUST NOT duplicate types.
  3. Public API surface per component = the files in Public API below. Everything under src/internal/ (and any other module not listed in Public API) is internal and other crates MUST NOT use it.
  4. Tests live in each crate's own tests/ directory (Rust convention). Workspace-level end-to-end tests live at tests/e2e/ (the workspace root, not under any crate).
  5. Stream-based wiring: tokio channels carrying shared data types are passed into actor constructors by the composition root (crates/autopilot). This keeps Layer 2 actors free of sibling imports — they receive Receiver<Frame>, Receiver<GimbalState>, etc. from shared::models without importing the crate that produces them.
  6. Sink traits in shared: where one component must push into another's transport (e.g. operator_bridge pushes POIs through telemetry_stream), the receiving side implements a trait defined in shared::contracts (TelemetrySink, MavlinkSink, etc.). The producing side depends only on the trait, not on the receiving crate.

Per-Component Mapping

Component: shared

  • Epic: AZ-626 (Bootstrap & Initial Structure — shared crate lands as part of AZ-640 initial structure task)
  • Directory: crates/shared/
  • Public API:
    • crates/shared/src/lib.rs (re-exports the submodules listed below)
    • crates/shared/src/models/mod.rs (Frame, BoundingBox, Detection, DetectionBatch, MovementCandidate, Tier2Evidence, VlmAssessment, POI, MapObject, MapObjectObservation, MapObjectsBundle, IgnoredItem, Coordinate, Geofence, MissionItem, MissionWaypoint, OperatorCommand, GimbalState)
    • crates/shared/src/config/mod.rs (Config, ConfigLoader, per-component typed sections)
    • crates/shared/src/error.rs (AutopilotError, Result<T>)
    • crates/shared/src/health.rs (ComponentHealth, AggregatedHealth, HealthLevel)
    • crates/shared/src/observability/mod.rs (tracing init, log-field constants per observability.md §2)
    • crates/shared/src/clock.rs (MonoClock, WallClock, ClockSource)
    • crates/shared/src/contracts/mod.rs (TelemetrySink, MavlinkSink, VlmProvider, OperatorCommandSink)
  • Internal: none — shared has no internal/ subtree; everything in shared is part of its public API by design.
  • Owns (exclusive write during implementation): crates/shared/**
  • Imports from: (none — Layer 1)
  • Consumed by: every other component crate + the autopilot binary

  • Epic: AZ-637
  • Directory: crates/mavlink_layer/
  • Public API:
    • crates/mavlink_layer/src/lib.rs (MavlinkLayer, MavlinkHandle, MavlinkConnection, public message types re-exported from shared::models)
  • Internal:
    • crates/mavlink_layer/src/internal/codec/* (MAVLink v2 encode/decode for the §7.7 surface only)
    • crates/mavlink_layer/src/internal/transport/udp.rs
    • crates/mavlink_layer/src/internal/transport/serial.rs
    • crates/mavlink_layer/src/internal/heartbeat.rs
    • crates/mavlink_layer/src/internal/retry.rs
  • Owns: crates/mavlink_layer/**
  • Imports from: shared
  • Consumed by: mission_executor, telemetry_stream (via constructor-injected Receiver<MavlinkTelemetry> or via the MavlinkSink trait)

Component: mission_client

  • Epic: AZ-638
  • Directory: crates/mission_client/
  • Public API:
    • crates/mission_client/src/lib.rs (MissionClient, MissionClientHandle::pull_mission(), post_middle_waypoint(), pull_mapobjects(), push_mapobjects(), health())
  • Internal:
    • crates/mission_client/src/internal/missions_api/* (REST client + retry + auth)
    • crates/mission_client/src/internal/mapobjects_sync/* (pre-flight GET + post-flight POST bundles)
    • crates/mission_client/src/internal/schema/* (schema-version validation against mission-schema)
  • Owns: crates/mission_client/**
  • Imports from: shared
  • Consumed by: mission_executor (for mission lifecycle), mapobjects_store (for hydrate/dump indirectly through mission_executor orchestration)

Component: frame_ingest

  • Epic: AZ-627
  • Directory: crates/frame_ingest/
  • Public API:
    • crates/frame_ingest/src/lib.rs (FrameIngest, FrameIngestHandle, ConsumerId)
    • FrameIngestHandle::subscribe() -> Receiver<Frame> — raw broadcast receiver (no per-consumer accounting)
    • FrameIngestHandle::subscribe_as(ConsumerId) -> FrameReceiver — receiver with per-consumer lag accounting
    • FrameIngestHandle::publisher() -> Arc<FramePublisher> — direct publisher handle for the composition root
    • FrameIngestHandle::dropped_frames(ConsumerId) -> u64, publishes_total() -> u64
    • FrameIngestHandle::health() -> ComponentHealth
  • Internal:
    • crates/frame_ingest/src/internal/publisher.rs (FramePublisher, FrameReceiver, PublisherStats)
    • crates/frame_ingest/src/internal/rtsp_client.rs
    • crates/frame_ingest/src/internal/decoder.rs
    • crates/frame_ingest/src/internal/timestamp.rs
  • Owns: crates/frame_ingest/**
  • Imports from: shared
  • Consumed by: detection_client, movement_detector, telemetry_stream (all via composition-root-wired Receiver<Frame>)

Component: detection_client

  • Epic: AZ-628
  • Directory: crates/detection_client/
  • Public API:
    • crates/detection_client/src/lib.rs (DetectionClient, DetectionClientConfig, DetectionClientHandle, DetectionEvent, ConnectionState, Tier1DegradationReason)
    • DetectionClient::run(frame_rx: Receiver<Frame>) -> (JoinHandle, DetectionClientHandle) — spawns the gRPC supervisor task
    • DetectionClientHandle::subscribe_events() -> Receiver<DetectionEvent> — broadcast stream of batches, schema errors, model-version changes, Tier-1 degradation transitions
    • DetectionClientHandle::health() -> ComponentHealth
    • DetectionClientHandle::stats() -> Arc<DetectionStats>, latency_p50/p99(), connection_state(), shutdown()
  • Internal:
    • crates/detection_client/build.rs (tonic-build for the gRPC proto)
    • crates/detection_client/proto/detections.proto (vendored copy of ../detections contract per architecture.md §10)
    • crates/detection_client/src/internal/runtime.rs (supervisor + bi-directional stream session)
    • crates/detection_client/src/internal/budget.rs (drop-oldest in-flight tracker)
    • crates/detection_client/src/internal/latency.rs (sliding-window p99 + degradation latch)
    • crates/detection_client/src/internal/stats.rs (lock-free atomic counters)
    • crates/detection_client/src/internal/proto.rs (generated tonic/prost types)
  • Owns: crates/detection_client/**
  • Imports from: shared
  • Consumed by: scan_controller (subscribes to events), telemetry_stream (via composition-root-wired Receiver<DetectionBatch> for operator overlay)

Component: gimbal_controller

  • Epic: AZ-634
  • Directory: crates/gimbal_controller/
  • Public API:
    • crates/gimbal_controller/src/lib.rs (GimbalController, GimbalControllerHandle::set_pose(...), zoom(level), state() -> GimbalState, state_stream() -> Receiver<GimbalState>, health())
  • Internal:
    • crates/gimbal_controller/src/internal/a40_protocol/* (ViewPro A40 UDP vendor protocol — encode, decode, CRC)
    • crates/gimbal_controller/src/internal/smooth_pan.rs (smooth-pan path-tracking primitive)
  • Owns: crates/gimbal_controller/**
  • Imports from: shared
  • Consumed by: scan_controller (handle), movement_detector (via constructor-injected Receiver<GimbalState>), frame_ingest (constructor-injected Receiver<GimbalState> for timestamp annotation if needed)

Component: semantic_analyzer

  • Epic: AZ-630
  • Directory: crates/semantic_analyzer/
  • Public API:
    • crates/semantic_analyzer/src/lib.rs (SemanticAnalyzer, SemanticAnalyzerHandle::analyze(roi) -> Result<Tier2Evidence>, health())
  • Internal:
    • crates/semantic_analyzer/src/internal/primitive_graph/* (path, branch-pile, entrance, road graph reasoner)
    • crates/semantic_analyzer/src/internal/roi_cnn.rs (TensorRT ROI CNN wrapper)
    • crates/semantic_analyzer/src/internal/scoring/* (path-freshness, endpoint, concealment)
  • Owns: crates/semantic_analyzer/**
  • Imports from: shared
  • Consumed by: scan_controller (handle)

Component: vlm_client

  • Epic: AZ-631
  • Directory: crates/vlm_client/
  • Public API:
    • crates/vlm_client/src/lib.rs (VlmClient implementing shared::contracts::VlmProvider; VlmClient::with_default() returns the no-op impl returning VlmAssessment { status: vlm_disabled }; real impl is gated behind feature = "vlm")
  • Internal:
    • crates/vlm_client/src/internal/uds_client.rs (Unix-domain socket IPC + peer-credential check)
    • crates/vlm_client/src/internal/schema_validate.rs (VlmAssessment schema validation)
    • crates/vlm_client/src/internal/prompt.rs (bounded prompt + payload size enforcement)
  • Owns: crates/vlm_client/**
  • Imports from: shared
  • Consumed by: scan_controller (via the shared::contracts::VlmProvider trait — never directly)

Component: mapobjects_store

  • Epic: AZ-633
  • Directory: crates/mapobjects_store/
  • Public API:
    • crates/mapobjects_store/src/lib.rs (MapObjectsStore, MapObjectsStoreHandle::classify(Detection) -> Classification, apply_decline(POI), dump_pending() -> MapObjectsBundle, hydrate(MapObjectsBundle), set_sync_state(SyncState), health())
  • Internal:
    • crates/mapobjects_store/src/internal/h3_index/* (h3rs wrapper + k-ring queries)
    • crates/mapobjects_store/src/internal/engine/mod.rs (StorageEngine trait — pluggable for Q3)
    • crates/mapobjects_store/src/internal/engine/in_memory_snapshot.rs (default impl: in-memory + JSON snapshot on flush)
    • crates/mapobjects_store/src/internal/diff.rs (NEW / MOVED / EXISTING / REMOVED-CANDIDATE classification)
    • crates/mapobjects_store/src/internal/ignored.rs
  • Owns: crates/mapobjects_store/**
  • Imports from: shared
  • Consumed by: scan_controller, operator_bridge, mission_executor (for hydrate at pre-flight + dump_pending at post-flight)

Component: movement_detector

  • Epic: AZ-629
  • Directory: crates/movement_detector/
  • Public API:
    • crates/movement_detector/src/lib.rs (MovementDetector, MovementDetectorHandle::candidates() -> Receiver<MovementCandidate>, health(); constructor takes Receiver<Frame>, Receiver<GimbalState>, Receiver<MavlinkTelemetry>)
  • Internal:
    • crates/movement_detector/src/internal/ego_motion.rs (homography-based ego-motion estimate)
    • crates/movement_detector/src/internal/optical_flow/* (classical CV path)
    • crates/movement_detector/src/internal/learned_cv/* (fallback per Q14 — behind feature = "learned_cv")
    • crates/movement_detector/src/internal/zoom_bands.rs (per-zoom-band threshold tables)
    • crates/movement_detector/src/internal/telemetry_sync.rs (frame ↔ gimbal ↔ UAV skew gate)
  • Owns: crates/movement_detector/**
  • Imports from: shared
  • Consumed by: scan_controller (consumes the MovementCandidate stream)

Component: telemetry_stream

  • Epic: AZ-639
  • Directory: crates/telemetry_stream/
  • Public API:
    • crates/telemetry_stream/src/lib.rs (TelemetryStream implementing shared::contracts::TelemetrySink; TelemetryStreamHandle::commands() -> Receiver<OperatorCommand>, health(); constructor takes Receiver<Frame>, Receiver<DetectionBatch>, Receiver<MavlinkTelemetry>, Receiver<BboxOverlay>)
  • Internal:
    • crates/telemetry_stream/src/internal/uplink/* (modem push: protocol per ../_docs/04_system_design_clarifications.md — Q2)
    • crates/telemetry_stream/src/internal/downlink/* (operator-command receive path)
    • crates/telemetry_stream/src/internal/encode/* (frame + telemetry + bbox-overlay serialisation)
  • Owns: crates/telemetry_stream/**
  • Imports from: shared
  • Consumed by: operator_bridge (via the TelemetrySink trait in shared::contracts; commands consumed via constructor-injected Receiver<OperatorCommand>)

Component: operator_bridge

  • Epic: AZ-635
  • Directory: crates/operator_bridge/
  • Public API:
    • crates/operator_bridge/src/lib.rs (OperatorBridge, OperatorBridgeHandle::surface_poi(POI) -> OperatorDecision, middle_waypoint_hints() -> Receiver<MiddleWaypointHint>, target_follow_events() -> Receiver<TargetFollowEvent>, health(); constructor takes Arc<dyn TelemetrySink> and Receiver<OperatorCommand>)
  • Internal:
    • crates/operator_bridge/src/internal/auth/* (OperatorCommand envelope validation — signature, replay protection, session validation; scheme stubbed pending Q9)
    • crates/operator_bridge/src/internal/audit_log.rs (persistent audit log writer for /var/lib/autopilot/audit/)
    • crates/operator_bridge/src/internal/decision_window.rs (confidence-scaled timeout: 40 % → 30 s, 100 % → 120 s linear)
  • Owns: crates/operator_bridge/**
  • Imports from: shared, mapobjects_store
  • Consumed by: scan_controller (for surface_poi), mission_executor (consumes middle_waypoint_hints stream)

Component: mission_executor

  • Epic: AZ-636
  • Directory: crates/mission_executor/
  • Public API:
    • crates/mission_executor/src/lib.rs (MissionExecutor, MissionExecutorHandle::start(Mission), insert_middle_waypoint(Coordinate), failsafe_trigger(FailsafeKind), state() -> ExecutorState, health(); constructor takes Receiver<MiddleWaypointHint> from operator_bridge)
  • Internal:
    • crates/mission_executor/src/internal/multirotor/fsm.rs (DISCONNECTED → … → LAND)
    • crates/mission_executor/src/internal/fixed_wing/fsm.rs (DISCONNECTED → … → WAIT_AUTO → LAND)
    • crates/mission_executor/src/internal/geofence/* (INCLUSION + EXCLUSION enforcement)
    • crates/mission_executor/src/internal/failsafe/ladder.rs (lost-link LinkOk → LinkDegraded → LinkLost → LinkLostInFollow)
    • crates/mission_executor/src/internal/battery_thresholds.rs (RTL floor, hard floor)
    • crates/mission_executor/src/internal/bit.rs (pre-flight built-in self-test; orchestrates pre-flight mapobjects_store.hydrate(mission_client.pull_mapobjects(...)))
    • crates/mission_executor/src/internal/middle_waypoint.rs (re-upload sequence on operator confirm)
    • crates/mission_executor/src/internal/post_flight.rs (orchestrates post-flight mission_client.push_mapobjects(mapobjects_store.dump_pending()))
  • Owns: crates/mission_executor/**
  • Imports from: shared, mavlink_layer, mission_client, mapobjects_store
  • Consumed by: scan_controller (for failsafe_trigger and insert_middle_waypoint)

Component: scan_controller

  • Epic: AZ-632
  • Directory: crates/scan_controller/
  • Public API:
    • crates/scan_controller/src/lib.rs (ScanController, ScanControllerHandle::tick(), submit_operator_cmd(OperatorCommand), state() -> ScanState, health(); constructor takes Receiver<DetectionBatch>, Receiver<MovementCandidate>, Receiver<Frame> plus handles for mapobjects_store, gimbal_controller, mission_executor, semantic_analyzer, operator_bridge, and Arc<dyn VlmProvider>)
  • Internal:
    • crates/scan_controller/src/internal/state_machine/mod.rs (ZoomedOut, ZoomedIn { roi, hold_started_at }, TargetFollow { target_id, started_at })
    • crates/scan_controller/src/internal/state_machine/transitions.rs
    • crates/scan_controller/src/internal/poi_queue/* (priority queue + ≤5 POIs/min cap + confidence × proximity × age ordering)
    • crates/scan_controller/src/internal/behaviour_tree/* (per system-flows.md §F4)
    • crates/scan_controller/src/internal/timeouts.rs (operator-decision window, POI timeouts, VLM waits)
    • crates/scan_controller/src/internal/frame_rate_guard.rs (suppress zoom-in transitions below ≥10 fps; surface yellow health)
  • Owns: crates/scan_controller/**
  • Imports from: shared, mapobjects_store, gimbal_controller, mission_executor, semantic_analyzer, operator_bridge
  • Consumed by: autopilot (composition root)

Component: autopilot (binary, composition root)

  • Epic: AZ-626 (Bootstrap & Initial Structure — the binary scaffold is part of AZ-640)
  • Directory: crates/autopilot/
  • Public API: this is a [[bin]] crate — it exposes no library API.
  • Internal:
    • crates/autopilot/src/main.rs (CLI parse, config load, tracing init, build component instances, run)
    • crates/autopilot/src/runtime.rs (build channels, wire actors, owns the Vec<JoinHandle>, shutdown orchestration)
    • crates/autopilot/src/health_server.rs (HTTP /health endpoint per containerization.md §7)
    • crates/autopilot/src/bit_runner.rs (invokes mission_executor.bit() and gates startup)
  • Owns: crates/autopilot/**
  • Imports from: shared + every Layer 2 actor crate + every Layer 3 coordinator + scan_controller
  • Consumed by: nothing — this is the binary

Shared / Cross-Cutting

All cross-cutting concerns live as modules inside the single crates/shared/ crate (Rust convention prefers a single shared crate over many tiny ones; the module boundaries inside shared:: enforce conceptual separation).

shared::models

  • Path: crates/shared/src/models/
  • Purpose: the canonical entity catalogue from _docs/02_document/data_model.md. One submodule per entity grouping (frame.rs, detection.rs, movement.rs, tier2.rs, vlm.rs, poi.rs, mapobject.rs, mission.rs, operator.rs, gimbal.rs).
  • Owned by: AZ-640 initial structure task (under epic AZ-626).
  • Consumed by: every component crate + the autopilot binary.

shared::config

  • Path: crates/shared/src/config/
  • Purpose: TOML loader (per containerization.md §6), typed per-component sections, environment-variable overlay, secrets resolution (via path to EnvironmentFile=).
  • Owned by: AZ-640 initial structure task.
  • Consumed by: every component crate.

shared::error

  • Path: crates/shared/src/error.rs
  • Purpose: AutopilotError enum + Result<T> = std::result::Result<T, AutopilotError> alias.
  • Owned by: AZ-640 initial structure task.
  • Consumed by: every crate.

shared::health

  • Path: crates/shared/src/health.rs
  • Purpose: ComponentHealth, HealthLevel { Green, Yellow, Red, Disabled }, AggregatedHealth — each component exposes its own health() -> ComponentHealth; autopilot::health_server aggregates per containerization.md §7.
  • Owned by: AZ-640 initial structure task.
  • Consumed by: every component + the binary's health server.

shared::observability

  • Path: crates/shared/src/observability/
  • Purpose: tracing-subscriber init (JSON to stdout); log-field constants for the §2 fields in observability.md; span helpers for frame trace + POI trace.
  • Owned by: AZ-640 initial structure task.
  • Consumed by: every component (for spans and counters).

shared::clock

  • Path: crates/shared/src/clock.rs
  • Purpose: MonoClock (monotonic, authoritative for telemetry-skew compensation and tick budgets), WallClock (bound to GPS time once locked, NTP at boot), ClockSource { Gnss, Host, Coast }. Drift > 200 ms → yellow health.
  • Owned by: AZ-640 initial structure task.
  • Consumed by: every component that timestamps anything (frame_ingest, movement_detector, scan_controller, operator_bridge audit log, mapobjects_store).

shared::contracts

  • Path: crates/shared/src/contracts/
  • Purpose: trait definitions for cross-component coupling that we want to keep import-free:
    • TelemetrySinkpush_frame, push_telemetry, push_overlay (impl: telemetry_stream)
    • MavlinkSinksend (impl: mavlink_layer; lets mission_executor depend on a trait rather than the concrete crate when convenient)
    • VlmProviderassess(roi) -> VlmAssessment (impl: vlm_client; default no-op impl returns vlm_disabled)
    • OperatorCommandSinkdispatch(OperatorCommand) (lets the composition root forward decoded commands from telemetry_stream to operator_bridge without coupling them)
  • Owned by: AZ-640 initial structure task.
  • Consumed by: operator_bridge (TelemetrySink), scan_controller (VlmProvider), mission_executor (may use MavlinkSink), telemetry_stream + vlm_client (implement the traits).

Allowed Dependencies (layering)

Read top-to-bottom; an upper layer may import from a lower layer but NEVER the reverse. Same-layer imports are explicitly listed in each component's Imports from.

Layer Components May import from
5. Composition autopilot (binary) 1, 2, 3, 4
4. Brain scan_controller 1, 2, 3
3. Coordinators operator_bridge, mission_executor 1, 2
2. Actors / Transports / Storage mavlink_layer, mission_client, frame_ingest, detection_client, movement_detector, semantic_analyzer, vlm_client, mapobjects_store, gimbal_controller, telemetry_stream 1
1. Shared / Foundation shared/* (none)

Violations of this table are Architecture findings in code-review and are High severity. Specifically:

  • A Layer 2 actor MAY NOT import a sibling Layer 2 actor. Stream dependencies (e.g. movement_detector consuming Frame) are wired via constructor-injected channels by the composition root; sink dependencies (e.g. operator_bridge pushing into telemetry_stream) are bridged via a trait in shared::contracts.
  • A Layer 3 coordinator MAY import any Layer 2 actor whose handle it directly calls. operator_bridge imports mapopjects_store for apply_decline. mission_executor imports mavlink_layer, mission_client, and mapobjects_store.
  • A Layer 3 coordinator MAY NOT import another Layer 3 coordinator. mission_executor consumes MiddleWaypointHint from operator_bridge via a constructor-injected Receiver<MiddleWaypointHint> wired by the composition root.

Layout Conventions (reference)

Language Root Per-component path Public API file Test path
Rust crates/ crates/<component>/ crates/<component>/src/lib.rs crates/<component>/tests/ (crate-level) + tests/e2e/ (workspace-level)

Self-verification

  • Every component in _docs/02_document/components/ has a Per-Component Mapping entry (13 components + shared + autopilot binary).
  • Every shared / cross-cutting concern has a Shared section entry (models, config, error, health, observability, clock, contracts).
  • Layering table covers every component, with shared at the bottom and autopilot binary at the top.
  • No component's Imports from list points at a higher layer. (scan_controller Layer 4 → Layers 1, 2, 3; coordinators Layer 3 → Layers 1, 2; actors Layer 2 → Layer 1 only.)
  • Paths follow Rust's crates/<component>/ convention.
  • No two components own overlapping paths — each Owns glob is rooted at a distinct crates/<component>/**.