azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-21 09:41:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-322] C10 DescriptorBatcher (faiss-cpu, OOM halve-retry)

Browse Source 
Implements the C10 internal phase that walks every C6 tile, embeds
through C2's backbone via the AZ-321-produced engine, and rebuilds
the AZ-306 FAISS HNSW index in one atomic write.

- DescriptorBatcher with halve-and-retry OOM recovery (default 1 retry)
- BackboneEmbedder Protocol + C7EngineBackboneEmbedder default impl
- DescriptorBatchError for OOM / dim-mismatch / missing-output failures
- Empty-corpus surfaces as outcome=failure with explicit hint to run C11
- Per-10% progress callback + DEBUG logs (no engine bytes leaked)
- Consumer-side Protocol cuts (TilesByBboxBatchQuery, TilePixelOpener,
  DescriptorIndexRebuilder) so c10 stays within AZ-270 lint
- runtime_root.c10_factory adds build_descriptor_batcher + three
  C6->C10 adapters
- 16 unit tests covering AC-1..AC-10 + 2 NFRs + 4 supplemental
  (Protocol conformance, query pass-through, handle release, config)

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-13 04:20:47 +03:00
parent 3b7265757b
commit f01a5058ab
12 changed files with 1733 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_document/module-layout.md
+5 -3
View File
@@ -209,14 +209,16 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec
- **Epic**: AZ-252 (E-C10 Cache Provisioner)
- **Directory**: `src/gps_denied_onboard/components/c10_provisioning/`
- **Public API**:
- `__init__.py` (re-exports `CacheProvisioner`, `Manifest`, `EngineCacheEntry`, plus AZ-321 surface: `EngineCompiler`, `BackboneSpec`, `EngineCompileRequest`, `EngineCompileResult`, `CompileOutcome`, `EngineCompileSummary`, `CompileEngineCallable`, `BackboneConfig`, `C10ProvisioningConfig`)
- `interface.py` (`CacheProvisioner` Protocol)
- `__init__.py` (re-exports `CacheProvisioner`, `Manifest`, `EngineCacheEntry`, plus AZ-321 surface: `EngineCompiler`, `BackboneSpec`, `EngineCompileRequest`, `EngineCompileResult`, `CompileOutcome`, `EngineCompileSummary`, `CompileEngineCallable`, `BackboneConfig`, `C10ProvisioningConfig`, plus AZ-322 surface: `DescriptorBatcher`, `BackboneEmbedder`, `C7EngineBackboneEmbedder`, `C10BatcherConfig`, `CorpusFilter`, `DescriptorBatchReport`, `ProgressEvent`, `TileBboxRecord`, `BatcherTile`, `TilesByBboxBatchQuery`, `TilePixelOpener`, `DescriptorIndexRebuilder`, `DescriptorBatchError`)
- `interface.py` (`CacheProvisioner` Protocol, `BackboneEmbedder` Protocol — AZ-322)
- Config block: `C10ProvisioningConfig` (registered on import)
- **Internal**:
- `engine_compiler.py` (AZ-321; per-model TRT compile + hardware-tied cache reuse + `CompileEngineCallable` structural cut of the C7 InferenceRuntime)
- `config.py` (AZ-321; `BackboneConfig` + `C10ProvisioningConfig` dataclasses)
- `descriptor_batcher.py` (AZ-322; `DescriptorBatcher` + DTOs + consumer-side Protocols `TilesByBboxBatchQuery` / `TilePixelOpener` / `DescriptorIndexRebuilder`)
- `c7_engine_embedder.py` (AZ-322; `C7EngineBackboneEmbedder` adapter wrapping AZ-297 `InferenceRuntime` + AZ-321 engine path)
- `default_provisioner.py` (engine compile + descriptors + manifest + content-hash gate, pending)
- Composition root: `runtime_root/c10_factory.py` (`build_engine_compiler`, `build_backbone_specs`)
- Composition root: `runtime_root/c10_factory.py` (`build_engine_compiler`, `build_backbone_specs`, `build_manifest_builder`, `build_manifest_verifier`, `build_descriptor_batcher` + the C6→C10 adapters `c6_tile_metadata_store_to_tiles_batch_query`, `c6_tile_store_to_pixel_opener`, `c6_descriptor_index_to_rebuilder`)
- **Owns**: `src/gps_denied_onboard/components/c10_provisioning/**`, `tests/unit/c10_provisioning/**`
- **Imports from**: `_types` (cross-component DTOs `EngineCacheEntry`, `BuildConfig`, `PrecisionMode`, `OptimizationProfile`, `HostCapabilities`, `TileMetadata`, etc.), `_types.inference_errors` (AZ-507 typed-error envelope for `EngineBuildError` + `CalibrationCacheError`), `helpers.sha256_sidecar`, `helpers.engine_filename_schema`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`. The `InferenceRuntime.compile_engine` surface (c7) and the `TileMetadataStore.query_by_bbox` surface (c6) are obtained via constructor-injected consumer-side structural Protocol cuts (the `CompileEngineCallable` cut already lives in `engine_compiler.py`; AZ-323 / AZ-324 will define analogous `query_by_bbox` cuts inside `c10_provisioning/`). NEVER `from gps_denied_onboard.components.c6_tile_cache import ...` or `from gps_denied_onboard.components.c7_inference import ...` inside `c10_provisioning/*.py`.
- **Consumed by**: `c12_operator_tooling`, `runtime_root` (operator binary only — excluded from airborne via `BUILD_C10_PROVISIONING=OFF` for airborne build per ADR-002)