# Legacy: WPF Era of Azaion (`annotations` predecessor) > **Source of truth for this doc:** `suite/annotations-research/` — a clone of > `suite/annotations` checked out at commit `22529c2 "Revert add MediaFile"` > (Mon Nov 17 2025), which is the LAST commit before the **big refactoring** > (`e7ea5a8`) that started decoupling the WPF UI from the backend in > preparation for the WPF→.NET API conversion (`9e7dc29` / > `fbbe556 refactor .net project to API`). > > This document captures the system as it existed when the **annotation tool, > the inference engine, the loader, and the UI were a single Windows desktop > application** with .NET WPF in front and Cython sidecar processes behind it, > all running on one machine. > > The current `azaion/ui` repo is the **React rewrite of the front-end half** > of that legacy stack. The Cython parts and the .NET service code became > separate suite submodules (`detections/`, `loader/`, `annotations/` (now > .NET API only), `flights/`, etc.). This file exists so future maintainers > can understand where features in the React UI come from, why some shapes > in the data look the way they do, and what is intentionally NOT being > ported. --- ## 1. Top-level layout (`Azaion.Suite.sln`) The legacy repository was a single Visual Studio solution containing eight .NET projects plus two Cython projects: | # | Project | Type | Role | |---|----------------------|---------------------|--------------------------------------------------------------| | 1 | `Azaion.Suite` | .NET 8 WPF (exe) | Application host. DI container, config, module registry, key handler. | | 2 | `Azaion.LoaderUI` | .NET 8 WPF (exe) | Login screen. Launches `Azaion.Suite.exe` with encrypted creds. | | 3 | `Azaion.Annotator` | .NET 8 WPF (lib) | Main annotation window: video/image canvas, bounding boxes, AI detect. | | 4 | `Azaion.Dataset` | .NET 8 WPF (lib) | Dataset Explorer window: thumbnail grid, class distribution, validation. | | 5 | `Azaion.Common` | .NET 8 (lib) | **Tangled core**: WPF user controls + LinqToDB models + RabbitMQ + HTTP + DTOs all in one assembly. | | 6 | `Azaion.CommonSecurity` | .NET 8 (lib) | AES helpers. Credentials persisted to disk encrypted. | | 7 | `Azaion.Test` | .NET 8 test project | Unit tests for utilities (intervals, throttle, parallel, tile processing). | | 8 | `Dummy` | placeholder dir | empty. | | C1 | `Azaion.Inference` | Cython (Python) | YOLO inference (ONNX / TensorRT). Separate process, ZeroMQ link to .NET. | | C2 | `Azaion.Loader` | Cython (Python) | Encrypted resource fetcher + hardware fingerprinting. Separate process, ZeroMQ link to .NET. | There was no internet-facing API. Everything ran on **one Windows machine** (operator laptop / OrangePi / Jetson). ``` +----------------------------- one Windows host -----------------------------+ | | | Azaion.LoaderUI.exe | | | | | | encrypts creds -> spawns Azaion.Suite.exe -c | | v | | Azaion.Suite.exe (WPF) | | | | | | Microsoft.Extensions.Hosting + DI | | | - registers Azaion.Annotator, Azaion.Dataset windows | | | - registers Annotation/Gallery/Inference/GpsMatcher Services | | | - registers AzaionApi (HttpClient -> remote installer/aux APIs) | | | - registers LoaderClient + InferenceClient + GpsMatcherClient | | | (all are ZeroMQ DealerSocket clients) | | | | | |--- LinqToDB --------------> SQLite file (annotations.db) | | |--- ZeroMQ Dealer ---------> Azaion.Loader.exe (Cython) | | | | | | | +---> downloads encrypted resources | | | from remote API | | | | | |--- ZeroMQ Dealer ---------> Azaion.Inference.exe (Cython) | | | | | | | +---> ONNX / TensorRT inference | | | | | +--- RabbitMQ.Stream client -> remote RabbitMQ (annotation sync) | | | +----------------------------------------------------------------------------+ ``` ## 2. Boot sequence 1. User runs `Azaion.LoaderUI.exe` (the launcher / login window). 2. `Login.LoginClick` → calls `IAzaionApi.Login` (HTTP) for installer-version check, then spawns the **external** `Azaion.Loader` Cython process and talks to it over ZeroMQ (`tcp://127.0.0.1:`): - `CommandType.Login` → loader stores credentials and a hardware-derived key (`hardware_service.pyx`). - `CommandType.CheckResource` → loader verifies it can decrypt the cached encrypted resource bundle. 3. `Login` AES-encrypts `ApiCredentials` (`Azaion.CommonSecurity`) and starts `Azaion.Suite.exe -c ` then closes itself. 4. `Azaion.Suite.App.Start(creds)`: - Builds a Serilog logger. - Builds `IConfiguration` from three JSON streams: a local `config.json`, plus `config.system.json` and `config.secured.json` fetched from disk via `LoaderClient.LoadFile(...)` (the Cython loader decrypts them on the fly). - Configures the DI container (`Microsoft.Extensions.Hosting`): - `IConfigUpdater`, `Annotator`, `DatasetExplorer`, `HelpWindow`, `MainSuite` - `IDbFactory`, `IAnnotationService`, `FailsafeAnnotationsProducer`, `IGalleryService` - `IInferenceClient`/`IInferenceService` (ZMQ → Cython inference) - `IGpsMatcherClient`/`IGpsMatcherService` (ZMQ → GPS matcher service) - `ISatelliteDownloader` - `IAzaionApi` (HTTP client to remote API for installer + assets) - `IAzaionModule` registrations (`AnnotatorModule`, `DatasetExplorerModule`) - MediatR with assemblies from Annotator, DatasetExplorer, Common. - Calls `Annotation.Init(directoriesConfig, detectionClassesDict)` — populates **static** state on the `Annotation` entity so that LinqToDB hydrated rows know how to compute `ImagePath` / `LabelPath` / `ThumbPath` / `Colors` / `ClassName`. (This static coupling is exactly what the `e7ea5a8` "big refactoring" set out to remove.) - Hooks a global preview-key handler that publishes a MediatR `KeyEvent` (with throttle) for any keyboard input that is not in a `TextBox`. - Shows `MainSuite` (the module switcher window). ## 3. Module system `Azaion.Suite.MainSuite` is a small chrome window with a left-hand `ListView` of modules. Each module implements: ```csharp public interface IAzaionModule { string Name { get; } // localized display name string SvgIcon { get; } // inline SVG markup Type MainWindowType { get; } // WPF Window subclass WindowEnum WindowEnum { get; } // identifier } ``` Two implementations existed at this commit: - `AnnotatorModule` → `Azaion.Annotator.Annotator` - `DatasetExplorerModule` → `Azaion.Dataset.DatasetExplorer` `MainSuite` shows the icon, opens the corresponding `Window` from the DI container, and tracks open windows in a `Dictionary` so clicking the same module twice activates instead of recreating. The `IAzaionModule` extension point is the seed of what became, in the post-refactor world, the **left-hand top-level navigation in the React SPA**: Flights, Annotations, Dataset, Admin, Settings. ## 4. The Annotator window (`Azaion.Annotator.Annotator`) The annotation surface (`Annotator.xaml.cs`, ~600 lines) is the heaviest part of the legacy app. It owned, all in one window: - **Video/image playback**: `LibVLCSharp` `MediaPlayer` for video, image decoding for stills. - **Canvas editor**: a custom WPF `Canvas` with `CanvasEditor` from `Azaion.Common.Controls` for click-and-drag bounding boxes, 8-handle resize, multi-select with Ctrl, zoom with Ctrl+wheel, pan with Ctrl+drag, crosshair cursor with active-class hint. - **Time-windowed annotation overlay** during video playback: an `IntervalTree` keyed by `[Time - 50ms, Time + 150ms]`; on each VLC position update, all overlapping intervals render and the rest clear. - **Detection class strip** (`Azaion.Common.Controls.DetectionClasses`): data grid of class colour + number + name, with PhotoMode switcher (Regular=0, Winter=20, Night=40); class number pressed via keyboard (1–9); class colour mixed into the bounding-box label. - **Annotation list** (right sidebar): `DataGrid` over the in-process `IntervalTree`, gradient-coloured by detection class, double-click seeks the video and zooms. - **Frame-by-frame controls**: 1, 5, 10, 30, 60-frame stepping computed from the video's FPS; play/pause/stop; mute and volume. - **AI Detect** (`R` key or button): spawns the Cython inference process via `IInferenceClient` (ZMQ) and streams progress into a modal `AutodetectDialog`. - **Camera config side panel** (`Azaion.Common.Controls.CameraConfigControl`): altitude / focal length / sensor width — used to compute GSD-based bounds for valid detection sizes. - **GPS panel** (`Azaion.Annotator.Controls.MapMatcher`): toggleable below the canvas; ties into `IGpsMatcherClient` which talks to a separate GPS-denied positioning Cython process. - **Help window** (`HelpWindow.xaml` + `HelpTexts.cs`): annotation quality guidelines. - **Ukrainian / English localisation** via `translations.json`. Everything in this list is owned by the same `Annotator` partial class. There is no view-model boundary; XAML code-behind directly: - queries `IDbFactory` for `AnnotationsDb`, then runs LinqToDB queries against the `Annotation`, `Detection`, `MediaFile`, `AnnotationQueueRecord` tables; - mutates the canvas, the data grid, the VLC media player, and the GPS panel; - publishes MediatR notifications (`AnnotationCreatedEvent`, `AnnotationsDeletedEvent`, `KeyEvent`, `SetStatusTextEvent`, `AnnotatorControlEvent`, `LoadErrorEvent`) which downstream services like `AnnotationService.OnAnnotationCreated` react to (e.g. to enqueue a sync message into the local SQLite buffer table for later RabbitMQ publish). This is the central tangle. The same class talks to the database, the network (RabbitMQ via mediator), the inference process, the file system, and the WPF visual tree. ## 5. The Dataset Explorer window (`Azaion.Dataset.DatasetExplorer`) Mirrors the annotator, but for browsing: - Thumbnail grid (virtualised) keyed by `Annotation.ThumbPath`, regenerated by `IGalleryService`. - Filter bar: date range, flight, status (`AnnotationStatus`: None / Created / Edited / Validated). - Class distribution chart (`Controls/ClassDistribution.xaml`): horizontal bars, one per `DetectionClass`, coloured with the class colour. - Inline editor tab — same `CanvasEditor` from `Azaion.Common.Controls`, reused. - Bulk validation: select multiple thumbnails, press `V`, status becomes `Validated`. - Local keyboard handlers: `1–9` (class), `Enter` (save), `Del` (delete selected), `X` (delete all), `V` (validate), arrow keys + PageUp/PageDown for navigation, `Esc` to close the editor. ## 6. `Azaion.Common`: the everything-bag This is the assembly that the post-refactor split tries hardest to undo. At commit `22529c2` it was a single .NET project containing: | Folder | Concern | |---------------|------------------------------------------------------------------------------------------------------------------------| | `Controls/` | **WPF user controls**: `CanvasEditor`, `NumericUpDown`, `DetectionClasses`, `CameraConfigControl`, `DetectionLabelPanel`, `UpdatableProgressBar`, `DetectionControl`. | | `Database/` | LinqToDB models + `AnnotationsDb : DataConnection` + `DbFactory` + `SchemaMigrator` + `AnnotationsDbSchemaHolder`. | | `DTO/` | App config sections (`AppConfig`, `LoaderClientConfig`, `InferenceClientConfig`, `GpsDeniedConfig`, `MapConfig`, `QueueConfig`, `AIRecognitionConfig`, `ThumbnailConfig`, `UIConfig`, `AnnotationConfig`, `CameraConfig`, `DirectoriesConfig`), domain enums (`AffiliationEnum`, `RoleEnum`, `WindowEnum`, `Direction`, `PlaybackControlEnum`), and shared shapes (`ApiCredentials`, `BusinessExceptionDto`, `LoginResponse`, `RemoteCommand`, `User`, `DetectionClass`, `LabelInfo`, `AnnotationResult`, `AnnotationThumbnail`, `ClusterDistribution`, `Coordinates`, `SatTile`, `DownloadTilesResult`, `SelectionState`, `FormState`, `ExternalClientsConfig`). | | `Events/` | MediatR notifications (`AnnotationCreatedEvent`, `AnnotationsDeletedEvent`, `KeyEvent`, `SetStatusTextEvent`, `AnnotatorControlEvent`, `LoadErrorEvent`). | | `Exceptions/` | `BusinessException`. | | `Extensions/` | Helpers — `Geo`, `ParallelExt`, `ResilienceExt`, `ThrottleExtensions`, `IntervalTree`-related, `Bitmap`, `Color`, `Cancellation`, `Queryable`, `Graphics`, `Size`, `String`, `DirectoryInfo`, `DenseDateTimeConverter`, `EnumExtensions`, `ServiceCollectionExtensions`. | | `Services/` | `AnnotationService` (RabbitMQ.Stream + LinqToDB + MediatR), `FailsafeAnnotationsProducer`, `GalleryService` (thumbnails), `AuthProvider`, `TileProcessor`, `SatelliteDownloader`, `LoaderClient` (ZMQ), `GpsMatcher/*` (ZMQ + service + event handler + events), `Inference/InferenceClient` (ZMQ), `Inference/InferenceService` (orchestrates inference jobs), `Inference/InferenceServiceEventHandler`, `Inference/InferenceServiceEvents`, `Cache`, `HashExtensions`. | | `Constants.cs`| `CONFIG_PATH`, suffixes, file naming conventions, the `FailsafeAppConfig` builder. | | `Security.cs` | AES-256-CFB credentials encryption / decryption. Key is time-derived for local on-disk storage; symmetrical on both ends of the loader handoff. | Concrete examples of the tangle, taken straight from this commit: - `Azaion.Common.Database.Annotation` carries `[IgnoreMember] System.Windows.Media.Color` in its computed `Colors` projection. A "database model" that imports `System.Windows.Media`. The DTO assembly cannot exist outside WPF. - `Annotation.Init(DirectoriesConfig, Dictionary)` is a **static initializer** that the application calls once at startup. Hydrated entities then read the static `_labelsDir`, `_imagesDir`, `_thumbDir`, `DetectionClassesDict` to compute their own paths and class names. Two annotation databases or two configurations cannot coexist in the same process. - `AppConfig` aggregates ten config sections including `UIConfig`, but is also passed to the queue producer, the loader client, the inference client, the satellite downloader. There is no clear seam between "app-host concerns" and "business concerns". - `AnnotationService` is constructed with `IDbFactory`, `FailsafeAnnotationsProducer`, `QueueConfig`, `UIConfig`, `IGalleryService`, `IMediator`, `IAzaionApi`, `ILogger`. It runs a `RabbitMQ.Stream.Consumer` *inside its constructor* via `Task.Run(...).Wait()`, publishes MediatR events into the WPF dispatcher, and uses `_imageAccessSemaphore` and `_messageProcessingSemaphore` to serialize cross-thread SQLite writes. Lifecycle and threading model are baked in. The follow-on commits (`e7ea5a8`, `9e7dc29`, `fbbe556`) split this into proper layers: repositories with interfaces, an `AnnotationPathResolver` service replacing the static fields on `Annotation`, a separate `Azaion.Common.Database.AnnotationRepository` + `IAnnotationRepository`, removal of `AnnotationsDbSchemaHolder`, and finally the move from a WPF client to a containerised .NET API exposing REST + SSE. ## 7. Cython sidecars ### `Azaion.Inference` Standalone Python project compiled with Cython (`build_inference.cmd` → PyInstaller → `azaion-inference.spec`). Top-level modules at `22529c2`: ``` ai_availability_status ai_config annotation classes.json constants_inf file_data inference inference_engine onnx_engine loader_client main_inference remote_command_handler_inf ``` It exposes a ZeroMQ DealerSocket port. The .NET side (`InferenceClient` in `Azaion.Common.Services.Inference`) sends MessagePack-serialised `RemoteCommand` envelopes; the Cython side dispatches to either the ONNX or TensorRT engine, reads inputs from the local file system, and streams back `DetectionEvent`-shaped progress. State the engine reports (mapped 1:1 to the React UI's `AIAvailabilityStatus`): | Value | Name | Meaning | |-------|-------------|--------------------------------------------------------| | 0 | None | Initial. | | 10 | Downloading | Pulling weights from Admin API / CDN via `loader_client`. | | 20 | Converting | ONNX → TensorRT (TensorRT devices only). | | 30 | Uploading | Uploading converted engine back to API for caching. | | 200 | Enabled | Inference engine ready. | | 300 | Warning | Recoverable, the engine may come back. | | 500 | Error | Failed to initialize. | ### `Azaion.Loader` Same shape — Cython, ZeroMQ DealerSocket, separate process. Modules: ``` api_client cdn_manager constants credentials file_data hardware_service main_loader remote_command remote_command_handler security ``` It is the only component in the legacy stack with internet access. It authenticates the user against the remote API, downloads encrypted resource bundles (model checkpoints, `config.system.json`, `config.secured.json`), and decrypts them on demand using a key derived from `email + password + hardware_id` (`security.pyx` + `hardware_service.pyx`). The .NET side never sees the raw resource files until the loader has already decrypted them. This Loader is exactly the component documented in `suite/_docs/00_top_level_architecture.md` under **Binary Split Security**. The 3 KB key fragment, the encrypted on-device archive, and the `SHA384(fragment + hw_hash + creds)` derivation all originate here. ## 8. Data model (LinqToDB → SQLite) | Table | Purpose | |-------------------------------|------------------------------------------------------------------------------------------| | `Annotations` | Per-frame label set: `Name`, `MediaHash`, `OriginalMediaName`, `Time`, `CreatedDate`, `CreatedEmail`, `CreatedRole`, `Source`, `AnnotationStatus`, `ValidateDate`, `ValidateEmail`, `Detections[]`, `Milliseconds`, `Lat`, `Lon`. | | `Detections` | Bounding box rows: `ClassNumber`, geometry, `Confidence`. | | `MediaFiles` | Files indexed by hash. Used to dedupe + drive the media list. | | `AnnotationQueueRecord` (`AnnotationsQueueRecords` table) | Local **failsafe outbox** for RabbitMQ publication. `FailsafeAnnotationsProducer` drains this every 10 s. | Schema is created/migrated in process by `SchemaMigrator` against a SQLite file pointed to by `DirectoriesConfig`. ## 9. Annotation sync (edge → central) The legacy code already had the eventual edge-to-central sync wired in: ``` Annotator window // user creates annotation │ MediatR: AnnotationCreatedEvent ▼ AnnotationService // local SQLite write │ ├─► Annotations row ├─► AnnotationQueueRecord row (unless SilentDetection) │ ▼ FailsafeAnnotationsProducer // BackgroundService-style task │ MessagePack + Gzip, retry on failure ▼ RabbitMQ.Stream "azaion-annotations" // remote │ └─► consumed by ai-queue-handler / Admin API in the remote tier ``` The React UI inherits the *protocol* (RabbitMQ stream, MessagePack + Gzip, dedupe by `Annotation.Name`) but no longer owns it — it runs in the new `annotations/` .NET API submodule of the suite. ## 10. What survived into the new world The following concepts are direct ports of the legacy WPF design and should be implemented in the React UI exactly the same way: - **Module switcher** with localized name + SVG icon → top navigation bar (Flights, Annotations, Dataset, Admin, Settings). - **Detection-class strip** with class colour, number, name, and PhotoMode switcher (Regular / Winter / Night, offsets 0/20/40). `yoloId = classId + photoModeOffset`. - **Canvas editor**: bounding-box draw / 8-handle resize / Ctrl multi-select / Ctrl+wheel zoom / Ctrl+drag pan / crosshair with active-class hint / normalized-coordinate clamping. - **Annotation row gradient** in the side list: a left-to-right gradient composed of each detection's class colour, opacity proportional to `Confidence`. Empty annotation → `#40DDDDDD` background. - **Affiliation icons** (Friendly / Hostile / Unknown / None) and **combat readiness** indicator (Ready / NotReady / Unknown) drawn next to the bounding-box label. - **Time-windowed annotation rendering** during video playback: `Before = 50 ms`, `After = 150 ms`, lookup via interval tree. - **Frame-by-frame stepping** in fixed counts (1, 5, 10, 30, 60), computed from `1 / fps`. - **Localized class names** (`DetectionClass.UIName` carried alongside the English `Name`). - **Camera config per session**: altitude / focal length / sensor width drives GSD-based detection-size validation. - **GPS-denied panel toggle** under the canvas (now implemented as the GPS-Denied mode of the Flights page in the React UI). - **Help window** with the six annotation quality rules. - **Color scheme**: dark navy/blue primary (`#343a40`), orange accents (`#fd7e14`), dark gray background (`#1e1e1e`), green success (`#40c057`), blue primary buttons (`#228be6`), red danger (`#fa5252`). - **Confirmation dialogs** for delete-media / delete-selected / delete-all / deactivate-user. - **Resizable panel widths** persisted per user. ## 11. What is intentionally NOT being ported - The DI host inside the UI process. The React app does not own a service container, RabbitMQ consumer, SQLite database, or background worker. All of that now lives in the per-service .NET / Python / Cython submodules. - LibVLCSharp. The browser's native `