ui/_docs/legacy/wpf-era.md

Legacy: WPF Era of Azaion (annotations predecessor)

Source of truth for this doc: suite/annotations-research/ — a clone of suite/annotations checked out at commit 22529c2 "Revert add MediaFile" (Mon Nov 17 2025), which is the LAST commit before the big refactoring (e7ea5a8) that started decoupling the WPF UI from the backend in preparation for the WPF→.NET API conversion (9e7dc29 / fbbe556 refactor .net project to API).

This document captures the system as it existed when the annotation tool, the inference engine, the loader, and the UI were a single Windows desktop application with .NET WPF in front and Cython sidecar processes behind it, all running on one machine.

The current azaion/ui repo is the React rewrite of the front-end half of that legacy stack. The Cython parts and the .NET service code became separate suite submodules (detections/, loader/, annotations/ (now .NET API only), flights/, etc.). This file exists so future maintainers can understand where features in the React UI come from, why some shapes in the data look the way they do, and what is intentionally NOT being ported.

---

1. Top-level layout (Azaion.Suite.sln)

The legacy repository was a single Visual Studio solution containing eight .NET projects plus two Cython projects:

#	Project	Type	Role
1	Azaion.Suite	.NET 8 WPF (exe)	Application host. DI container, config, module registry, key handler.
2	Azaion.LoaderUI	.NET 8 WPF (exe)	Login screen. Launches Azaion.Suite.exe with encrypted creds.
3	Azaion.Annotator	.NET 8 WPF (lib)	Main annotation window: video/image canvas, bounding boxes, AI detect.
4	Azaion.Dataset	.NET 8 WPF (lib)	Dataset Explorer window: thumbnail grid, class distribution, validation.
5	Azaion.Common	.NET 8 (lib)	Tangled core: WPF user controls + LinqToDB models + RabbitMQ + HTTP + DTOs all in one assembly.
6	Azaion.CommonSecurity	.NET 8 (lib)	AES helpers. Credentials persisted to disk encrypted.
7	Azaion.Test	.NET 8 test project	Unit tests for utilities (intervals, throttle, parallel, tile processing).
8	Dummy	placeholder dir	empty.
C1	Azaion.Inference	Cython (Python)	YOLO inference (ONNX / TensorRT). Separate process, ZeroMQ link to .NET.
C2	Azaion.Loader	Cython (Python)	Encrypted resource fetcher + hardware fingerprinting. Separate process, ZeroMQ link to .NET.

There was no internet-facing API. Everything ran on one Windows machine (operator laptop / OrangePi / Jetson).

+----------------------------- one Windows host -----------------------------+
|                                                                            |
|  Azaion.LoaderUI.exe                                                       |
|     |                                                                      |
|     |  encrypts creds  ->  spawns Azaion.Suite.exe -c <encrypted>          |
|     v                                                                      |
|  Azaion.Suite.exe (WPF)                                                    |
|     |                                                                      |
|     |  Microsoft.Extensions.Hosting + DI                                   |
|     |     - registers Azaion.Annotator, Azaion.Dataset windows             |
|     |     - registers Annotation/Gallery/Inference/GpsMatcher Services     |
|     |     - registers AzaionApi (HttpClient -> remote installer/aux APIs)  |
|     |     - registers LoaderClient + InferenceClient + GpsMatcherClient    |
|     |       (all are ZeroMQ DealerSocket clients)                          |
|     |                                                                      |
|     |--- LinqToDB --------------> SQLite file (annotations.db)             |
|     |--- ZeroMQ Dealer ---------> Azaion.Loader.exe (Cython)               |
|     |                                |                                     |
|     |                                +---> downloads encrypted resources   |
|     |                                      from remote API                 |
|     |                                                                      |
|     |--- ZeroMQ Dealer ---------> Azaion.Inference.exe (Cython)            |
|     |                                |                                     |
|     |                                +---> ONNX / TensorRT inference       |
|     |                                                                      |
|     +--- RabbitMQ.Stream client -> remote RabbitMQ (annotation sync)       |
|                                                                            |
+----------------------------------------------------------------------------+

2. Boot sequence

1. User runs Azaion.LoaderUI.exe (the launcher / login window).
2. Login.LoginClick → calls IAzaionApi.Login (HTTP) for installer-version check, then spawns the external Azaion.Loader Cython process and talks to it over ZeroMQ (tcp://127.0.0.1:<port>):
   • CommandType.Login → loader stores credentials and a hardware-derived key (hardware_service.pyx).
   • CommandType.CheckResource → loader verifies it can decrypt the cached encrypted resource bundle.
3. Login AES-encrypts ApiCredentials (Azaion.CommonSecurity) and starts Azaion.Suite.exe -c <encrypted> then closes itself.
4. Azaion.Suite.App.Start(creds):
   • Builds a Serilog logger.
   • Builds IConfiguration from three JSON streams: a local config.json, plus config.system.json and config.secured.json fetched from disk via LoaderClient.LoadFile(...) (the Cython loader decrypts them on the fly).
   • Configures the DI container (Microsoft.Extensions.Hosting):
     • IConfigUpdater, Annotator, DatasetExplorer, HelpWindow, MainSuite
     • IDbFactory, IAnnotationService, FailsafeAnnotationsProducer, IGalleryService
     • IInferenceClient/IInferenceService (ZMQ → Cython inference)
     • IGpsMatcherClient/IGpsMatcherService (ZMQ → GPS matcher service)
     • ISatelliteDownloader
     • IAzaionApi (HTTP client to remote API for installer + assets)
     • IAzaionModule registrations (AnnotatorModule, DatasetExplorerModule)
     • MediatR with assemblies from Annotator, DatasetExplorer, Common.
   • Calls Annotation.Init(directoriesConfig, detectionClassesDict) — populates static state on the Annotation entity so that LinqToDB hydrated rows know how to compute ImagePath / LabelPath / ThumbPath / Colors / ClassName. (This static coupling is exactly what the e7ea5a8 "big refactoring" set out to remove.)
   • Hooks a global preview-key handler that publishes a MediatR KeyEvent (with throttle) for any keyboard input that is not in a TextBox.
   • Shows MainSuite (the module switcher window).

3. Module system

Azaion.Suite.MainSuite is a small chrome window with a left-hand ListView of modules. Each module implements:

public interface IAzaionModule
{
    string Name { get; }              // localized display name
    string SvgIcon { get; }           // inline SVG markup
    Type MainWindowType { get; }      // WPF Window subclass
    WindowEnum WindowEnum { get; }    // identifier
}

Two implementations existed at this commit:

• AnnotatorModule → Azaion.Annotator.Annotator
• DatasetExplorerModule → Azaion.Dataset.DatasetExplorer

MainSuite shows the icon, opens the corresponding Window from the DI container, and tracks open windows in a Dictionary<WindowEnum, Window> so clicking the same module twice activates instead of recreating.

The IAzaionModule extension point is the seed of what became, in the post-refactor world, the left-hand top-level navigation in the React SPA: Flights, Annotations, Dataset, Admin, Settings.

4. The Annotator window (Azaion.Annotator.Annotator)

The annotation surface (Annotator.xaml.cs, ~600 lines) is the heaviest part of the legacy app. It owned, all in one window:

• Video/image playback: LibVLCSharp MediaPlayer for video, image decoding for stills.
• Canvas editor: a custom WPF Canvas with CanvasEditor from Azaion.Common.Controls for click-and-drag bounding boxes, 8-handle resize, multi-select with Ctrl, zoom with Ctrl+wheel, pan with Ctrl+drag, crosshair cursor with active-class hint.
• Time-windowed annotation overlay during video playback: an IntervalTree<TimeSpan, Annotation> keyed by [Time - 50ms, Time + 150ms]; on each VLC position update, all overlapping intervals render and the rest clear.
• Detection class strip (Azaion.Common.Controls.DetectionClasses): data grid of class colour + number + name, with PhotoMode switcher (Regular=0, Winter=20, Night=40); class number pressed via keyboard (1–9); class colour mixed into the bounding-box label.
• Annotation list (right sidebar): DataGrid over the in-process IntervalTree, gradient-coloured by detection class, double-click seeks the video and zooms.
• Frame-by-frame controls: 1, 5, 10, 30, 60-frame stepping computed from the video's FPS; play/pause/stop; mute and volume.
• AI Detect (R key or button): spawns the Cython inference process via IInferenceClient (ZMQ) and streams progress into a modal AutodetectDialog.
• Camera config side panel (Azaion.Common.Controls.CameraConfigControl): altitude / focal length / sensor width — used to compute GSD-based bounds for valid detection sizes.
• GPS panel (Azaion.Annotator.Controls.MapMatcher): toggleable below the canvas; ties into IGpsMatcherClient which talks to a separate GPS-denied positioning Cython process.
• Help window (HelpWindow.xaml + HelpTexts.cs): annotation quality guidelines.
• Ukrainian / English localisation via translations.json.

Everything in this list is owned by the same Annotator partial class. There is no view-model boundary; XAML code-behind directly:

• queries IDbFactory for AnnotationsDb, then runs LinqToDB queries against the Annotation, Detection, MediaFile, AnnotationQueueRecord tables;
• mutates the canvas, the data grid, the VLC media player, and the GPS panel;
• publishes MediatR notifications (AnnotationCreatedEvent, AnnotationsDeletedEvent, KeyEvent, SetStatusTextEvent, AnnotatorControlEvent, LoadErrorEvent) which downstream services like AnnotationService.OnAnnotationCreated react to (e.g. to enqueue a sync message into the local SQLite buffer table for later RabbitMQ publish).

This is the central tangle. The same class talks to the database, the network (RabbitMQ via mediator), the inference process, the file system, and the WPF visual tree.

5. The Dataset Explorer window (Azaion.Dataset.DatasetExplorer)

Mirrors the annotator, but for browsing:

• Thumbnail grid (virtualised) keyed by Annotation.ThumbPath, regenerated by IGalleryService.
• Filter bar: date range, flight, status (AnnotationStatus: None / Created / Edited / Validated).
• Class distribution chart (Controls/ClassDistribution.xaml): horizontal bars, one per DetectionClass, coloured with the class colour.
• Inline editor tab — same CanvasEditor from Azaion.Common.Controls, reused.
• Bulk validation: select multiple thumbnails, press V, status becomes Validated.
• Local keyboard handlers: 1–9 (class), Enter (save), Del (delete selected), X (delete all), V (validate), arrow keys + PageUp/PageDown for navigation, Esc to close the editor.

6. Azaion.Common: the everything-bag

This is the assembly that the post-refactor split tries hardest to undo. At commit 22529c2 it was a single .NET project containing:

Folder	Concern
Controls/	WPF user controls: CanvasEditor, NumericUpDown, DetectionClasses, CameraConfigControl, DetectionLabelPanel, UpdatableProgressBar, DetectionControl.
Database/	LinqToDB models + AnnotationsDb : DataConnection + DbFactory + SchemaMigrator + AnnotationsDbSchemaHolder.
DTO/	App config sections (AppConfig, LoaderClientConfig, InferenceClientConfig, GpsDeniedConfig, MapConfig, QueueConfig, AIRecognitionConfig, ThumbnailConfig, UIConfig, AnnotationConfig, CameraConfig, DirectoriesConfig), domain enums (AffiliationEnum, RoleEnum, WindowEnum, Direction, PlaybackControlEnum), and shared shapes (ApiCredentials, BusinessExceptionDto, LoginResponse, RemoteCommand, User, DetectionClass, LabelInfo, AnnotationResult, AnnotationThumbnail, ClusterDistribution, Coordinates, SatTile, DownloadTilesResult, SelectionState, FormState, ExternalClientsConfig).
Events/	MediatR notifications (AnnotationCreatedEvent, AnnotationsDeletedEvent, KeyEvent, SetStatusTextEvent, AnnotatorControlEvent, LoadErrorEvent).
Exceptions/	BusinessException.
Extensions/	Helpers — Geo, ParallelExt, ResilienceExt, ThrottleExtensions, IntervalTree-related, Bitmap, Color, Cancellation, Queryable, Graphics, Size, String, DirectoryInfo, DenseDateTimeConverter, EnumExtensions, ServiceCollectionExtensions.
Services/	AnnotationService (RabbitMQ.Stream + LinqToDB + MediatR), FailsafeAnnotationsProducer, GalleryService (thumbnails), AuthProvider, TileProcessor, SatelliteDownloader, LoaderClient (ZMQ), GpsMatcher/* (ZMQ + service + event handler + events), Inference/InferenceClient (ZMQ), Inference/InferenceService (orchestrates inference jobs), Inference/InferenceServiceEventHandler, Inference/InferenceServiceEvents, Cache, HashExtensions.
Constants.cs	CONFIG_PATH, suffixes, file naming conventions, the FailsafeAppConfig builder.
Security.cs	AES-256-CFB credentials encryption / decryption. Key is time-derived for local on-disk storage; symmetrical on both ends of the loader handoff.

Concrete examples of the tangle, taken straight from this commit:

• Azaion.Common.Database.Annotation carries [IgnoreMember] System.Windows.Media.Color in its computed Colors projection. A "database model" that imports System.Windows.Media. The DTO assembly cannot exist outside WPF.
• Annotation.Init(DirectoriesConfig, Dictionary<int, DetectionClass>) is a static initializer that the application calls once at startup. Hydrated entities then read the static _labelsDir, _imagesDir, _thumbDir, DetectionClassesDict to compute their own paths and class names. Two annotation databases or two configurations cannot coexist in the same process.
• AppConfig aggregates ten config sections including UIConfig, but is also passed to the queue producer, the loader client, the inference client, the satellite downloader. There is no clear seam between "app-host concerns" and "business concerns".
• AnnotationService is constructed with IDbFactory, FailsafeAnnotationsProducer, QueueConfig, UIConfig, IGalleryService, IMediator, IAzaionApi, ILogger. It runs a RabbitMQ.Stream.Consumer inside its constructor via Task.Run(...).Wait(), publishes MediatR events into the WPF dispatcher, and uses _imageAccessSemaphore and _messageProcessingSemaphore to serialize cross-thread SQLite writes. Lifecycle and threading model are baked in.

The follow-on commits (e7ea5a8, 9e7dc29, fbbe556) split this into proper layers: repositories with interfaces, an AnnotationPathResolver service replacing the static fields on Annotation, a separate Azaion.Common.Database.AnnotationRepository + IAnnotationRepository, removal of AnnotationsDbSchemaHolder, and finally the move from a WPF client to a containerised .NET API exposing REST + SSE.

7. Cython sidecars

Azaion.Inference

Standalone Python project compiled with Cython (build_inference.cmd → PyInstaller → azaion-inference.spec). Top-level modules at 22529c2:

ai_availability_status   ai_config           annotation
classes.json             constants_inf       file_data
inference                inference_engine    onnx_engine
loader_client            main_inference      remote_command_handler_inf

It exposes a ZeroMQ DealerSocket port. The .NET side (InferenceClient in Azaion.Common.Services.Inference) sends MessagePack-serialised RemoteCommand envelopes; the Cython side dispatches to either the ONNX or TensorRT engine, reads inputs from the local file system, and streams back DetectionEvent-shaped progress.

State the engine reports (mapped 1:1 to the React UI's AIAvailabilityStatus):

Value	Name	Meaning
0	None	Initial.
10	Downloading	Pulling weights from Admin API / CDN via loader_client.
20	Converting	ONNX → TensorRT (TensorRT devices only).
30	Uploading	Uploading converted engine back to API for caching.
200	Enabled	Inference engine ready.
300	Warning	Recoverable, the engine may come back.
500	Error	Failed to initialize.

Azaion.Loader

Same shape — Cython, ZeroMQ DealerSocket, separate process. Modules:

api_client            cdn_manager        constants
credentials           file_data          hardware_service
main_loader           remote_command     remote_command_handler
security

It is the only component in the legacy stack with internet access. It authenticates the user against the remote API, downloads encrypted resource bundles (model checkpoints, config.system.json, config.secured.json), and decrypts them on demand using a key derived from email + password + hardware_id (security.pyx + hardware_service.pyx). The .NET side never sees the raw resource files until the loader has already decrypted them.

This Loader is exactly the component documented in suite/_docs/00_top_level_architecture.md under Binary Split Security. The 3 KB key fragment, the encrypted on-device archive, and the SHA384(fragment + hw_hash + creds) derivation all originate here.

8. Data model (LinqToDB → SQLite)

Table	Purpose
Annotations	Per-frame label set: Name, MediaHash, OriginalMediaName, Time, CreatedDate, CreatedEmail, CreatedRole, Source, AnnotationStatus, ValidateDate, ValidateEmail, Detections[], Milliseconds, Lat, Lon.
Detections	Bounding box rows: ClassNumber, geometry, Confidence.
MediaFiles	Files indexed by hash. Used to dedupe + drive the media list.
AnnotationQueueRecord (AnnotationsQueueRecords table)	Local failsafe outbox for RabbitMQ publication. FailsafeAnnotationsProducer drains this every 10 s.

Schema is created/migrated in process by SchemaMigrator against a SQLite file pointed to by DirectoriesConfig.

9. Annotation sync (edge → central)

The legacy code already had the eventual edge-to-central sync wired in:

Annotator window          // user creates annotation
   │ MediatR: AnnotationCreatedEvent
   ▼
AnnotationService         // local SQLite write
   │
   ├─► Annotations row
   ├─► AnnotationQueueRecord row (unless SilentDetection)
   │
   ▼
FailsafeAnnotationsProducer // BackgroundService-style task
   │ MessagePack + Gzip, retry on failure
   ▼
RabbitMQ.Stream "azaion-annotations"  // remote
   │
   └─► consumed by ai-queue-handler / Admin API in the remote tier

The React UI inherits the protocol (RabbitMQ stream, MessagePack + Gzip, dedupe by Annotation.Name) but no longer owns it — it runs in the new annotations/ .NET API submodule of the suite.

10. What survived into the new world

The following concepts are direct ports of the legacy WPF design and should be implemented in the React UI exactly the same way:

• Module switcher with localized name + SVG icon → top navigation bar (Flights, Annotations, Dataset, Admin, Settings).
• Detection-class strip with class colour, number, name, and PhotoMode switcher (Regular / Winter / Night, offsets 0/20/40). yoloId = classId + photoModeOffset.
• Canvas editor: bounding-box draw / 8-handle resize / Ctrl multi-select / Ctrl+wheel zoom / Ctrl+drag pan / crosshair with active-class hint / normalized-coordinate clamping.
• Annotation row gradient in the side list: a left-to-right gradient composed of each detection's class colour, opacity proportional to Confidence. Empty annotation → #40DDDDDD background.
• Affiliation icons (Friendly / Hostile / Unknown / None) and combat readiness indicator (Ready / NotReady / Unknown) drawn next to the bounding-box label.
• Time-windowed annotation rendering during video playback: Before = 50 ms, After = 150 ms, lookup via interval tree.
• Frame-by-frame stepping in fixed counts (1, 5, 10, 30, 60), computed from 1 / fps.
• Localized class names (DetectionClass.UIName carried alongside the English Name).
• Camera config per session: altitude / focal length / sensor width drives GSD-based detection-size validation.
• GPS-denied panel toggle under the canvas (now implemented as the GPS-Denied mode of the Flights page in the React UI).
• Help window with the six annotation quality rules.
• Color scheme: dark navy/blue primary (#343a40), orange accents (#fd7e14), dark gray background (#1e1e1e), green success (#40c057), blue primary buttons (#228be6), red danger (#fa5252).
• Confirmation dialogs for delete-media / delete-selected / delete-all / deactivate-user.
• Resizable panel widths persisted per user.

11. What is intentionally NOT being ported

• The DI host inside the UI process. The React app does not own a service container, RabbitMQ consumer, SQLite database, or background worker. All of that now lives in the per-service .NET / Python / Cython submodules.
• LibVLCSharp. The browser's native <video> element with a frame-accurate seeking shim handles playback.
• ZeroMQ DealerSockets. The browser only speaks HTTP and SSE. Inference, GPS-matching, satellite tile fetching, and loader requests are all exposed as REST endpoints by their respective suite services.
• The static Annotation.Init(...) initializer. Path/colour computation becomes selector logic over the API DTOs, with no static state.
• The Azaion.Common god-assembly. Each concern is now a separate suite submodule with its own repo, Dockerfile, and OpenAPI document.
• The Azaion.LoaderUI external-process handoff with encrypted creds on the command line. The browser performs POST /auth/login against the Admin API and stores a JWT.
• The Cython Loader and the binary-split key-fragment dance. That whole protocol is server-side now (loader/ submodule) and the React UI is not involved beyond showing a progress screen.

12. How to read the research copy

cd /Users/obezdienie001/dev/azaion/suite/annotations-research
git status                  # detached at 22529c2 "Revert add MediaFile"
git log --oneline -n 5      # see surrounding commits

The folder is a plain clone of suite/annotations and is not wired into the suite's .gitmodules, so the parent repository ignores it.

If you want to compare the WPF-era code to the immediately following "big refactoring" commit, the comparison is:

git log --oneline --reverse 22529c2..e7ea5a8     # there is only e7ea5a8 itself
git diff 22529c2 e7ea5a8 -- Azaion.Annotator     # what the cleanup changed
git diff 22529c2 e7ea5a8 -- Azaion.Common        # the big assembly split prep

Two commits later (fbbe556 / 9e7dc29) the WPF projects disappear entirely and are replaced by a containerised .NET API — that is the state currently checked out in suite/annotations.