azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-04-23 02:06:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

initial structure implemented

Browse Source 
docs -> _docs
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-12-01 14:20:56 +02:00
parent 9134c5db06
commit abc26d5c20
360 changed files with 3881 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_components/14_result_manager/14.01_feature_frame_result_persistence.md
+46
View File
@@ -0,0 +1,46 @@
# Feature: Frame Result Persistence
## Description
Handles atomic persistence and real-time publishing of individual frame processing results. Ensures consistency between `frame_results` and `waypoints` tables through transactional updates, and triggers SSE events for live client updates.
## Component APIs Implemented
### `update_frame_result(flight_id: str, frame_id: int, result: FrameResult) -> bool`
Persists frame result atomically (frame_results + waypoints tables) and publishes via SSE.
### `publish_waypoint_update(flight_id: str, frame_id: int) -> bool`
Internal method to trigger waypoint visualization update after persistence.
## External Services Used
- **F03 Flight Database**: `execute_transaction()` for atomic multi-table updates
- **F15 SSE Event Streamer**: `send_frame_result()` for real-time client notification
## Internal Methods
| Method | Purpose |
|--------|---------|
| `_build_frame_transaction(flight_id, frame_id, result)` | Constructs DB transaction with frame_results INSERT/UPDATE and waypoints UPDATE |
| `_update_flight_statistics(flight_id)` | Updates flight-level statistics after frame persistence |
## Unit Tests
| Test | Description |
|------|-------------|
| `test_update_frame_result_new_frame` | New frame result stored in frame_results table |
| `test_update_frame_result_updates_waypoint` | Waypoint table updated with latest position |
| `test_update_frame_result_transaction_atomic` | Both tables updated or neither (rollback on failure) |
| `test_update_frame_result_triggers_sse` | F15.send_frame_result() called on success |
| `test_update_frame_result_refined_flag` | Existing result updated with refined=True |
| `test_publish_waypoint_fetches_latest` | Fetches current data before publishing |
| `test_publish_waypoint_handles_transient_error` | Retries on transient F15 errors |
| `test_publish_waypoint_logs_on_db_unavailable` | Logs error and continues on DB failure |
## Integration Tests
| Test | Description |
|------|-------------|
| `test_per_frame_processing_e2e` | Process frame → stored in F03 → SSE event sent via F15 |
| `test_statistics_updated_after_frame` | Flight statistics reflect new frame count and confidence |
_docs/02_components/14_result_manager/14.02_feature_result_retrieval.md
+44
View File
@@ -0,0 +1,44 @@
# Feature: Result Retrieval
## Description
Provides read access to frame results for REST API endpoints and SSE reconnection replay. Supports full flight result retrieval with statistics and incremental change detection for efficient reconnection.
## Component APIs Implemented
### `get_flight_results(flight_id: str) -> FlightResults`
Retrieves all frame results and computed statistics for a flight.
### `get_changed_frames(flight_id: str, since: datetime) -> List[int]`
Returns frame IDs modified since a given timestamp for incremental updates.
## External Services Used
- **F03 Flight Database**: Query frame_results and waypoints tables
## Internal Methods
| Method | Purpose |
|--------|---------|
| `_compute_flight_statistics(frames)` | Calculates total_frames, processed_frames, refined_frames, mean_confidence |
| `_query_frames_by_timestamp(flight_id, since)` | Queries frame_results with updated_at > since filter |
## Unit Tests
| Test | Description |
|------|-------------|
| `test_get_flight_results_returns_all_frames` | All frames for flight_id returned |
| `test_get_flight_results_includes_statistics` | FlightStatistics computed correctly |
| `test_get_flight_results_empty_flight` | Returns empty frames list with zero statistics |
| `test_get_flight_results_performance` | < 200ms for 2000 frames |
| `test_get_changed_frames_returns_modified` | Only frames with updated_at > since returned |
| `test_get_changed_frames_empty_result` | Returns empty list when no changes |
| `test_get_changed_frames_includes_refined` | Refined frames included in changed set |
## Integration Tests
| Test | Description |
|------|-------------|
| `test_incremental_update_after_disconnect` | Client reconnects → get_changed_frames returns frames since disconnect |
| `test_get_results_matches_stored_data` | Retrieved results match what was persisted via update_frame_result |
_docs/02_components/14_result_manager/14.03_feature_batch_refinement_updates.md
+48
View File
@@ -0,0 +1,48 @@
# Feature: Batch Refinement Updates
## Description
Handles batch updates for frames that have been retrospectively improved through factor graph optimization or chunk merging. Receives GPS-converted coordinates from F02.2 (which handles ENU→GPS conversion) and updates both persistence and SSE clients.
## Component APIs Implemented
### `mark_refined(flight_id: str, refined_results: List[RefinedFrameResult]) -> bool`
Updates results for frames refined by factor graph optimization (loop closure, etc.).
### `update_results_after_chunk_merge(flight_id: str, refined_results: List[RefinedFrameResult]) -> bool`
Updates results for frames affected by chunk merge into main trajectory.
## External Services Used
- **F03 Flight Database**: Batch update frame_results and waypoints within transaction
- **F15 SSE Event Streamer**: `send_refinement()` for each updated frame
## Internal Methods
| Method | Purpose |
|--------|---------|
| `_build_batch_refinement_transaction(flight_id, refined_results)` | Constructs batch transaction for multiple frame updates |
| `_publish_refinement_events(flight_id, frame_ids)` | Sends SSE refinement events for all updated frames |
## Unit Tests
| Test | Description |
|------|-------------|
| `test_mark_refined_updates_all_frames` | All frames in refined_results updated |
| `test_mark_refined_sets_refined_flag` | refined=True set on all updated frames |
| `test_mark_refined_updates_gps_coordinates` | GPS coordinates match RefinedFrameResult values |
| `test_mark_refined_triggers_sse_per_frame` | F15.send_refinement() called for each frame |
| `test_mark_refined_updates_waypoints` | Waypoint table updated with new positions |
| `test_chunk_merge_updates_all_frames` | All merged frames updated |
| `test_chunk_merge_sets_refined_flag` | refined=True for merged frames |
| `test_chunk_merge_triggers_sse` | SSE events sent for merged frames |
| `test_batch_transaction_atomic` | All updates succeed or all rollback |
## Integration Tests
| Test | Description |
|------|-------------|
| `test_batch_refinement_e2e` | 100 frames processed → 40 refined → all updated in F03 → SSE events sent |
| `test_chunk_merge_e2e` | Chunk merged → affected frames updated → clients notified |
| `test_refined_frames_in_get_results` | get_flight_results returns refined=True for updated frames |
_docs/02_components/14_result_manager/14._component_result_manager.md
+342
View File
@@ -0,0 +1,342 @@
# Result Manager
## Interface Definition
**Interface Name**: `IResultManager`
### Interface Methods
```python
class IResultManager(ABC):
@abstractmethod
def update_frame_result(self, flight_id: str, frame_id: int, result: FrameResult) -> bool:
"""
Atomic update:
1. Saves result to frame_results table.
2. Updates waypoint in waypoints table.
3. All within a single transaction via F03.
"""
pass
@abstractmethod
def publish_waypoint_update(self, flight_id: str, frame_id: int) -> bool:
pass
@abstractmethod
def get_flight_results(self, flight_id: str) -> FlightResults:
pass
@abstractmethod
def mark_refined(self, flight_id: str, refined_results: List[RefinedFrameResult]) -> bool:
"""
Updates results for frames that have been retrospectively improved.
Args:
flight_id: Flight identifier
refined_results: List of RefinedFrameResult containing frame_id and GPS-converted coordinates
(caller F02.2 converts ENU to GPS before calling this method)
"""
pass
@abstractmethod
def get_changed_frames(self, flight_id: str, since: datetime) -> List[int]:
pass
@abstractmethod
def update_results_after_chunk_merge(self, flight_id: str, refined_results: List[RefinedFrameResult]) -> bool:
"""
Updates results for frames affected by chunk merge.
Args:
flight_id: Flight identifier
refined_results: List of RefinedFrameResult with GPS-converted coordinates
(caller F02.2 converts ENU to GPS before calling this method)
"""
pass
```
## Component Description
### Responsibilities
- Result consistency and publishing.
- **Atomic Updates**: Ensures consistency between normalized `waypoints` and denormalized `frame_results` via transaction requests to F03.
### Scope
- Result state management
- Flight Database integration (waypoint storage)
- SSE event triggering
- Incremental update detection
- Result persistence
## API Methods
### `update_frame_result(flight_id: str, frame_id: int, result: FrameResult) -> bool`
**Description**: Persists and publishes the result of a processed frame.
**Called By**:
- Main processing loop (after each frame)
- F10 Factor Graph (after refinement)
**Input**:
```python
flight_id: str
frame_id: int
result: FrameResult:
gps_center: GPSPoint
altitude: float
heading: float
confidence: float
timestamp: datetime
refined: bool
objects: List[ObjectLocation] # From external detector
```
**Output**: `bool` - True if updated
**Processing Flow**:
1. Construct DB transaction:
- Insert/Update `frame_results`.
- Update `waypoints` (latest position).
2. Call `F03.execute_transaction()`.
3. If success: call `F15.send_frame_result()`.
4. Update flight statistics.
**Test Cases**:
1. New frame result → stored and published
2. Refined result → updates existing, marks refined=True
---
### `publish_waypoint_update(flight_id: str, frame_id: int) -> bool`
**Description**: Specifically triggers an update for the waypoint visualization.
**Called By**:
- Internal (after update_frame_result)
**Input**:
```python
flight_id: str
frame_id: int
```
**Output**: `bool` - True if updated successfully
**Processing Flow**:
1. Fetch latest data.
2. Call `F15.send_frame_result()` (or a specific lightweight event).
3. Handle errors (retry if transient)
**Test Cases**:
1. Successful update → Waypoint stored in database
2. Database unavailable → logs error, continues
---
### `get_flight_results(flight_id: str) -> FlightResults`
**Description**: Retrieves all results for a flight.
**Called By**:
- F01 REST API (results endpoint)
- Testing/validation
**Input**: `flight_id: str`
**Output**:
```python
FlightResults:
flight_id: str
frames: List[FrameResult]
statistics: FlightStatistics
```
**Test Cases**:
1. Get all results → returns complete trajectory
---
### `mark_refined(flight_id: str, refined_results: List[RefinedFrameResult]) -> bool`
**Description**: Updates results for frames that have been retrospectively improved (e.g., after loop closure or chunk merge).
**Called By**:
- F02.2 Flight Processing Engine (after factor graph optimization)
**Input**:
```python
flight_id: str
refined_results: List[RefinedFrameResult] # GPS-converted results from F02.2
```
**Output**: `bool`
**Important**: F14 does NOT call F10 or F13. The caller (F02.2) performs the following steps before calling mark_refined():
1. F02.2 gets refined poses from F10.get_trajectory(flight_id) (ENU coordinates)
2. F02.2 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
3. F02.2 constructs RefinedFrameResult objects with GPS coordinates
4. F02.2 calls F14.mark_refined() with the GPS-converted results
**Processing Flow**:
1. For each refined_result in refined_results:
- Extract frame_id, gps_center, confidence from RefinedFrameResult
- Update result with refined=True via F03 Flight Database (part of transaction)
- Update waypoint via F03 Flight Database (part of transaction)
- Call F15 SSE Event Streamer.send_refinement()
**Test Cases**:
1. Batch refinement → all frames updated and published
2. GPS coordinates match converted values
---
### `get_changed_frames(flight_id: str, since: datetime) -> List[int]`
**Description**: Gets frames changed since timestamp (for incremental updates).
**Called By**:
- F15 SSE Event Streamer (for reconnection replay)
**Input**:
```python
flight_id: str
since: datetime
```
**Output**: `List[int]` - Frame IDs changed since timestamp
**Test Cases**:
1. Get changes → returns only modified frames
2. No changes → returns empty list
---
### `update_results_after_chunk_merge(flight_id: str, refined_results: List[RefinedFrameResult]) -> bool`
**Description**: Updates results for frames affected by chunk merging into main trajectory.
**Called By**:
- F02.2 Flight Processing Engine (after chunk merge completes)
**Input**:
```python
flight_id: str
refined_results: List[RefinedFrameResult] # GPS-converted results for merged frames
```
**Output**: `bool` - True if updated successfully
**Important**: F14 does NOT call F10, F13, or F11. F02.2 coordinates the chunk merge workflow:
1. F11 returns merge result to F02.2 (direct call return, not event)
2. F02.2 gets updated poses from F10.get_trajectory(flight_id) (ENU coordinates)
3. F02.2 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
4. F02.2 constructs RefinedFrameResult objects
5. F02.2 calls F14.update_results_after_chunk_merge() with GPS-converted results
**Processing Flow**:
1. For each refined_result in refined_results:
- Extract frame_id, gps_center, confidence from RefinedFrameResult
- Update result with refined=True via F03 Flight Database (part of transaction)
- Update waypoint via F03 Flight Database (part of transaction)
- Call F15 SSE Event Streamer.send_refinement()
**Test Cases**:
1. **Chunk merge updates**: All merged frames updated and published
2. **GPS accuracy**: Updated GPS matches optimized poses
## Integration Tests
### Test 1: Per-Frame Processing
1. Process frame 237
2. update_frame_result() → stores result
3. Verify publish_waypoint_update() called (F03.update_waypoint())
4. Verify F15 SSE event sent
### Test 2: Batch Refinement
1. Process 100 frames
2. Factor graph refines frames 10-50
3. mark_refined([10-50]) → updates all
4. Verify Flight Database updated (F03.batch_update_waypoints())
5. Verify SSE refinement events sent
### Test 3: Incremental Updates
1. Process frames 1-100
2. Client disconnects at frame 50
3. Client reconnects
4. get_changed_frames(since=frame_50_time)
5. Client receives frames 51-100
## Non-Functional Requirements
### Performance
- **update_frame_result**: < 50ms
- **publish_waypoint_update**: < 100ms (non-blocking)
- **get_flight_results**: < 200ms for 2000 frames
### Reliability
- Result persistence survives crashes
- Guaranteed at-least-once delivery to Flight Database
- Idempotent updates
## Dependencies
### Internal Components
- **F03 Flight Database**: Must support transactional updates.
- **F15 SSE Event Streamer**: For real-time result streaming.
**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02.2) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies.
### External Dependencies
- None
## Data Models
### FrameResult
```python
class ObjectLocation(BaseModel):
object_id: str
pixel: Tuple[float, float]
gps: GPSPoint
class_name: str
confidence: float
class FrameResult(BaseModel):
frame_id: int
gps_center: GPSPoint
altitude: float
heading: float
confidence: float
timestamp: datetime
refined: bool
objects: List[ObjectLocation]
updated_at: datetime
```
### RefinedFrameResult
```python
class RefinedFrameResult(BaseModel):
"""
GPS-converted frame result provided by F02.2 after coordinate transformation.
F02.2 converts ENU poses from F10 to GPS using F13 before passing to F14.
"""
frame_id: int
gps_center: GPSPoint # GPS coordinates (converted from ENU by F02.2)
confidence: float
heading: Optional[float] = None # Updated heading if available
```
### FlightResults
```python
class FlightStatistics(BaseModel):
total_frames: int
processed_frames: int
refined_frames: int
mean_confidence: float
processing_time: float
class FlightResults(BaseModel):
flight_id: str
frames: List[FrameResult]
statistics: FlightStatistics
```