add chunking

This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-27 03:43:19 +02:00
parent 4f8c18a066
commit 2037870f67
43 changed files with 7041 additions and 4135 deletions
@@ -0,0 +1,242 @@
# SSE Event Streamer
## Interface Definition
**Interface Name**: `ISSEEventStreamer`
### Interface Methods
```python
class ISSEEventStreamer(ABC):
@abstractmethod
def create_stream(self, flight_id: str, client_id: str) -> StreamConnection:
pass
@abstractmethod
def send_frame_result(self, flight_id: str, frame_result: FrameResult) -> bool:
pass
@abstractmethod
def send_search_progress(self, flight_id: str, search_status: SearchStatus) -> bool:
pass
@abstractmethod
def send_user_input_request(self, flight_id: str, request: UserInputRequest) -> bool:
pass
@abstractmethod
def send_refinement(self, flight_id: str, frame_id: int, updated_result: FrameResult) -> bool:
pass
@abstractmethod
def close_stream(self, flight_id: str, client_id: str) -> bool:
pass
```
## Component Description
### Responsibilities
- Server-Sent Events broadcaster for real-time results
- Stream per-frame processing results to clients
- Send refinement updates asynchronously
- Request user input when processing blocked
- Handle client connections and reconnections
- Event replay from last received event
### Scope
- SSE protocol implementation
- Event formatting and sending
- Connection management
- Client reconnection handling
- Multiple concurrent streams per flight
## API Methods
### `create_stream(flight_id: str, client_id: str) -> StreamConnection`
**Description**: Creates SSE connection for a client.
**Called By**: F01 REST API (GET /stream endpoint)
**Output**:
```python
StreamConnection:
stream_id: str
flight_id: str
client_id: str
last_event_id: Optional[str]
```
**Event Types**:
- `frame_processed`
- `frame_refined`
- `search_expanded`
- `user_input_needed`
- `processing_blocked`
- `route_api_updated`
- `route_completed`
**Test Cases**:
1. Create stream → client receives keepalive pings
2. Multiple clients → each gets own stream
---
### `send_frame_result(flight_id: str, frame_result: FrameResult) -> bool`
**Description**: Sends frame_processed event.
**Called By**: F13 Result Manager
**Event Format**:
```json
{
"event": "frame_processed",
"id": "frame_237",
"data": {
"frame_id": 237,
"gps": {"lat": 48.123, "lon": 37.456},
"altitude": 800.0,
"confidence": 0.95,
"heading": 87.3,
"timestamp": "2025-11-24T10:30:00Z"
}
}
```
**Test Cases**:
1. Send event → all clients receive
2. Client disconnected → event buffered for replay
---
### `send_search_progress(flight_id: str, search_status: SearchStatus) -> bool`
**Description**: Sends search_expanded event.
**Called By**: F11 Failure Recovery Coordinator
**Event Format**:
```json
{
"event": "search_expanded",
"data": {
"frame_id": 237,
"grid_size": 9,
"status": "searching"
}
}
```
---
### `send_user_input_request(flight_id: str, request: UserInputRequest) -> bool`
**Description**: Sends user_input_needed event.
**Called By**: F11 Failure Recovery Coordinator
**Event Format**:
```json
{
"event": "user_input_needed",
"data": {
"request_id": "uuid",
"frame_id": 237,
"candidate_tiles": [...]
}
}
```
---
### `send_refinement(flight_id: str, frame_id: int, updated_result: FrameResult) -> bool`
**Description**: Sends frame_refined event.
**Called By**: F13 Result Manager
**Event Format**:
```json
{
"event": "frame_refined",
"data": {
"frame_id": 237,
"gps": {"lat": 48.1235, "lon": 37.4562},
"refined": true
}
}
```
---
### `close_stream(flight_id: str, client_id: str) -> bool`
**Description**: Closes SSE connection.
**Called By**: F01 REST API (on client disconnect)
## Integration Tests
### Test 1: Real-Time Streaming
1. Client connects
2. Process 100 frames
3. Client receives 100 frame_processed events
4. Verify order and completeness
### Test 2: Reconnection with Replay
1. Client connects
2. Process 50 frames
3. Client disconnects
4. Process 50 more frames
5. Client reconnects with last_event_id
6. Client receives frames 51-100
### Test 3: User Input Flow
1. Processing blocks
2. send_user_input_request()
3. Client receives event
4. Client responds with fix
5. Processing resumes
## Non-Functional Requirements
### Performance
- **Event latency**: < 500ms from generation to client
- **Throughput**: 100 events/second
- **Concurrent connections**: 1000+ clients
### Reliability
- Event buffering for disconnected clients
- Automatic reconnection support
- Keepalive pings every 30 seconds
## Dependencies
### Internal Components
- None (receives calls from other components)
### External Dependencies
- **FastAPI** or **Flask** SSE support
- **asyncio**: For async event streaming
## Data Models
### StreamConnection
```python
class StreamConnection(BaseModel):
stream_id: str
flight_id: str
client_id: str
created_at: datetime
last_event_id: Optional[str]
```
### SSEEvent
```python
class SSEEvent(BaseModel):
event: str
id: Optional[str]
data: Dict[str, Any]
```