# OKVIS2 v2 upstream patch: expose tracking-stats accessor (feature counts + parallax + MRE)

**Task**: AZ-952_okvis2_upstream_tracking_stats_patch
**Name**: OKVIS2 v2 upstream patch — expose tracking-stats accessor (feature counts + parallax + MRE)
**Description**: Sibling to AZ-951 (covariance + ADR). AZ-943's implementation attempt on 2026-05-29 confirmed that the four tracking-stats fields the `Okvis2Backend` C++ binding must fill have no source in OKVIS2 v2's public `setOptimisedGraphCallback` arg list. `TrackingState` (`okvis/ViInterface.hpp` lines 167-174) carries only `id`, `isKeyframe`, `trackingQuality` (enum: Good/Marginal/Lost), `recognisedPlace`, `isFullGraphOptimising`, `currentKeyframeId` — NONE of the five tracking-stats fields the binding needs.
**Complexity**: 3 SP
**Dependencies**: AZ-951 (SOFT — same ADR, same patch-mechanism decision; can land in either order, but combining patches is easier if scheduled together), AZ-332 (the AZ-332 Plan-phase pin this work deviates from alongside AZ-951), AZ-592 (parent placeholder)
**Component**: c1_vio (epic AZ-254 / E-C1); also touches `cpp/okvis2/` (upstream wrapper)
**Tracker**: AZ-952 (https://denyspopov.atlassian.net/browse/AZ-952)
**Parent Epic**: AZ-254 (E-C1)
**Blocks**: AZ-943 AC-4 (tracked/new/lost feature counts + mean_parallax + mre_px fields)

Jira AZ-952 is the authoritative spec; this file is the in-workspace mirror.

## Goal

**Blocks AZ-943 AC-4** (the five tracking-stats fields). The Python facade (`okvis2.py` `_build_vio_output`) consumes all five (line 393-399: `FeatureQuality(tracked=..., new=..., lost=..., mean_parallax=..., mre_px=...)`). The `tracked` field also feeds the `_classify_state(vio_output.feature_quality)` DEGRADED-state classifier (line 241) — placeholders would systematically misclassify health.

Five fields with no source in the public callback surface:

- `tracked_features` (int) — not in callback args; computed inside `okvis::Frontend` during matching.
- `new_features` (int) — same.
- `lost_features` (int) — same.
- `mean_parallax` (double, px) — not in callback args; computed inside `okvis::Frontend` keyframe selection.
- `mre_px` (double, mean reprojection error) — not in callback args; ceres optimisation byproduct on the realtimeGraph.

## Scope

1. **Patch file** under `cpp/okvis2/patches/expose_tracking_stats.patch` (or merge into a single combined patch with AZ-951's covariance patch — scheduler decides):
   - Add `void ViSlamBackend::getLatestTrackingStats(StateId id, int& tracked, int& newCount, int& lost, double& meanParallaxPx, double& mreReprojectionPx) const` — reads from the relevant private members (`frontend_` / `realtimeGraph_` / `multiFrames_`) via a single batched accessor.
     - `tracked` = count of landmark observations for state `id` that were also observed in the most recent prior keyframe.
     - `newCount` = count of landmark observations for state `id` that were NOT observed in any prior frame.
     - `lost` = count of landmarks observed in the prior keyframe but absent from `id`.
     - `meanParallaxPx` = mean keypoint pixel displacement between `id` and the most recent prior keyframe, over the `tracked` matched set.
     - `mreReprojectionPx` = mean per-observation reprojection residual from the realtimeGraph optimisation, over all observations attached to `id`.

2. **CMake glue**: same mechanism as AZ-951's covariance patch (vendored header overrides vs in-place git apply vs forked submodule — decided at AZ-951 scheduling). If AZ-951 lands first, this ticket reuses that mechanism; if scheduled in parallel, mechanism is decided together.

3. **Verification path**: compile-clean evidence comes via AZ-944 (Linux CI BUILD_OKVIS2=ON flip). Local macOS gets no compile evidence (project policy).

## Acceptance Criteria

- **AC-1**: `cpp/okvis2/patches/expose_tracking_stats.patch` (or the combined patch with AZ-951's content) exists, applies cleanly to the vendored upstream at `cpp/okvis2/upstream/`, and the total patch surface across this + AZ-951 stays ≤ 200 lines of diff (keeps future rebase cost low).
- **AC-2**: After patch application, `okvis::ViSlamBackend::getLatestTrackingStats(...)` is a public method that fills the five out-params with finite values for any valid `StateId` of a state that has been through realtimeGraph optimisation. For state IDs that have not yet been optimised, all five are set to documented sentinel values (zeros + warning log).
- **AC-3**: All five values are computed from the realtimeGraph's actual matched-observation set; no placeholders, no defaults. Code comment in the patch explains the derivation for each field.
- **AC-4**: ADR-XXX from AZ-951 is updated to cite this ticket alongside the covariance accessor work, so the deviation-from-pin rationale documents the FULL telemetry exposure surface.
- **AC-5**: Local task spec for AZ-943 is updated to call `backend().getLatestTrackingStats(state.id, ...)` inside the `setOptimisedGraphCallback` lambda (the AZ-943 implementation, post-unblock).

## Open Decisions (resolve at scheduling)

1. **Combined vs separate patch file**: ship as `expose_telemetry.patch` (one file covering both AZ-951's covariance + this ticket's tracking-stats) or as two separate `.patch` files. Default proposal: one combined patch with two logical commits inside it.
2. **Sentinel semantics for unoptimised states**: zeros + warning log (proposed default) vs raise OKVIS2-side exception (binding rethrows as `OkvisOptimizationException`).
3. **Parallax denominator edge case**: when `tracked == 0` (no matches at all), `meanParallaxPx` is undefined. Proposed default: emit NaN + warning log; binding then short-circuits to DEGRADED health state. Scheduler may choose 0.0 instead.

## Out of scope

- 6×6 pose covariance accessor — covered by AZ-951 (sibling ticket; same ADR, same patch mechanism).
- The ADR creation itself — owned by AZ-951; this ticket extends the ADR's scope rather than creating a separate one.
- Submitting the patch to upstream OKVIS2 maintainers (tracked as a follow-up issue after both tickets land locally).
- The downstream AZ-943 binding update — owned by AZ-943, which is currently blocked-by this ticket.

## References

- AZ-943 implementation attempt (2026-05-29): proved the five tracking-stats fields have no source in OKVIS2 v2's public callback surface.
- AZ-592 line 24 (incorrect spec assumption): "derive tracked_features + mean_parallax from TrackingState" — superseded by this ticket; TrackingState does NOT carry these.
- `cpp/okvis2/upstream/okvis_common/include/okvis/ViInterface.hpp` lines 167-174: TrackingState definition (only 6 fields, none of them tracking-stats).
- `cpp/okvis2/upstream/okvis_ceres/include/okvis/ViSlamBackend.hpp`: no public tracking-stats accessor anywhere.
- AZ-848 (jetson_eskf_out_of_order_regression): downstream EKF assumes finite, computed VIO telemetry; placeholders would mislead its diagnostic surface.
- meta-rule.mdc "Real Results, Not Simulated Ones" — the constraint that forces this path over a placeholder.
- Sibling ticket: AZ-951 (covariance + ADR).
