# OKVIS2 v2 upstream patch: expose 6×6 pose covariance accessor (+ ADR for pin deviation)

**Task**: AZ-951_okvis2_upstream_covariance_patch
**Name**: OKVIS2 v2 upstream patch — expose 6×6 pose covariance accessor (+ ADR for pin deviation)
**Description**: Land the documented "approach (b) upstream patch" escape hatch from AZ-592 (line 30) and AZ-943 (Implementation Notes "Tiny upstream patch"). AZ-943's implementation attempt on 2026-05-29 confirmed that the proposed "approach (a) in-binding subclass workaround" is structurally impossible: `ThreadedSlam::estimator_` is declared `private` (not `protected`), and `ViSlamBackend` has no public covariance accessor anywhere in the v2 upstream headers.
**Complexity**: 3 SP
**Dependencies**: AZ-332 (the AZ-332 Plan-phase pin this work deviates from), AZ-592 (parent placeholder that originally offered this approach as option (a) in its line 30-31)
**Component**: c1_vio (epic AZ-254 / E-C1); also touches `cpp/okvis2/` (upstream wrapper) and `_docs/03_implementation/architecture/decisions/` (ADR)
**Tracker**: AZ-951 (https://denyspopov.atlassian.net/browse/AZ-951)
**Parent Epic**: AZ-254 (E-C1)
**Blocks**: AZ-943 AC-4 (pose_covariance_6x6 field)

Jira AZ-951 is the authoritative spec; this file is the in-workspace mirror.

## Goal

Land the documented "approach (b) upstream patch" escape hatch. **Blocks AZ-943 AC-4** (`pose_covariance_6x6` field). The Python facade (`okvis2.py` `_build_vio_output`) shape-checks the 6×6 covariance; downstream EKF in C2 treats it as Kalman gain weight, so an identity placeholder would lie about VIO uncertainty (contradicts AZ-848 ESKF out-of-order analysis).

## Scope

1. **ADR-XXX (pin deviation rationale)** under `_docs/03_implementation/architecture/decisions/`:
   - Title: "OKVIS2 v2 upstream patch — expose ViSlamBackend pose-covariance accessor for Okvis2Backend C++ binding"
   - Decision: deviate from AZ-332 Plan-phase fixed pin to land a small, surgical, documented patch.
   - Alternatives considered: (1) keep pin + ship placeholder covariance (violates meta-rule "Real Results, Not Simulated Ones"); (2) hard fork OKVIS2 (rejected — too much surface); (3) upstream the patch as a follow-up contribution to the OKVIS2 maintainers (recommended).
   - Consequences: future upstream rebases must reapply; patch is small and self-contained to minimise that cost.

2. **Patch file** under `cpp/okvis2/patches/expose_covariance.patch`:
   - Make `ThreadedSlam::estimator_` reachable from the binding: either add `public okvis::ViSlamBackend& backend()` to `ThreadedSlam` OR change `estimator_` from `private` to `protected`. Recommend the public accessor — cleaner API surface, less invasive.
   - Add `Eigen::Matrix<double, 6, 6> ViSlamBackend::computeCovariance6x6(StateId id) const` — wraps `ceres::Covariance::Compute` over the `realtimeGraph_`'s ceres::Problem for the pose parameter block at `id`. Returns a documented failure-sentinel (identity * large scale + warning log) when the covariance computation is rank-deficient; binding then flags the output as low-confidence.

3. **CMake glue** in `cpp/okvis2/CMakeLists.txt`: apply the patch via the chosen mechanism (decided at scheduling — see Open Decisions).

4. **Verification path**: compile-clean evidence comes via AZ-944 (Linux CI BUILD_OKVIS2=ON flip). Local macOS gets no compile evidence (project policy).

## Acceptance Criteria

- **AC-1**: ADR-XXX exists under `_docs/03_implementation/architecture/decisions/`, follows the project's existing ADR template, and cites AZ-332 (Plan-phase pin), AZ-592 (parent placeholder), AZ-943 (the blocked binding ticket), and this ticket's Jira key.
- **AC-2**: `cpp/okvis2/patches/expose_covariance.patch` exists, applies cleanly to the vendored upstream at `cpp/okvis2/upstream/`, and the patch surface is ≤ 100 lines of diff (keeps future rebase cost low).
- **AC-3**: After patch application, `okvis::ThreadedSlam` has a public method that returns a reference / pointer to its `okvis::ViSlamBackend` member.
- **AC-4**: After patch application, `okvis::ViSlamBackend` has a public `Eigen::Matrix<double, 6, 6> computeCovariance6x6(StateId id) const` method backed by `ceres::Covariance::Compute`. Behaviour:
  - Success: returns the 6×6 marginalised pose covariance.
  - Failure (rank-deficient / non-converged / wrong state ID): returns a documented failure sentinel and emits a single warning log per occurrence — NO exception thrown into the binding (the binding layer decides whether to surface this as an OkvisOptimizationException).
- **AC-5**: Patch mechanism (in-place `git apply` vs vendored header overrides vs forked submodule) is chosen at scheduling and documented in the patch's commit message + ADR-XXX.
- **AC-6**: Local task spec for AZ-943 is updated to call `backend().computeCovariance6x6(state.id)` inside the `setOptimisedGraphCallback` lambda (the AZ-943 implementation, post-unblock).

## Open Decisions (resolve at scheduling)

1. **Patch application mechanism**: in-place `git apply` (simplest, but touches vendored source) vs vendored header overrides under `cpp/okvis2/include_overrides/` (most transparent in code review) vs forked submodule (heaviest, only if patch grows large). Default proposal: vendored header overrides.
2. **Covariance failure semantics**: silent identity sentinel + log (proposed default; binding flags output as low-confidence) vs raise an OKVIS2-side exception (then binding rethrows as `OkvisOptimizationException`).

## Out of scope

- Tracking-stats telemetry (tracked/new/lost feature counts, mean_parallax, mre_px) — separate sibling ticket (AZ-952); this one is covariance-only because the two pieces have different upstream surface area and risk profiles.
- Submitting the patch to upstream OKVIS2 maintainers (tracked as a follow-up issue on the upstream GitHub mirror after this lands locally).
- The downstream AZ-943 binding update — owned by AZ-943, which is currently blocked-by this ticket.

## References

- AZ-943 implementation attempt (2026-05-29): proved "approach (a) subclass workaround" infeasible — `ThreadedSlam::estimator_` is `private`, `ViSlamBackend` has no public covariance accessor.
- AZ-592 line 30-31: offered this exact approach as fallback when (a) fails.
- AZ-943 Implementation Notes "Tiny upstream patch": defers to this approach explicitly.
- `cpp/okvis2/upstream/okvis_multisensor_processing/include/okvis/ThreadedSlam.hpp` line 254: `private: okvis::ViSlamBackend estimator_;`
- `cpp/okvis2/upstream/okvis_ceres/include/okvis/ViSlamBackend.hpp`: no public covariance accessor anywhere.
- meta-rule.mdc "Real Results, Not Simulated Ones" — the constraint that forces this path over a placeholder.
- Sibling ticket: AZ-952 (tracking-stats accessor).
