azaion/satellite-provider

Fork 0

mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 08:51:13 +00:00

Files

T

Oleksandr Bezdieniezhnykh af4219fce6

ci/woodpecker/push/01-test Pipeline was successful

Details

ci/woodpecker/push/02-build-push Pipeline was successful

Details

[AZ-500] Cycle 4 Steps 12-15 sync (test-spec / docs / security / perf)

Step 12 (Test-Spec Sync) - cycle-update mode
  - traceability-matrix: 8 AZ-500 AC rows + .NET 10 runtime
    restriction supersession + Cycle-4 coverage shape note
    (no new tests; ACs verified by re-running existing 78-test
    suite + build pipeline + manifest grep)

Step 13 (Update Docs) - task mode
  - FINAL_report, 00_discovery, architecture, module-layout,
    api_program, tests_unit: .NET 8 -> .NET 10 / C# 12 -> 14 /
    Swashbuckle 6.6.2 -> 10.1.7 + Microsoft.OpenApi 2.x
    refactor note in api_program; Serilog.AspNetCore 8.0.3
    fallback documented inline per AZ-500 Risk #4
  - deployment/{containerization, ci_cd_pipeline}: Docker
    aspnet/sdk:8.0 -> :10.0
  - ripple_log_cycle4: empty import-graph ripple recorded
    (Program.cs is entry point; ParameterDescriptionFilter only
    consumed by Program.cs; csproj/global.json/Dockerfile have
    no import edges)

Step 14 (Security Audit) - resume mode
  - dependency_scan_cycle4: AZ-500 19-package delta scanned;
    cycle-3 D1+D3 (CVE-2026-26130) closed by major-version
    bump; cycle-3 D2 (Test.Sdk 17.8.0 NuGet.Frameworks flag)
    carried over - explicitly out of AZ-500 scope
  - security_report_cycle4: PASS_WITH_WARNINGS (only carry-over
    Medium open; AZ-500 introduced 0 new Critical/High); cycle-3
    static_analysis/owasp_review/infrastructure_review carried
    forward unchanged (AZ-500 made no source-level edits to
    those surfaces)

Step 15 (Performance Test) - perf mode, full default-param run
  - perf_2026-05-12_cycle4: 7 Pass + 1 Unverified (PT-08 hit
    pre-existing scripts/run-performance-tests.sh:417 grep-
    pipefail bug, NOT a .NET 10 regression)
  - PT-07 warm p95 = 301ms (7.7x improvement vs cycle-3 short
    variant - .NET 10 pipeline + N=20 dilution); cold p95 =
    2782ms (-14%); PT-06 90ms (-49%)
  - AZ-500 NFR (Performance) MET for 7/8 scenarios
  - Cycle-3 perf-harness leftover updated with replay #3
    results; STAYS OPEN per AZ-500 Constraint (deletes only on
    fully clean run)

Recommended follow-up PBIs (out of cycle-4 scope, surfaced for
the backlog):
  - 1 SP fix scripts/run-performance-tests.sh:416-417 grep-
    pipefail (replace grep -o ... | wc -l with grep -c ... ||
    true) - unblocks PT-08 + closes the cycle-3 perf leftover
  - 3 SP migrate WithOpenApi(...) callsites to ASP.NET Core 10
    minimal-API metadata extensions (clears 8 ASPDEPR002
    warnings; recorded in batch_01_cycle4_review.md)
  - 1 SP Microsoft.OpenApi 2.x nullable cleanup (CS8604 in
    ParameterDescriptionFilter.cs:25)
  - 1 SP bump Microsoft.NET.Test.Sdk 17.8.0 -> 17.13.0+
    (closes cycle-3 D2 NuGet.Frameworks transitive flag)

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-05-12 06:05:29 +03:00

6.0 KiB

Raw Blame History

Perf Run — Cycle 4 (post AZ-500 .NET 10 migration)

Date: 2026-05-12T04:50:00Z Run label: cycle4 — full default-parameter run Trigger: autodev existing-code Step 15 (Performance Test gate); AZ-500 NFR (Performance) requires the full PT-01..PT-08 harness against the migrated .NET 10 build. Runner: scripts/run-performance-tests.sh (default params: PERF_REPEAT_COUNT=20, PERF_UAV_BATCH_SIZE=10) System under test: docker-compose up -d --build against mcr.microsoft.com/dotnet/aspnet:10.0 (post AZ-500); api healthy on :18980, swagger 301, anonymous request 401. Build: SatelliteProvider.IntegrationTests Release, .NET 10.0.103 SDK, 0 errors / 11 warnings (carried-over NU1902 IdentityModel + CA2227 — both unrelated to AZ-500).

Results

#	Scenario	Verdict	Observed	Threshold	Source of threshold
PT-01	Tile download (cold)	PASS	3207ms	≤ 30000ms	`_docs/02_document/tests/performance-tests.md`
PT-02	Cached tile retrieval	PASS	259ms	≤ 500ms	`_docs/02_document/tests/performance-tests.md`
PT-03	Region 200m / z18	PASS	2200ms	≤ 60000ms	`_docs/02_document/tests/performance-tests.md`
PT-04	Region 500m / z18 + stitch	PASS	2139ms	≤ 120000ms	`_docs/02_document/tests/performance-tests.md`
PT-05	5 concurrent regions	PASS	2611ms	≤ 300000ms	`_docs/02_document/tests/performance-tests.md`
PT-06	Route creation (2 points)	PASS	90ms	≤ 5000ms	`_docs/02_document/tests/performance-tests.md`
PT-07	Region request distribution (N=20, cold + warm)	PASS	cold p50=2208ms, p95=2782ms · warm p50=88ms, p95=301ms	warm p95 < cold p95	AZ-484 / AZ-492
PT-08	UAV batch upload (batch=10, N=20)	Unverified	— (script crashed at fixture-generation step before any latency capture)	— (would have been: batch p95 ≤ 2000ms per AZ-488)	not measurable this run

Scenarios: 7 Pass · 0 Warn · 0 Fail · 1 Unverified

Comparison vs. cycle-3 (replay #2 short variant, 2026-05-12T02:21:00Z, REPEAT_COUNT=2)

Scenario	Cycle-3 short	Cycle-4 full	Delta	Note
PT-01	2538ms	3207ms	+27% (both ≪ 30000ms)	within normal cold-path noise; full run picks first uncached tile from a different geographic cell
PT-02	195ms	259ms	+33% (both ≪ 500ms)	within normal warm-path noise
PT-03	384ms	2200ms	(both ≪ 60000ms)	full run includes Google Maps round-trips; cycle-3 short variant hit the cached path
PT-04	2202ms	2139ms	−3%	essentially flat
PT-05	3258ms	2611ms	−20%	small improvement; could be .NET 10 ASP.NET pipeline gain or timing noise
PT-06	178ms	90ms	−49%	2x faster; consistent with .NET 10 GC + JIT improvements on small-allocation paths
PT-07 cold p95	3241ms	2782ms	−14%	improvement
PT-07 warm p95	2340ms	301ms	−87% (7.7x improvement)	dominant signal: larger sample size (N=20 vs N=2) dilutes first-touch outliers; also benefits from .NET 10 cached-path gains
PT-08	unmeasurable (script bug)	unmeasurable (same script bug)	n/a	pre-existing `scripts/run-performance-tests.sh:417` grep-pipefail issue; documented in cycle-3 perf-harness leftover

No scenario regressed beyond its threshold. AZ-500 NFR (Performance — "must not regress beyond the existing thresholds") is MET for 7 of 8 scenarios, with PT-08 unmeasurable due to a pre-existing script bug (the underlying production handler's perf is healthy — cycle-3 captured one batch at 99ms, well below the 2000ms threshold, and AZ-500 didn't change the AZ-488 hot path).

Verdict (perf-mode skill rubric)

Per-scenario classification: 7 Pass + 1 Unverified — not blocking; surface in report so coverage gap is visible.
No Warn or Fail anywhere in the run.
AZ-500 perf NFR: MET. The "must not regress beyond existing thresholds" gate is satisfied for every scenario where a measurement was possible; the one Unverified scenario is blocked by a script-instrumentation bug, not by a runtime/perf change.

Step 15 verdict: PASS_WITH_UNVERIFIED. Auto-chain to Step 16 (Deploy) is permitted per the perf-mode skill.

Outstanding items

Cycle-3 perf-harness leftover (_docs/_process_leftovers/2026-05-12_perf-cycle3-harness-execution.md): updated with replay #3 results. Stays OPEN per AZ-500 Constraint ("leftover file is deleted ONLY when the full perf script runs cleanly"). PT-08 still cannot be measured end-to-end until the script-fix PBI lands.
Follow-up PBI (recommended, 1 SP): fix scripts/run-performance-tests.sh:416-417 grep-pipefail. Replace grep -o '"status":"X"' file | wc -l with grep -c '"status":"X"' file || true. After this lands, the cycle-3 leftover can be deleted on the next perf run (and PT-08 batch p95 measurement becomes available going forward).
Follow-up PBI (recommended, 3 SP): migrate the 8 WithOpenApi(...) callsites in Program.cs to ASP.NET Core 10's minimal-API metadata extensions. Clears the 8 ASPDEPR002 deprecation warnings. Already filed in _docs/03_implementation/reviews/batch_01_cycle4_review.md. Not perf-related — quality/maintainability — but listed here for traceability since both PBIs surfaced from the AZ-500 cycle.

Self-verification

All scenarios from _docs/02_document/tests/performance-tests.md exercised (PT-01..PT-08).
Each Pass scenario verified against its threshold from the cited spec source.
Cycle-to-cycle delta computed for trend tracking — no scenario regressed beyond threshold.
PT-08 Unverified classification justified (instrumentation failure, not perf failure; reproduced across two consecutive replays at the same line).
Cross-referenced AZ-500 NFR (Performance) — MET for 7/8 scenarios; the 1 Unverified is documented and blocked by a pre-existing, unrelated script bug.
Cycle-3 perf-harness leftover updated with replay #3 results; remains OPEN per AZ-500 Constraint.

6.0 KiB Raw Blame History Unescape Escape