[AZ-504] Fix grep | wc -l pipefail crash in PT-08 batch counting

scripts/run-performance-tests.sh:416-417 used `grep -o ... | wc -l`
to count `"status":"accepted"` and `"status":"rejected"` markers in
the PT-08 batch response. On the happy path (rejected=0) grep -o
exits 1, and under `set -o pipefail` + `set -e` (line 16) the
pipeline killed the script before reaching any of PT-08's reporting
code — reproducing twice in the cycle-3 perf-harness leftover
(replay #2 + #3 post-AZ-500).

Fix: neutralise grep's no-match exit locally with `|| true` on the
grep stage of each pipeline. `grep -o | wc -l` is kept (not swapped
for `grep -c`) because the PT-08 response is compact JSON — all
items live on one line, so `grep -c` would always return 1 and lose
occurrence-count semantics. An 8-line comment explains why grep
cannot fail for I/O at this code path (file is curl-written, HTTP
200 gated).

AC-1 + AC-2 verified in-place against a standalone harness under
`set -e -o pipefail` (compact-JSON, mixed-status, edge-empty
cases). AC-3 + AC-4 are Step 15 (Performance Test) obligations by
spec design — the leftover deletion (AC-4) is "in the same commit"
as the green full perf run.

Batch report: _docs/03_implementation/batch_01_cycle5_report.md.
Code review: _docs/03_implementation/reviews/batch_01_cycle5_review.md
— PASS, no findings.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-12 16:32:36 +03:00
parent 8e509b550c
commit ab437a15df
4 changed files with 101 additions and 2 deletions
@@ -0,0 +1,60 @@
# Batch Report
**Batch**: 01 (cycle 5)
**Tasks**: AZ-504 — Perf script: fix grep | wc -l pipefail crash in PT-08
**Date**: 2026-05-12
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|---------------|-------|-------------|--------|
| AZ-504_perf_script_grep_pipefail_fix | Done (AC-1/AC-2 in-place verified; AC-3/AC-4 deferred to Step 15) | 1 file | pass (in-place harness, 4 cases under `set -e -o pipefail`) | 2/4 covered now; 2/4 deferred to Step 15 by spec design | None |
## Changes
`scripts/run-performance-tests.sh:416-417` — replaced
```bash
accepted=$(grep -o '"status":"accepted"' "$PERF_TMP_DIR/pt08_resp.json" | wc -l | tr -d ' ')
rejected=$(grep -o '"status":"rejected"' "$PERF_TMP_DIR/pt08_resp.json" | wc -l | tr -d ' ')
```
with
```bash
# AZ-504: grep exits 1 on zero matches. Under `set -o pipefail` (line 16)
# that kills the assignment and crashes the script on the happy path
# (rejected=0). Neutralise the no-match case locally with `|| true` so
# the pipeline still produces a count. The response is compact JSON
# (one line, all items) so `grep -o | wc -l` is required to count
# occurrences — `grep -c` would only count matching lines (=1). The
# file is guaranteed-readable here (curl wrote it earlier in this
# iteration on the HTTP 200 branch), so grep cannot fail for I/O.
accepted=$({ grep -o '"status":"accepted"' "$PERF_TMP_DIR/pt08_resp.json" || true; } | wc -l | tr -d ' ')
rejected=$({ grep -o '"status":"rejected"' "$PERF_TMP_DIR/pt08_resp.json" || true; } | wc -l | tr -d ' ')
```
**Why `|| true` on grep instead of `grep -c`**: PT-08 response is compact JSON written by ASP.NET Core's default serializer — all items live on a single line of the response body. `grep -c` counts MATCHING LINES, so it would return 1 regardless of whether there are 1 or 10 accepted items (Risk 1 of the task spec). `grep -o` + `wc -l` preserves occurrence semantics. The `|| true` is the minimum local neutralisation of pipefail and is justified by the `coderule.mdc` exception "If an error is truly safe to ignore, log it or comment why" — the 8-line comment explains why grep cannot fail for I/O at this code path (file is curl-written, HTTP-200-gated).
**Shellcheck pass over the file (Risk 2 of task spec)**: surveyed all `grep ... | wc -l` and `grep -[oc]` sites. Only two `grep -o ... | wc -l` sites exist in the script — both at lines 416-417 (the ones fixed). The third `grep -o` site (line 141, region status polling) is already protected by `head -1 || true` and is not vulnerable. No other defensive work required.
## AC Test Coverage
| AC | Status | Where verified |
|----|--------|----------------|
| AC-1 — PT-08 completes on zero-rejected response | **Covered** | Standalone harness reproduces the exact pipeline under `set -e -o pipefail` with `accepted=2 rejected=0`; pipeline returns count `0` without script exit. |
| AC-2 — PT-08 completes on zero-accepted response (defensive) | **Covered** | Same harness with `accepted=0 rejected=2`; pipeline returns count `0` for accepted without script exit. |
| AC-3 — PT-08 summary line prints in full run | **Deferred to Step 15** | Requires `docker-compose up -d --build` + full default-parameter `./scripts/run-performance-tests.sh` (PERF_REPEAT_COUNT=20 PERF_UAV_BATCH_SIZE=10). This is exactly the Step 15 (Performance Test) gate for cycle 5. The spec is staged: AC-4 GIVEN clause depends on AC-3 + "the full perf run is green", so the natural verification environment is Step 15. |
| AC-4 — Leftover deletion on green full run | **Deferred to Step 15** | By spec design — AC-4 says the leftover file is deleted "in the same commit" as the green full perf run. That commit IS the Step 15 deliverable. Confirmed in the perf-cycle3 leftover's "Replay attempt #4" entry. |
**Spec-Gap?** No. AC-3 + AC-4 are not gaps for Step 10 — they are explicit Step 15 deliverables baked into the spec itself (AC-4 GIVEN clause). Step 10 (this batch) delivers the script fix that makes Step 15 possible.
## Test Strategy Note
This project has no shell-script unit test infrastructure (no BATS, no `scripts/test_*.sh`). The established pattern is "the script is the test" — `scripts/run-tests.sh` and `scripts/run-performance-tests.sh` ARE the verification surface. Adding BATS for a 1-SP single-line fix would be infra creep explicitly disallowed by `coderule.mdc` ("Avoid boilerplate and unnecessary indirection ... follow established project patterns"). The standalone harness recorded above is repeatable on demand and produces evidence equivalent to a BATS smoke test.
## Code Review Verdict: PASS
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Next Batch: AZ-503 — Tile identity → UUIDv5 + integer UPSERT + bulk-list endpoint (3 SP)
@@ -0,0 +1,31 @@
# Code Review Report
**Batch**: 01 (cycle 5) — AZ-504 (1 SP)
**Date**: 2026-05-12
**Verdict**: PASS
## Phase Summary
| Phase | Result |
|-------|--------|
| 1. Context Loading | OK — task spec read; intent = tolerate grep exit 1 on no-match while preserving occurrence-count semantics; do NOT change PT-08 threshold or production code. |
| 2. Spec Compliance | OK — AC-1 + AC-2 verified in standalone harness under `set -e -o pipefail`. AC-3 + AC-4 staged by spec design (AC-4 GIVEN clause couples them to the Step 15 full perf run). No Spec-Gap. |
| 3. Code Quality | OK — `\|\| true` is locally scoped, accompanied by an 8-line comment explaining why grep cannot fail for I/O at this code path; coderule.mdc allows this with comment. Mirror-image accepted/rejected lines are intentional, not duplication worth extracting. |
| 4. Security Quick-Scan | OK — no SQL, no command injection, no secrets, no input validation touch. |
| 5. Performance Scan | OK — same `grep -o \| wc -l` complexity as before; `\|\| true` adds zero overhead on the happy path. |
| 6. Cross-Task Consistency | N/A — single-task batch. |
| 7. Architecture Compliance | N/A — shell script, not in any C# component; no layer / Public API / cycle / duplicate-symbol concerns. |
## Findings
None.
## Notes
- Risk 1 (compact JSON vs `grep -c`) was verified empirically against `SatelliteProvider.Common/DTO/UavTileBatchUploadResponse.cs` — the ASP.NET Core default serializer emits the full `Items` list on a single line. `grep -c` would have collapsed all occurrences to a line count of 1. The chosen fix (`grep -o ... \|\| true \| wc -l`) preserves occurrence semantics. The 8-line in-script comment captures this so a future maintainer doesn't re-introduce `grep -c`.
- Risk 2 (other vulnerable `grep ... \| wc -l` sites) — Grep pass over the whole script found only the two sites named in the spec (416, 417). The third `grep -o` site (line 141, region status polling) is protected by `head -1 \|\| true`. No other defensive work required.
- Test infra note — no BATS/shell-test framework in this project. Adding one for a 1-SP fix is infra creep. The standalone harness in the batch report is the established equivalent verification.
## Verdict
**PASS** — no findings of any severity. Proceed to commit. AC-3 + AC-4 are Step 15 obligations, not Step 10 gaps.
+10 -2
View File
@@ -413,8 +413,16 @@ else
continue continue
fi fi
accepted=$(grep -o '"status":"accepted"' "$PERF_TMP_DIR/pt08_resp.json" | wc -l | tr -d ' ') # AZ-504: grep exits 1 on zero matches. Under `set -o pipefail` (line 16)
rejected=$(grep -o '"status":"rejected"' "$PERF_TMP_DIR/pt08_resp.json" | wc -l | tr -d ' ') # that kills the assignment and crashes the script on the happy path
# (rejected=0). Neutralise the no-match case locally with `|| true` so
# the pipeline still produces a count. The response is compact JSON
# (one line, all items) so `grep -o | wc -l` is required to count
# occurrences — `grep -c` would only count matching lines (=1). The
# file is guaranteed-readable here (curl wrote it earlier in this
# iteration on the HTTP 200 branch), so grep cannot fail for I/O.
accepted=$({ grep -o '"status":"accepted"' "$PERF_TMP_DIR/pt08_resp.json" || true; } | wc -l | tr -d ' ')
rejected=$({ grep -o '"status":"rejected"' "$PERF_TMP_DIR/pt08_resp.json" || true; } | wc -l | tr -d ' ')
PT08_ACCEPTED=$((PT08_ACCEPTED + accepted)) PT08_ACCEPTED=$((PT08_ACCEPTED + accepted))
PT08_REJECTED=$((PT08_REJECTED + rejected)) PT08_REJECTED=$((PT08_REJECTED + rejected))