mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-04 10:48:20 +00:00
Producer. test-impact-artifact.yml publishes, on every push to main, the commit's team artifact bundle: its frozen graph, built incrementally from the previous bundle, and its per-test coverage map, re-run only for the suites the push selects (scripts/ci/test-impact-publish.sh). A weekly run rebuilds both from scratch and fails when the incremental graph's topology differs from the full one. Bundles are Actions artifacts kept 30 days, never committed. Shadow. The report-only test-impact-shadow job also builds the checkout and runs `test-impact select` against the base side of the PR merge commit (scripts/ci/test-impact-select.sh). A bundle is used only when it comes from a successful main-branch producer run for exactly that commit (scripts/ci/test-impact-bundle.sh). The job stays continue-on-error and outside ci-ok. Narrowing switch, PR CI only. While the repository variable TEST_IMPACT_MODE is `gate`, a select-tests job turns the engine's answer into the tests the test-unix and test-windows legs run. Unset, nothing changes. It fails open by construction: - scripts/test-impact/selection.py is the only place an answer becomes a runner filter. Only a well-formed selected lane whose filter matches its own suite list narrows; run-all answers, a lane that selects nothing, unknown schemas or tokens, and any crash run everything. - `test` passes a selection only from a successful, narrowed select-tests run; ci-ok does not need select-tests. A broken selection costs minutes, never coverage. - scripts/run-tests-parallel.sh launches only the selected suites and keeps every guard on the selected list. A selected suite the build does not register fails the run, unless the engine saw its registration as conditional; then it is dropped with a notice, exactly as a full run of that build never runs it. Each runner judges its own tokens, so a selected test that does not exist fails its suite. - The shard manifest records the selection's digest, and verify-shard-union.sh requires every shard of a leg to agree on it. - Dry runs and releases never pass a selection (lane-wiring contract). Also fixed on the way: - publish crashed on any run-all push: a run-all lane carries "suites": null. - the git-facts hostile-diff probe re-ran the runner with the parent's CBM_TEST_ONLY_FILE still set; under a narrowed run naming its suite it re-ran the spawning test, which spawned again. The child now clears it (a new test fails without that). - the select summary crashed on the same null suites. Contracts: tests/test_test_impact_selection.sh (Step 0z8), tests/test_harness_selection.sh (Step 2c, the real runner through the real harness), and new cases in the lane-gate and lane-wiring contracts. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
103 lines
4.6 KiB
Bash
103 lines
4.6 KiB
Bash
#!/usr/bin/env bash
|
|
# test-impact-select.sh — the PR selection with the team artifact: what
|
|
# `codebase-memory-mcp test-impact select` runs for this change.
|
|
#
|
|
# Shadow (default): REPORT-ONLY; pr.yml's test-impact-shadow job runs it
|
|
# continue-on-error and ci-ok does not need it.
|
|
# --gate DIR (pr.yml's select-tests job, only when the repository variable
|
|
# TEST_IMPACT_MODE is `gate`): also turns the answer into what the test legs
|
|
# run (scripts/test-impact/selection.py) — DIR/test-only.txt and
|
|
# DIR/optional-suites.txt for a narrowed selection — and writes
|
|
# `selection=narrowed` or `selection=all` to $GITHUB_OUTPUT. Anything but a
|
|
# well-formed narrowed answer, a lane that selects nothing included, is `all`.
|
|
#
|
|
# The base is the base side of the PR merge commit (HEAD^1 by default). Its
|
|
# bundle is fetched with scripts/ci/test-impact-bundle.sh, which only accepts
|
|
# the main-branch producer's own run for exactly that commit; only then does
|
|
# the selection pass --artifact-verified. Without a bundle the engine builds
|
|
# the graph itself and admits no coverage: every reached suite runs whole.
|
|
#
|
|
# Usage: scripts/ci/test-impact-select.sh --binary PATH --out DIR [--base REV]
|
|
# [--platform LABEL] [--gate DIR]
|
|
# Writes DIR/selection-engine.json and DIR/summary-engine.md (appended
|
|
# to $GITHUB_STEP_SUMMARY when set).
|
|
# Exit: 0 = selection written · 1 = the tooling broke · 2 = usage error.
|
|
set -euo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
|
BINARY="" OUT="" BASE="HEAD^1" PLATFORM="linux-x86_64-clang21" GATE=""
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--binary) BINARY="$2"; shift 2 ;;
|
|
--out) OUT="$2"; shift 2 ;;
|
|
--base) BASE="$2"; shift 2 ;;
|
|
--platform) PLATFORM="$2"; shift 2 ;;
|
|
--gate) GATE="$2"; shift 2 ;;
|
|
*) sed -n '2,24p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2 ;;
|
|
esac
|
|
done
|
|
[ -n "$BINARY" ] && [ -n "$OUT" ] || exit 2
|
|
|
|
mkdir -p "$OUT"
|
|
WORK="$(mktemp -d)"
|
|
trap 'rm -rf "$WORK"' EXIT
|
|
export CBM_RUNTIME_DIR="$(mktemp -d /tmp/cbm-rt.XXXXXX)" # sun_path must fit
|
|
export CBM_CACHE_DIR="$WORK/cache"
|
|
mkdir -p "$WORK/engine" && chmod 700 "$WORK/engine"
|
|
base="$(git -C "$ROOT" rev-parse "$BASE")"
|
|
|
|
artifact=()
|
|
bundle_rc=0
|
|
"$ROOT/scripts/ci/test-impact-bundle.sh" --fetch "$base" "$WORK/bundle" || bundle_rc=$?
|
|
if [ "$bundle_rc" -eq 0 ]; then
|
|
artifact=(--artifact "$WORK/bundle" --artifact-commit "$base" --artifact-verified)
|
|
elif [ "$bundle_rc" -ne 3 ]; then
|
|
# No bundle means no coverage: more tests run, never fewer. Say so.
|
|
echo "::warning::test-impact bundle fetch failed (exit $bundle_rc): selecting without coverage"
|
|
fi
|
|
started=$(date +%s)
|
|
"$BINARY" test-impact select --repo "$ROOT" --base "$base" --work "$WORK/engine" \
|
|
--platform "$PLATFORM" ${artifact[@]+"${artifact[@]}"} > "$OUT/selection-engine.json"
|
|
elapsed=$(( $(date +%s) - started ))
|
|
|
|
python3 - "$OUT/selection-engine.json" "$elapsed" "${#artifact[@]}" > "$OUT/summary-engine.md" <<'PY'
|
|
import json, sys
|
|
answer = json.load(open(sys.argv[1]))
|
|
coverage = answer.get("receipt", {}).get("coverage", {})
|
|
lines = ["### test-impact engine", "",
|
|
f"- decision: `{answer.get('decision')}` in {sys.argv[2]} s",
|
|
f"- team artifact: {'fetched' if sys.argv[3] != '0' else 'none'}; coverage "
|
|
f"`{coverage.get('state')}` {coverage.get('rejection_reasons') or ''}"]
|
|
for lane in answer.get("lanes") or []:
|
|
suites = lane.get("suites") or []
|
|
whole = [s["suite"] for s in suites if s.get("mode") == "whole"]
|
|
narrowed = {s["suite"]: len(s.get("tests") or []) for s in suites if s.get("mode") == "tests"}
|
|
lines.append(f"- lane `{lane.get('lane')}` ({lane.get('decision')}): {len(whole)} whole, "
|
|
f"{len(narrowed)} narrowed ({sum(narrowed.values())} tests)")
|
|
print("\n".join(lines))
|
|
PY
|
|
|
|
if [ -n "$GATE" ]; then
|
|
mkdir -p "$GATE"
|
|
selection=all
|
|
filter_rc=0
|
|
python3 "$ROOT/scripts/test-impact/selection.py" filter "$OUT/selection-engine.json" \
|
|
--lane unit --out "$GATE/test-only.txt" --optional-out "$GATE/optional-suites.txt" ||
|
|
filter_rc=$?
|
|
if [ "$filter_rc" -eq 0 ]; then
|
|
selection=narrowed
|
|
echo "- PR CI runs $(wc -l < "$GATE/test-only.txt" | tr -d ' ') selection tokens" \
|
|
>> "$OUT/summary-engine.md"
|
|
else
|
|
rm -f "$GATE/test-only.txt" "$GATE/optional-suites.txt"
|
|
echo "- PR CI runs every test (selection exit $filter_rc)" >> "$OUT/summary-engine.md"
|
|
fi
|
|
if [ -n "${GITHUB_OUTPUT:-}" ]; then
|
|
echo "selection=$selection" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
fi
|
|
cat "$OUT/summary-engine.md"
|
|
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
cat "$OUT/summary-engine.md" >> "$GITHUB_STEP_SUMMARY"
|
|
fi
|