mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-04 10:48:20 +00:00
ci(test-impact): give the engine full history; publish maps with an incomplete suite
The test-impact engine certifies the history it diffs and refuses a shallow
clone ("Graft or shallow state prevents history certification"). The shadow
job, the gate-mode select-tests job and the main-branch producer all checked
out with fetch-depth 2. So every shadow answer since #2458 was run_all:
GIT_UNAVAILABLE, then POLICY_UNAVAILABLE, because the merge base's
.codebase-memory.json is read through the same git facts. That was 24 of 24
measured PR runs. All three jobs now check out full history (about 400 MiB,
these jobs only).
The producer never published a bundle either. coverage-map.py exits 1 when a
suite fails under coverage: the map is still written and that suite's tests
are incomplete. test-impact-publish.sh ran it under set -e, so the publish
aborted. The failing suite was test_impact_git. Its hostile-diff probe
re-enters the runner as a child, which inherited CBM_TEST_COVERAGE_DIR,
started a coverage run of its own, and refused the per-test LLVM_PROFILE_FILE
it inherits.
Two changes:
- The probe now clears CBM_TEST_COVERAGE_DIR, as it already clears
CBM_TEST_ONLY_FILE. The child's profile then goes to the spawning test, like
any other child process.
- publish treats exit 1 as a written, conservative map (incomplete tests run
whole), with a warning that names the suite. It fails only on exit 2.
Proof:
- Depth-2 clone at f9ce3865: test-impact select answers run_all
[GIT_UNAVAILABLE, POLICY_UNAVAILABLE] with "Graft or shallow state ...", as
in CI. Same commit with full history: graph certified, run_all
[RUNNER_REACHED] (the change reaches the runner's main).
- Coverage runner (Homebrew LLVM, macOS), test_impact_git: before, 43 passed
and 2 failed ("a coverage run starts with LLVM_PROFILE_FILE=/dev/null");
after, 45 passed. coverage-map.py --suite test_impact_git: 45 tests,
0 incomplete, exit 0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
1 parent
c202539e21
commit
674a731fcc
4 files changed
+28
-7
No files matched your search
@@ -65,8 +65,9 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
# The PR merge commit and its base parent: the change to select for.
|
||||
fetch-depth: 2
|
||||
# Full history: the engine certifies the history it diffs and refuses
|
||||
# a shallow clone, so a depth-2 checkout always answered `all`.
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
- name: Build this checkout's engine
|
||||
run: |
|
||||
@@ -281,8 +282,11 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
# The PR merge commit and its base parent: the diff to predict for.
|
||||
fetch-depth: 2
|
||||
# Full history: the engine (test-impact select) certifies the history
|
||||
# it diffs and refuses a shallow clone ("Graft or shallow state
|
||||
# prevents history certification"), which made every shadow answer
|
||||
# run_all (GIT_UNAVAILABLE). About 400 MiB, this job only.
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
- name: Fetch the latest released binary
|
||||
id: release
|
||||
|
||||
@@ -38,7 +38,9 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
fetch-depth: 2
|
||||
# Full history: `test-impact select/publish` certify the history they
|
||||
# read and refuse a shallow clone.
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install build deps and LLVM 21 (coverage toolchain)
|
||||
|
||||
@@ -56,9 +56,19 @@ fi
|
||||
map() { # out [suite...]
|
||||
local out="$1"; shift
|
||||
local args=()
|
||||
local rc=0
|
||||
for suite in "$@"; do args+=(--suite "$suite"); done
|
||||
python3 "$ROOT/scripts/test-impact/coverage-map.py" --runner "$RUNNER" --out "$out" \
|
||||
${LLVM_BIN:+--llvm-bin "$LLVM_BIN"} ${args[@]+"${args[@]}"}
|
||||
${LLVM_BIN:+--llvm-bin "$LLVM_BIN"} ${args[@]+"${args[@]}"} || rc=$?
|
||||
# 1 = a suite failed or timed out under coverage: the map is still written
|
||||
# and that suite's tests are `incomplete`, which only ever selects MORE
|
||||
# tests (an incomplete test runs whole). Publish it, but say which suite.
|
||||
# 2 = usage or toolchain error: no trustworthy map, fail.
|
||||
if [ "$rc" -eq 1 ]; then
|
||||
echo "::warning::coverage map: a suite failed under coverage; its tests stay incomplete ($(python3 -c 'import json,sys; print(", ".join(s["suite"] for s in json.load(open(sys.argv[1]))["suites"] if s["exit"] != 0))' "$out/meta.json"))"
|
||||
return 0
|
||||
fi
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
OBSERVED=""
|
||||
|
||||
@@ -64,9 +64,14 @@ int tf_maybe_run_git_facts_diff_probe(int argc, char **argv) {
|
||||
* Re-enter the existing exact-diff test; the parent process remains untouched.
|
||||
* An inherited CBM_TEST_ONLY_FILE (a narrowed CI run) is cleared: the runner
|
||||
* unions it with CBM_TEST_ONLY, and one naming this suite would re-run the
|
||||
* spawning test, which spawns again. */
|
||||
* spawning test, which spawns again. An inherited CBM_TEST_COVERAGE_DIR (a
|
||||
* coverage-map run) is cleared too: with it the child would start a coverage
|
||||
* run of its own, which refuses the per-test LLVM_PROFILE_FILE it inherits.
|
||||
* Without it the child's profile goes to that file, credited to the spawning
|
||||
* test like any other child process. */
|
||||
if (cbm_setenv("GIT_DIFF_OPTS", "--unified=999", 1) != 0 ||
|
||||
cbm_setenv("CBM_TEST_ONLY_FILE", "", 1) != 0 ||
|
||||
cbm_unsetenv("CBM_TEST_COVERAGE_DIR") != 0 ||
|
||||
cbm_setenv(
|
||||
"CBM_TEST_ONLY",
|
||||
"test_impact_git:test_git_facts_diff_uses_merge_base_zero_context_and_nul_metadata",
|
||||
|
||||
Reference in new issue
Block a user