ci: non-product PRs (T0b) run the contract steps too

Under R1, a T0b push (test-infrastructure/, lint config, graph-ui, memwaste
tooling) selected lint and no test leg, so the static contract steps that
police exactly those files -- the VM scripts, the compose and run.sh wiring,
the venue-parity rules -- ran nowhere. Replayed over September, 13 of the 18
T0b pushes ran no contract step at all.

The R1 table now gives T0b lint + contracts, so those PRs get the
contracts job (scripts/test.sh --contracts-only) as T0a PRs do. The
fixture table's six T0b cases expect it, and the history replay asserts
that every push runs the contract steps (the contracts lane or any test
leg). Comments that described the job as docs-only are updated.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-09-30 23:48:22 +02:00
1 parent d3efe3baec
commit 16e2732819
7 files changed
+32 -18

No files matched your search

+4 -4
View File
@@ -261,10 +261,10 @@ jobs:
${{ runner.temp }}/test-impact/changed-files.txt
if-no-files-found: warn
# ── Docs-only PRs (tier T0a) select no test leg, so this runs the static
# contract steps of the canonical test leg on their own (they police
# docs surfaces too: language counts, version metadata). Every test leg
# runs the same steps, so the job exists only when no leg does. ──
# ── Docs-only and non-product PRs (tiers T0a, T0b) run the static contract
# steps of the canonical test leg on their own: they police docs
# surfaces (language counts, version metadata), test-infrastructure/ and
# lint config too. Every test leg runs the same steps. ──
contracts:
needs: [changes]
if: ${{ !cancelled() && needs.changes.result == 'success' && contains(fromJSON(needs.changes.outputs.lanes), 'contracts') }}
+1 -1
View File
@@ -17,7 +17,7 @@ codes) and rejects unknown flags with exit 2 + `Please consult --help.`
| leg | entry | what a run gives you |
|---|---|---|
| **test** | `test.sh` | DEFAULT = the venue leg: static contracts (Step 0a–0r) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. `--suites a,b` = iteration mode (incremental rebuild, subset, seconds). `--tsan` = the ThreadSanitizer leg. `--contracts-only` = the static contract steps alone (PR CI's docs-only job). CLANGARM64 gets CI's trap-UBSan flags automatically. |
| **test** | `test.sh` | DEFAULT = the venue leg: static contracts (Step 0a–0r) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. `--suites a,b` = iteration mode (incremental rebuild, subset, seconds). `--tsan` = the ThreadSanitizer leg. `--contracts-only` = the static contract steps alone (PR CI's contracts job for docs-only and non-product changes). CLANGARM64 gets CI's trap-UBSan flags automatically. |
| **package wrappers** | `ci/test-package-wrappers.sh` | Go, npm, and PyPI runtime-set publication/lock suites on the current host. CI runs the same entry on Linux and Windows so both platform lock implementations gate. |
| **build** | `build.sh` | CLEAN production runtime set (native executable + authenticated integration asset; `--with-ui` adds one content-addressed UI pack). ccache via `env.sh` makes repeats fast; `CCACHE_COMPILERCHECK=content` guarantees a hit is byte-identical to a cold compile — never stale. `--version`, `STATIC=1`, `BUILD_DIR=`. |
| **lint** | `lint.sh` | clang-tidy + cppcheck + clang-format (+ no-skips policy). `--ci` = the CI gate set (no clang-tidy). Drives the same make targets as `make lint`/`lint-ci`. |
+3 -2
View File
@@ -251,14 +251,15 @@ FULL = {"codeql-gate", "diag", "license-gate", "lint", "lint-mem", "lsan-macos",
"smoke-mac", "smoke-ubuntu", "smoke-win", "tsan-arm", "tsan-mac", "tsan-x86",
"unix-arm64", "unix-macos-intel", "unix-macos14", "unix-x86", "windows",
"windows-guards"}
UNIVERSE = FULL | {"contracts"} # contracts: the docs-only job; every test leg runs them too
UNIVERSE = FULL | {"contracts"} # contracts: the T0a/T0b job; every test leg runs them too
LINT = {"lint", "lint-mem"}
CORE_LEGS = {"unix-x86", "unix-macos14", "windows", "shard-completeness"}
TEST_LEGS = {"unix-x86", "unix-arm64", "unix-macos14", "unix-macos-intel", "windows"}
ALWAYS = {"security-static"}
R1 = {
"T0a-docs": {"contracts"},
"T0b-nonproduct": LINT,
# The contract steps police test-infrastructure/ and lint config too.
"T0b-nonproduct": LINT | {"contracts"},
"T1-tests": LINT | CORE_LEGS,
"T2-extract": LINT | CORE_LEGS | {"codeql-gate", "tsan-x86", "smoke-ubuntu"},
"T3-mid": LINT | CORE_LEGS | {"codeql-gate", "tsan-x86", "smoke-ubuntu"},
+16 -3
View File
@@ -9,7 +9,9 @@
# its documented deviation, and a deviation may only ever raise the tier
# (the fail-safe direction) -- never quietly select less than was measured;
# 2. every real failure of the month (22, fixtures/2026-09-real-failures.tsv)
# had its catching lane selected for its push.
# had its catching lane selected for its push;
# 3. every push runs the static contract steps (the contracts lane or any
# test leg) -- they police docs, test-infrastructure/ and lint config.
# Pure data, one selector process (--batch over the distinct file sets).
#
# Usage: scripts/test-impact/replay-selector.sh [repo-root]
@@ -17,7 +19,7 @@ set -euo pipefail
case "${1:-}" in
-h | --help)
sed -n '2,15p' "$0" | sed 's/^# \{0,1\}//'
sed -n '2,17p' "$0" | sed 's/^# \{0,1\}//'
exit 0
;;
-*)
@@ -111,6 +113,16 @@ for run_id, pr, sha, fileset, analysis, expected, note in rows("2026-09-pushes.t
else:
deviated += 1
# The static contract steps police docs, test-infrastructure/ and lint config,
# so every push runs them: in the contracts job or inside any test leg.
CONTRACT_CARRIERS = {"contracts", "unix-x86", "unix-arm64", "unix-macos14",
"unix-macos-intel", "windows"}
uncovered = collections.Counter(d["tier"] for d in pushes.values()
if not set(d["lanes"]) & CONTRACT_CARRIERS)
for tier, count in sorted(uncovered.items()):
failures.append(f"{count} {tier} push(es) run no contract step (neither the "
f"contracts lane nor a test leg)")
caught = 0
for run_id, attempt, pr, sha, lane, why in rows("2026-09-real-failures.tsv", 6):
decision = pushes.get(run_id)
@@ -129,7 +141,8 @@ if failures:
sys.exit(1)
print(f"selector history replay OK: {len(pushes)} pushes, tier as expected for all "
f"({matched} equal to the analysis, {deviated} documented fail-safe deviations); "
f"{caught} real failures, every catching lane selected")
f"{caught} real failures, every catching lane selected; every push runs the "
f"contract steps")
print(" pushes per tier: " + ", ".join(f"{t} {tiers[t]}" for t in TIERS))
print(f" full runs: {sum(1 for d in pushes.values() if d['full'])}")
PY
+1 -1
View File
@@ -32,7 +32,7 @@ Modes:
--contracts-only
Only the static contract steps (Step 0*) of the default leg:
no compiler, no build, no suites. PR CI runs this for
docs-only changes, where no test leg is selected.
docs-only and non-product changes (tiers T0a, T0b).
Options:
--arch ARCH Force target arch (arm64 | x86_64), e.g. under Rosetta.
+6 -6
View File
@@ -5,12 +5,12 @@
# "FULL -a -b" = every PR lane except a and b).
docs-only README.md docs/BENCHMARK.md T0a-docs false contracts security-static
issue-template-is-docs .github/ISSUE_TEMPLATE/bug.yml T0a-docs false contracts security-static
local-infra test-infrastructure/run.sh T0b-nonproduct false lint lint-mem security-static
lint-config scripts/memory-core-baseline.txt T0b-nonproduct false lint lint-mem security-static
graph-ui-adds-guards graph-ui/src/App.tsx T0b-nonproduct false lint lint-mem security-static smoke-ubuntu windows-guards
memwaste-tools-add-memwaste scripts/memwaste.sh T0b-nonproduct false lint lint-mem memwaste security-static
non-gating-script-runs-contracts-leg scripts/benchmark-index.sh T0b-nonproduct false lint lint-mem security-static shard-completeness unix-x86
test-impact-tooling scripts/test-impact/replay-selector.sh T0b-nonproduct false lint lint-mem security-static shard-completeness unix-x86
local-infra test-infrastructure/run.sh T0b-nonproduct false contracts lint lint-mem security-static
lint-config scripts/memory-core-baseline.txt T0b-nonproduct false contracts lint lint-mem security-static
graph-ui-adds-guards graph-ui/src/App.tsx T0b-nonproduct false contracts lint lint-mem security-static smoke-ubuntu windows-guards
memwaste-tools-add-memwaste scripts/memwaste.sh T0b-nonproduct false contracts lint lint-mem memwaste security-static
non-gating-script-runs-contracts-leg scripts/benchmark-index.sh T0b-nonproduct false contracts lint lint-mem security-static shard-completeness unix-x86
test-impact-tooling scripts/test-impact/replay-selector.sh T0b-nonproduct false contracts lint lint-mem security-static shard-completeness unix-x86
tests-only tests/test_store.c T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows
test-fixture tests/fixtures/cpp_include/a.h T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows
contract-test tests/test_version_metadata_contract.sh T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows
Can't render this file because it contains an unexpected character in line 3 and column 58.
+1 -1
View File
@@ -7,7 +7,7 @@
# (and "standard"): those runs execute every job, sharded, exactly as
# before lane selection. Only pr.yml asks for the PR shard profile.
# 2. pr.yml feeds the selector's lanes to every reusable workflow, runs the
# docs-only contracts job, and hands ci-ok both the selection and the
# T0a/T0b contracts job, and hands ci-ok both the selection and the
# run's job list (actions: read) -- ci-ok needs `contracts` but never the
# report-only memwaste / shadow jobs, and the shadow is continue-on-error.
# 3. every lane of scripts/ci/select-lanes.sh --list-lanes is served by a