mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-06 17:49:47 +00:00
Martin Vogel
dbc6ac486b
Five suites (grammar_regression grammar_labels pipeline lang_contract grammar_probe_e) were excluded from the MSan lane for "stack-overflow under instrumentation". That was never deep recursion. MSan aborted while REPORTING real uninitialized reads: - every abort was on the main thread. MSan's "T<n>" is the OS thread id and equalled the pid in all five x86-64 aborts (CI job 91868259558); the old block read it as a worker-thread ordinal, so the worker-stack knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything; - with the default slow unwinder (the image's MSan-instrumented libunwind) a report exhausted the main thread's 256 MiB stack, and the overflow handler then failed with "nested bug in the same thread"; - with fast_unwind_on_fatal=1 the same test prints the use-of-uninitialized-value report instead (measured on one tree and test: slow unwind aborts at the stack limit, fast unwind reports wit/scanner.c:13). The two reads are fixed in the previous commits of this series, so: - MSAN_OPTIONS defaults to fast_unwind_on_fatal=1 (the build already keeps frame pointers); - MSAN_EXCLUDE is down to cli, whose install-path failure is a separate, still-undiagnosed cause; - the known-red block, the header (it still claimed CI runs without exclusions) and the MSAN_ORIGINS and stack comments say what is true; - the partial-lane warning and the exclude-name typo check lived in a branch behind `elif true` that never ran. They now run in the per-suite path, the check as a case match instead of a grep -q pipe that pipefail can misreport; - the comments in scripts/ci/msan-lane.sh and above the test-msan job in _test.yml no longer say the five suites overflow. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
59 lines
2.4 KiB
Bash
Executable File
59 lines
2.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# MemorySanitizer lane — the canonical entry, used by every venue.
|
|
#
|
|
# MSan needs every linked library instrumented or reads of memory those
|
|
# libraries wrote report as uninitialized, so the lane runs inside an image
|
|
# carrying an MSan-built libc++/libc++abi/libunwind and zlib
|
|
# (test-infrastructure/Dockerfile.msan). Building that image is the expensive
|
|
# part, hence the buildx layer cache.
|
|
#
|
|
# Locally the same lane is reached through the compose service:
|
|
# ./test-infrastructure/run.sh msan
|
|
# which shares scripts/msan.sh with this path — one harness, both venues.
|
|
#
|
|
# Usage: scripts/ci/msan-lane.sh [build|run|all] (default: all)
|
|
set -euo pipefail
|
|
|
|
cd "$(dirname "$0")/../.."
|
|
|
|
CACHE_DIR="${MSAN_BUILDX_CACHE:-/tmp/.buildx-msan}"
|
|
IMAGE="${MSAN_IMAGE:-cbm-msan:ci}"
|
|
BUILDER="${MSAN_BUILDER:-msan-builder}"
|
|
MODE="${1:-all}"
|
|
|
|
build_image() {
|
|
echo "=== MSan image (buildx, cached layers) ==="
|
|
# A builder may already exist from a previous step or a retried job.
|
|
docker buildx create --use --name "$BUILDER" 2>/dev/null || docker buildx use "$BUILDER"
|
|
docker buildx build \
|
|
--cache-from "type=local,src=$CACHE_DIR" \
|
|
--cache-to "type=local,dest=$CACHE_DIR,mode=max" \
|
|
-f test-infrastructure/Dockerfile.msan \
|
|
-t "$IMAGE" --load test-infrastructure/
|
|
}
|
|
|
|
run_suite() {
|
|
# MSAN_EXCLUDE is intentionally NOT passed, so the container inherits the
|
|
# single authoritative list defined in scripts/msan.sh (which documents each
|
|
# excluded suite and the cause it is excluded for).
|
|
#
|
|
# This leg used to force it empty to settle whether the local arm64
|
|
# exclusions were an aarch64 artifact. They were not: `cli` fails for an
|
|
# install-path reason, `incremental` was a shadow-memory RSS artifact (now
|
|
# fixed in the test), and the five suites that "overflowed" on both
|
|
# architectures were hiding real uninitialized reads whose reports crashed
|
|
# under the slow unwinder. With fast unwinding (scripts/msan.sh) those reports
|
|
# print, the reads are fixed, and the five suites are back in the lane.
|
|
# Only `cli` is still excluded, and scripts/msan.sh warns that the lane is
|
|
# partial while anything is.
|
|
echo "=== MSan suite (exclusions per scripts/msan.sh) ==="
|
|
docker run --rm -v "$PWD:/src" -w /src "$IMAGE"
|
|
}
|
|
|
|
case "$MODE" in
|
|
build) build_image ;;
|
|
run) run_suite ;;
|
|
all) build_image; run_suite ;;
|
|
*) echo "usage: $0 [build|run|all]" >&2; exit 2 ;;
|
|
esac
|