test(repro): honest-RED #581, honest safety-net #431 (false-guard audit)

repro_issue581 was a FALSE GUARD. A 3-node fixture over 150 search_graph calls
cannot move process RSS (observed factor=1.00), so rss_end <= 3.0x rss_warmup
passed even on the leaking build, and an explicit PASS() escape fired when RSS
read 0. #581 is OPEN: there is no SQLITE_CHECKPOINT_TRUNCATE anywhere and
cbm_mem_collect is never called in the query path (only after index/delete). The
green was a lie. Replace the vacuous assertion with an honest FAIL that documents
the leak is unfixed and not reproducible in this tier (needs a real
WAL/committed-pages reproduction tier plus the fix).

repro_issue431 is actually FIXED: install_vscode_profile_configs (cli.c:3211)
scans Code/User/profiles/ and plans a per-profile mcp.json, and the test passes
via the real assertion. Its detection-unmet branch returned a VACUOUS PASS that
would hide a future regression; change it to SKIP_PLATFORM so a host where
detection cannot fire is honestly inconclusive, not falsely green.

Repro board: 311 passed, 1 failed (the honest #581 RED).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-06-28 19:53:44 +02:00
1 parent 006ea01d60
commit 25d2fa8af2
2 files changed
+34 -42

No files matched your search

+10 -7
View File
@@ -109,14 +109,17 @@ TEST(repro_issue431_vscode_profile_inherits_mcp_json) {
/* --- Precondition: VSCode is detected --- */
cbm_detected_agents_t agents = cbm_detect_agents(tmpdir);
if (!agents.vscode) {
/* Detection failed in the temp tree — adjust path derivation.
* On non-Apple Linux the detection reads cbm_app_config_dir() which
* is process-global (not home-relative), so detection may return false
* for a synthetic tmpdir home. The bug still exists, but we cannot
* demonstrate it via the plan-based oracle without detection firing.
* Mark the test as an expected skip on this platform/config. */
/* #431 IS FIXED: install_vscode_profile_configs() (cli.c:3211) scans
* Code/User/profiles/ and plans a per-profile mcp.json, so the assertion
* below passes as a genuine regression guard whenever detection fires
* (which it does for this fixture). This branch is only reached if
* cbm_detect_agents() cannot see the fixture home on some platform — in
* which case the fix cannot be VERIFIED here. Skip honestly rather than
* vacuously PASS (would hide a future regression) or FAIL (would red a
* fixed bug). */
th_rmtree(tmpdir);
PASS(); /* precondition unmet — non-blocking; bug still open */
SKIP_PLATFORM("VSCode detection did not fire for the synthetic fixture "
"home; cannot verify the #431 per-profile install here");
}
/* --- Run the install plan oracle (dry-run, no mutations) --- */
+24 -35
View File
@@ -245,46 +245,35 @@ TEST(repro_issue581_query_rss_stable) {
free(args);
rh_cleanup(&lp, store);
// If RSS is not measurable (cbm_mem_rss() returns 0 and no Linux fallback),
// skip the growth assertion -- an unmeasurable RSS cannot produce a
// meaningful signal. This avoids a false PASS masking a real leak on
// platforms where our RSS API is unavailable.
if (rss_warmup == 0 || rss_end == 0) {
printf(" NOTE: RSS not measurable on this platform/build; "
"growth assertion skipped (inconclusive, not a pass)\n");
PASS();
if (rss_warmup > 0 && rss_end > 0) {
printf(" rss_warmup_kb=%zu rss_end_kb=%zu factor=%.2f threshold=%.1f\n", rss_warmup / 1024,
rss_end / 1024, (double)rss_end / (double)rss_warmup, LEAK_FACTOR);
} else {
printf(" NOTE: RSS not measurable on this platform/build\n");
}
printf(" rss_warmup_kb=%zu rss_end_kb=%zu factor=%.2f threshold=%.1f\n",
rss_warmup / 1024, rss_end / 1024,
(double)rss_end / (double)rss_warmup,
LEAK_FACTOR);
// PRIMARY assertion: end RSS must not exceed LEAK_FACTOR x warmup RSS.
// HONEST RED — this guard is currently VACUOUS and #581 is OPEN.
//
// RED condition (current code):
// SQLite WAL + mimalloc retained pages grow each iteration.
// Over 150 iterations the cumulative growth pushes rss_end above
// LEAK_FACTOR * rss_warmup.
// ASSERT fires -> RED.
// This fixture CANNOT reproduce the leak: a 3-node graph over 150
// search_graph calls allocates far too little to move process RSS (observed
// factor=1.00), so the old "rss_end <= 3.0 x rss_warmup" assertion passed
// even on the leaking build. A green here would mean "leak fixed" while the
// leak is unfixed — a false guard that violates the tests-are-guards rule
// (green <=> fixed). So it stays RED.
//
// GREEN condition (after fix):
// Periodic compaction (cbm_mem_collect + WAL TRUNCATE checkpoint) keeps
// rss_end near rss_warmup. factor stays <1.5 comfortably.
// Turning this GREEN legitimately requires BOTH:
// (a) a real reproduction tier — a long-running MCP session issuing
// thousands of ops against a LARGE graph, measuring the SQLite WAL
// file size and mimalloc committed pages DIRECTLY (not process-RSS
// jitter) so the monotonic growth is actually observable; AND
// (b) the fix — periodic SQLITE_CHECKPOINT_TRUNCATE + cbm_mem_collect() in
// the MCP query loop / idle eviction (see the header + #581).
//
// We report the ratio in the failure message so the fixer can see the
// growth slope without needing a profiler.
size_t rss_limit = (size_t)(rss_warmup * LEAK_FACTOR);
if (rss_end > rss_limit) {
printf(" BUG #581 reproduced: RSS grew %.2fx after %d search_graph calls "
"(warmup=%zu kB end=%zu kB limit=%zu kB)\n",
(double)rss_end / (double)rss_warmup,
ITER_TOTAL - ITER_WARMUP,
rss_warmup / 1024, rss_end / 1024, rss_limit / 1024);
}
ASSERT(rss_end <= rss_limit);
PASS();
// Until both land this is an honest "not fixed / not provable here" RED, not
// a false green.
FAIL("bug #581 (query-path memory leak) is OPEN and cannot be reproduced in "
"this fixture (RSS factor ~1.0 even when leaking) — needs a real "
"WAL/committed-pages reproduction tier plus the fix");
}
// -- Suite ------------------------------------------------------------------