Files
codebase-memory-mcp/tests
Anton Standrik a2ac0540c1 fix(test): fail the index benchmark when its index is refused
bench_finish keeps the private root on a zero exit status, but the index
call swallowed its failure (`|| echo '{"error":"index failed"}'`), so a
refused index exited 0 and, with CBM_BENCH_KEEP_RUNTIME set, kept an
empty root and wrote runtime-root.txt for an index that does not exist.

benchmark-index.sh now records the CLI's exit status, keeps 00-index.json
valid JSON on failure ({"error":"index failed","exit":N}), and exits 1
once every per-run file — the three timing files included — is written,
when the index failed or reported no project (the search-graph twin's own
test). bench_finish then cleans up as its comment always promised, and
the caller's loop notices. The index call keeps its stderr in a
run-private temp file, the twin's shape, and the failure prints it under
`--- index stderr ---` with the first 500 bytes of the response: for a
one-shot CLI, stderr is the only channel a refusal is reported on. A
stale runtime-root.txt from an earlier run in the same results directory
is removed at start, so a failed run leaves no handoff at all.

The contract test's fixture answers `cli index_repository` with a
minimal envelope when CBM_BENCH_PROBE_INDEX_OK is set, so the keep run
covers the successful-index handoff and asserts project.txt; the
unflagged run asserts a non-zero exit and the surfaced cause after a
refused index; a new keep run with a refused index asserts a non-zero
exit, the timing files, no surviving root and no handoff, a pre-seeded
stale one included. Against the previous head the test fails on
"benchmark-index exited 0 after a refused index".

Signed-off-by: Anton Standrik <astandrik@yandex-team.ru>
2026-10-27 12:57:37 +03:00
..