mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-04 10:48:20 +00:00
test(cli): stop a negative probe from spending 15s proving silence
The macos-15-intel leg killed the cli suite at its 900s wall clock while reporting 7537 passed, 0 failed. Two separate things were wrong. The suite was misclassified. cli spends 497s of the 900s default on macos-14, the FASTEST macOS runner, while macos-15-intel in the same matrix is 2.4-3.6x slower on comparable suites (daemon_runtime 842s vs 349s, stack_overflow_b 277s vs 76s). 497s at that ratio cannot fit. daemon_runtime, at 842s on that same runner, passes only because it was already in SLOW_SUITES. cli belongs there too -- it is deterministic and simply large, which is the only thing that list is allowed to mean. The drain test was also waiting on a clock instead of on the system. Profiling the suite per test (315 tests, 300s wall / 166s cpu) put cli_install_into_host_namespace_still_drains_host_cohort at 42s wall for 19s of cpu -- 23s idle, the largest single idle block, where the next worst was 11s. The existing cli_install_force_quiesces_active_cohort_before_replacing_binary drains a cohort too and idles 0.5s, which is what pointed at the difference. The cost was the host_serving probe. Its generous budget exists for the POSITIVE question -- is this daemon still up? -- where a slow reply on a loaded runner must not be misread as drained; that fixed a real flake and is kept. Asked in the negative it inverts: no reply is coming, so the full 15s is spent establishing silence and ASSERT_FALSE is decided by the timeout expiring rather than by the daemon's behaviour. The drain is already proven positively in that test -- install returns 0 only after the activation completed, and the host child exits CLI_SCOPE_HOST_DRAINED -- so the probe only has to confirm it. Also shortens the drained child's teardown budget, which retried service_free / lease_release / manager_free against 10s on a path where the activation had already torn the service down. Behaviour at the deadline is unchanged; it is reached sooner when nothing is wedged. Measured at only -3s on its own, kept because it is correct and free. Local, same machine, ASan+UBSan: the drain test 42.0s -> 25.8s (-38%), the suite 300s -> 279s, 315 passed before and after -- no test removed, no assertion weakened. PR #2188, which runs the same leg without these two tests, passed macos-15-intel in 37m34s, so the suite was not already over budget on main. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
1 parent
9a460b389c
commit
fa99ff64eb
2 files changed
+58
-8
No files matched your search
@@ -26,7 +26,19 @@ SUITE_NAME = re.compile(r"^[a-z0-9_]+$")
|
||||
SUMMARY = re.compile(r"^ (?P<passed>[0-9]+) passed")
|
||||
FAILED = re.compile(r"(?:^|, )(?P<failed>[0-9]+) failed")
|
||||
SKIPPED = re.compile(r"(?:^|, )(?P<skipped>[0-9]+) skipped")
|
||||
SLOW_SUITES = frozenset(("incremental", "store_arch", "daemon_runtime"))
|
||||
# Suites whose honest runtime does not fit the default per-suite budget, and so
|
||||
# get --slow-timeout instead. This is a statement about SIZE, never about
|
||||
# flakiness: every suite here is deterministic and simply long, and a racy suite
|
||||
# must be made deterministic rather than given more clock.
|
||||
#
|
||||
# `cli` joined the list because the classification had gone stale, not because
|
||||
# anything regressed. It spends 497s of the 900s default on macos-14 -- the
|
||||
# FASTEST macOS runner -- while the macos-15-intel runner in the same matrix is
|
||||
# 2.4-3.6x slower on comparable suites (daemon_runtime 842s vs 349s,
|
||||
# stack_overflow_b 277s vs 76s). 497s at that ratio cannot fit, so the suite was
|
||||
# killed at 900s and reported as hung. daemon_runtime, at 842s on that same
|
||||
# runner, survives only because it was already listed here.
|
||||
SLOW_SUITES = frozenset(("incremental", "store_arch", "daemon_runtime", "cli"))
|
||||
POLL_SECONDS = 0.05
|
||||
|
||||
# WHY: the Windows descendant probe below is a cold `powershell.exe` + CIM
|
||||
|
||||
Reference in new issue
Block a user