nprasad2077/codebase-memory-mcp
1
0 Fork 0
mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-04 18:58:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,306 Commits 624 Branches 56 Tags
main
Commit Graph
8 Commits
Author SHA1 Message Date
Martin Vogel 57b557ced8 fix(test-impact): close the role table's fail-opens, follow literals through macros 
The first commit of this branch is the Smart CI change as of the rebase
onto 9f85f104. A review of it found places where the engine could run
fewer tests than a change needs; each now runs everything or selects the
right tests, with a test that fails when its fix is reverted.

Engine:
- A `#define NAME "literal"` line counted as naming a role but selected
  nobody, so the tests that start the blocking-git probe through their
  marker macros would not have run. The scan now follows NAME, as a whole
  word, to the code that uses it, through chains of up to 4 macros; a
  define it cannot follow runs everything.
- main dispatches through the role table, so no edge leads from main to a
  handler. A table entry the engine cannot read, or whose handler is not
  exactly one function, was skipped; it now runs everything.
- In a runner file, only a test case names a role's literal. The runner's
  own helper that reads the marker no longer counts as a spawner, so a
  role only the runner names runs everything again.
- A handler reached first by a parse gap or by the spawn lift was never
  lifted. Role lifts and parse gaps now run to a fixpoint, before the
  runner check and again after the spawn lift.
- A reached sink that main calls (the blocking-git probe) counted as a way
  into main and hid the real one: #2644 now names the start-up path main
  -> cbm_index_supervisor_capture_build_fingerprint.
- No pointer past the end of a role line; the RUNNER_REACHED note clamps
  its prefix.

CI:
- pkg/npm/README.md, which test_cli reads, runs the test legs instead of
  the docs tier.
- The build rule lists the harness scripts the test legs run:
  run-test-wave.py, path-safety.sh, clean.sh.
- The revert guard rejects the ancestor's coverage when a diff fails.
- The vetoed selection is uploaded (test-selection-vetoed).

Docs: SMART_CI.md states the new fail-safe cases, the version that lacks
the new config keys, the selection base, and that the narrowed leg builds
with gcc while the coverage comes from clang.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-17 20:44:27 +02:00
Martin Vogel 7f6f567840 wip: Smart CI narrows PR CI for real (pre-rebase checkpoint) 
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-17 14:48:52 +02:00
Martin Vogel 7aa384b401 fix(graph): platform variants keep their path QNs and are related by variant_key 
42a93202 gave a non-static C-family function under a platform directory
(arch/x86/, src/win/, ...) or file suffix a platform-neutral QN, so the
platform variants of one function shared a node. On the Linux kernel that
merged only 644 of 17,413 such definitions; the other 16,769 lost their
path for nothing (`arch/arm64/mm/init.c` mem_init became
`arch.mm.init.mem_init`, and a lookup by its path QN found nothing),
arches missing from the token list kept their paths, and calls bound to
merged nodes produced impossible arch -> other-arch CALLS.

Every definition keeps its path-derived QN again (the v0.11.0 identity).
The platform-neutral QN is now a key: definitions carrying one get a
`variant_key` node property, and nodes of a project sharing a key are one
variant group. The test-impact walk (store_impact `follow_variant_keys`,
enabled by the test-impact engine) treats a reached variant as reaching its
whole group, at seeding and at every step, so a change confined to the
Windows variant still selects the tests that reach the POSIX variant's
callers. The key is a pure function of a definition's own path, name and
linkage: incremental re-parsing of the changed file writes what a full
index writes. Same-QN variants (#if/#else, Rust cfg, Go build tags,
.c/.h pairs) and their `variants` property are unchanged. No new edge type.

Proof (macOS arm64):
- Linux kernel subset (arch, include, kernel, mm, lib; 27,005 files):
  arch functions whose QN is not under their own path 12,873 -> 0;
  arch/A -> arch/B CALLS 10,353 -> 7,101; 581 variant groups (1,815
  members), each one of the former cross-file merges; the path QN of
  arch/arm64's mem_init resolves.
- This repository: 13 groups (test_impact_tree_{posix,windows},
  mem_override_{posix,win}); before, 22 of 49 platform-path functions had a
  stripped QN.
- New/rewritten tests (RED on the old tree or without the walk
  expansion): conditional_variants cv_c_platform_files,
  cv_c_platform_directories, cv_incremental_platform_files;
  test_impact_engine
  test_impact_engine_platform_variants_reach_each_others_callers;
  store_impact impact_walk_crosses_platform_variants_only_when_asked;
  cv_incremental_variant_partners re-fixtured to Go build tags.
- 18 affected suites: 2710 passed, 4 Windows-only skips.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-13 18:33:40 +02:00
Martin Vogel 8887437e6d fix(test): build the test-impact suites on Windows 
Two Windows-only compile errors, hidden until now because the Windows leg
stopped at its contract steps before building the runner:

- test_test_impact.c calls cbm_find_cli under _WIN32 without including
  cli/cli.h (an implicit declaration, an error in clang).
- test_test_impact_engine.c canonicalizes the fixture's temp dir with
  realpath, which Windows does not have. It now uses _fullpath there, with
  forward slashes like the paths the fixture builds from it, the same split
  test_daemon_ipc.c makes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-06 03:53:49 +02:00
Martin Vogel 42a9320200 graph: one identity per conditional definition, SPAWNS edges, no cross-language guesses 
The test-impact engine selects tests by walking the graph, so it can only
be as right as the graph. Three gaps made it wrong:

One identity for a definition's mutually exclusive variants
- A definition written once per #if/#else branch, per #[cfg], per
  platform file (foo_linux.c / foo_win.c, platform directories) or per Go
  build tag is ONE node. The graph buffer keeps every variant's span in a
  `variants` node property (definition labels only), so a change in any
  variant maps to the definition.
- Extraction now sees definitions inside conditional blocks where it
  skipped them: C++/C#/ObjC class members under #if (one cursor walk,
  conditional block types per language in lang_specs), Elixir defs under
  if/unless, Fortran .F90/.F95/.F03/.F08, Rust #[cfg] twins (no more
  `name#cfg(...)` QNs that lost every call), C platform files and
  directories (non-static file-scope functions).
- Haskell: the vendored scanner reads only an #if group's first branch.
  The grammar stays upstream; every further branch is parsed as its own
  same-length view (cpp_branch_views.c) and adds what the raw parse
  missed. haskell/bytestring: no new partial parses, +27 functions,
  IMPORTS 322 -> 329, 131 definitions with variant spans.
- Incremental closure repair re-parses every file of such a definition
  when one of them changes (variant partners through DEFINES), so a
  repair equals a full index.

No cross-language name guesses
- A name-only resolution (unique_name, suffix_match, qualified_suffix,
  field_type_hint, and same_module across files that only share a path
  stem) never binds a symbol another language defines.
- Bash imports are the files a script sources: every top-level command
  was read as an import of its name (`set -e` imported "set", bound to a
  TSX method in 137 scripts here).
- This repository: cross-language CALLS 5,916 -> 5,091; the remainder are
  C/C++ pairs, builtins and same-stem module collisions (recorded).

SPAWNS edges (per language, like the HTTP client patterns)
- A call that starts a program (subprocess.run, exec.Command,
  child_process.spawn, posix_spawn, system, Process.Start, Command::new,
  ... about 30 languages, through the file's own imports) gets a SPAWNS
  edge to a Process node named after the program (literal first word, or
  <dynamic>) instead of a CALLS guess. No target resolution. Both
  resolvers ask at the same point (parity test).

The engine follows the graph
- Seeds map changed lines through every variant span; a test written
  once per #if branch is one test; when a change reaches a non-test
  `main`, the tests that spawn processes are reached too (the process
  runs main, and main reaches everything).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-06 02:44:56 +02:00
Martin Vogel 6a29a19c23 fix(test-impact): build and run on Linux GCC, git before 2.44, LeakSanitizer 
PR CI on the first push of these commits found what no local leg had
built yet:

- test_impact_result_validate.c: initialize the count GCC's
  -Wmaybe-uninitialized flags (-Werror on every Linux build: smoke,
  shadow, analyze).
- tests: split one-line `if` chains GCC reports as misleading
  indentation, and drop a redundant `= {0}` it reads as missing braces
  (the GCC test runner did not compile).
- tests/test_test_impact_engine.c: realpath() into a PATH_MAX buffer;
  glibc's _FORTIFY_SOURCE aborts on anything smaller, as
  test_index_resilience.c already notes.
- test_impact_git.c: pass --no-lazy-fetch only when the git executable
  takes it (git 2.44+, probed once per session with the option on
  --version). Older git rejected it and every engine command failed;
  Ubuntu 24.04 ships git 2.43. `-c protocol.allow=never` already turns a
  lazy fetch into an error there, which is what the option asks for.
- test_impact_engine.c: the selection and publish job threads release
  their thread-local node-type bitset cache before they end, as the
  parallel workers and the MCP index thread do. LeakSanitizer reported it
  (488 bytes in 13 allocations, test-diag); the Linux arm64 ASan leg
  reproduced exactly that and is clean with the fix.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-05 23:55:25 +02:00
Martin Vogel 36e5874525 feat(test-impact): team artifact bundles and admitted coverage 
A main commit can now publish its test-impact bundle — the commit's frozen
graph, its per-test coverage map and receipt.json binding both — and a PR
selection given the bundle of its merge base, verified by the caller,
builds its graph incrementally from the bundle's graph and narrows suites
by the admitted map. Without a verified bundle nothing changes: the graph
is built from scratch and every reached suite runs whole.

Producer and consumer:
- `codebase-memory-mcp test-impact publish|select` (in process, no
  daemon): publish builds the bundle of the checked-out commit, from the
  previous bundle's graph when that bundle verifies; select is the
  selection with optional --artifact/--artifact-commit/--artifact-verified.
  The MCP surface is unchanged.
- receipt.json binds the exact imported graph (graph digest), the graph's
  content and topology, the map's three files and a compatibility digest
  (platform label + the bytes of test_impact.coverage.compatibility_paths).
- A bundle is used only when the caller verified its source, it is the
  merge base's own, and its imported graph is the one the receipt names;
  the map is admitted by the existing receipt policy (source, ancestry,
  content, age, compatibility, metadata). Refusals become rejection
  reasons in the answer's receipt.
- C_eff: a changed function the map knows contributes its own id; one it
  does not know (new, platform-only, a type or macro) contributes its
  nearest mapped callers on the static inbound closure; deleted names
  contribute every mapped function of that name.

Graph:
- the frozen build accepts a base candidate and repairs it through the
  incremental closure route; any doubt declines to the full build. Pinned
  to one worker like the full frozen build.
- incremental closure repair skipped the interprocedural complexity pass:
  re-parsed functions lost transitive_loop_depth/recursive and callers kept
  stale values. One traversal now serves the in-memory pass and a new
  store-level recompute that runs before the importance rescore.
- test-impact's frozen builds use a fixed display Git context: the per-run
  pin path and the commit ids made two builds of one tree differ and made
  every commit a semantic-input change, so repair always declined.
- graph content and topology digests (store_content_digest.c): the exact
  graph digest binds one database (ids, db_uid, generation), never equal
  across builds; the content digest keys rows by what they say, the
  topology digest by what selection reads.

Measured on this repository (frozen publish, macOS arm64): a two-file C
change repairs in 18-20 s instead of 75-80 s with the topology of the full
build; a docs-only change repairs with identical content; selections with
and without the bundle are identical on all seven replayed pairs
(two repaired incrementally, five declined to the full build). Recorded, not
fixed: closure repair still differs from a full build in edge PROPERTIES
(argument quoting and key order, CONFIGURES args) and in the corpus-wide
SIMILAR_TO/SEMANTICALLY_RELATED edges; selection reads neither, so the
topology digest is the incremental-equals-full check and the receipt
carries both.

Tooling: scripts/test-impact/coverage-merge.py builds a push's incremental
coverage map (re-run suites verbatim, the rest carried with ids remapped
onto the new image's function table), with its contract as test.sh Step
0z7.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-05 18:52:26 +02:00
Martin Vogel c5abb64943 feat(test-impact): detect_changes scope:"tests" end to end 
One request pins the change (merge base .. HEAD), builds the graph and the
runnable test registry of exactly the HEAD snapshot, seeds the graph from
the change, walks what depends on it and answers which tests to run
(cbm.test_impact). Missing or doubtful evidence never narrows: it becomes a
run-all answer that names why.

Snapshot (classified ledger -> frozen graph -> registry):
- one shared language classifier for discovery and pinned blobs (raw bytes
  on every platform); the classified ledger feeds the frozen build, which
  indexes exactly the listed files and records partial parses as per-file
  PARSE_GAP instead of refusing the graph;
- the pinned tree skips symlinks and submodule links (counted), as
  discovery never indexes them;
- the test model scopes uncertainty to the suites of its file (they run
  whole), reads line splices where they occur, treats a mid-file
  `#ifndef X` + `#define X` as an ordinary condition, and no longer reads a
  plain identifier named like a macro as an invocation. On this repository
  the registry is now narrowable and matches a text census exactly
  (8,365 cases, 141 runner suites).

Seeding and reach (port of the measured replay, rule B):
- hunk -> overlapping definitions, comment-only hunks dropped, pure
  deletions -> enclosing definition, no-definition hunks -> whole file,
  deleted names found by text, header changes propagated through the
  include closure and the headers' macros (pastes included), file-level
  escalation for unclassifiable header lines;
- inbound fixpoint walk over CALLS, ASYNC_CALLS, USAGE, CALL_REFERENCE,
  READS (+ routes); cases, suites and the runner's entry points are sinks;
  a reached runner main runs everything (RUNNER_REACHED);
- a parse gap whose text names a reached definition reaches the
  definition around it (or its file);
- macro-registered tests run their suites whole; cases and suites map to
  graph nodes by their start line.
The line patterns reproduce the reference regexes exactly on 1,383,850
lines of this repository and 37,167,926 kernel lines; on 24 replayed
changes the engine selects at least what the reference selected (13
identical, the rest only conservative supersets on the same graph).

Also:
- configured test declarations that cannot be mapped degrade their file
  (no configured roles, a test_declarations diagnostic) instead of
  aborting the whole index;
- the selector keeps other runners' suites out of the selection, and the
  result codec judges the model by narrowability like the selector;
- a change to .codebase-memory.json runs everything (built-in rule);
- .codebase-memory.json with the c-cbm preset and this repository's rules.

Gate fixes (suites, lint-ci, Windows cross-compile):
- the request runs on its own thread with a default-size stack (daemon
  tool calls run on 256 KiB threads, where the in-process build died);
- a request-supplied config_path is no longer followed by an unlink of an
  uninitialized path buffer;
- the inventory's frame-local reader no longer stays reachable from the
  caller's source after the filter call;
- FILE_ID_INFO needs _WIN32_WINNT >= 0x0602; the pinned-tree header pins
  it before any system header (toolchains defaulting lower failed);
- memory-core ratchet: the win_utf8 converters gain core-backed
  `_in(cls, ...)` forms over one implementation (the legacy names keep
  the C library for unmigrated callers), the Git facts module uses them,
  and the Windows spawn path releases everything through one failure
  exit; pipeline, extraction and config allocations that this work
  touched moved to the core in matched pairs; baselines tightened;
- the branch's C files formatted with the project's clang-format.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-10-05 02:15:08 +02:00