The lint job's single-process cppcheck pass now takes 22-29 minutes:
the src tree grew about 60 % in files in ten days (210 files / 7.4 MB on
2026-10-04, 335 / 11.2 MB since 2026-10-12) while the cppcheck flags
stayed the same. Three runs on 2026-10-13 (38330125249, 38345850562,
38370789949) were cancelled at the 30-minute cap with cppcheck still
running and every earlier lint step clean. A cancelled lint is a hard
stop for the release graph, and the dry run's tests need lint to pass.
45 minutes puts the bound back where only a genuine hang reaches it.
Parallel cppcheck (-j) would be faster, but it changes cppcheck's
whole-program checks; it is left for its own change with a
findings-equivalence proof.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A file and the directory beside it with the same stem (genproto.sh next
to genproto/) got one qualified name: the file's Module QN equals the
directory's Folder QN, and the graph buffer keeps the structural node, so
the file's Module was dropped, and with it the edges that pointed at it.
On 17 infrastructure repositories this lost 27 Modules (shell scripts,
YAML and JSON manifests, a Makefile, Rust, Elixir, Python, Terraform).
One side now yields, decided once per run from the discovered files:
- the Folder is spelled proj.pkg.foo/ (no module QN ends in a slash);
- where the directory is a Java/Go package it IS that package's module
and Go imports resolve to it, so the Folder keeps its QN and the file's
Module is spelled proj.pkg.foo#sh instead.
Symbol QNs do not change.
The closure-repair planner declines when a deletion touches a collision
on either side, since the spelling depends on the file set. The kustomize
resolver binds a directory entry to its Folder even when a same-stem
manifest sits beside it. CBM_SEMANTIC_INDEX_VERSION 7 -> 8, so existing
indexes rebuild once.
Measured on main 56700bdb over the 17 corpora: colliding files without a
Module 27 -> 0 (19 marked Folders, 8 #ext Modules). k8s.io resolves 3
more kustomize entries, and deckhouse's `source shell_lib.sh` imports
land on the script instead of the directory.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The #961/#1989 rescue re-adopts definitions from the preprocessed second
pass when they sit in a raw parse error, and drops a copy only when its QN
equals a raw one. When the two parses scope the same definition
differently, both copies survived:
- Linux lib/tests/randstruct_kunit.c: a macro-generated struct body makes
the raw parse nest every later struct inside one ERROR region; the flat
expanded copies were adopted next to the mis-nested raw ones (5
duplicate groups).
- coreclr/rapidjson: the expanded parse nests utilcode.h classes wrongly
or loses rapidjson's scopes, and both copies survived.
One rule at one decision point: an expanded and a raw definition with the
same label, name and start line but different QNs are one definition. The
raw copy wins and whatever the expanded pass attributed to the expanded QN
moves to it -- unless
- the raw copy's nesting exists only because of a raw ERROR region that
the expanded parse resolved (the randstruct shape, six conditions in
cbm_misnested_raw_twin): then the flat expanded copy replaces the raw
one and the raw children (Fields, nested defs, call/usage scopes) are
re-parented to it; or
- the raw copy's QN is already held by a raw definition of another label
(coreclr inc/holder.h: the raw parse reads a FORCEINLINE constructor as
a Function of the class's scope-less QN): then the expanded copy keeps
its own, correct QN instead of vanishing into the Function's node.
Proof (macOS arm64):
- randstruct_kunit.c: 5 -> 0 duplicate groups, 12 Fields re-parented to
the flat structs, CALLS/USAGE/WRITES unchanged.
- dotnet/runtime native subset (src/coreclr + src/native): 700 -> 183
groups; the remaining groups are no raw/expanded pairs at one site.
- redis: node and edge sets byte-identical.
- New tests (RED without the change): extraction
extract_c_misnested_raw_struct_replaced_by_flat_copy,
extract_cpp_preprocessed_rescue_skips_raw_def_on_same_line,
extract_cpp_rescue_keeps_expanded_class_when_raw_qn_is_taken (fixture
delta-minimized from holder.h).
- conditional_variants, extraction, c_lsp, pipeline, edge_types_probe,
parse_coverage, lsp_resolution_probe: 1854 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
42a93202 gave a non-static C-family function under a platform directory
(arch/x86/, src/win/, ...) or file suffix a platform-neutral QN, so the
platform variants of one function shared a node. On the Linux kernel that
merged only 644 of 17,413 such definitions; the other 16,769 lost their
path for nothing (`arch/arm64/mm/init.c` mem_init became
`arch.mm.init.mem_init`, and a lookup by its path QN found nothing),
arches missing from the token list kept their paths, and calls bound to
merged nodes produced impossible arch -> other-arch CALLS.
Every definition keeps its path-derived QN again (the v0.11.0 identity).
The platform-neutral QN is now a key: definitions carrying one get a
`variant_key` node property, and nodes of a project sharing a key are one
variant group. The test-impact walk (store_impact `follow_variant_keys`,
enabled by the test-impact engine) treats a reached variant as reaching its
whole group, at seeding and at every step, so a change confined to the
Windows variant still selects the tests that reach the POSIX variant's
callers. The key is a pure function of a definition's own path, name and
linkage: incremental re-parsing of the changed file writes what a full
index writes. Same-QN variants (#if/#else, Rust cfg, Go build tags,
.c/.h pairs) and their `variants` property are unchanged. No new edge type.
Proof (macOS arm64):
- Linux kernel subset (arch, include, kernel, mm, lib; 27,005 files):
arch functions whose QN is not under their own path 12,873 -> 0;
arch/A -> arch/B CALLS 10,353 -> 7,101; 581 variant groups (1,815
members), each one of the former cross-file merges; the path QN of
arch/arm64's mem_init resolves.
- This repository: 13 groups (test_impact_tree_{posix,windows},
mem_override_{posix,win}); before, 22 of 49 platform-path functions had a
stripped QN.
- New/rewritten tests (RED on the old tree or without the walk
expansion): conditional_variants cv_c_platform_files,
cv_c_platform_directories, cv_incremental_platform_files;
test_impact_engine
test_impact_engine_platform_variants_reach_each_others_callers;
store_impact impact_walk_crosses_platform_variants_only_when_asked;
cv_incremental_variant_partners re-fixtured to Go build tags.
- 18 affected suites: 2710 passed, 4 Windows-only skips.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Doc-section -> code candidates (doc_link_candidates, #2574/#2610) are the
documentation-in-the-graph feature of this release. They now run by
default for full indexes and not for moderate or fast ones; an explicit
`doc_candidates` argument, or a `doc_candidates` setting the user set,
overrides the default either way in every mode.
- pipeline: the switch is unset/on/off; unset means on only when the
effective mode is full.
- MCP: forwards only an explicit choice (argument, then a user-set config
value) to daemon jobs and workers; daemon auto-index and watcher/UI jobs
now carry a user-set value too (their workers have no config, so
`doc_candidates false` was ignored there).
- fast + doc_candidates on runs the semantic pass's corpus phases and the
doc phase only: candidates, and still no SIMILAR_TO or
SEMANTICALLY_RELATED edges.
- config: `doc_candidates` joins the config key table (default `auto`);
`config set doc_candidates ...` was refused as an unknown key before.
README, docs/CONFIGURATION.md and the MCP schema describe the default.
Proof (macOS arm64, django fb11376):
full 11,304 rows · moderate 0 · fast 0 · full+off 0 · moderate+on 1 ·
fast+on 1 (moderate/fast discovery skips docs/) -- nodes and edges are
identical between on and off in every mode; fast+on adds 0 semantic
edges.
- New tests (RED on the old code): pipeline
pipeline_doc_candidates_default_follows_mode,
pipeline_doc_candidates_explicit_wins_over_mode,
pipeline_doc_candidates_fast_on_legacy_incremental; mcp
tool_index_repository_doc_candidates_follow_mode,
tool_index_repository_doc_candidates_forwarded_only_when_explicit;
daemon_application
daemon_application_programmatic_index_forwards_set_doc_candidates.
- mcp 374 passed (4 Windows-only skips), daemon_application 73 passed,
pipeline/store_arch/cli/runtime_settings/index_policy/pipeline_frozen
green on the release tree.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
doclink_rst.c sorted d.toks unconditionally; the vector stays NULL until
the first token, and qsort's base is declared nonnull even for zero
elements. GCC UBSan on Linux amd64 reported it in rst_section_text_cut and
diagrams_rst ("null pointer passed as argument 1, which is declared to
never be null", doclink_rst.c:1767) -- the same class as the k8s pass fix.
Sort only when there are at least two tokens, as the file already does for
its section titles.
Proof: doc_links_rst on Linux amd64 (GCC, ASan+UBSan) reports the
violation without the change and none with it; doc_links_rst + diagrams on
macOS: 27 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The preprocessed second pass rebuilt the expanded source with simplecpp's
TokenList::stringify(), which puts one space between every pair of tokens:
`template<> struct bitonic<int64_t, AVX2>` became
`bitonic < int64_t , AVX2 >`. Every QN builder takes the class segment from
node text, so a definition the #961 rescue re-adopted from that buffer
(vxsort's INLINE/NOINLINE macros break the raw parse on every method line)
got a second, token-spaced QN, and the QN-equality dedup kept both. CALLS
split across the two spellings (dotnet/runtime: 902 compact + 996 spaced
into the bitonic sorters).
The expanded buffer now keeps the source spacing: tokens that touched in
the source (same file and line, no macro expansion between them) stay
touching, every other gap is still one space, and the line structure and
#line directives are unchanged, so every line remap holds. One fix point
instead of a normaliser at each QN builder; no vendored code changes.
Proof (macOS arm64, dotnet/runtime 60108ba6, two identical runs each):
- duplicate definition groups 706 -> 217 (v0.11.0: 545); nodes with a
token-spaced QN 495 -> 0; CALLS into bitonic 902 + 996 -> 998, one
spelling. The remaining groups are coreclr gc.cpp's SVR/WKS builds (184)
and 33 scope disagreements between the two parses.
- Side effect: rapidjson document.h's GenericValue members now exist once,
with the raw parse's scope (the token-spaced copy that carried the
nested scope is gone); `operator[]` / `operator()` in the expanded pass
now parse as single tokens.
- New test (RED without the change): extraction
extract_cpp_template_specialization_preprocessed_qn_single_spelling.
Three c_lsp collision fixtures recalibrated to the new byte offsets
(comment padding and offsets only, assertions unchanged).
- extraction, c_lsp, pipeline, edge_types_probe, parse_coverage,
conditional_variants, lsp_resolution_probe: 1852 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A struct member in the short-name registry turned a unique header function
into a two-candidate name: `put_page(page)` beside the arm64
`kvm_pgtable_mm_ops.put_page` member. The lookup then answered by
suffix_match, and the #725 cross-language guard refused it because .h is
C++ in the extension table -- the call vanished, not even a CALL_REFERENCE
remained. On the Linux kernel every header function with a same-named
member elsewhere lost its callers: put_page 345 -> 9, resource_size
1,159 -> 4, alloc_skb 419 -> 4 (v0.11.0 -> main).
A bare C call (no `.`/`->`) can name neither a struct member -- reached
only through its object, cbm_call_onto_field_policy already refuses every
Field it lands on -- nor a method, which C does not have. The CALLS
resolvers (sequential and parallel twins) now ask
cbm_registry_resolve_call_lang(), which sets those candidates aside before
the name strategies run; the per-file resolve cache keys on that choice.
References and every other language are unchanged.
Setting aside only the Fields was not enough: redis's libc `read()` then
became the unique name of a C++ Qt adapter's `read` Method (62 false edges
in v0.11.0). Exempting C/C++ from the #725 guard instead opened 375k
C -> header suffix_match edges, ~40 % of them false on redis -- rejected.
Proof (macOS arm64, Linux kernel 2d2338c93):
- put_page 9 -> 344, resource_size 4 -> 1,158, alloc_skb 4 -> 417,
cpu_online 5 -> 188, device_unlock 1 -> 140, dma_unmap_sgtable 2 -> 77
(v0.11.0: 345 / 1,159 / 419 / 198 / 140 / 77).
- CALLS 1,845,215 -> 1,924,199 (+4.3 %); a seeded sample of 300 gained
edges: 300 have a bare `name(` call in the caller's span.
- redis: +8 call pairs; libc read/write no longer bind project members.
- New tests (RED without the change): registry
c_bare_call_does_not_weigh_field_namesakes; pipeline
pipeline_c_call_reaches_header_function_beside_field (+ _parallel).
- registry, pipeline, extraction, extraction_imports, c_lsp,
edge_types_probe, lsp_resolution_probe, complexity, parallel,
incremental: 2134 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#2513 (77bc4927) computes `url_context` -- is the callee an HTTP client or a
global fetch? -- for every positional argument of every call, in every
language, through cbm_service_pattern_match: a linear strstr over every
service pattern. Only the URL-folding functions read it, and they return
NULL outside Python and JS/TS; the one other reader is the template-string
branch. On the C# corpus (dotnet/runtime, 451k functions) extraction went
from 24.7 s to 37.4 s (+51 %); a macOS sample put the difference in
handle_calls -> cbm_service_pattern_match -> match_qn -> strstr.
A per-call url_context_t now computes the answer at most once and only when
an argument needs it: the folding paths ask only in folding languages, the
template-string branch asks for itself.
Proof (macOS arm64, Apple M5 Pro):
- C# parallel_extract 37.4 s -> 24.2 s, wall 127-132 s -> 119.7 s (the
pre-#2513 level, 24.7 s / 121.9 s).
- Byte-identical graphs: the full node and edge sets (properties included)
hash the same before and after on django (Python), typescript (TS) and
csharp, HTTP_CALLS included (457 / 283 / 39).
- extraction, extraction_imports, pipeline, parallel, infrascan and
edge_types_probe: 1009 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The C# and Java Tier-2 registry builders register types, finalize, then
register the rest from `defs + type_count`. With no definitions of that
language the array is NULL, and NULL + 0 is undefined: clang's UBSan
reports "applying zero offset to null pointer" at cs_lsp.c:3886 on the
macOS legs of dry run 38321896354 (reached by
incremental_k8s_edits_reach_other_files, a project without C# files).
The second registration now runs only when there is something to register;
both finalize calls are unchanged.
cs_lsp, cs_lsp_bench, java_lsp, java_lsp_coverage, lsp_resolution_probe:
529 passed (macOS). The report reproduces on CI's macOS clang, not on the
local toolchain, so CI's macOS legs are the check.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two reds of dry run 38321896354 on main 56700bdb, both reproduced locally:
- pass_k8s.c sorted empty vectors with qsort(NULL, 0, ...). glibc declares
qsort's base nonnull, so GCC's UBSan reports it on Linux (pass_k8s.c:832,
:1009); macOS does not flag it. Every qsort in the file now goes through
k8s_qsort, which skips arrays of fewer than two elements.
- incremental_deleted_spawner_drops_its_process (cb497a82) deleted a shell
spawner and a Python spawner in one step and expected closure repair.
The Python namespace admission (#2531, merged after it) declines closure
repair for any deleted Python module -- its names leave its importers'
namespaces -- and pipeline_python_namespace_admission_deleted_and_empty
pins that. The spawner test now deletes in two steps: the shell script
through closure repair (its Process goes, 2 stay), then the Python file
through the full route (1 stays). Both admission rules stay covered.
pipeline_semantic_manifest_repro 49 passed; pipeline, infrascan, spawns,
incremental and the pipeline_* suites 607 passed (macOS).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cx_pyrange_exact_control counted edges with a "source":"lsp" property
that CALLS edges never carry, so every control failed and
complexity_py_module_ranges_preserve_calls_with_bounded_work reported
zero controls. Count edges whose strategy is lsp_*, the way the pyext
control and the pipeline tests recognize an LSP resolution.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_scope_lookup_local returns the binding, not its type, so the range
tests passed a const CBMVarBinding * where a const CBMType * was
expected and the test runner did not compile
(-Werror,-Wincompatible-pointer-types). Type checks now go through
pylsp_local_type, Hidden keeps the binding so the package case still
asserts that nothing is bound, and shadow_only checks the type of its
binding.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A call no project symbol resolves, whose callee an import binds to an
external library, is now spelled through that import and classified with
the existing service patterns: `send` under
`from paho.mqtt import publish as send` is paho.mqtt.publish (ASYNC_CALLS
to its topic), `navigate` under `import { get as navigate } from 'axios'`
is axios.get (a typed GET HTTP_CALLS edge). Before, such calls had no
service edge at all.
- CBMImport gains `member`: the exported name an ES named import binds
(`get` in `{ get as navigate }`), NULL for default, namespace and other
imports and for every other language. Filled by process_named_imports,
carried through result compaction (and so through spill), counted in the
extraction census with the other import strings.
- One helper, cbm_pipeline_external_import_service, serves both call
resolvers: a per-file index (local name -> binding import), built on the
first unresolved call that asks, then one hash lookup per call.
- Precision: only unresolved calls, after the callee-name, route-suffix and
global-fetch fallbacks; never a relative import, an import an IMPORTS
edge materialized (a project module), a name bound by two different
imports, or a Python callee rebound as a parameter. An HTTP edge needs a
request verb in the spelled name, so urllib.parse.urljoin is not a
request. Python, JavaScript, TypeScript and TSX.
Tests: extraction_imports covers the member and its survival through
compaction; a pipeline test runs both resolvers over a typed GET, a typed
POST and a publish through aliases, and over controls that must stay
silent (urljoin, lodash and json aliases, a local definition and a
parameter shadowing the import, a materialized project package named
httpx). Each guard was reverted once and its control went red.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CBMCallArg gained url_value after the probe's positional initializer was
written, so -Wmissing-field-initializers (-Werror) stopped the test build
once this branch met main. Spell the new field out as NULL.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Refs #2291, #706, #1147. Preserve raw argument values and carry URL projections separately through compaction and spill.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Preserve registry array order and full-scan fallback when optional query
storage is unavailable. Keep temporary range storage scoped to each module
operation and shared type indexes in the registry owner arena.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cx_pyext_control looked for a "source":"lsp" property that CALLS edges
never carry, so the control count was always zero and
complexity_py_submodule_probe_avoids_registry_scan failed even though
every cx_pyext_use() kept its local call. An LSP resolution records an
lsp_* strategy (lsp_direct for this call), the same check the pipeline
tests use.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #1245 by Andrew Hundt: the two exclusive-reservation
helpers and their two call sites, hand-ported onto main's install.ps1.
Nothing else from that file is taken -- the PR's copy also carries an
older TLS block that would re-break Windows 10 (#1856), so a wholesale
take was not an option.
Two paths the installer used were guessable, and one of them adopted
whatever it found:
$TmpDir = Join-Path $env:TEMP "cbm-install-$(Get-Random)"
New-Item -ItemType Directory -Path $TmpDir -Force
Get-Random is seeded per process and its space is small, and -Force on an
existing directory is a no-op that returns it. A directory planted at a
guessed name became the staging area -- and the owner-only DACL the
installer applies is set only afterwards, on a directory someone else
created. The new helper names the directory with a GUID and creates it
with -ErrorAction Stop, so a collision is a retry, never an adoption.
$InstallerTmp = "$InstallerDest.new"
was a fixed path in the install directory, reservable by anyone who could
write there first. The sibling is now reserved with FileMode.CreateNew and
FileShare.None on an unguessable name, which is atomic, before anything is
copied into it.
One deliberate difference from the PR: the sibling reservation runs INSIDE
the existing try, and the cleanup checks that a path was assigned. That
block is documented as best-effort ("a failure here still leaves a working
install"), and a reservation that exhausts its 32 attempts should be one
more best-effort miss, not a script abort under ErrorActionPreference=Stop.
Verified: pure ASCII (PS 5.1 reads a BOM-less .ps1 as ANSI), LF endings
matching main, both old call sites gone from live code.
NOT verified: this file was not parsed by PowerShell. There is no pwsh on
the macOS host, the Windows VM is down, and CI's Windows guards only check
that `update` prints the install.ps1 command -- they never execute it.
Opened as a draft for that reason; it needs one PowerShell parse before it
merges.
Co-authored-by: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
lookup_func_self guards a NULL qualified_name in its indexed bucket scan
and its post-finalize tail scan, but not in the linear scan an
unfinalized registry takes. Iterating a fallback chain checks every base
entry for shadowing against the head, so a head holding a func without a
qualified name crashed in strcmp (ASan SEGV in
pylsp_prefix_order_matches_iterator_and_chain_shadowing). The linear
scan now skips such entries like the other two.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both passes now emit byte-identical HTTP_CALLS and ASYNC_CALLS
properties, so the vendor-types parity fingerprint hashes them whole
again instead of trimming the "args" member. The precondition change
from the previous fixture correction stays.
Refs #1186
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The parallel pass appends the call's ,"args":[...] member to every
HTTP_CALLS and ASYNC_CALLS edge (finalize_and_emit). The sequential pass
appended args to CALLS only, so the same service call produced
different edge properties depending on which pass indexed the repo, and
create_data_flows had no caller_args for small (sequential) repos.
Split the args-and-closing-brace step out of finalize_and_emit as
cbm_pipeline_close_call_props and let the sequential
emit_http_async_edge build its props in the same CBM_SZ_2K buffer and
finish them with that function. Field order, expression sanitizing,
escaping and truncation are therefore one implementation; CALLS edges
and their "line" member are unchanged in both passes.
Refs #1186
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendor-types test never got past its extraction precondition, so
two of its assertions had never run against production:
- The precondition looked for the import alias in the callee text of
`GuzzleClient::get` and `KafkaProducer::publish`. The extractor names
a scoped call by its method alone and keeps only a literal `Route::`
scope (#952). The aliased scope is now proven on the occurrence text
at the call's site span, which identifies the same call just as
strictly.
- The sequential/parallel parity fingerprint hashed the HTTP_CALLS and
ASYNC_CALLS properties whole. The parallel emitter appends the call's
"args" member last; the sequential pass leaves route/service
properties without it by design (calls_emit_edge). Those two edge
types are now compared with only that trailing member removed; their
type, endpoints and every other property must still agree.
Refs #1186
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two gaps let a call on a vendor class escape the #1186 evidence:
- A vendor class the PHP LSP knows from its seeded library data (for
example GuzzleHttp.Client) resolved as php_static_resolved or
php_method_typed. Its target has no graph node either, but without the
php_external_receiver strategy the pipeline fell through to a
name-only registry binding. Methods found on a class reached through a
vendor `use` now carry php_external_receiver as well.
- A vendor call skips the registry, and the extractor names a scoped
call by its method alone (only a literal `Route::` scope is kept), so
`GuzzleClient::get('/x')` reached the route-verb fallback and became a
route registration instead of an HTTP call. When the callee text names
no service, the service patterns now also see the vendor method the
LSP proved (`GuzzleHttp.Client.get`, `Kafka.Producer.publish`), in
both the sequential and the parallel pass.
Refs #1186
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
