Cross-file LSP for Java rebuilt a type registry for EVERY file from a def
set the module filter reduces by a constant factor, not to a constant
size: the JVM filter branch includes every def sharing the file's
namespace, so per-file work tracks the corpus (measured defs_per_file
1,292 -> 5,031 in lockstep as defs_total went 179k -> 689k). That makes
the pass O(files x corpus_defs) — 87% of a Java index and the bulk of
the v0.9.0 -> v0.10.x slowdown.
Three changes, one architecture (the pattern Go/Python/C/C#/TS already
use):
1. cbm_java_build_cross_registry — the JVM def universe (Java + Kotlin,
for mixed source roots) built ONCE per run, sealed read-only, shared
across resolve workers. Wired into both pipeline.c and
pipeline_incremental.c.
2. Two-phase registration inside that build: all TYPES first, finalize
(hash buckets exist), then FUNCS. Func registration parses signatures,
and type-name qualification via cbm_registry_lookup_type is a LINEAR
scan until finalize — a single mixed pass measured 0.44 ms/def at
689k defs, a ~300 s sequential build that erased the sharing win.
Two-phase: 306 s -> 2.8 s. Stable partition order because overload
ties resolve to the first registered QN match.
3. cbm_run_java_lsp_cross_with_registry resolves each file against the
shared base through an overlay holding exactly THIS FILE's defs
(register_local_func_or_type_from_file). Own-file scope is the load-
bearing choice: patch_one_method refines signatures from the AST and
must write a private copy (its types live in the per-file arena, the
base is sealed), and any wider scope re-imports the quadratic — a
module/namespace-scoped overlay measured ratio 4.00 on the growing-
package corpus, own-file measures 2.00.
Elasticsearch corpus (46,477 files), same host, CBM_PROFILE=1:
v0.10.5 this change v0.9.0
wall 419.9 s 85.6 s (4.9x) 61.5 s
cross-LSP CPU 6,195,780ms 52,513ms (118x) 110,344 ms
us_per_file 207,448 1,770 3,722
us_per_file_per_kdef 300 2 —
nodes 693,100 693,100 —
edges 5,646,235 5,649,949 —
Cross-LSP CPU now beats v0.9.0. Nodes are identical; edges +0.066%,
consistent with the shared base resolving cross-package targets the old
per-file namespace/import filter could not see, plus source-order
independence from the two-phase build.
Guarded by the complexity suite's shared-package gate (RED at ratio 4.00
on the pre-change tree, 2.00 after — see the suite commit).
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Indexing large OSS during the perf sweep crashed three LSP resolvers, all by stack overflow in their recursive resolve walks (macOS crash reports): elasticsearch — SIGSEGV in bind_lambda_args under deep recursive java_resolve_calls_in_node frames; bitcoin — SIGSEGV in cbm_type_substitute via c_adl_resolve under deep c_resolve_calls_in_node frames; microsoft/TypeScript — SIGBUS under an unbounded lookup_member_type cycle (cyclic type graph in reallyLargeFile.ts). The expression-eval guards (eval_depth) did not cover these walks. Each resolver now has a depth-guarded entry (walk_depth cap 512 for the Java/C AST walks, member_depth cap 64 for TS member lookup); past the cap the subtree resolves as unknown — graceful degradation instead of a crash. Reproduce-first: prod CLI probes crashed pre-fix (nested-call Java rc=139, nested-call C++ rc=139); fork+exit-signal regression tests added (TS cyclic shape needs a real cross-file registry, so it is verified at the real-repo tier — microsoft/TypeScript now indexes clean). Side-finding documented in the sweep report: Java BLOCK nesting >=3000 is minutes-slow (separate pathology, not this fix).
Martin Vogel
Martin Vogel