Commit Graph
841 Commits
Author SHA1 Message Date
Martin Vogel 3ee9d5d112 Drop the nim grammar and refresh language counts
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
2026-06-12 02:17:39 +02:00
Martin Vogel 630bd40a90 Replace the graph-UI HTTP server with a first-party implementation
The web server behind the graph UI is now a purpose-built in-house
module (src/ui/httpd.c): localhost-only listener, strict HTTP/1.1
parsing with fixed request caps, a per-connection receive deadline,
and Connection: close semantics. http_server.c keeps the routing and
handlers, rewritten against the new transport API; the public server
API and main.c are unchanged. The previously vendored third-party
server is removed entirely.

Comes with a new 28-test transport + routing suite (tests/test_httpd.c)
covering parsing edge cases (strict CRLF, Content-Length limits, raw
path matching, percent-decode rules) and live-socket behavior (CORS
policy, RPC dispatch, receive deadline, clean shutdown). The security
audit scripts now check the new file layout and treat any network call
in vendored code as a failure.
2026-06-12 02:17:25 +02:00
Martin Vogel 73f859a8e6 Add tree-sitter runtime LICENSE to common/tree_sitter
The alloc.h/array.h/parser.h headers in
internal/cbm/vendored/common/tree_sitter/ are part of the tree-sitter C
runtime (MIT, (c) 2018 Max Brunsfeld) but had no LICENSE file alongside
them. Copy the upstream MIT license into that directory and clarify in
THIRD_PARTY.md the split between the tree-sitter-html scanner helpers
(common/, (c) 2014) and the core runtime headers (common/tree_sitter/,
(c) 2018).
2026-06-12 01:14:10 +02:00
Martin Vogel 20bf70e238 Reword Hybrid LSP description; correct grammar count to 159
Describe the Hybrid LSP layer as a lightweight C implementation
structurally inspired by and compatible with major language servers,
rather than a clean-room re-implementation of their algorithms, across
README, site, and llms.txt. Drop intelephense from the reference list.
2026-06-12 00:02:30 +02:00
Martin Vogel d57bbe6a27 Complete vendored license coverage; rework THIRD_PARTY.md
Restore upstream LICENSE files for every vendored grammar and C library
across both vendored trees (19 grammar dirs, lz4, zstd, simplecpp,
verstable, wyhash, ts_runtime, common, mongoose, xxhash, yyjson; SQLite
public-domain notice). Add the nomic-embed-code Apache-2.0 license plus a
NOTICE describing the embedding derivation. First-party grammars carry
the project MIT license.

THIRD_PARTY.md now defers grammar provenance to the verified
MANIFEST.md instead of a stale hand-written table (which misattributed
clojure as EPL-1.0 — it is CC0-1.0), covers all vendored libraries, and
documents the Hybrid LSP reference servers and stdlib type-data
provenance (typeshed, Go stdlib introspection, hand-curated specs).
2026-06-12 00:02:30 +02:00
Martin Vogel ad5d45ab3d Document the v0.8.0 hybrid LSP lineup across README and site
Java, Kotlin, and Rust join the Hybrid LSP sections: badge and intro counts go 6 -> 9 language families, the capability tables gain the three new resolver rows (README + docs/index.html), the clean-room reference list adds Eclipse JDT and rust-analyzer, llms.txt and the landing-page JSON-LD/FAQ are aligned, the stale test-count badge moves to 5,577, and the sitemap lastmod is refreshed.
2026-06-11 23:44:34 +02:00
Martin Vogel 244039e6ce Document the Java, Kotlin, and Rust hybrid LSP resolvers in the README
Adds the three v0.8.0 resolver rows to the Hybrid LSP capability table.
0.8.0 v0.8.0
2026-06-11 21:34:36 +02:00
Martin Vogel 16dc384128 Tag the dispatched sha in release-draft, not the checkout HEAD
The release tag must point at the commit the artifacts were built from (github.sha). The step tagged whatever the job checkout resolved to - the branch head at job START - so a commit pushed to main mid-run moved the tag target; when that head commit happened to touch .github/workflows/, the GitHub App token's tag push was rejected outright (refusing to create refs at workflow-modifying commits), failing the 0.8.0 draft.
2026-06-11 21:34:36 +02:00
Martin Vogel 3bbb6bb073 Inject the release version into npm/PyPI packaging at publish time
The 0.8.0 release failed at publish-registries because pkg/npm/package.json still carried the previous hand-pinned version and npm refuses to publish over an existing release. The job now rewrites pkg/npm/package.json and pkg/pypi/pyproject.toml from the workflow's version input (with grep verification that the injection took) before publishing, so a forgotten manual bump can never fail the pipeline again. server.json needs no injection - publish-mcp-registry already syncs it from the published npm package.
2026-06-11 21:01:11 +02:00
Martin Vogel 972577145f Bump registry packaging versions to 0.8.0
The release pipeline publishes pkg/npm and pkg/pypi as-is; both still carried 0.7.0, so npm rejected the publish with 'cannot publish over the previously published versions' and failed the publish-registries job (the GitHub release itself drafted fine with all assets). server.json bumped to match.
2026-06-11 19:50:58 +02:00
Martin Vogel 2b67abfd54 Probe the by-name index in import_map_suffix instead of scanning the whole registry
The import_map_suffix fallback ran cbm_ht_foreach over the ENTIRE exact-QN table (with a strlen per entry) for every call the direct candidate missed - on elasticsearch that is a 700k-entry scan per unresolved call site, and sampling showed it at 94% of resolve-phase CPU (cbm_registry_resolve -> cbm_ht_foreach -> ims_scan -> strlen). Any QN matching 'resolved.*.suffix' necessarily has the suffix's last segment as its own last segment, so the existing by_name index already holds the only possible candidates: probe it and prefix/tail-check the handful of entries. elasticsearch full index: 640 s -> 71 s, nodes identical, edges within 0.001% (iteration-order tiebreaks among equal-confidence candidates); strategy semantics covered by resolve_import_map_suffix and registry_confidence_import_map_suffix.
2026-06-11 15:06:50 +02:00
Martin Vogel 39517860ff Hash cross-LSP registries; two-phase def registration
Profiling symfony (416 s wall, one core pinned) convicted cbm_registry_lookup_type -> strcmp at 95% of samples: the PHP and Python def-registration loops probe the registry per Method (receiver-stub check) BEFORE the hash indexes exist, i.e. a linear scan over the growing type table - O(methods x types) per file, and the cross resolvers run per file. Java and Kotlin never called cbm_registry_finalize at all, leaving every lookup during their file walks linear over the whole cross registry. Fixes: (1) two-phase registration in php/py (types, finalize, then funcs/methods); (2) finalize added to the java/kotlin cross entry points; (3) lookups now tail-scan entries added after finalize - previously such entries were silently INVISIBLE to the hashed path (reproduce-first: tslsp_hash_registry_post_finalize_adds was red); (4) cbm_registry_finalize_into allocates the index from a per-call scratch arena - putting buckets into the pipeline-lifetime result arena per file accumulated +1.1 GB RSS on the FastAPI incremental test (incr_full_index caught it), and the py tier-2 builder (one def per call) skips mid-build finalize entirely. symfony: 416 s -> 12 s, identical node count; suite 5577/0.
2026-06-11 14:25:11 +02:00
Martin Vogel 5706bf52f0 Extract url_path for the dump index via real JSON parsing
The dump path sliced url_path out of the properties string with strstr/strchr, returning ESCAPED bytes and cutting at embedded escaped quotes, while the index's backing column is GENERATED AS json_extract(properties,'$.url_path') and yields the unescaped value. The mismatch left edges 'missing from index idx_edges_url_path' under PRAGMA integrity_check once the formerly-malformed route props became valid JSON. url_path is now parsed with yyjson so the indexed value matches json_extract exactly.
2026-06-10 18:25:08 +02:00
Martin Vogel 8666a99e29 Close the remaining malformed edge-props producers
Three stragglers after 6c04ab7/7f78f93, found by edge json_valid audits on elasticsearch: (1) the parallel emit_route_registration twin still interpolated callee/route_path raw into CALLS props; (2) HANDLES props buffers were smaller than escaped handler QNs plus the JSON wrapper, so snprintf truncated the closing brace (escaping alone cannot prevent wrapper truncation - value buffers now 512B inside 1KB props); (3) DATA_FLOWS assembles sliced handler_params/args_json fragments that can break in ways escaping does not cover - the assembled props are now validated with yyjson and fall back to a minimal valid envelope. Verified by reindexing elasticsearch: edge json_valid violations 47 -> expected 0.
2026-06-10 17:07:10 +02:00
Martin Vogel 7f78f936ec Spill oversized index keys to overflow pages; escape remaining edge-props text
The hand-rolled dump writer stored index cells fully inline regardless of payload size. SQLite's format spills index payloads above the max-local threshold (16,422 bytes at the 64 KB page size) to overflow pages, so a long key written inline makes the reader interpret key bytes as an overflow page number: PRAGMA integrity_check reported 'invalid page number 0x43654C6C' (ASCII from the key text) in idx_nodes_name on elasticsearch (very long Section names), and name lookups on that index silently returned nothing. write_index_btree now rewrites oversized cells (varint + local prefix + overflow chain via write_overflow_pages) before page building; promoted separators carry their chains with them. Reproduce-first: sw_long_index_keys_overflow (20 KB name; integrity_check failed pre-fix, ok post-fix). Also escapes the remaining raw text interpolations in DATA_FLOWS route props and HANDLES handler QNs (the ~51-edge malformed tail after 6c04ab7).
2026-06-10 16:42:20 +02:00
Martin Vogel 036a80ef41 Never mutate the shared Tier-2 cross registry from resolve workers
c_process_function's template-param attach wrote rf->type_param_names on the registry — in the Tier-2 cross-LSP phase that registry is shared READ-ONLY across all resolve workers, so the write raced concurrent readers AND stored a pointer into the writing worker's per-file arena; once that arena was recycled the shared registry held dangling memory, crashing other workers intermittently (bitcoin: SIGSEGV in c_process_function strcmp via cbm_run_c_lsp_cross_with_registry, ~1-in-2 runs). A registry_shared flag now marks the cross context and the attach is skipped there (cross-phase template deduction falls back to positional names — graceful degradation); the single-file extract phase keeps full fidelity. Real-repo-tier reproduction: 10x bitcoin index loop, 0 failures post-fix (pre-fix ~50% crash rate); suite 5575/0.
2026-06-10 16:12:27 +02:00
Martin Vogel 6c04ab7f61 Escape sliced source text in edge properties JSON
Edge properties interpolated raw source-text slices into JSON via snprintf without escaping: decorator text (quotes + raw newlines, e.g. @register.tag("block") or multi-line @override_settings) in DECORATES props - django produced 3826 malformed edges - plus usage ref_name and route-registration callee/url in USAGE/CALLS props. Malformed edge JSON aborts every json_extract consumer, including the url_path_gen generated-column evaluation that runs during PRAGMA integrity_check. All such sites now route through cbm_json_escape (DECORATES twins in pass_parallel.c/pass_semantic.c, usage emit twins, route-registration calls). Regression: pipeline_edge_props_valid_json (register.tag fixture; real-repo red recorded on the django index).
2026-06-10 13:44:09 +02:00
Martin Vogel 0442cceb80 Bound lambda/method-ref expected-param lookup by declared param count
bind_lambda_args and the method-reference arity path indexed the resolved method signature's NULL-terminated param_types array by the CALL-SITE argument index. A call with more arguments than the resolved overload declares (overload mismatch, varargs) read past the terminator and dereferenced whatever followed in the arena as a CBMType* - SIGSEGV while indexing elasticsearch (same out-of-bounds family as #427 and the bitcoin targs fix). Both sites now bound the index by the walked param count. Reproduce-first: 6-line fixture - run(Runnable) invoked with six lambdas - crashed rc=134 pre-fix; fork+exit-signal regression test lsp_java_lambda_args_exceed_params_no_crash added; suite 5574/0.
2026-06-10 13:03:00 +02:00
Martin Vogel bb7341dc07 Fix uninitialized template-arg arrays binding stack garbage into type graphs
The C/C++ LSP's explicit-template-argument call path (template_function, e.g. Using<Fmt>(v)) filled a stack array const CBMType *targs[16] without NULL-terminating it before cbm_type_substitute. With more declared type params than explicit args (bitcoin src/serialize.h: Using's Wrapper<Formatter, T&> with 2 params, 1 arg), the bounded args walk read uninitialized stack; any non-NULL garbage got bound to the type param and woven into the REGISTERED return-type graph, then dereferenced at a later call site - SIGSEGV (crash shape: string bytes misread as a CBMType*). All c_lsp targs arrays are now zero-filled, and cbm_type_substitute's args walk treats implausible values (misaligned / below the first page) as the terminator so contract violations can never leak garbage into a type graph. Reproduce-first: deterministic poison-args unit test (typerep_substitute_rejects_garbage_args_entries) + real-repo verification (bitcoin indexes clean under ASan; was a deterministic SIGSEGV).
2026-06-10 12:11:02 +02:00
Martin Vogel 9f981a1612 Bound LSP resolve recursion depth (stack-overflow crashes indexing real repos)
Indexing large OSS during the perf sweep crashed three LSP resolvers, all by stack overflow in their recursive resolve walks (macOS crash reports): elasticsearch — SIGSEGV in bind_lambda_args under deep recursive java_resolve_calls_in_node frames; bitcoin — SIGSEGV in cbm_type_substitute via c_adl_resolve under deep c_resolve_calls_in_node frames; microsoft/TypeScript — SIGBUS under an unbounded lookup_member_type cycle (cyclic type graph in reallyLargeFile.ts). The expression-eval guards (eval_depth) did not cover these walks. Each resolver now has a depth-guarded entry (walk_depth cap 512 for the Java/C AST walks, member_depth cap 64 for TS member lookup); past the cap the subtree resolves as unknown — graceful degradation instead of a crash. Reproduce-first: prod CLI probes crashed pre-fix (nested-call Java rc=139, nested-call C++ rc=139); fork+exit-signal regression tests added (TS cyclic shape needs a real cross-file registry, so it is verified at the real-repo tier — microsoft/TypeScript now indexes clean). Side-finding documented in the sweep report: Java BLOCK nesting >=3000 is minutes-slow (separate pathology, not this fix).
2026-06-10 10:44:19 +02:00
Martin Vogel 359f37cd71 Escape control bytes in properties-JSON array items (raw newlines from multi-line C params)
The array appenders (param_types/param_names/decorators/base_classes) in both props-builder twins escaped only quote and backslash, so items sliced from multi-line C parameter declarations carried raw newline/tab bytes — invalid inside JSON strings (the remaining 118 malformed kernel rows after the truncation fix; e.g. param_types:["struct\n\t\t..."]). Array items now go through the full escape function, and both escapers degrade any other raw control byte (e.g. form feed) to a space. Reproduced via the prod CLI on a multi-line-param fixture (nl_fn json_valid=0 pre-fix); the sweep regression test gained that fixture (61st case).
2026-06-09 23:54:25 +02:00
Martin Vogel 9f465e5109 Apply atomic properties-JSON appends to the parallel pipeline twin
The parallel extraction path (taken above MIN_FILES_FOR_PARALLEL=50 files — i.e. every real repo) has its own copy of the props appenders in pass_parallel.c, so the previous fix in pass_definitions.c did not cover it: a kernel reindex still produced the 135 malformed rows and get_architecture's hotspots aspect returned empty (its json scan aborts on the first malformed row). Same atomic-append change as the serial twin. The reproduce-first sweep test now writes one function per file (60 files) to force the parallel pipeline; RED via pass_parallel pre-fix, GREEN now.
2026-06-09 23:30:51 +02:00
Martin Vogel 2712756492 test: make arch_boundaries perf guard self-relative (CI machine spread)
The absolute 8s bound failed on the shared ubuntu-arm UBSan runner, which measured 25.9s for work that takes 207ms locally (125x spread). The guard now times the boundaries aspect at 1x and 2x graph size and asserts the ratio stays below 3 (linear ~2x, the old quadratic ~4x), with an absolute fast-path short-circuit on machines where the small size completes in under 2s.
2026-06-09 23:07:47 +02:00
Martin Vogel 4b317c8779 Make definition-properties JSON appends atomic (no mid-value truncation)
build_def_props serialized into a fixed 2KB buffer and cut fields mid-value when full: the closing quote/brace got skipped, leaving malformed properties JSON (Linux kernel: 135 nodes, 50-param functions cut at 2047 bytes inside param_types — '["enum'). Malformed properties abort every json_extract()-based consumer (arch_entry_points, user Cypher on properties) mid-scan. Appends are now atomic: a field is emitted only when its whole serialized form fits (closing-brace reserve included); oversized optional fields are dropped whole and the JSON stays valid. Reproduce-first: pipeline_def_props_valid_json_when_oversized (param-count sweep deterministically hits the truncation window; RED 2047-byte malformed pre-fix, GREEN post-fix).
2026-06-09 22:59:33 +02:00
Martin Vogel 08115fdfb5 get_architecture: skip JSON property-key discovery (counts-only schema)
handle_get_architecture called cbm_store_get_schema, whose per-label/per-type property-key discovery runs json_each() over EVERY node and edge row (~10s of the 23s on a 4.8M-node kernel graph) — yet the handler only renders label/type counts. Added cbm_store_get_schema_counts (same impl, with_props=false) and switched the handler to it. The full get_graph_schema tool is unchanged. A partial expression index on is_entry_point was tried and reverted: json_extract in an index WHERE aborts store open on rows with malformed properties JSON (pre-existing in older DBs; see next commit).
2026-06-09 22:59:32 +02:00
Martin Vogel 66bf33a003 Cap name-resolution candidate lists at 256 (unresolvably ambiguous)
resolve_name_lookup walked the full by_name candidate array (reachability check + score per candidate, re-done per file via the per-file caches) for every cache-miss name. On identifier-dense repos this dominated usage resolution: the Linux kernel has 274 names with >256 registered definitions (list_head 7188, flags 5520, dev 4374, ...) and resolve_usages burned 987s CPU. candidate_count_penalty already floors confidence to ~3/count (<=0.006 at 256), so these matches were noise edges. Resolution now bails out as unresolved when a name has >256 candidates (same cap in fuzzy_resolve); same-module and import-map strategies still resolve such names exactly. Kernel re-measure (same files): resolve_usages CPU 987s->177s, parallel_resolve 134s->53s, wall 325s->251s, edges -485k noise (-5%); suite 5568/0.
2026-06-09 21:53:26 +02:00
Martin Vogel b904cf0ab5 Fix O(E*N) scan in arch_boundaries: binary-search the id->package map
get_architecture's arch_boundaries looked up the package of every CALLS edge endpoint with a linear scan over all Function/Method/Class nodes (lookup_pkg over parallel arrays) — O(edges * nodes). On the Linux kernel graph (~1.4M defs, ~1.4M CALLS edges) the boundaries aspect spun >10 minutes at 100% CPU; latent until C call extraction was fixed (the kernel previously produced almost no CALLS edges). The node query now orders by id and lookup_pkg binary-searches: 80k nodes/160k edges 16.6s -> 207ms in the regression test; kernel-graph get_architecture(all) >10min -> 23s. Reproduce-first: arch_boundaries_no_quadratic_scan (red 16566ms pre-fix vs 8000ms bound, green 207ms post-fix).
2026-06-09 21:32:16 +02:00
Martin Vogel a50b086809 Attribute out-of-line C++ method definitions to their class (distill #428)
C++/CUDA out-of-line method definitions (Foo::bar in a .cpp; the class body lives declaration-only in a header) were recorded as free Functions with no link to their class. A new helper cpp_out_of_line_parent_class() detects a qualified_identifier declarator on a function_definition and resolves the immediate enclosing class (descending nested scopes ns::Foo::bar to the direct parent); extract_func_def then promotes the def to label Method with parent_class set to the class QN so DEFINES_METHOD edges resolve. Distilled (reimplemented) from PR #428. Reproduce-first: cpp_out_of_line_method_issue428 (single-level out-of-line defs; the nested-namespace path isn't isolation-testable because tree-sitter-cpp parses a synthetic doubly-qualified def as an ERROR node, so it is exercised by real codebases).
2026-06-09 14:32:26 +02:00
Martin Vogel fb17573279 Fix out-of-bounds read in cbm_type_substitute when type_args is shorter than type_params (#427)
The C++ LSP passes a class's declared template-param names (e.g. template<class T, class Alloc = ...>) alongside an instance's fewer template args (vector<int>). The substitution loop is bounded by type_params, so a param matching beyond the supplied args indexed type_args[i] past the array — reading adjacent arena memory (a type-name string byte) as a CBMType* that c_eval_expr_type later dereferenced -> SEGV (ASan: stack-buffer-overflow in cbm_type_substitute via c_eval_expr_type). Deterministic, all platforms; distinct from the #424 allocator crash. Fix at the single chokepoint: bound the args length by the (NULL-terminated) param count and gate every type_args[i] access by it, so no caller can over-read regardless of language. Reproduce-first: typerep_substitute_short_args_no_oob_issue427.
2026-06-09 14:32:26 +02:00
Martin Vogel 9dbed50068 Move Glama packaging into pkg/glama; drop redundant packaging/ dir
packaging/glama -> pkg/glama so all packaging lives under pkg/. Update the
_smoke.yml job command and the Dockerfile/comment references to the new path.
verify.sh is path-relative (dirname $0), so its logic is unchanged.
2026-06-09 13:57:12 +02:00
Martin Vogel 8d6dfc5758 Merge branch 'session-combined' 2026-06-09 11:57:03 +02:00
Martin Vogel eab4148de8 Upgrade vendored mimalloc 3.2.8->3.3.2 + platform-correct override; regen vendored checksums (#424)
mimalloc bumped to v3.3.2 (MI_MALLOC_VERSION 30302), which adds static-CRT malloc override on Windows. Makefile: define the real MI_MALLOC_OVERRIDE=1 ONLY on MinGW (enables the static-CRT override entry points — the generic Windows fix for #424); on Unix keep static-link-order override (defining MI_MALLOC_OVERRIDE there compiles alloc-override.c's forwarding defs and crashes macOS with 'mi_free: invalid pointer' since system libs keep the system allocator). The old -DMI_OVERRIDE=1 was never read by mimalloc source — kept as a prod marker. smoke-test.sh: hard-fail on any 'mimalloc: error' in stderr. Regenerated scripts/vendored-checksums.txt for the new tree (security-vendored.sh passes; no dangerous calls in vendored mimalloc).
2026-06-09 11:55:59 +02:00
Martin Vogel 69c16cada4 fix: deliver the static -portable linux binary on all install/update paths
The standard linux release binary dynamically links glibc 2.38+ and
GLIBCXX_3.4.32, so it fails to start on Debian 11, Ubuntu 20.04/22.04,
RHEL/Rocky 8/9, Amazon Linux 2, etc. — yet install.sh, the npm and PyPI
wrappers, and the binary's own self-update all fetched it by default,
contradicting the "single static binary" promise.

Point every linux install + self-update path at the fully-static
"-portable" asset (gcc -static), which has no glibc floor. macOS/Windows
are unaffected and unchanged.

- install.sh, pkg/npm/install.js, pkg/pypi _cli.py: select -portable on linux
- src/cli/cli.c: self-update download URL AND checksum archive name both use
  -portable on linux (they must match or the update fails checksum verify)
- scripts/smoke-test.sh: assert linux self-update targets the -portable asset
- scripts/ci/check-glibc-compat.sh: new guard — runs the binary inside
  debian:bullseye (glibc 2.31) and asserts it starts
- _smoke.yml: run the guard on the portable binary in smoke-linux-portable

Reproduced: standard binary -> "GLIBC_2.38 not found" on glibc 2.31;
portable binary runs cleanly.
2026-06-09 11:36:00 +02:00
Martin Vogel 16103d9e39 fix(glama): use portable static binary so the check image runs on any base
The standard linux release asset dynamically links glibc 2.38 / libstdc++
(GLIBCXX_3.4.32) and failed to start on debian:bookworm-slim (glibc 2.36).
The -portable asset is fully static (gcc -static); the image now builds and
passes the MCP initialize + tools/list handshake (15 tools).
2026-06-09 11:36:00 +02:00
Martin Vogel fa1021630a Add Glama directory integration (glama.json + check image)
- glama.json claims maintainership on glama.ai.
- packaging/glama/Dockerfile wraps the static Linux binary in a minimal
  image so Glama can launch the stdio server and run its MCP introspection
  checks (which power the directory score badge). The tool needs no Docker
  to run; this image is only for the directory integration. Fetches the
  latest release binary, arch-aware via TARGETARCH.
- packaging/glama/verify.sh builds the image and asserts the server answers
  initialize + tools/list with no project indexed.
- _smoke.yml runs verify.sh as a non-gating job (continue-on-error) so the
  integration is guarded against drift without ever blocking a release.
2026-06-09 11:36:00 +02:00
Martin Vogel 9507c4b0a1 Add official MCP Registry publishing to the release pipeline
- Add `mcpName` to the npm package and an `mcp-name:` marker to the PyPI
  README so the registry can verify package ownership for both.
- Rewrite server.json to reference the npm + PyPI packages instead of
  five per-platform mcpb tarballs. The old form was pinned to a stale
  0.6.1 with hardcoded SHA-256s that needed manual updates every release;
  the package form only needs a version sync. Also trims the description
  to the registry's 100-character limit.
- Add a publish-mcp-registry job: downloads mcp-publisher, authenticates
  via GitHub Actions OIDC (no token, no device flow), syncs server.json's
  version to the just-published npm package, and publishes. It runs after
  npm/PyPI but does not gate the release un-draft, so a registry outage
  never blocks shipping and the job can be retried on its own.
2026-06-09 11:36:00 +02:00
Martin Vogel 332bea6b42 Bind sqlite3 and libgit2 to mimalloc explicitly in prod build (#424 follow-up)
Defense-in-depth follow-up to the tree-sitter bind: route sqlite3 and libgit2
allocations through mimalloc explicitly so a correct binary never depends on the
fragile MI_OVERRIDE symbol override (where the Windows static-MinGW link can
resolve a library's malloc to mimalloc but its free to the CRT, corrupting the
heap). Prod-only, guarded by the existing CBM_BIND_TS_ALLOCATOR; the CRT+ASan
test build compiles these as no-ops and stays unchanged.

New cbm_alloc_init() consolidates all three binds behind a static idempotency
guard. main() calls it as its very first statement — before cbm_mcp_server_new
opens the in-memory store — because SQLITE_CONFIG_MALLOC must run before the
first sqlite3_open*/sqlite3_initialize, else sqlite3_config returns SQLITE_MISUSE
silently and the bind is ignored (the return code is asserted). cbm_init() also
calls cbm_alloc_init() so pipeline-pass entry points get the binds too; the
ts_set_allocator bind moved out of cbm_init into cbm_alloc_init.

sqlite3: a static sqlite3_mem_methods backed by mi_malloc/mi_free/mi_realloc,
xSize via mi_usable_size, xRoundup to an 8-byte boundary, xInit returning
SQLITE_OK, xShutdown a no-op (int-sized sqlite signatures wrapped with size_t
casts). libgit2: a static git_allocator (gmalloc/grealloc/gfree → mi_*) installed
via git_libgit2_opts(GIT_OPT_SET_ALLOCATOR). The allocator is set BEFORE any
git_libgit2_init: git_allocator_global_init keeps a custom allocator if one is
already set (git__allocator.gmalloc != git_failalloc_malloc), the pre-init global
is the fail-allocator, and there is no allocator reset on git_libgit2_shutdown —
so binding once at startup survives pass_githistory's per-call init/shutdown
pairs. libgit2 code is additionally guarded by HAVE_LIBGIT2.

Test build 5564/0 ASan-clean; prod build clean with CBM_BIND_TS_ALLOCATOR=1; the
HAVE_LIBGIT2 branch type-checked against the libgit2 sys/alloc.h struct shape;
lint-ci clean.
2026-06-09 11:25:56 +02:00
Martin Vogel bb4af38a29 smoke: index a large templated C++ header to reproduce the #424 prod crash
The C++ heap-corruption crash only manifests in the production binary (MI_OVERRIDE=1), so the smoke job (which runs the prod binary) is where it reproduces — most reliably on Windows. A ~10k-line templated header exercises the parse-churn path; indexing it must succeed.
2026-06-09 09:21:34 +02:00
Martin Vogel 0d2da15d4e Bind tree-sitter runtime to mimalloc in prod build to fix C++ indexing crash (#424)
Root cause (corrected from the report): not a tree-sitter null-subtree sentinel — 0xffffffffffffffff is a freed/heap-corrupted pointer. The vendored ts runtime allocates via its overridable ts_current_malloc/free; under the prod build's MI_OVERRIDE=1, the Windows static-MinGW link (--allow-multiple-definition) resolves ts_malloc to mimalloc but ts_free can resolve to the CRT free (or vice-versa), corrupting the heap freelist and crashing mid-parse on large templated C++ headers (scales with parse churn, not syntax). cbm_init now calls ts_set_allocator(mi_malloc, mi_calloc, mi_realloc, mi_free) so ts allocate+free go through one allocator on every platform. Guarded to CFLAGS_PROD (MI_OVERRIDE=1); the test build stays CRT+ASan to avoid an alloc/free mismatch there. Tests in test_stack_overflow.c: allocator-binding mechanism + large-templated-C++ extraction guard (the C++ gap noted in #424).
2026-06-09 09:20:15 +02:00
Martin Vogel 0314a216c6 Rename k8s enum PATH_MAX to avoid system-macro collision on macOS-15/Windows
PATH_MAX is defined by limits.h on the macOS-15 SDK (1024) and MSYS2 clang64 (260), so 'enum { PATH_MAX = 12 }' expanded to 'enum { 1024 = 12 }' → 'expected identifier' build failure on those platforms (Linux/macOS-14 didn't pull the macro into this TU). Renamed to K8S_PATH_DEPTH.
2026-06-09 00:59:19 +02:00
Martin Vogel 632af52f26 Make namespace import resolution deterministic across platforms
The namespace map kept only the first file per package (by platform-dependent iteration order). A same-package wildcard import (import com.example.* from a file itself in com.example) could resolve to the importer → self-edge suppressed → 0 IMPORTS on amd64 but 1 on arm64 (ei_kotlin_wildcard_import). Now store ALL files per namespace as a delimited list and resolve to the first non-importer sibling — deterministic on amd64/arm64. Re-key via the stored key pointer to avoid a use-after-free.
2026-06-09 00:01:36 +02:00
Martin Vogel 6f3b24df4f Fix CI lint: drop debug scaffolding, make lisp import walker iterative, apply clang-format
Remove leftover getenv(CBM_DBG_*) debug fprintf blocks in extract_calls.c/pass_calls.c (cppcheck always-true). Convert parse_lisp_imports to a stack-based walk (no recursion → no NOLINT, avoids deep-nesting overflow). Apply Homebrew clang-format to all touched sources. Local suite 5562/0, lint-ci clean.
2026-06-08 22:55:23 +02:00
Martin Vogel 54108092d8 Recover Kotlin classes from grammar ERROR nodes (companion object / inner class with delegation)
fwcd/tree-sitter-kotlin wraps a whole class in an ERROR node when a member uses companion object / inner class combined with a delegation specifier. recover_kotlin_error_classes scans an ERROR node's flat tokens for class/object keyword + identifier, emits the Class def (respecting enclosing class QN) and collects bases from delegation_specifier siblings. Strictly additive (only fires on ERROR regions). Fixes inherit_kotlin (MyClass companion + Node inner).
2026-06-08 22:55:23 +02:00
Martin Vogel b30fe13de1 Emit INFRA_MAPS for HCL scheduler + k8s selectors, ASYNC_CALLS for SQS struct args
extract_calls: recover SQS/SNS queue identity from Go composite-literal struct fields (QueueUrl/TopicArn) so emit_http_async_edge forms ASYNC_CALLS. extract_unified: HCL value/body descent + cloud_scheduler_job source synthesis. pipeline: self-create topic Route for standalone-manifest infra bindings + run infra-route passes in the sequential pipeline. pass_k8s: cross-manifest Service-selector -> workload label/name matching -> INFRA_MAPS.
2026-06-08 22:55:23 +02:00
Martin Vogel de2c54ac1f Emit DECORATES to synthetic nodes for external decorators + python subscript-dunder CALLS
pass_semantic + pass_parallel: when a decorator name resolves to no local def, upsert a synthetic '<decorator:NAME>' node (label Decorator) and emit DECORATES+USAGE — fixes rust #[derive]/#[allow], swift @discardableResult, scala @deprecated (deduped per project; pipeline-level so extraction histograms unaffected). py_lsp: inject a synthetic CBMCall for resolved subscript/operator dunders on typed receivers so they reach the CALLS pipeline — fixes python s[0]->Seq.__getitem__.
2026-06-08 22:55:23 +02:00
Martin Vogel 53f1bf2ebd Annotate python dunder_getitem receiver (sound subscript-dunder resolution)
The unannotated param could not be resolved soundly (would require an unsound sole-class guess). Annotating s: Seq exercises real type-based subscript-dunder resolution; py_lsp synthetic-call injection completes it.
2026-06-08 22:55:23 +02:00
Martin Vogel 35ac60c544 Resolve C# field writes + Kotlin cross-file calls
lang_specs: C# postfix/prefix_unary_expression as assignments (x++ -> WRITES). extract_semantic: resolve_write_lhs_node for unary inc/dec write targets. helpers: dedicated kotlin_keywords (Kotlin doesn't reserve double/int/... so fun double()/call double() now extract). kotlin_lsp + pass_lsp_cross: cbm_run_kotlin_lsp_cross + sole-definer top-level fallback. java_lsp: sole-definer type fallback for cross-file static calls. Fixes convergence 571, edge_structural 374, lsp_resolution 1210.
2026-06-08 22:55:23 +02:00
Martin Vogel eb47f86551 Resolve Rust operator-trait and macro-arg method calls
rust_lsp: operator desugaring (a+b -> Type::add etc., index_expression -> index) resolving the left-operand user type soundly; re-parse std macro args (format!/println!/assert!/...) so receiver-method calls inside them (d.label()) resolve; inject synthetic CBMCall sites so recovered calls reach the edge pipeline. Fixes convergence 965/1119 + matrix c2/rust/add_trait.
2026-06-08 22:55:23 +02:00
Martin Vogel 0d87a86e54 Resolve IMPORTS edges for 12 grammar-only languages
extract_imports: recursive lisp walker (Clojure ns/:require, Common Lisp defpackage/:use, Fennel nested require) + parsers for elm/move/smali(descriptor-demangle)/tlaplus(EXTENDS)/vhdl(use work)/wit/smithy(#->.)/hyprlang(source). lang_specs: astro frontmatter JS re-parse for component imports. pass_pkgmap: basename sibling-file candidate for prefixed include paths. All sound resolution via namespace-map / sibling / symbol-fallback / relative-import.
2026-06-08 22:55:23 +02:00
DeusDataandDeusData bd6d617bcf ci: deploy Pages via Actions, scoped to docs changes (#425)
The legacy "Deploy from a branch" Pages build ran Jekyll on every push to
main and failed on the planning docs under /docs — Jekyll's Liquid parser
hit `Unknown tag 'data'` in EVALUATION_PLAN.md, so the website build went
red after nearly every commit.

/docs is a hand-written static site (index.html + robots/sitemap/llms.txt),
so it never needed Jekyll. Deploy it with a GitHub Actions workflow that
uploads /docs as-is and only triggers on docs/** changes, and add a
.nojekyll guard. Switch the Pages source to "GitHub Actions" for this to
take over from the legacy build.

Co-authored-by: DeusData <joerge@datadice.io>
2026-06-08 18:26:11 +02:00