Martin Vogel
1b79558975
A plain '&' inside JSX -- a URL query in an attribute (href="https://fonts.googleapis.com/css2?family=Inter&display=swap"), or text such as <p>Tom & Jerry</p> -- produced a tree-sitter ERROR node in .js, .jsx and .tsx files. The file was flagged parse_partial in the coverage view, and the error recovery turned words from the URL (create-next-app -> create/next/app) into identifiers that resolved to unrelated symbols as false USAGE edges. Root cause is upstream tree-sitter-javascript, which the tsx dialect builds on: unescaped_{double,single}_jsx_string_fragment is /([^"&]|&[^#A-Za-z])+/, so '&' is only accepted before a space/digit or as a complete html_character_reference (& / &). jsx_text is scanned by the external scanner, which stops at every '&', and nothing in _jsx_child accepted a '&' that is not a reference. So '&' + letter in a string, and any lone '&' in JSX text, had no valid token. Fix: javascript and tsx become self-maintained forks of the pins we already vendored (javascript @ 58404d8cf191; typescript tsx @ 75b3874edb2d on its javascript base @ 3a837b6f3658), following the arkts model. One patch (tools/tree-sitter-javascript/patches/ jsx-lone-ampersand.patch) is applied to both javascript bases: - a one-character '&' token at lexical precedence -1, aliased to the existing string_fragment (in strings) and jsx_text (in text); the lower precedence keeps & / & lexing as html_character_reference - the string-fragment regexes exclude a '&' directly before the closing quote, so "a&" no longer runs past the end of the string Node types are unchanged (same type/named set). The external scanners stay upstream bytes. Parsers regenerated with tree-sitter-cli 0.25.10 (ABI 15; tsx moves from ABI 14, and its tree_sitter/ headers now match arkts). Without the patch both fork dirs regenerate upstream exactly. The upstream corpus passes unchanged: javascript 116/116; tsx gives the same results as unpatched upstream. Proof on vercel/next.js examples/ (canary 725705bd, 1,722 js/jsx/tsx files): parse_partial 92 -> 47 (tsx 33 -> 0, jsx 2 -> 0, js 12 -> 2; the two left are Babel do-expressions and Flow types), no newly flagged file, code-graph nodes 14,739 -> 14,739, edges 24,773 -> 24,657: the 115 USAGE edges removed were all false cross-example edges from URL fragments, plus a 4 SIMILAR_TO / +3 SEMANTICALLY_RELATED reshuffle from the corrected ASTs. Registered like arkts: MANIFEST.md fork rows + notes (ABI tally recounted), audit FORKS map, THIRD_PARTY.md, vendored checksums. LICENSEs stay upstream's MIT byte-identical. Tests: parse_coverage gains a .js/.jsx/.tsx matrix (double/single-quoted attribute, '&' before the quote, '&' + letter/space/digit in text, plus &/& controls) and a guard that broken JSX is still flagged. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
11 KiB
Third-Party Licenses
This project vendors third-party code. We are grateful to the authors and
maintainers of these projects for making their work freely available.
Every vendored component directory carries the upstream LICENSE
(or COPYING / NOTICE) file alongside the sources.
Tree-sitter Runtime
The tree-sitter C runtime is vendored in internal/cbm/vendored/ts_runtime/.
- Project: tree-sitter
- License: MIT
- Copyright: (c) 2018–2024 Max Brunsfeld
Local modification (internal/cbm/vendored/ts_runtime/src/stack.c, #913): a
single CBM patch bounds the recursive ambiguity-merge in stack_node_add_link
at CBM_TS_STACK_MERGE_MAX_DEPTH (512). Deeply nested grammar-ambiguous input
(e.g. Perl f(f(f(...)))) otherwise recurses once per level on the native C
stack and overflows it during parsing (SIGSEGV on the ~1 MB Windows thread
stack, and even the 8 MB POSIX stack at extreme depth) before any extractor
runs. Past the cap the ambiguity is left on the GLR stack instead of merged —
still a valid parse, never a wrong one — mirroring the existing
MAX_LINK_COUNT bail-out. The change is clearly marked // CBM patch: inline.
On re-vendor (e.g. ts_runtime → 0.26.x): re-apply this bound.
The shared scanner helpers in internal/cbm/vendored/common/ (scanner.h,
tag.h) originate from
tree-sitter-html (MIT,
(c) 2014 Max Brunsfeld) and carry that project's LICENSE in
internal/cbm/vendored/common/.
The core runtime headers in internal/cbm/vendored/common/tree_sitter/
(alloc.h, array.h, parser.h) are part of the tree-sitter C runtime
(tree-sitter, MIT,
(c) 2018 Max Brunsfeld) and carry their own LICENSE in that directory.
Tree-sitter Grammars
160 pre-generated parsers are vendored in internal/cbm/vendored/grammars/<lang>/
(generated parser.c plus scanner.c where applicable, compiled statically).
Each grammar is the work of its upstream authors and each grammar directory
contains the upstream LICENSE file.
The canonical provenance record — upstream repository, pinned commit, and
cross-registry verification status for every grammar — is
internal/cbm/vendored/grammars/MANIFEST.md.
License summary:
- Nearly all grammars are MIT-licensed.
clojure(sogaiu/tree-sitter-clojure) is CC0-1.0;fennelis CC0-1.0;jinja2andjustare Apache-2.0;pineis ISC (declared by its upstream).- The grammars authored in-house for this project (
chialisp,cobol,form,janet,magma,protobuf,wolfram) are MIT under the project's own license, (c) DeusData. Each ships the repository's own LICENSE, byte-identical to the root copy; they carry no third-party copyright because there is no third party.chialispis a generic s-expression grammar for the Chia smart-coin language, written for this project because no usable public grammar exists; its source and corpus tests live intools/tree-sitter-chialisp/. - Nine further grammars (
arkts,assembly,cfml,cfscript,dotenv,javascript,pine,qml,tsx) are self-maintained forks that retain their original upstream authors' licenses — see the manifest for per-grammar provenance.arktsis a first-party derivative of tree-sitter/tree-sitter-typescript (MIT, (c) 2017 Max Brunsfeld; on the tree-sitter-javascript base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions; its grammar source lives intools/tree-sitter-arkts/.javascript(tree-sitter/tree-sitter-javascript, MIT, (c) 2014 Max Brunsfeld) andtsx(tree-sitter/tree-sitter-typescript'stsxdialect, MIT, (c) 2017 Max Brunsfeld, on the same javascript base) are the upstream grammars with one (c) 2026 DeusData patch that lets a lone&appear in JSX strings and text; their grammar sources live intools/tree-sitter-javascript/andtools/tree-sitter-tsx/.
tree-sitter-plsql
- Project: AndreasMaierDe/tree-sitter-plsql
- License: MIT
- Copyright: (c) 2022 AndreasMaierDe
- Vendored at:
internal/cbm/vendored/grammars/plsql/ - Pinned commit:
28aebef209be - Notes: Community-maintained grammar for Oracle PL/SQL; not in the
nvim-treesitter or Helix registries (
community-nichein the manifest). No external scanner. One local patch:parser.c's#include <tree_sitter/parser.h>is changed to the quoted form used by every other vendored grammar, as documented ininternal/cbm/vendored/grammars/MANIFEST.md. PL/SQL support was originally contributed in PR #1033 by Oğuz (@ouzsrcm).
tree-sitter-objectscript (UDL + routine)
- Project: intersystems/tree-sitter-objectscript
- License: MIT
- Copyright: (c) 2025 InterSystems Corporation
- Vendored at:
internal/cbm/vendored/grammars/objectscript_udl/,internal/cbm/vendored/grammars/objectscript_routine/ - Pinned commit:
a7ffcdf - Notes: InterSystems-maintained grammar for the ObjectScript language (InterSystems IRIS / Caché). Vendor-maintained; not in nvim-treesitter or Helix registries. Each
scanner.c's upstream#include "../../common/scanner.h"is repointed to a per-directoryobjectscript_common.hcopied from upstreamcommon/scanner.h; two loop counters in that copy are widened fromuint8_ttointas documented ininternal/cbm/vendored/grammars/MANIFEST.md.
Vendored C/C++ Libraries
| Library | Path | License | Project |
|---|---|---|---|
| SQLite 3 | vendored/sqlite3/ |
Public Domain | sqlite.org |
| mimalloc | vendored/mimalloc/ |
MIT | microsoft/mimalloc |
| yyjson | vendored/yyjson/ |
MIT | ibireme/yyjson |
| xxHash | vendored/xxhash/ |
BSD-2-Clause | Cyan4973/xxHash |
| TRE | vendored/tre/ |
BSD-2-Clause | laurikari/tre |
| LZ4 | internal/cbm/vendored/lz4/ |
BSD-2-Clause (library files) | lz4/lz4 |
| Zstandard | internal/cbm/vendored/zstd/ |
BSD-3-Clause (dual BSD / GPLv2 — BSD selected) | facebook/zstd |
| simplecpp | internal/cbm/vendored/simplecpp/ |
0BSD | danmar/simplecpp |
| Verstable | internal/cbm/vendored/verstable/ |
MIT | JacksonAllan/Verstable |
| wyhash | internal/cbm/vendored/wyhash/ |
Unlicense (public domain) | wangyi-fudan/wyhash |
Local modifications to these libraries are documented next to the
vendored sources (currently only SQLite: vendored/sqlite3/PATCHES.md,
raising the Unix VFS MAX_PATHNAME ceiling from 512 to 4096 to match
CBM's 4 KiB path support). Patches must be reapplied on every upstream
refresh and are covered by scripts/vendored-checksums.txt.
The graph-UI HTTP server is a first-party implementation
(src/ui/httpd.c + src/ui/http_server.c) — no third-party HTTP library
is used.
Embedded Model Data
Semantic vector search uses static token embeddings derived from the
nomic-embed-code model, vendored in vendored/nomic/:
- Model: nomic-ai/nomic-embed-code
- License: Apache License 2.0
- Copyright: (c) Nomic AI
See vendored/nomic/NOTICE for the exact derivation procedure
(per-token inference + int8 quantization via scripts/extract_nomic_vectors.py).
Hybrid LSP — Reference Language Servers
The Hybrid LSP layer (internal/cbm/lsp/) is an original C implementation
written for this project. It contains no source code from any language
server. Its type-resolution behavior is structurally inspired by, and
validated for output compatibility against, the published behavior of the
following language servers and language specifications. They are listed here
as acknowledgment; their licenses are noted for reference:
| Language | Reference implementation / specification | Upstream license |
|---|---|---|
| TypeScript / JavaScript | tsserver (microsoft/TypeScript), typescript-go | Apache-2.0 |
| Python | pyright | MIT |
| Go | gopls (golang/tools) | BSD-3-Clause |
| PHP | PHP language reference + Composer PSR-4 autoloading specification | — |
| C# | Roslyn (dotnet/roslyn) | MIT |
| C / C++ | clangd (llvm/llvm-project) | Apache-2.0 WITH LLVM-exception |
| Java | Java Language Specification; output parity with Eclipse JDT LS | EPL-2.0 (reference only) |
| Kotlin | Kotlin language specification; fwcd/kotlin-language-server | MIT |
| Rust | rust-analyzer | MIT OR Apache-2.0 |
Standard-library type data
The stdlib type registries in internal/cbm/lsp/generated/ were produced as
follows:
- Python (
python_stdlib_data.c) — generated from python/typeshed type stubs (commita7912d521e16ff63caf7a8b64b9072542be36777), Apache-2.0, (c) the typeshed contributors. The generator isscripts/gen-py-stdlib.py. - Go (
go_stdlib_data.c) — generated by introspecting the public API of the Go standard library (golang/go, BSD-3-Clause). - Java, Kotlin, C#, PHP, C/C++, Rust — hand-curated from public API documentation and language specifications; no upstream source code was extracted or transcribed.
Embedded Graph UI
Release binaries built with --with-ui embed the compiled graph-ui/
frontend bundle. Its npm dependencies (React, three.js, @react-three/*,
radix-ui, lucide-react, tailwindcss, and friends) are all under permissive
licenses (MIT / ISC / Apache-2.0 / Zlib); the exact set is recorded in
graph-ui/package.json and graph-ui/package-lock.json, and the per-package
license texts of the production bundle are appended to the
THIRD_PARTY_NOTICES.md shipped inside the -ui release archives
(generated by scripts/gen-ui-licenses.py).