Martin Vogel
0df990aa60
Routing .ets through the plain TypeScript grammar produced no definition node for any @Component struct (0/19 across the OpenHarmony sample corpus), parsed struct members as statement labels and build() as a call expression, and registered ArkUI builtins (Column, Row, ListItem, ...) as user function definitions with 28% fabricated cross-file CALLS edges. This adds a dedicated ArkTS grammar that keeps everything TypeScript already handles and models the ArkTS syntax TS cannot parse. Grammar (tools/tree-sitter-arkts/, generated parser vendored to internal/cbm/vendored/grammars/arkts/, ABI 15 via tree-sitter-cli 0.25.10): first-party derivative of tree-sitter/tree-sitter-typescript v0.23.2 @ 75b3874edb2d (the commit our vendored typescript/tsx come from) on the tree-sitter-javascript v0.23.1 base @ 3a837b6f3658 (the version upstream's package-lock pins). LICENSE carries the upstream MIT text verbatim plus a derivative notice; the external scanner is a symbol-rename trampoline over a byte-identical copy of the already-vendored typescript scanner header. ArkTS additions: decorated struct component declarations with class-shaped bodies; UI-DSL trailing-closure component calls with attribute chains after the closing brace; decorators on function declarations (@Builder, @Extend(T), @Styles); @Extend/@Styles leading-dot attribute-chain bodies; anonymous stateStyles style blocks; import lazy named imports. Wiring: CBM_LANG_ARKTS enum + lang spec (struct_declaration as a class kind labeled Struct, public_field_definition fields), .ets extension, LANG_NAMES/userconfig entries, grammar shim, and TS-family joins across the extractors and the name-resolution guards (parse_es_imports, JS keywords/test-file/func-kind/scope tables, TS heritage/param/receiver/ namespace/type-ref/channel/usage paths, js_ts_family and the weak-method- match suppression). Field definitions now also carry their decorators, so @State/@Prop/@Link members are findable with their roles. Tests: grammar corpus (35 cases), per-language regression + label golden + import net + extension cases, seven ArkTS extraction tests including the phantom-builtin regression (ArkUI builtins must never be definitions), and capability-ledger / call-node-manifest / call-argument-matrix rows. On the 14-file OpenHarmony sample corpus the TS-grammar baseline had 12/14 files with parse errors (231 ERROR nodes); the arkts grammar parses 14/14 clean, and an end-to-end index yields all 19 components as Struct nodes, 68 methods, 119 fields, and zero phantom builtin definitions. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
9.1 KiB
Third-Party Licenses
This project vendors third-party code. We are grateful to the authors and
maintainers of these projects for making their work freely available.
Every vendored component directory carries the upstream LICENSE
(or COPYING / NOTICE) file alongside the sources.
Tree-sitter Runtime
The tree-sitter C runtime is vendored in internal/cbm/vendored/ts_runtime/.
- Project: tree-sitter
- License: MIT
- Copyright: (c) 2018–2024 Max Brunsfeld
Local modification (internal/cbm/vendored/ts_runtime/src/stack.c, #913): a
single CBM patch bounds the recursive ambiguity-merge in stack_node_add_link
at CBM_TS_STACK_MERGE_MAX_DEPTH (512). Deeply nested grammar-ambiguous input
(e.g. Perl f(f(f(...)))) otherwise recurses once per level on the native C
stack and overflows it during parsing (SIGSEGV on the ~1 MB Windows thread
stack, and even the 8 MB POSIX stack at extreme depth) before any extractor
runs. Past the cap the ambiguity is left on the GLR stack instead of merged —
still a valid parse, never a wrong one — mirroring the existing
MAX_LINK_COUNT bail-out. The change is clearly marked // CBM patch: inline.
On re-vendor (e.g. ts_runtime → 0.26.x): re-apply this bound.
The shared scanner helpers in internal/cbm/vendored/common/ (scanner.h,
tag.h) originate from
tree-sitter-html (MIT,
(c) 2014 Max Brunsfeld) and carry that project's LICENSE in
internal/cbm/vendored/common/.
The core runtime headers in internal/cbm/vendored/common/tree_sitter/
(alloc.h, array.h, parser.h) are part of the tree-sitter C runtime
(tree-sitter, MIT,
(c) 2018 Max Brunsfeld) and carry their own LICENSE in that directory.
Tree-sitter Grammars
160 pre-generated parsers are vendored in internal/cbm/vendored/grammars/<lang>/
(generated parser.c plus scanner.c where applicable, compiled statically).
Each grammar is the work of its upstream authors and each grammar directory
contains the upstream LICENSE file.
The canonical provenance record — upstream repository, pinned commit, and
cross-registry verification status for every grammar — is
internal/cbm/vendored/grammars/MANIFEST.md.
License summary:
- Nearly all grammars are MIT-licensed.
clojure(sogaiu/tree-sitter-clojure) is CC0-1.0;fennelis CC0-1.0;jinja2andjustare Apache-2.0;pineis ISC (declared by its upstream).- The grammars authored in-house for this project (
cobol,form,janet,magma,protobuf,wolfram) are MIT under the project's own license, (c) DeusData. Seven further grammars (arkts,assembly,cfml,cfscript,dotenv,pine,qml) are self-maintained forks that retain their original upstream authors' licenses — see the manifest for per-grammar provenance.arktsis a first-party derivative of tree-sitter/tree-sitter-typescript (MIT, (c) 2017 Max Brunsfeld; on the tree-sitter-javascript base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions; its grammar source lives intools/tree-sitter-arkts/.
tree-sitter-objectscript (UDL + routine)
- Project: intersystems/tree-sitter-objectscript
- License: MIT
- Copyright: (c) 2025 InterSystems Corporation
- Vendored at:
internal/cbm/vendored/grammars/objectscript_udl/,internal/cbm/vendored/grammars/objectscript_routine/ - Pinned commit:
a7ffcdf - Notes: InterSystems-maintained grammar for the ObjectScript language (InterSystems IRIS / Caché). Vendor-maintained; not in nvim-treesitter or Helix registries. Each
scanner.c's upstream#include "../../common/scanner.h"is repointed to a per-directoryobjectscript_common.hcopied from upstreamcommon/scanner.h; two loop counters in that copy are widened fromuint8_ttointas documented ininternal/cbm/vendored/grammars/MANIFEST.md.
Vendored C/C++ Libraries
| Library | Path | License | Project |
|---|---|---|---|
| SQLite 3 | vendored/sqlite3/ |
Public Domain | sqlite.org |
| mimalloc | vendored/mimalloc/ |
MIT | microsoft/mimalloc |
| yyjson | vendored/yyjson/ |
MIT | ibireme/yyjson |
| xxHash | vendored/xxhash/ |
BSD-2-Clause | Cyan4973/xxHash |
| TRE | vendored/tre/ |
BSD-2-Clause | laurikari/tre |
| LZ4 | internal/cbm/vendored/lz4/ |
BSD-2-Clause (library files) | lz4/lz4 |
| Zstandard | internal/cbm/vendored/zstd/ |
BSD-3-Clause (dual BSD / GPLv2 — BSD selected) | facebook/zstd |
| simplecpp | internal/cbm/vendored/simplecpp/ |
0BSD | danmar/simplecpp |
| Verstable | internal/cbm/vendored/verstable/ |
MIT | JacksonAllan/Verstable |
| wyhash | internal/cbm/vendored/wyhash/ |
Unlicense (public domain) | wangyi-fudan/wyhash |
Local modifications to these libraries are documented next to the
vendored sources (currently only SQLite: vendored/sqlite3/PATCHES.md,
raising the Unix VFS MAX_PATHNAME ceiling from 512 to 4096 to match
CBM's 4 KiB path support). Patches must be reapplied on every upstream
refresh and are covered by scripts/vendored-checksums.txt.
The graph-UI HTTP server is a first-party implementation
(src/ui/httpd.c + src/ui/http_server.c) — no third-party HTTP library
is used.
Embedded Model Data
Semantic vector search uses static token embeddings derived from the
nomic-embed-code model, vendored in vendored/nomic/:
- Model: nomic-ai/nomic-embed-code
- License: Apache License 2.0
- Copyright: (c) Nomic AI
See vendored/nomic/NOTICE for the exact derivation procedure
(per-token inference + int8 quantization via scripts/extract_nomic_vectors.py).
Hybrid LSP — Reference Language Servers
The Hybrid LSP layer (internal/cbm/lsp/) is an original C implementation
written for this project. It contains no source code from any language
server. Its type-resolution behavior is structurally inspired by, and
validated for output compatibility against, the published behavior of the
following language servers and language specifications. They are listed here
as acknowledgment; their licenses are noted for reference:
| Language | Reference implementation / specification | Upstream license |
|---|---|---|
| TypeScript / JavaScript | tsserver (microsoft/TypeScript), typescript-go | Apache-2.0 |
| Python | pyright | MIT |
| Go | gopls (golang/tools) | BSD-3-Clause |
| PHP | PHP language reference + Composer PSR-4 autoloading specification | — |
| C# | Roslyn (dotnet/roslyn) | MIT |
| C / C++ | clangd (llvm/llvm-project) | Apache-2.0 WITH LLVM-exception |
| Java | Java Language Specification; output parity with Eclipse JDT LS | EPL-2.0 (reference only) |
| Kotlin | Kotlin language specification; fwcd/kotlin-language-server | MIT |
| Rust | rust-analyzer | MIT OR Apache-2.0 |
Standard-library type data
The stdlib type registries in internal/cbm/lsp/generated/ were produced as
follows:
- Python (
python_stdlib_data.c) — generated from python/typeshed type stubs (commita7912d521e16ff63caf7a8b64b9072542be36777), Apache-2.0, (c) the typeshed contributors. The generator isscripts/gen-py-stdlib.py. - Go (
go_stdlib_data.c) — generated by introspecting the public API of the Go standard library (golang/go, BSD-3-Clause). - Java, Kotlin, C#, PHP, C/C++, Rust — hand-curated from public API documentation and language specifications; no upstream source code was extracted or transcribed.
Embedded Graph UI
Release binaries built with --with-ui embed the compiled graph-ui/
frontend bundle. Its npm dependencies (React, three.js, @react-three/*,
radix-ui, lucide-react, tailwindcss, and friends) are all under permissive
licenses (MIT / ISC / Apache-2.0 / Zlib); the exact set is recorded in
graph-ui/package.json and graph-ui/package-lock.json, and the per-package
license texts of the production bundle are appended to the
THIRD_PARTY_NOTICES.md shipped inside the -ui release archives
(generated by scripts/gen-ui-licenses.py).