Martin Vogelanddarkverbito
706bb2ce81
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:
* comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
* `(include foo.clib)` was rejected because of the dot in the filename;
* `(defconstant NAME <expr>)` accepted only a primitive after the name.
So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.
It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.
Extraction, and the four decisions in it:
* `Constant` is admitted to the cross-file name registry
(cbm_label_is_registry_symbol). Chialisp's condition codes live as
`(defconstant ...)` in a shared .clib and are referenced by name from every
puzzle; labelled but unregistered they are seeded nowhere, and roughly half
of a Chialisp graph would be present as nodes yet never the target of an
edge. All three registry-seeding sites route through that one predicate, so
they cannot diverge. No other language emits `Constant`, so the blast radius
is exactly Chialisp. Pinned by a two-file graph test that goes red when the
label is removed from the predicate.
* `export` / `namespace` are NOT definition heads. `(export foo)` names a
function already defined in the same file, so treating it as a def head
mints a second node for one symbol. They stay in the not-a-call filter.
* A parameter list is a `list` too, but it binds rather than invokes. Without
that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
* `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
recording it would invent a dependency on a file that does not exist.
Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.
Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.
Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.
Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
10 KiB
Third-Party Licenses
This project vendors third-party code. We are grateful to the authors and
maintainers of these projects for making their work freely available.
Every vendored component directory carries the upstream LICENSE
(or COPYING / NOTICE) file alongside the sources.
Tree-sitter Runtime
The tree-sitter C runtime is vendored in internal/cbm/vendored/ts_runtime/.
- Project: tree-sitter
- License: MIT
- Copyright: (c) 2018–2024 Max Brunsfeld
Local modification (internal/cbm/vendored/ts_runtime/src/stack.c, #913): a
single CBM patch bounds the recursive ambiguity-merge in stack_node_add_link
at CBM_TS_STACK_MERGE_MAX_DEPTH (512). Deeply nested grammar-ambiguous input
(e.g. Perl f(f(f(...)))) otherwise recurses once per level on the native C
stack and overflows it during parsing (SIGSEGV on the ~1 MB Windows thread
stack, and even the 8 MB POSIX stack at extreme depth) before any extractor
runs. Past the cap the ambiguity is left on the GLR stack instead of merged —
still a valid parse, never a wrong one — mirroring the existing
MAX_LINK_COUNT bail-out. The change is clearly marked // CBM patch: inline.
On re-vendor (e.g. ts_runtime → 0.26.x): re-apply this bound.
The shared scanner helpers in internal/cbm/vendored/common/ (scanner.h,
tag.h) originate from
tree-sitter-html (MIT,
(c) 2014 Max Brunsfeld) and carry that project's LICENSE in
internal/cbm/vendored/common/.
The core runtime headers in internal/cbm/vendored/common/tree_sitter/
(alloc.h, array.h, parser.h) are part of the tree-sitter C runtime
(tree-sitter, MIT,
(c) 2018 Max Brunsfeld) and carry their own LICENSE in that directory.
Tree-sitter Grammars
160 pre-generated parsers are vendored in internal/cbm/vendored/grammars/<lang>/
(generated parser.c plus scanner.c where applicable, compiled statically).
Each grammar is the work of its upstream authors and each grammar directory
contains the upstream LICENSE file.
The canonical provenance record — upstream repository, pinned commit, and
cross-registry verification status for every grammar — is
internal/cbm/vendored/grammars/MANIFEST.md.
License summary:
- Nearly all grammars are MIT-licensed.
clojure(sogaiu/tree-sitter-clojure) is CC0-1.0;fennelis CC0-1.0;jinja2andjustare Apache-2.0;pineis ISC (declared by its upstream).- The grammars authored in-house for this project (
chialisp,cobol,form,janet,magma,protobuf,wolfram) are MIT under the project's own license, (c) DeusData. Each ships the repository's own LICENSE, byte-identical to the root copy; they carry no third-party copyright because there is no third party.chialispis a generic s-expression grammar for the Chia smart-coin language, written for this project because no usable public grammar exists; its source and corpus tests live intools/tree-sitter-chialisp/. - Seven further grammars (
arkts,assembly,cfml,cfscript,dotenv,pine,qml) are self-maintained forks that retain their original upstream authors' licenses — see the manifest for per-grammar provenance.arktsis a first-party derivative of tree-sitter/tree-sitter-typescript (MIT, (c) 2017 Max Brunsfeld; on the tree-sitter-javascript base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions; its grammar source lives intools/tree-sitter-arkts/.
tree-sitter-plsql
- Project: AndreasMaierDe/tree-sitter-plsql
- License: MIT
- Copyright: (c) 2022 AndreasMaierDe
- Vendored at:
internal/cbm/vendored/grammars/plsql/ - Pinned commit:
28aebef209be - Notes: Community-maintained grammar for Oracle PL/SQL; not in the
nvim-treesitter or Helix registries (
community-nichein the manifest). No external scanner. One local patch:parser.c's#include <tree_sitter/parser.h>is changed to the quoted form used by every other vendored grammar, as documented ininternal/cbm/vendored/grammars/MANIFEST.md. PL/SQL support was originally contributed in PR #1033 by Oğuz (@ouzsrcm).
tree-sitter-objectscript (UDL + routine)
- Project: intersystems/tree-sitter-objectscript
- License: MIT
- Copyright: (c) 2025 InterSystems Corporation
- Vendored at:
internal/cbm/vendored/grammars/objectscript_udl/,internal/cbm/vendored/grammars/objectscript_routine/ - Pinned commit:
a7ffcdf - Notes: InterSystems-maintained grammar for the ObjectScript language (InterSystems IRIS / Caché). Vendor-maintained; not in nvim-treesitter or Helix registries. Each
scanner.c's upstream#include "../../common/scanner.h"is repointed to a per-directoryobjectscript_common.hcopied from upstreamcommon/scanner.h; two loop counters in that copy are widened fromuint8_ttointas documented ininternal/cbm/vendored/grammars/MANIFEST.md.
Vendored C/C++ Libraries
| Library | Path | License | Project |
|---|---|---|---|
| SQLite 3 | vendored/sqlite3/ |
Public Domain | sqlite.org |
| mimalloc | vendored/mimalloc/ |
MIT | microsoft/mimalloc |
| yyjson | vendored/yyjson/ |
MIT | ibireme/yyjson |
| xxHash | vendored/xxhash/ |
BSD-2-Clause | Cyan4973/xxHash |
| TRE | vendored/tre/ |
BSD-2-Clause | laurikari/tre |
| LZ4 | internal/cbm/vendored/lz4/ |
BSD-2-Clause (library files) | lz4/lz4 |
| Zstandard | internal/cbm/vendored/zstd/ |
BSD-3-Clause (dual BSD / GPLv2 — BSD selected) | facebook/zstd |
| simplecpp | internal/cbm/vendored/simplecpp/ |
0BSD | danmar/simplecpp |
| Verstable | internal/cbm/vendored/verstable/ |
MIT | JacksonAllan/Verstable |
| wyhash | internal/cbm/vendored/wyhash/ |
Unlicense (public domain) | wangyi-fudan/wyhash |
Local modifications to these libraries are documented next to the
vendored sources (currently only SQLite: vendored/sqlite3/PATCHES.md,
raising the Unix VFS MAX_PATHNAME ceiling from 512 to 4096 to match
CBM's 4 KiB path support). Patches must be reapplied on every upstream
refresh and are covered by scripts/vendored-checksums.txt.
The graph-UI HTTP server is a first-party implementation
(src/ui/httpd.c + src/ui/http_server.c) — no third-party HTTP library
is used.
Embedded Model Data
Semantic vector search uses static token embeddings derived from the
nomic-embed-code model, vendored in vendored/nomic/:
- Model: nomic-ai/nomic-embed-code
- License: Apache License 2.0
- Copyright: (c) Nomic AI
See vendored/nomic/NOTICE for the exact derivation procedure
(per-token inference + int8 quantization via scripts/extract_nomic_vectors.py).
Hybrid LSP — Reference Language Servers
The Hybrid LSP layer (internal/cbm/lsp/) is an original C implementation
written for this project. It contains no source code from any language
server. Its type-resolution behavior is structurally inspired by, and
validated for output compatibility against, the published behavior of the
following language servers and language specifications. They are listed here
as acknowledgment; their licenses are noted for reference:
| Language | Reference implementation / specification | Upstream license |
|---|---|---|
| TypeScript / JavaScript | tsserver (microsoft/TypeScript), typescript-go | Apache-2.0 |
| Python | pyright | MIT |
| Go | gopls (golang/tools) | BSD-3-Clause |
| PHP | PHP language reference + Composer PSR-4 autoloading specification | — |
| C# | Roslyn (dotnet/roslyn) | MIT |
| C / C++ | clangd (llvm/llvm-project) | Apache-2.0 WITH LLVM-exception |
| Java | Java Language Specification; output parity with Eclipse JDT LS | EPL-2.0 (reference only) |
| Kotlin | Kotlin language specification; fwcd/kotlin-language-server | MIT |
| Rust | rust-analyzer | MIT OR Apache-2.0 |
Standard-library type data
The stdlib type registries in internal/cbm/lsp/generated/ were produced as
follows:
- Python (
python_stdlib_data.c) — generated from python/typeshed type stubs (commita7912d521e16ff63caf7a8b64b9072542be36777), Apache-2.0, (c) the typeshed contributors. The generator isscripts/gen-py-stdlib.py. - Go (
go_stdlib_data.c) — generated by introspecting the public API of the Go standard library (golang/go, BSD-3-Clause). - Java, Kotlin, C#, PHP, C/C++, Rust — hand-curated from public API documentation and language specifications; no upstream source code was extracted or transcribed.
Embedded Graph UI
Release binaries built with --with-ui embed the compiled graph-ui/
frontend bundle. Its npm dependencies (React, three.js, @react-three/*,
radix-ui, lucide-react, tailwindcss, and friends) are all under permissive
licenses (MIT / ISC / Apache-2.0 / Zlib); the exact set is recorded in
graph-ui/package.json and graph-ui/package-lock.json, and the per-package
license texts of the production bundle are appended to the
THIRD_PARTY_NOTICES.md shipped inside the -ui release archives
(generated by scripts/gen-ui-licenses.py).