Recognize explicit C-family code in .inc files before extraction while
preserving BitBake recipe bodies and ObjectScript markers. Exercise disk
and byte classifiers, callback/linkage extraction and fast/full indexing.
Signed-off-by: knewstimek <knewstimek@users.noreply.github.com>
.comp, .geom, .tesc and .tese are the glslang stage extensions next to the
already-mapped .vert and .frag; Vulkan compute shaders were not indexed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Leonardo trindade miranda <tmonestudio@gmail.com>
Signed-off-by: mbrl <mbrl@local>
Brings #2421 (dep-info .d files kept out of the D grammar, direction
approved in #2414) up to date with main a2227157.
One textual conflict, in the SUITE(pipeline) RUN_TEST list of
tests/test_pipeline.c: main added
pipeline_stem_folder_collision_keeps_both_nodes and
pipeline_kustomize_entry_beside_same_stem_file right after
pipeline_structure_nodes, where this branch adds
pipeline_d_dep_info_leaves_no_nodes. Resolved by keeping all three
RUN_TEST lines, main's first.
src/discover/language.c, src/discover/discover.h, tests/test_discover.c
and tests/test_language.c merged cleanly. The merged tree adds exactly
this branch's 310 lines to main and deletes nothing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Dockerfile-variant check read the character after "Dockerfile" /
"Containerfile" before it knew the name held that base, so every name
shorter than the base was read past its end. GCC's AddressSanitizer
caught it once the doc-link layer began asking the language of short
code file names (a.go): 27 to 51 suites aborted per Linux shard. The
base is compared first now.
The test passes each short name in a heap block of exactly its size;
under GCC ASan it fails without the fix (heap-buffer-overflow in
lang_dockerfile_variant).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dockerfiles were Module + ENV/ARG Variables only: the 261 FROM and
114 COPY --from lines of the infra corpora joined nothing, and build
files named for a variant were not indexed at all.
- pass_docker (new, end of the K8s pass): FROM <image> and
COPY --from=<image> -> DEPENDS_ON {via: from | copy-from, image,
stage, line} from the Dockerfile's Module to an Image node. ARG
defaults are expanded (global ARGs in FROM, a stage's ARG / ENV in
COPY); a stage (by name or index), scratch and an ARG without a value
are no image.
- Image (new label, as approved): one node per repository, normalised
(docker.io for no registry host, library/ for an official image, a
registry host and port kept); tag and digest stay on the edge.
- Detection: Dockerfile.<x>, Dockerfile-<x>, Dockerfile_<x>,
Containerfile[.<x>] and <x>.Dockerfile are Dockerfiles, unless the
suffix is a language extension of its own (Dockerfile.j2 stays a
Jinja template) or a BuildKit ignore file.
- QNs: a Dockerfile.<variant> keeps its suffix (fqn.c and helpers.c,
which must agree); dropping it gave the variant the plain
Dockerfile's QN, merging both Modules. Covered by this branch's
index-format bump.
- Incremental: an Image no DEPENDS_ON / USAGE edge touches any more is
dropped on both routes, like a Process without SPAWNS; the
unreferenced-node GC takes a list of edge types.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- OpenTofu configuration (`.tofu`) and variable files (`.tfvars`) had no
language and were never indexed (cloud-foundation-fabric: 104 `.tofu`,
208 `.tfvars`).
- Every HCL `block` became a Class, nested ones included: a resource's
`lifecycle`, `dynamic`/`content` and argument blocks were entities next
to the resource (cloud-foundation-fabric: `dynamic` 1,342, `content`
205 beside 898 resources). Only a block directly in the file's body --
resource, data, module, variable, output, locals, provider, terraform,
Terragrunt's include/dependency -- is a Class now.
- A file's own assignments (a `.tfvars` file, Terragrunt `inputs`) sit
one level below the root the variable walk scanned; none was extracted.
They are Variables now.
On real input (HCL Classes / .tofu+.tfvars files / HCL Variables), against
main: cloud-foundation-fabric 6,324 -> 3,919 / 0 -> 312 / 0 -> 1,067,
kubernetes/k8s.io 2,102 -> 1,590 / 0 -> 11 / 0 -> 42, kubespray
892 -> 804 / 0 -> 13 / 0 -> 174, eks-blueprints 924 -> 733,
kube-hetzner 757 -> 683, deckhouse 976 -> 830; the terragrunt example
keeps its 31 top-level blocks and gains 5 input Variables.
Not changed here: the remaining HCL USAGE edges are generic name guesses
(a `range` call resolves to a Class in <python-builtins>, module outputs
to same-named outputs of unrelated modules), and the name-similarity
config links now also see the 104 OpenTofu files.
The nested-block, .tofu and .tfvars probes and the extension unit test
fail without the change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
lang_probe_bytes_follow_the_name, lang_classify_matches_every_content_rule
and the unreadable-content test now include .d: a 4 KB probe, a cargo
dep-info line that is not indexed and D source that stays D, through both
cbm_language_classify and cbm_disambiguate_d. With lang_probe_d disabled,
the first two fail, as does discover_d_dep_info_not_indexed_as_dlang.
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
main moved file classification into cbm_language_classify (c5abb649),
which discovery and the test-impact inventory share, so the .d hook in
detect_file_language conflicted. discover.c is main's. The probe is now
a content rule in language.c: cbm_language_probe_bytes asks 4 KB for a
.d name that maps to D, and cbm_language_classify runs lang_d_text on
those bytes. cbm_disambiguate_d reads raw bytes through lang_read_head,
as the other probes do.
Over 22,720 local .d files (10,408 dep-info, 12,312 D), classification
is unchanged from 006f2a9, and the path probe, discovery and the byte
classifier agree on every file.
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
One request pins the change (merge base .. HEAD), builds the graph and the
runnable test registry of exactly the HEAD snapshot, seeds the graph from
the change, walks what depends on it and answers which tests to run
(cbm.test_impact). Missing or doubtful evidence never narrows: it becomes a
run-all answer that names why.
Snapshot (classified ledger -> frozen graph -> registry):
- one shared language classifier for discovery and pinned blobs (raw bytes
on every platform); the classified ledger feeds the frozen build, which
indexes exactly the listed files and records partial parses as per-file
PARSE_GAP instead of refusing the graph;
- the pinned tree skips symlinks and submodule links (counted), as
discovery never indexes them;
- the test model scopes uncertainty to the suites of its file (they run
whole), reads line splices where they occur, treats a mid-file
`#ifndef X` + `#define X` as an ordinary condition, and no longer reads a
plain identifier named like a macro as an invocation. On this repository
the registry is now narrowable and matches a text census exactly
(8,365 cases, 141 runner suites).
Seeding and reach (port of the measured replay, rule B):
- hunk -> overlapping definitions, comment-only hunks dropped, pure
deletions -> enclosing definition, no-definition hunks -> whole file,
deleted names found by text, header changes propagated through the
include closure and the headers' macros (pastes included), file-level
escalation for unclassifiable header lines;
- inbound fixpoint walk over CALLS, ASYNC_CALLS, USAGE, CALL_REFERENCE,
READS (+ routes); cases, suites and the runner's entry points are sinks;
a reached runner main runs everything (RUNNER_REACHED);
- a parse gap whose text names a reached definition reaches the
definition around it (or its file);
- macro-registered tests run their suites whole; cases and suites map to
graph nodes by their start line.
The line patterns reproduce the reference regexes exactly on 1,383,850
lines of this repository and 37,167,926 kernel lines; on 24 replayed
changes the engine selects at least what the reference selected (13
identical, the rest only conservative supersets on the same graph).
Also:
- configured test declarations that cannot be mapped degrade their file
(no configured roles, a test_declarations diagnostic) instead of
aborting the whole index;
- the selector keeps other runners' suites out of the selection, and the
result codec judges the model by narrowability like the selector;
- a change to .codebase-memory.json runs everything (built-in rule);
- .codebase-memory.json with the c-cbm preset and this repository's rules.
Gate fixes (suites, lint-ci, Windows cross-compile):
- the request runs on its own thread with a default-size stack (daemon
tool calls run on 256 KiB threads, where the in-process build died);
- a request-supplied config_path is no longer followed by an unlink of an
uninitialized path buffer;
- the inventory's frame-local reader no longer stays reachable from the
caller's source after the filter call;
- FILE_ID_INFO needs _WIN32_WINNT >= 0x0602; the pinned-tree header pins
it before any system header (toolchains defaulting lower failed);
- memory-core ratchet: the win_utf8 converters gain core-backed
`_in(cls, ...)` forms over one implementation (the legacy names keep
the C library for unmigrated callers), the Git facts module uses them,
and the Windows spawn path releases everything through one failure
exit; pipeline, extraction and config allocations that this work
touched moved to the core in matched pairs; baselines tightened;
- the branch's C files formatted with the project's clang-format.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
is_dep_rule_line split targets only on space, tab and CR, so valid D such as
"import\fstd.stdio : writeln;" kept the keyword and the dotted module name in
one path-like target and was dropped as dep-info. Vertical tab, form feed
and the UTF-8 separators U+2028/U+2029 now split targets too.
Over 12,062 real D files and 4,494 real files under deps/ dirs, no
classification changes.
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
is_dep_rule_line took any '.' or '/' before the rule colon as a path, so
valid D such as "@1.0:" (a float UDA) and "public/**/:" (a comment between
tokens) was dropped as dep-info. A target holding '@' or a comment opener
("/*", "/+", "//") is now D.
Over 12,062 real D files and 4,494 real files under deps/ dirs, no
classification changes. A dep-info file whose target path contains '@' now
stays D, as it does on main.
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
.d mapped to D unconditionally, so make-style dependency files written by
cargo, rustc, gcc/clang -MD and CMake were parsed with the D grammar and
walked at superlinear cost. Add cbm_disambiguate_d, a 4 KB first-line sniff
in the shape of the existing .frm/.cls/.m disambiguators, that returns
CBM_LANG_COUNT only for a make rule whose targets look like paths and keeps D
on any doubt.
A 200-file cargo dep-info corpus goes from 108.9 s to 1.27 s to index; 0 of
12,062 real D files are misclassified.
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
A 4.7 MB binary Godot resource (piko_walk_mesh.res) never finished
indexing: `.res` maps to ReScript, so the resource was parsed as ReScript
source and the definitions pass spun. Three defects, each fixed:
1. Quadratic usage walk (the hang). The ReScript `let` binding check
(is_first_named_part_of, shared with Julia, Typst, Elm, PureScript
and Nickel) climbed from every named leaf with ts_node_parent, which
descends from the root and scans each level's children. A binary
file parses into a flat error tree (~170k root children per MB), so
each climb cost O(root children) and the file O(n^2): 1 MB spent
46 s in cbm_extract_unified, 4.7 MB never returned. The check now
climbs the unified walk's own cursor (O(1) per hop), like the other
occurrence classifiers; the node-based climb stays only for callers
outside the walk. 1 MB: 46 s -> 0.03 s; 4.7 MB: 2.2 s end to end.
2. Quadratic lexing in error recovery. tree-sitter retries the lexer at
every byte of an unparseable stretch with every external token
valid; the ReScript scanner's template-string loop then ran to the
next backtick/$/backslash/NUL (end of file if none) and discarded
the result. That cost O(n^2) inside lexing, where the parse budget's
progress callback never runs, so such files were dropped by the
5 s CPU budget instead of parsed -- plain text too (100 KB of
non-ReScript text: 15 s -> 0.18 s). Local scanner patch: skip the
template branch when NEWLINE is also valid, which only the ERROR
state allows. Valid files parse to identical trees. Recorded in
vendored/grammars/MANIFEST.md.
3. Binary content parsed as source. `.res` is also Godot's binary
resource format and the Windows compiled-resource format. Following
the .m/.cls/.frm/.inc/.cfc precedent, cbm_disambiguate_res reads the
first 4 KB and reports a file holding a NUL byte as unsupported;
ReScript text never contains one.
Tests (deterministic work counters, no clocks):
- extraction::extract_rescript_let_bindings_use_the_walk_cursor --
zero root-descending parent hops at 128 and 1024 statements
(before: 515 / 4099).
- complexity::complexity_rescript_error_recovery_lexing_is_linear --
lexer bytes pulled through a chunked TSInput, 4 KB vs 8 KB, ratio
must be linear (before 3.99, after 2.00).
- language::lang_res_binary_resource_unsupported and
lang_res_rescript_stays_rescript.
Fixes#2176
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The XML documents that make up a .NET repository project system were
absent from EXT_TABLE, so .csproj, .props, .targets, .resx, .plist,
.xaml and the app manifests were never indexed and their content was
unreachable through search_code. tree_sitter_xml is already wired up,
so these only needed rows in the table.
Fixes#2229
Signed-off-by: yasmoradi <msynk.dev@gmail.com>
Visual Basic 6 shares .frm with FORM and .cls with Apex/ObjectScript, so VB6
forms were parsed by the FORM grammar (0 defs) and class modules by the Apex
grammar (junk nodes, extract timeouts). Recognise the VB6 export header
(VERSION x.yy / VERSION 1.0 CLASS / Begin VB. / Attribute VB_Name) and report
such files as unsupported (CBM_LANG_COUNT) until a Visual Basic grammar exists:
new cbm_disambiguate_frm(), VB6 check added to cbm_disambiguate_cls() ahead of
its ObjectScript scan. Defaults on any doubt are unchanged (FORM / Apex).
Short-term step agreed in #721; the grammar follows separately.
Refs #721
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Tokarzewski <bartlomiej.tokarzewski@gmail.com>
Follow-up to #1824, which mapped Blazor `.razor` to the C# host and extracted
`@page` routes from it. `.cshtml` is the same Razor syntax with the same C#
host and was equally unmapped: an ASP.NET Core MVC or Razor Pages application
produced no nodes at all for its entire view tree.
`@page` matters more here than it does for components. On a Blazor component
the directive is an optional annotation; in Razor Pages it is the directive
that *defines* a page, so the routable surface of a Razor Pages app lives
exclusively in this file type.
That is why the extension mapping alone is not enough. `cbm_path_is_razor()`
hardcoded a single `.razor` suffix, so a `.cshtml` file would have been
discovered and parsed but skipped by the directive scan -- present in the
graph, with none of its routes. Widened to a suffix table covering both.
Scope is deliberately the Razor family and nothing else:
- `.vbhtml` is excluded because there is no VB.NET language in this repo,
so there is no host to map it to.
- `.aspx` / `.ascx` are excluded because Web Forms is a different templating
syntax (`<%@ %>`, `runat="server"`) with no `@page` directive, so neither
the C# error recovery nor the directive scan applies.
Best-effort on the same terms as `.razor`: the C# grammar recovers the `@{ }`
and `@functions` blocks, and the surrounding markup lands in ERROR regions
reported through parse_partial.
Tests: the extension mapping; a Razor Page asserting the extracted route path
and method; and a negative case for a layout with no `@page`, since most
`.cshtml` files are layouts and partials and a spurious Route node per view
would be worse than the missing ones this fixes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A .razor file had no entry in EXT_TABLE, so discovery skipped it entirely
and a Blazor application produced no graph nodes for any of its
components. The only way in was an undocumented extra_extensions entry in
a per-project .codebase-memory.json.
Map .razor to CBM_LANG_CSHARP. This is best-effort by design: the C#
grammar recovers the @code block, while the surrounding markup lands in
ERROR regions and is reported through parse_partial. That is strictly
more than the Module-and-imports the other markup-hosted languages
(Vue, Svelte, Astro) extract today, and it needs no new grammar.
Covered by tests/test_language.c:lang_ext_razor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: The1nk <23087600+The1nk@users.noreply.github.com>
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:
* comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
* `(include foo.clib)` was rejected because of the dot in the filename;
* `(defconstant NAME <expr>)` accepted only a primitive after the name.
So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.
It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.
Extraction, and the four decisions in it:
* `Constant` is admitted to the cross-file name registry
(cbm_label_is_registry_symbol). Chialisp's condition codes live as
`(defconstant ...)` in a shared .clib and are referenced by name from every
puzzle; labelled but unregistered they are seeded nowhere, and roughly half
of a Chialisp graph would be present as nodes yet never the target of an
edge. All three registry-seeding sites route through that one predicate, so
they cannot diverge. No other language emits `Constant`, so the blast radius
is exactly Chialisp. Pinned by a two-file graph test that goes red when the
label is removed from the predicate.
* `export` / `namespace` are NOT definition heads. `(export foo)` names a
function already defined in the same file, so treating it as a def head
mints a second node for one symbol. They stay in the not-a-call filter.
* A parameter list is a `list` too, but it binds rather than invokes. Without
that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
* `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
recording it would invent a dependency on a file that does not exist.
Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.
Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.
Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.
Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #1412 by @allanoepping. Fixes a real recall hole: tag-dialect
.cfc components were routed to the cfscript grammar (extracting nothing),
and <cfscript> bodies inside tag-dialect files are opaque cf_script_content
to the cfml grammar — whole legacy components produced zero functions.
Fix 1 — his .cfc dialect sniffer, nearly verbatim: cbm_disambiguate_cfc()
reads a 16K head via cbm_fopen; <cfcomponent>/<cffunction> anywhere means
tag dialect; a leading <cfscript (after whitespace and <!--- ---> comments)
or no markup at all keeps the old CFSCRIPT routing, so script-dialect files
and read failures behave exactly as before. Wired in detect_file_language
beside the existing _m/_cls/_inc sniffers, with his 8 direct tests.
Fix 2 — his embedded-defs capability, re-based onto the shared machinery
that superseded his mechanism mid-flight (#1852, merged after his last
rebase): a cfml_embedded_imports spec row (cf_script_tag/cf_script_content
-> CFSCRIPT), CFML added to the embedded-imports dispatch, and the
extract_structure gate extended to CFML. Included ranges on the original
buffer give absolute coordinates for free, so his line-number assertions
hold verbatim, and calls inside blocks become extractable as a bonus. His
~90-line slice+shift implementation is deliberately NOT carried — it
mirrored the sanctioned shape of its day, which #1852 replaced.
One correctness guard the port needed: the Vue attribute resolver is now
gated to Vue — cf_script_tag carries no lang=/src= attributes, and running
the resolver would have silently rewritten CFML's embedded language from
CFSCRIPT to the resolver's JavaScript default.
His embedded extraction test (line remap + <cfsetting> ERROR-cascade case)
carries over verbatim and binds: reverting the spec/gate wiring fails it at
ASSERT(has_def(r, "Function", "greet")). Suites: extraction, language,
edge_imports, lang_contract, grammar_labels, grammar_regression — 637/0.
Co-authored-by: Allan Oepping <allan@intellipay.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vendor AndreasMaierDe/tree-sitter-plsql @ 28aebef209be (MIT, ABI 14, no
external scanner) and wire CBM_LANG_PLSQL end to end:
- extraction onto existing labels only: package spec/body -> Class,
procedure/function (create/definition/declaration) -> Function,
file -> Module; callee extraction for ref_call incl. the
package-qualified pkg.name form; identifier usage vocabulary with the
upstream 'parameter' kind (a ref_call ARGUMENT wrapper) carved out of
the whole-binding rule
- discover: .pks/.pkb/.pck/.pls/.plb/.plsql/.fnc/.trg/.bdy/.tps/.tpb
(.sql stays SQL and .prc stays FORM, both asserted negatively)
- ledgers: CALL_WITHOUT_REFERENCE_VOCAB(PLSQL), call-node manifest
ref_call row (historical total 219 -> 220), call-argument matrix B row
- tests: extraction triple (package body + package-qualified call,
standalone function, honest negative pin of the upstream
CREATE TYPE ... AS OBJECT ERROR-node limitation), grammar-regression
case, label golden Class:1,Function:1,Module:1, MUST_EXTRACT_DEFS,
extension mapping, bug-repro board battery row
- MANIFEST.md (community-niche verdict, local include-quote patch row),
THIRD_PARTY.md section, refreshed vendored checksums
PL/SQL support was contributed by Oguz (@ouzsrcm) in #1033; re-vendored
from upstream per vendoring policy with his language wiring distilled on
top.
Co-authored-by: Oğuz <oguzhan.saricam@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routing .ets through the plain TypeScript grammar produced no definition
node for any @Component struct (0/19 across the OpenHarmony sample corpus),
parsed struct members as statement labels and build() as a call expression,
and registered ArkUI builtins (Column, Row, ListItem, ...) as user function
definitions with 28% fabricated cross-file CALLS edges. This adds a
dedicated ArkTS grammar that keeps everything TypeScript already handles
and models the ArkTS syntax TS cannot parse.
Grammar (tools/tree-sitter-arkts/, generated parser vendored to
internal/cbm/vendored/grammars/arkts/, ABI 15 via tree-sitter-cli 0.25.10):
first-party derivative of tree-sitter/tree-sitter-typescript v0.23.2
@ 75b3874edb2d (the commit our vendored typescript/tsx come from) on the
tree-sitter-javascript v0.23.1 base @ 3a837b6f3658 (the version upstream's
package-lock pins). LICENSE carries the upstream MIT text verbatim plus a
derivative notice; the external scanner is a symbol-rename trampoline over
a byte-identical copy of the already-vendored typescript scanner header.
ArkTS additions: decorated struct component declarations with class-shaped
bodies; UI-DSL trailing-closure component calls with attribute chains after
the closing brace; decorators on function declarations (@Builder,
@Extend(T), @Styles); @Extend/@Styles leading-dot attribute-chain bodies;
anonymous stateStyles style blocks; import lazy named imports.
Wiring: CBM_LANG_ARKTS enum + lang spec (struct_declaration as a class
kind labeled Struct, public_field_definition fields), .ets extension,
LANG_NAMES/userconfig entries, grammar shim, and TS-family joins across
the extractors and the name-resolution guards (parse_es_imports, JS
keywords/test-file/func-kind/scope tables, TS heritage/param/receiver/
namespace/type-ref/channel/usage paths, js_ts_family and the weak-method-
match suppression). Field definitions now also carry their decorators, so
@State/@Prop/@Link members are findable with their roles.
Tests: grammar corpus (35 cases), per-language regression + label golden +
import net + extension cases, seven ArkTS extraction tests including the
phantom-builtin regression (ArkUI builtins must never be definitions), and
capability-ledger / call-node-manifest / call-argument-matrix rows. On the
14-file OpenHarmony sample corpus the TS-grammar baseline had 12/14 files
with parse errors (231 ERROR nodes); the arkts grammar parses 14/14 clean,
and an end-to-end index yields all 19 components as Struct nodes, 68
methods, 119 fields, and zero phantom builtin definitions.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
Helm chart helpers (.tpl) and Go templates produced no semantic graph — just a
flat key flood. Wire the existing gotemplate grammar for real structure:
- .tpl files map to the gotemplate language (Helm _helpers.tpl).
- {{ define "chart.fullname" }} becomes a Function node named after the
template (the name is a string literal, not an identifier, so it's resolved
in a dedicated walker branch).
- {{ include "x" . }} and {{ template "x" . }} resolve to the referenced
template name instead of the bare `include` builtin, so they form CALLS
edges to the define'd Function. template_action is added to the call set.
Lets you trace which charts include which named templates. Tests cover define
-> Function and include/template -> CALLS, plus the .tpl mapping.
Refs #338
Vendor the MIT-licensed cfmleditor/tree-sitter-cfml grammars and wire both
CFML dialects through the pipeline:
- .cfc components -> cfscript grammar (CBM_LANG_CFSCRIPT): a JS-like grammar, so
function/method declarations, calls and imports extract with the shared JS/TS
arrays. Components' functions become Function nodes.
- .cfm templates -> cfml tag grammar (CBM_LANG_CFML): tag-based (HTML-derived).
Embedded <cfscript> functions extract via function_declaration; tag-based
<cffunction name="..."> nodes are handled by a dedicated walker branch that
reads the name attribute (cf_function_tag has no name field). Shared scanner
headers are vendored at vendored/common/ so the grammar's ../../common/
includes resolve without editing vendored sources.
Adds an ASCII case-insensitive compare for CFML's case-insensitive attributes.
Tests cover both dialects and the extension mapping. LICENSEs vendored.
Closes#38
Vendor the MIT-licensed tree-sitter-qmljs grammar (a TypeScript superset with
declarative ui_* nodes) and wire .qml files through the full pipeline: language
enum, extension + display-name mapping, grammar shim, and a language spec that
reuses the JS/TS function/call/branch arrays and adds QML class/field/import
types (ui_inline_component, ui_property, ui_signal, ui_import).
QML files now produce Function nodes (incl. JS functions inside QML objects),
class/interface/enum nodes, imports and CALLS edges, so call chains through a
Qt project's .qml files can be traced. LICENSE vendored alongside the grammar.
Closes#42
Laravel Blade templates (*.blade.php) fell through to the single-extension
lookup and were mis-classified as PHP, so the PHP extractor tried to parse
Blade syntax. Add a built-in compound-extension table (checked before the
single-extension fallback and alongside user config) mapping .blade.php to
CBM_LANG_BLADE, so Blade files are discovered and parsed with the Blade
grammar. Combined with the now-non-blocking discovery gate (c29e6d5), this
resolves the "agent stuck on Blade files" report. Plain .php is unaffected.
.mjs (ES modules) and .cjs (CommonJS) were absent from the extension
table, so those files were never assigned a language — they weren't
indexed (no graph nodes) and search_code returned 0 for any pattern in
them. Add .mjs/.cjs → JavaScript and .mts/.cts → TypeScript.
Verified end-to-end: a function in a .mjs file is now extracted and found
by search_code. Closes#197.
Add automated extension-to-language mapping tests for all new grammars
that have file extensions. Removes conflicting extensions:
- .blade.php: compound extension, handled by userconfig system
- .env*: filenames, not extensions (no auto-discovery for dotfiles)
- .sv/.svh: already mapped to Verilog, SystemVerilog uses same grammar
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Martin Vogel
knewstimek
Leonardo trindade mirandaandClaude Opus 5.5
Nelson Spence
yasmoradi
TokarzewskiandClaude Opus 5
The1nkandClaude Opus 5
Anand Aiyer
CooperSheroy
Martin Vogel
test