mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-06 17:49:47 +00:00
Martin Vogel
1b79558975
A plain '&' inside JSX -- a URL query in an attribute (href="https://fonts.googleapis.com/css2?family=Inter&display=swap"), or text such as <p>Tom & Jerry</p> -- produced a tree-sitter ERROR node in .js, .jsx and .tsx files. The file was flagged parse_partial in the coverage view, and the error recovery turned words from the URL (create-next-app -> create/next/app) into identifiers that resolved to unrelated symbols as false USAGE edges. Root cause is upstream tree-sitter-javascript, which the tsx dialect builds on: unescaped_{double,single}_jsx_string_fragment is /([^"&]|&[^#A-Za-z])+/, so '&' is only accepted before a space/digit or as a complete html_character_reference (& / &). jsx_text is scanned by the external scanner, which stops at every '&', and nothing in _jsx_child accepted a '&' that is not a reference. So '&' + letter in a string, and any lone '&' in JSX text, had no valid token. Fix: javascript and tsx become self-maintained forks of the pins we already vendored (javascript @ 58404d8cf191; typescript tsx @ 75b3874edb2d on its javascript base @ 3a837b6f3658), following the arkts model. One patch (tools/tree-sitter-javascript/patches/ jsx-lone-ampersand.patch) is applied to both javascript bases: - a one-character '&' token at lexical precedence -1, aliased to the existing string_fragment (in strings) and jsx_text (in text); the lower precedence keeps & / & lexing as html_character_reference - the string-fragment regexes exclude a '&' directly before the closing quote, so "a&" no longer runs past the end of the string Node types are unchanged (same type/named set). The external scanners stay upstream bytes. Parsers regenerated with tree-sitter-cli 0.25.10 (ABI 15; tsx moves from ABI 14, and its tree_sitter/ headers now match arkts). Without the patch both fork dirs regenerate upstream exactly. The upstream corpus passes unchanged: javascript 116/116; tsx gives the same results as unpatched upstream. Proof on vercel/next.js examples/ (canary 725705bd, 1,722 js/jsx/tsx files): parse_partial 92 -> 47 (tsx 33 -> 0, jsx 2 -> 0, js 12 -> 2; the two left are Babel do-expressions and Flow types), no newly flagged file, code-graph nodes 14,739 -> 14,739, edges 24,773 -> 24,657: the 115 USAGE edges removed were all false cross-example edges from URL fragments, plus a 4 SIMILAR_TO / +3 SEMANTICALLY_RELATED reshuffle from the corrected ASTs. Registered like arkts: MANIFEST.md fork rows + notes (ABI tally recounted), audit FORKS map, THIRD_PARTY.md, vendored checksums. LICENSEs stay upstream's MIT byte-identical. Tests: parse_coverage gains a .js/.jsx/.tsx matrix (double/single-quoted attribute, '&' before the quote, '&' + letter/space/digit in text, plus &/& controls) and a guard that broken JSX is still flagged. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
53 lines
2.2 KiB
Diff
53 lines
2.2 KiB
Diff
--- a/grammar.js
|
|
+++ b/grammar.js
|
|
@@ -684,6 +684,9 @@
|
|
_jsx_child: $ => choice(
|
|
$.jsx_text,
|
|
$.html_character_reference,
|
|
+ // cbm fork (#1736): a lone '&' that does not start a complete
|
|
+ // html_character_reference is plain text, as in HTML/React.
|
|
+ alias($._jsx_lone_ampersand, $.jsx_text),
|
|
$._jsx_element,
|
|
$.jsx_expression,
|
|
),
|
|
@@ -749,6 +752,7 @@
|
|
repeat(choice(
|
|
alias($.unescaped_double_jsx_string_fragment, $.string_fragment),
|
|
$.html_character_reference,
|
|
+ alias($._jsx_string_lone_ampersand, $.string_fragment), // cbm fork (#1736)
|
|
)),
|
|
'"',
|
|
),
|
|
@@ -757,6 +761,7 @@
|
|
repeat(choice(
|
|
alias($.unescaped_single_jsx_string_fragment, $.string_fragment),
|
|
$.html_character_reference,
|
|
+ alias($._jsx_string_lone_ampersand, $.string_fragment), // cbm fork (#1736)
|
|
)),
|
|
'\'',
|
|
),
|
|
@@ -766,11 +771,21 @@
|
|
// We give names to the token() constructs containing a regexp
|
|
// so as to obtain a node in the CST.
|
|
//
|
|
- unescaped_double_jsx_string_fragment: _ => token.immediate(prec(1, /([^"&]|&[^#A-Za-z])+/)),
|
|
+ // cbm fork (#1736): '&' directly before the closing quote is excluded so the
|
|
+ // fragment cannot run past the end of the string (upstream: &[^#A-Za-z]).
|
|
+ unescaped_double_jsx_string_fragment: _ => token.immediate(prec(1, /([^"&]|&[^#A-Za-z"])+/)),
|
|
|
|
// same here
|
|
- unescaped_single_jsx_string_fragment: _ => token.immediate(prec(1, /([^'&]|&[^#A-Za-z])+/)),
|
|
+ unescaped_single_jsx_string_fragment: _ => token.immediate(prec(1, /([^'&]|&[^#A-Za-z'])+/)),
|
|
|
|
+ // cbm fork (#1736): a lone '&' that does not begin a complete
|
|
+ // html_character_reference (`?a=1&b=2`, `Tom &Jerry`). Precedence -1 keeps
|
|
+ // html_character_reference (precedence 0) winning for `&` / `&`; the
|
|
+ // lexer falls back to this one-character token only when no reference
|
|
+ // matches. Upstream leaves `&` + letter/`#` unmatched, producing an ERROR.
|
|
+ _jsx_lone_ampersand: _ => token(prec(-1, '&')),
|
|
+ _jsx_string_lone_ampersand: _ => token.immediate(prec(-1, '&')),
|
|
+
|
|
_jsx_attribute_value: $ => choice(
|
|
alias($._jsx_string, $.string),
|
|
$.jsx_expression,
|