diff --git a/BENCHMARK.md b/BENCHMARK.md index 5bf4bd514..ec42a7b73 100644 --- a/BENCHMARK.md +++ b/BENCHMARK.md @@ -2,7 +2,7 @@ ## Methodology -- **35 languages** (27 programming + 8 config/markup), 12 questions each (4 for config languages) +- **63 languages** (27 programming + 8 config/markup), 12 questions each (4 for config languages) - **Up to 5 attempts** per question with escalating retry strategies - **Real open-source repos** (medium to large: 78--49K nodes) - **Grading**: PASS (1.0) / PARTIAL (0.5) / FAIL (0.0), N/A excluded from denominator @@ -847,7 +847,7 @@ The tool handles 20K-node kernel subsystems without timeouts. Deep traces on wel ## Cross-Cutting Findings ### Strengths -1. **Zero indexing failures** across 35 repos of all sizes (78 to 49K nodes) +1. **Zero indexing failures** across 63 repos of all sizes (78 to 49K nodes) 2. **100% on 17 languages** -- half the languages achieved perfect scores 3. **No language below 62%** -- even the weakest performs core operations 4. **Massive codebases handled**: Django (49K nodes), Laravel (38K nodes), neovim (24K nodes) -- no performance issues diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 201af0c55..43131caf7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -37,7 +37,7 @@ golangci-lint run ./... ``` cmd/codebase-memory-mcp/ Entry point (MCP server + CLI + install/update) internal/ - lang/ Language specs (35 languages, tree-sitter node types) + lang/ Language specs (63 languages, tree-sitter node types) parser/ Tree-sitter grammar loading pipeline/ Multi-pass indexing pipeline httplink/ Cross-service HTTP route matching diff --git a/README.md b/README.md index a60d0945e..0bcc81854 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Parses source code with [tree-sitter](https://tree-sitter.github.io/tree-sitter/ ## Features -- **59 languages**: Python, Go, JavaScript, TypeScript, TSX, Rust, Java, C++, C#, C, PHP, Lua, Scala, Kotlin, Ruby, Bash, Zig, Elixir, Haskell, OCaml, Objective-C, Swift, Dart, Perl, Groovy, Erlang, R, Clojure, F#, Julia, Vim Script, Nix, Common Lisp, Elm, Fortran, CUDA, COBOL, Verilog, Emacs Lisp, HTML, CSS, SCSS, YAML, TOML, HCL, SQL, Dockerfile, JSON, XML, Markdown, Makefile, CMake, Protobuf, GraphQL, Vue, Svelte, Meson, GLSL, INI +- **63 languages**: Python, Go, JavaScript, TypeScript, TSX, Rust, Java, C++, C#, C, PHP, Lua, Scala, Kotlin, Ruby, Bash, Zig, Elixir, Haskell, OCaml, Objective-C, Swift, Dart, Perl, Groovy, Erlang, R, Clojure, F#, Julia, Vim Script, Nix, Common Lisp, Elm, Fortran, CUDA, COBOL, Verilog, Emacs Lisp, MATLAB, Lean 4, FORM, Magma, HTML, CSS, SCSS, YAML, TOML, HCL, SQL, Dockerfile, JSON, XML, Markdown, Makefile, CMake, Protobuf, GraphQL, Vue, Svelte, Meson, GLSL, INI - **Architecture overview**: `get_architecture` returns languages, packages, entry points, routes, hotspots, boundaries, layers, and clusters in a single call — instant codebase orientation - **Architecture Decision Records**: `manage_adr` persists architectural decisions (PURPOSE, STACK, ARCHITECTURE, PATTERNS, TRADEOFFS, PHILOSOPHY) across sessions with section filtering and validation - **Louvain community detection**: Discovers hidden functional modules across packages by clustering CALLS, HTTP_CALLS, and ASYNC_CALLS edges @@ -57,7 +57,7 @@ Claude Code formats and explains the results. **Why no built-in LLM?** Other code graph tools embed an LLM to translate natural language into graph queries. This means extra API keys, extra cost per query, and another model to configure. With MCP, the AI assistant you're already talking to *is* the query translator — no duplication needed. -**Token efficiency**: Compared to having an AI agent grep through your codebase file by file, graph queries return precise results in a single tool call. In benchmarks across 59 real-world repos (78 to 49K nodes), five structural queries consumed ~3,400 tokens via codebase-memory-mcp versus ~412,000 tokens via file-by-file exploration — a **99.2% reduction**. All 59 supported languages use the same efficient graph backend. +**Token efficiency**: Compared to having an AI agent grep through your codebase file by file, graph queries return precise results in a single tool call. In benchmarks across 63 real-world repos (78 to 49K nodes), five structural queries consumed ~3,400 tokens via codebase-memory-mcp versus ~412,000 tokens via file-by-file exploration — a **99.2% reduction**. All 63 supported languages use the same efficient graph backend. ## Performance @@ -663,7 +663,7 @@ make install # go install ## Language Benchmark -59 languages supported. Benchmarked against 59 real open-source repositories (78 to 49K nodes). 12 standardized questions per language. Grading: HIGH (1.0) / MEDIUM (0.5) / LOW (0.1). Overall: **76%** average MCP score across all languages (97% for explorer-based agents). +63 languages supported. Benchmarked against 63 real open-source repositories (78 to 49K nodes). 12 standardized questions per language. Grading: HIGH (1.0) / MEDIUM (0.5) / LOW (0.1). Overall: **76%** average MCP score across all languages (97% for explorer-based agents). | Tier | Score | Languages | |------|-------|-----------| @@ -682,8 +682,8 @@ See [`BENCHMARK.md`](BENCHMARK.md) for the full 35-language benchmark with per-q cmd/codebase-memory-mcp/ Entry point (MCP stdio server + CLI mode + install/update commands) internal/ store/ SQLite graph storage (nodes, edges, traversal, search, architecture, Louvain clustering) - lang/ Language specs (59 languages, tree-sitter node types) - cbm/ Vendored tree-sitter C grammars (59 languages) and AST extraction engine + lang/ Language specs (63 languages, tree-sitter node types) + cbm/ Vendored tree-sitter C grammars (63 languages) and AST extraction engine pipeline/ Multi-pass indexing (structure → definitions → calls → HTTP links → config links → communities → tests) httplink/ Cross-service HTTP route/call-site matching cypher/ Cypher query lexer, parser, planner, executor diff --git a/docs/index.html b/docs/index.html index 2fcb8242a..cec424d8e 100644 --- a/docs/index.html +++ b/docs/index.html @@ -4,19 +4,19 @@ codebase-memory-mcp — Code Knowledge Graph for AI Assistants - + - + - + @@ -210,7 +210,7 @@
fewer tokens
-
59
+
63
languages
@@ -322,8 +322,8 @@

Features

-

59 Languages

-

Python, Go, JS, TS, TSX, Rust, Java, C++, C#, C, PHP, Ruby, Kotlin, Scala, Zig, Elixir, Haskell, OCaml, Swift, Dart, and 39 more via vendored tree-sitter grammars.

+

63 Languages

+

Python, Go, JS, TS, TSX, Rust, Java, C++, C#, C, PHP, Ruby, Kotlin, Scala, Zig, Elixir, Haskell, OCaml, Swift, Dart, MATLAB, Lean 4, and 41 more via vendored tree-sitter grammars.

Call Graph Tracing

@@ -371,7 +371,7 @@ - Languages598-11 + Languages638-11 RuntimeSingle Go binaryNode.js (npx) Runtime dependencyNoneNode.js Embedded LLMNo (uses your MCP client)Yes (extra API key + cost) diff --git a/internal/cbm/regression_test.go b/internal/cbm/regression_test.go index 5a862199d..6dc8b41dc 100644 --- a/internal/cbm/regression_test.go +++ b/internal/cbm/regression_test.go @@ -1031,3 +1031,34 @@ func TestMagmaParse_Regression(t *testing.T) { t.Fatal(err) } } + +func TestMagmaImport_Regression(t *testing.T) { + src := []byte("load \"utils.mag\";\nload \"lib/helpers.mag\";\n") + r, err := ExtractFile(src, lang.Magma, "t", "main.mag") + if err != nil { + t.Fatal(err) + } + if len(r.Imports) < 2 { + t.Fatalf("expected at least 2 imports, got %d", len(r.Imports)) + } +} + +func TestMagmaCall_Regression(t *testing.T) { + src := []byte("function Foo(x)\n y := Bar(x);\n return y;\nend function;\n") + r, err := ExtractFile(src, lang.Magma, "t", "calls.mag") + if err != nil { + t.Fatal(err) + } + if len(r.Calls) == 0 { + t.Fatal("expected calls to be extracted, got 0") + } + found := false + for _, c := range r.Calls { + if c.CalleeName == "Bar" { + found = true + } + } + if !found { + t.Errorf("expected call to 'Bar', got calls: %v", r.Calls) + } +} diff --git a/internal/cbm/vendored/grammars/magma/parser.c b/internal/cbm/vendored/grammars/magma/parser.c index f6c018d66..1104291e6 100644 --- a/internal/cbm/vendored/grammars/magma/parser.c +++ b/internal/cbm/vendored/grammars/magma/parser.c @@ -13,10 +13,10 @@ #define ALIAS_COUNT 0 #define TOKEN_COUNT 93 #define EXTERNAL_TOKEN_COUNT 0 -#define FIELD_COUNT 1 +#define FIELD_COUNT 2 #define MAX_ALIAS_SEQUENCE_LENGTH 13 #define MAX_RESERVED_WORD_SET_SIZE 0 -#define PRODUCTION_ID_COUNT 3 +#define PRODUCTION_ID_COUNT 4 #define SUPERTYPE_COUNT 0 enum ts_symbol_identifiers { @@ -1068,22 +1068,27 @@ static const TSSymbolMetadata ts_symbol_metadata[] = { enum ts_field_identifiers { field_name = 1, + field_path = 2, }; static const char * const ts_field_names[] = { [0] = NULL, [field_name] = "name", + [field_path] = "path", }; static const TSMapSlice ts_field_map_slices[PRODUCTION_ID_COUNT] = { [1] = {.index = 0, .length = 1}, [2] = {.index = 1, .length = 1}, + [3] = {.index = 2, .length = 1}, }; static const TSFieldMapEntry ts_field_map_entries[] = { [0] = - {field_name, 0}, + {field_path, 1}, [1] = + {field_name, 0}, + [2] = {field_name, 1}, }; @@ -23176,8 +23181,8 @@ static const TSParseActionEntry ts_parse_actions[] = { [405] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_format_string, 4, 0, 0), [407] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sequence_expression, 3, 0, 0), [409] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_sequence_expression, 3, 0, 0), - [411] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 4, 0, 1), - [413] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 4, 0, 1), + [411] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 4, 0, 2), + [413] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 4, 0, 2), [415] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_index_expression, 4, 0, 0), [417] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_index_expression, 4, 0, 0), [419] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_set_expression, 3, 0, 0), @@ -23207,16 +23212,16 @@ static const TSParseActionEntry ts_parse_actions[] = { [467] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_anonymous_function, 5, 0, 0), [469] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_format_string, 5, 0, 0), [471] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_format_string, 5, 0, 0), - [473] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 5, 0, 1), - [475] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 5, 0, 1), + [473] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 5, 0, 2), + [475] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 5, 0, 2), [477] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_index_expression, 5, 0, 0), [479] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_index_expression, 5, 0, 0), [481] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_parenthesized_expression, 3, 0, 0), [483] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_parenthesized_expression, 3, 0, 0), [485] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_string_literal, 3, 0, 0), [487] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_string_literal, 3, 0, 0), - [489] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 3, 0, 1), - [491] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 3, 0, 1), + [489] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 3, 0, 2), + [491] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 3, 0, 2), [493] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_set_expression, 4, 0, 0), [495] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_set_expression, 4, 0, 0), [497] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_anonymous_function, 6, 0, 0), @@ -23227,16 +23232,16 @@ static const TSParseActionEntry ts_parse_actions[] = { [507] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_sequence_expression, 2, 0, 0), [509] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_tuple_expression, 2, 0, 0), [511] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_tuple_expression, 2, 0, 0), - [513] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 9, 0, 2), - [515] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 9, 0, 2), + [513] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 9, 0, 3), + [515] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 9, 0, 3), [517] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_case_statement, 7, 0, 0), [519] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 7, 0, 0), [521] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 7, 0, 0), [523] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 7, 0, 0), - [525] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 8, 0, 2), - [527] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 8, 0, 2), - [529] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 8, 0, 2), - [531] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 8, 0, 2), + [525] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 8, 0, 3), + [527] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 8, 0, 3), + [529] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 8, 0, 3), + [531] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 8, 0, 3), [533] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 8, 0, 0), [535] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 8, 0, 0), [537] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 8, 0, 0), @@ -23245,14 +23250,14 @@ static const TSParseActionEntry ts_parse_actions[] = { [543] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 8, 0, 0), [545] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 8, 0, 0), [547] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 8, 0, 0), - [549] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 9, 0, 2), - [551] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 9, 0, 2), + [549] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 9, 0, 3), + [551] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 9, 0, 3), [553] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 9, 0, 0), [555] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 9, 0, 0), [557] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 9, 0, 0), [559] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_for_statement, 9, 0, 0), - [561] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 10, 0, 2), - [563] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 10, 0, 2), + [561] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 10, 0, 3), + [563] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 10, 0, 3), [565] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 10, 0, 0), [567] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_for_statement, 10, 0, 0), [569] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 11, 0, 0), @@ -23269,8 +23274,8 @@ static const TSParseActionEntry ts_parse_actions[] = { [591] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_expression_statement, 2, 0, 0), [593] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_while_statement, 7, 0, 0), [595] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_while_statement, 7, 0, 0), - [597] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_load_statement, 3, 0, 0), - [599] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_load_statement, 3, 0, 0), + [597] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_load_statement, 3, 0, 1), + [599] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_load_statement, 3, 0, 1), [601] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_print_statement, 3, 0, 0), [603] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_print_statement, 3, 0, 0), [605] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_error_statement, 3, 0, 0), @@ -23289,12 +23294,12 @@ static const TSParseActionEntry ts_parse_actions[] = { [631] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_repeat_statement, 5, 0, 0), [633] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_require_statement, 5, 0, 0), [635] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_require_statement, 5, 0, 0), - [637] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 6, 0, 2), - [639] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 6, 0, 2), - [641] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 6, 0, 2), - [643] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 6, 0, 2), - [645] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 6, 0, 2), - [647] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 6, 0, 2), + [637] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 6, 0, 3), + [639] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 6, 0, 3), + [641] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 6, 0, 3), + [643] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 6, 0, 3), + [645] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 6, 0, 3), + [647] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 6, 0, 3), [649] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 6, 0, 0), [651] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 6, 0, 0), [653] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_while_statement, 6, 0, 0), @@ -23303,12 +23308,12 @@ static const TSParseActionEntry ts_parse_actions[] = { [659] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 6, 0, 0), [661] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 6, 0, 0), [663] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 6, 0, 0), - [665] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 7, 0, 2), - [667] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 7, 0, 2), - [669] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 7, 0, 2), - [671] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 7, 0, 2), - [673] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 7, 0, 2), - [675] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 7, 0, 2), + [665] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 7, 0, 3), + [667] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 7, 0, 3), + [669] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 7, 0, 3), + [671] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 7, 0, 3), + [673] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 7, 0, 3), + [675] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 7, 0, 3), [677] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_return_statement, 3, 0, 0), [679] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_return_statement, 3, 0, 0), [681] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym__type_expr, 1, 0, 0), diff --git a/internal/pipeline/pipeline.go b/internal/pipeline/pipeline.go index c2d5fe4f6..eb3c1833c 100644 --- a/internal/pipeline/pipeline.go +++ b/internal/pipeline/pipeline.go @@ -1270,7 +1270,13 @@ func (p *Pipeline) passImports() { // Try to find the target as a Module node first targetNode, _ := p.findNodeByQN(p.ProjectName, targetQN) if targetNode == nil { - // Try common suffixes: module QN might need .__init__ or similar + // Try treating import path as a relative file path (e.g. "utils.mag", "lib/helpers.h") + resolvedQN := fqn.ModuleQN(p.ProjectName, targetQN) + if resolvedQN != targetQN { + targetNode, _ = p.findNodeByQN(p.ProjectName, resolvedQN) + } + } + if targetNode == nil { logImportDrop(moduleQN, localName, targetQN) continue } diff --git a/tree-sitter-magma/grammar.js b/tree-sitter-magma/grammar.js index cafb2915e..50fb6d7b0 100644 --- a/tree-sitter-magma/grammar.js +++ b/tree-sitter-magma/grammar.js @@ -186,7 +186,7 @@ module.exports = grammar({ load_statement: $ => seq( 'load', - $._expression, + field('path', $._expression), ';', ), diff --git a/tree-sitter-magma/src/grammar.json b/tree-sitter-magma/src/grammar.json index c96d2a6c6..a0cf536db 100644 --- a/tree-sitter-magma/src/grammar.json +++ b/tree-sitter-magma/src/grammar.json @@ -856,8 +856,12 @@ "value": "load" }, { - "type": "SYMBOL", - "name": "_expression" + "type": "FIELD", + "name": "path", + "content": { + "type": "SYMBOL", + "name": "_expression" + } }, { "type": "STRING", diff --git a/tree-sitter-magma/src/node-types.json b/tree-sitter-magma/src/node-types.json index 98127f05a..e86133f66 100644 --- a/tree-sitter-magma/src/node-types.json +++ b/tree-sitter-magma/src/node-types.json @@ -1898,80 +1898,81 @@ { "type": "load_statement", "named": true, - "fields": {}, - "children": { - "multiple": false, - "required": true, - "types": [ - { - "type": "anonymous_function", - "named": true - }, - { - "type": "binary_expression", - "named": true - }, - { - "type": "boolean_literal", - "named": true - }, - { - "type": "call_expression", - "named": true - }, - { - "type": "comprehension", - "named": true - }, - { - "type": "format_string", - "named": true - }, - { - "type": "identifier", - "named": true - }, - { - "type": "index_expression", - "named": true - }, - { - "type": "member_expression", - "named": true - }, - { - "type": "number", - "named": true - }, - { - "type": "parenthesized_expression", - "named": true - }, - { - "type": "range_expression", - "named": true - }, - { - "type": "sequence_expression", - "named": true - }, - { - "type": "set_expression", - "named": true - }, - { - "type": "string_literal", - "named": true - }, - { - "type": "tuple_expression", - "named": true - }, - { - "type": "unary_expression", - "named": true - } - ] + "fields": { + "path": { + "multiple": false, + "required": true, + "types": [ + { + "type": "anonymous_function", + "named": true + }, + { + "type": "binary_expression", + "named": true + }, + { + "type": "boolean_literal", + "named": true + }, + { + "type": "call_expression", + "named": true + }, + { + "type": "comprehension", + "named": true + }, + { + "type": "format_string", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "index_expression", + "named": true + }, + { + "type": "member_expression", + "named": true + }, + { + "type": "number", + "named": true + }, + { + "type": "parenthesized_expression", + "named": true + }, + { + "type": "range_expression", + "named": true + }, + { + "type": "sequence_expression", + "named": true + }, + { + "type": "set_expression", + "named": true + }, + { + "type": "string_literal", + "named": true + }, + { + "type": "tuple_expression", + "named": true + }, + { + "type": "unary_expression", + "named": true + } + ] + } } }, { diff --git a/tree-sitter-magma/src/parser.c b/tree-sitter-magma/src/parser.c index f6c018d66..1104291e6 100644 --- a/tree-sitter-magma/src/parser.c +++ b/tree-sitter-magma/src/parser.c @@ -13,10 +13,10 @@ #define ALIAS_COUNT 0 #define TOKEN_COUNT 93 #define EXTERNAL_TOKEN_COUNT 0 -#define FIELD_COUNT 1 +#define FIELD_COUNT 2 #define MAX_ALIAS_SEQUENCE_LENGTH 13 #define MAX_RESERVED_WORD_SET_SIZE 0 -#define PRODUCTION_ID_COUNT 3 +#define PRODUCTION_ID_COUNT 4 #define SUPERTYPE_COUNT 0 enum ts_symbol_identifiers { @@ -1068,22 +1068,27 @@ static const TSSymbolMetadata ts_symbol_metadata[] = { enum ts_field_identifiers { field_name = 1, + field_path = 2, }; static const char * const ts_field_names[] = { [0] = NULL, [field_name] = "name", + [field_path] = "path", }; static const TSMapSlice ts_field_map_slices[PRODUCTION_ID_COUNT] = { [1] = {.index = 0, .length = 1}, [2] = {.index = 1, .length = 1}, + [3] = {.index = 2, .length = 1}, }; static const TSFieldMapEntry ts_field_map_entries[] = { [0] = - {field_name, 0}, + {field_path, 1}, [1] = + {field_name, 0}, + [2] = {field_name, 1}, }; @@ -23176,8 +23181,8 @@ static const TSParseActionEntry ts_parse_actions[] = { [405] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_format_string, 4, 0, 0), [407] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sequence_expression, 3, 0, 0), [409] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_sequence_expression, 3, 0, 0), - [411] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 4, 0, 1), - [413] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 4, 0, 1), + [411] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 4, 0, 2), + [413] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 4, 0, 2), [415] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_index_expression, 4, 0, 0), [417] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_index_expression, 4, 0, 0), [419] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_set_expression, 3, 0, 0), @@ -23207,16 +23212,16 @@ static const TSParseActionEntry ts_parse_actions[] = { [467] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_anonymous_function, 5, 0, 0), [469] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_format_string, 5, 0, 0), [471] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_format_string, 5, 0, 0), - [473] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 5, 0, 1), - [475] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 5, 0, 1), + [473] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 5, 0, 2), + [475] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 5, 0, 2), [477] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_index_expression, 5, 0, 0), [479] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_index_expression, 5, 0, 0), [481] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_parenthesized_expression, 3, 0, 0), [483] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_parenthesized_expression, 3, 0, 0), [485] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_string_literal, 3, 0, 0), [487] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_string_literal, 3, 0, 0), - [489] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 3, 0, 1), - [491] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 3, 0, 1), + [489] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_call_expression, 3, 0, 2), + [491] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_call_expression, 3, 0, 2), [493] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_set_expression, 4, 0, 0), [495] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_set_expression, 4, 0, 0), [497] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_anonymous_function, 6, 0, 0), @@ -23227,16 +23232,16 @@ static const TSParseActionEntry ts_parse_actions[] = { [507] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_sequence_expression, 2, 0, 0), [509] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_tuple_expression, 2, 0, 0), [511] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_tuple_expression, 2, 0, 0), - [513] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 9, 0, 2), - [515] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 9, 0, 2), + [513] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 9, 0, 3), + [515] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 9, 0, 3), [517] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_case_statement, 7, 0, 0), [519] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 7, 0, 0), [521] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 7, 0, 0), [523] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 7, 0, 0), - [525] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 8, 0, 2), - [527] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 8, 0, 2), - [529] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 8, 0, 2), - [531] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 8, 0, 2), + [525] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 8, 0, 3), + [527] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 8, 0, 3), + [529] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 8, 0, 3), + [531] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 8, 0, 3), [533] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 8, 0, 0), [535] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 8, 0, 0), [537] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 8, 0, 0), @@ -23245,14 +23250,14 @@ static const TSParseActionEntry ts_parse_actions[] = { [543] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 8, 0, 0), [545] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 8, 0, 0), [547] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 8, 0, 0), - [549] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 9, 0, 2), - [551] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 9, 0, 2), + [549] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 9, 0, 3), + [551] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 9, 0, 3), [553] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 9, 0, 0), [555] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 9, 0, 0), [557] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 9, 0, 0), [559] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_for_statement, 9, 0, 0), - [561] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 10, 0, 2), - [563] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 10, 0, 2), + [561] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 10, 0, 3), + [563] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 10, 0, 3), [565] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 10, 0, 0), [567] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_for_statement, 10, 0, 0), [569] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_for_statement, 11, 0, 0), @@ -23269,8 +23274,8 @@ static const TSParseActionEntry ts_parse_actions[] = { [591] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_expression_statement, 2, 0, 0), [593] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_while_statement, 7, 0, 0), [595] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_while_statement, 7, 0, 0), - [597] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_load_statement, 3, 0, 0), - [599] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_load_statement, 3, 0, 0), + [597] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_load_statement, 3, 0, 1), + [599] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_load_statement, 3, 0, 1), [601] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_print_statement, 3, 0, 0), [603] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_print_statement, 3, 0, 0), [605] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_error_statement, 3, 0, 0), @@ -23289,12 +23294,12 @@ static const TSParseActionEntry ts_parse_actions[] = { [631] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_repeat_statement, 5, 0, 0), [633] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_require_statement, 5, 0, 0), [635] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_require_statement, 5, 0, 0), - [637] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 6, 0, 2), - [639] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 6, 0, 2), - [641] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 6, 0, 2), - [643] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 6, 0, 2), - [645] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 6, 0, 2), - [647] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 6, 0, 2), + [637] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 6, 0, 3), + [639] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 6, 0, 3), + [641] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 6, 0, 3), + [643] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 6, 0, 3), + [645] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 6, 0, 3), + [647] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 6, 0, 3), [649] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_if_statement, 6, 0, 0), [651] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_if_statement, 6, 0, 0), [653] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_while_statement, 6, 0, 0), @@ -23303,12 +23308,12 @@ static const TSParseActionEntry ts_parse_actions[] = { [659] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_case_statement, 6, 0, 0), [661] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_try_statement, 6, 0, 0), [663] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_try_statement, 6, 0, 0), - [665] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 7, 0, 2), - [667] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 7, 0, 2), - [669] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 7, 0, 2), - [671] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 7, 0, 2), - [673] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 7, 0, 2), - [675] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 7, 0, 2), + [665] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_function_definition, 7, 0, 3), + [667] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_function_definition, 7, 0, 3), + [669] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_procedure_definition, 7, 0, 3), + [671] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_procedure_definition, 7, 0, 3), + [673] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_intrinsic_definition, 7, 0, 3), + [675] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_intrinsic_definition, 7, 0, 3), [677] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_return_statement, 3, 0, 0), [679] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_return_statement, 3, 0, 0), [681] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym__type_expr, 1, 0, 0),