From 6f3fdba92ae6c7e39179e424c66fce9b29797888 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Mon, 28 Sep 2026 20:49:30 +0200 Subject: [PATCH 01/15] fix(python): extract defs nested in a function body (#1548) Why The Python def walk stopped at every function_definition: it extracted the def and never looked at its body. Every def or class declared inside a function -- closures, decorator wrappers, the route handlers of a Flask or FastAPI app factory (`@app.route` inside create_app), local classes and their methods -- was absent from the graph. Blocks at module level (if / try / for / with) were transparent, so only function bodies were blind. The calls inside those defs were not re-attributed either: the call-scope walk named a nested def by its class or module only (`pkg.mod.nested`, a QN no node had), so the edges fell back to the file node, and inbound traces of the called function silently lost the caller. A def nested in a method got the method's own QN (`Tree.visit` for the `def visit` inside method Tree.visit), so its calls, recursion included, landed on the method. On an app-factory codebase this removes the whole HTTP layer. Fix - Extraction (extract_defs.c): after a Python def is extracted, its body is walked with the def's QN as the enclosing scope; methods' bodies are walked under the method QN. A nested def or class is qualified by the def it sits in -- pkg.mod.create_app.index, pkg.mod.Tree.visit.visit, pkg.mod.create_app.Settings.value -- the QN the Python LSP already gives the calls inside it (py_process_function). Decorated nested defs keep their decorators, so their route_path mints the Route and HANDLES. Lambdas and comprehensions hold no statements and stay as they are. - Call scope (extract_unified.c): for Python the nearest named scope, a def or a class, prefixes a nested def or class, so every call, usage and env read inside a nested def carries the nested def's QN. - A def nested in a function is a local of it (Python LEGB), so it must not become a global name: a Python symbol whose QN extends a Function/Method node is registered for exact lookups only, never in the short-name index (registry.c cbm_registry_add_for_language, used by the full, parallel and incremental registration sites alike). Without this, django's per-test `class MyModelAdmin` and every decorator's `def wrapper` became unique_name / suffix_match targets for unrelated calls. The incremental seed derives each stored file's language the way discovery does (name, then shebang), so it keeps the same symbols out. The import symbol-name fallback skips such locals too (pass_pkgmap.c), and the Python LSP no longer binds a def-local class module-wide by its short name (py_lsp.c). - A bare call now resolves lexically first (registry.c cbm_registry_resolve_lexical, strategy lexical_scope): the def or class bound in the nearest enclosing function, skipping class bodies, stopping at the module. The sequential and parallel call passes use it identically, and so does the handler lookup of a call-registered route: `app.add_url_rule("/ping", "ping", ping)` in create_app reaches the local ping. Proof (real code; every node and edge diffed, main 1f9b4db1 vs this) - pallets/flask examples/tutorial (flaskr, create_app factory): the nested `hello` handler is now create_app.hello with HANDLES to /hello; 4 nested defs/classes added, 0 nodes and 0 edges removed. - pallets/flask d73fa1c (tests declare handlers inside test functions): Function 532 -> 1068, Method 318 -> 381, Class 130 -> 209. The +678 equals the 678 unique nested def/class QNs Python's own ast finds. Handler HANDLES (decorators and add_url_rule) 32 -> 296. 0 nodes removed, 0 duplicate QNs. The 94 removed CALLS: 90 moved from the file node onto the nested def that makes the call, 4 replaced by the lexically bound local. - django fb11376 (local, 2,927 .py files): nodes 55,483 -> 60,052, Function 1,992 -> 3,027, Method 28,850 -> 29,491, Class 8,187 -> 11,073. 0 nodes removed and 0 duplicate QNs. Every added Function/Method/Class sits under a Function/Method of the same file, and they match ast's 4,567 nested QNs except for 5 methods declared in an `if` inside a local class body (class-body `if` is not walked for any class). CALLS 62,430 -> 63,815. Every added CALLS edge has a nested def at one end: 1,015 from nested defs, 1,219 lexical_scope into a local, and none from outside the local's function. Of the 849 removed: 507 moved from the file node onto the nested def, 334 replaced by the local the name binds, and 8 were false on main, all checked (e.g. the recursive `get_actions` nested in call_command had been suffix-matched to ModelAdmin.get_actions). HANDLES unchanged (135). - Samples checked against the source: AbstractBaseUser.check_password.setter -> set_password / save; View.as_view.view -> setup / dispatch; Apps.lazy_model_operation -> lazy_model_operation.apply_next_model (line 426); InstrumentedASGIRequest.META -> the local InstrumentedDict (line 279); flask test_provide_automatic_options_arg_disable.index -> GET /. Tests - extraction: extract_python_nested_def_qualified_by_enclosing_def, _calls_attributed_to_it, _decorated_nested_route_handler, _nested_class, _nested_def_named_like_its_method, _lambda_and_comprehension_stay_in_enclosing_def (control), _nested_def_env_access_attributed_to_it (all _issue1548). - pipeline: pipeline_python_app_factory_routes, _nested_def_is_function_ local, _local_classes_stay_in_their_def, _import_never_binds_a_local_def, _nested_defs_parallel (>= 50 files, CBM_WORKERS=4), _nested_def_local_after_incremental (all _issue1548). - registry: registry_python_function_local_is_exact_only, registry_resolve_lexical_walks_enclosing_defs (both _issue1548). The extraction and pipeline tests were RED on main 3/3 for the asserted reason and are green with the fix; the registry tests cover the new registry API. Each production piece was reverted on its own, and each revert turns at least one of these tests RED. Fixes #1548 Signed-off-by: Martin Vogel (cherry picked from commit 62fda43fd66200879db62a58903a32dab0dd1a20) --- internal/cbm/extract_defs.c | 66 ++++- internal/cbm/extract_unified.c | 33 ++- internal/cbm/lsp/py_lsp.c | 132 ++++++++-- src/pipeline/pass_calls.c | 22 +- src/pipeline/pass_parallel.c | 19 +- src/pipeline/pass_pkgmap.c | 32 +++ src/pipeline/pipeline.h | 17 +- src/pipeline/pipeline_incremental.c | 5 +- src/pipeline/pipeline_internal.h | 10 + src/pipeline/registry.c | 131 +++++++++- tests/test_extraction.c | 299 ++++++++++++++++++++++ tests/test_pipeline.c | 378 ++++++++++++++++++++++++++++ tests/test_registry.c | 92 +++++++ 13 files changed, 1192 insertions(+), 44 deletions(-) diff --git a/internal/cbm/extract_defs.c b/internal/cbm/extract_defs.c index 77e0bbca0..2b24de63f 100644 --- a/internal/cbm/extract_defs.c +++ b/internal/cbm/extract_defs.c @@ -6624,11 +6624,16 @@ static void extract_func_def(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec * scope. Note the call-scope side (compute_func_qn in extract_unified.c) is * NOT language-gated — it qualifies whenever a scope is pushed — so this gate * and the scope-push rule must move together, or a call QN names a def QN that - * was never minted and the edge is dropped at write. */ + * was never minted and the edge is dropped at write. + * + * Python: the only enclosing scope that reaches here is the def a nested + * def sits in (py_push_def_body), so `def index()` inside create_app is + * proj.mod.create_app.index. */ if (ctx->enclosing_class_qn && (ctx->language == CBM_LANG_CPP || ctx->language == CBM_LANG_CUDA || ctx->language == CBM_LANG_TYPESCRIPT || ctx->language == CBM_LANG_TSX || - ctx->language == CBM_LANG_ARKTS || ctx->language == CBM_LANG_NIX)) { + ctx->language == CBM_LANG_ARKTS || ctx->language == CBM_LANG_NIX || + ctx->language == CBM_LANG_PYTHON)) { def.qualified_name = cbm_arena_sprintf(a, "%s.%s", ctx->enclosing_class_qn, qn_name); } def.label = "Function"; @@ -11667,6 +11672,51 @@ static void push_class_body_children(TSNode node, const CBMLangSpec *spec, wd_st } } +/* Python: a def's body is a scope of its own, and a `def` or `class` statement + * in it is a definition like any other -- closures, decorator wrappers, the + * route handlers of an app factory (`@app.route` inside create_app), local + * classes. Walk the body with the def's QN as the enclosing scope, so a nested + * def is qualified by the def it sits in: proj.mod.create_app.index. That is + * the QN the Python LSP already gives the calls inside it (py_process_function) + * and the one the call-scope walk computes (extract_unified.c + * py_nearest_scope_qn). Only the body is walked: parameters, + * annotations and decorators hold expressions, never statements. A lambda or + * a comprehension holds no statements either, so neither becomes a def. */ +static void py_push_def_body(wd_stack_t *s, TSNode def_node, const char *def_qn) { + TSNode body = ts_node_child_by_field_name(def_node, TS_FIELD("body")); + if (def_qn && !ts_node_is_null(body)) { + wd_push(s, body, def_qn); + } +} + +/* Python methods are extracted by extract_class_methods, not by the walk, so + * their bodies are pushed here, under the QN push_method_def gave the method + * (class_qn.name), in source order. */ +static void py_push_method_bodies(CBMExtractCtx *ctx, TSNode class_node, const char *class_qn, + wd_stack_t *s) { + TSNode body = find_class_member_body(class_node, ctx->language); + if (ts_node_is_null(body) || !class_qn) { + return; + } + for (int i = (int)ts_node_named_child_count(body) - SKIP_CHAR; i >= 0; i--) { + TSNode method = ts_node_named_child(body, (uint32_t)i); + if (strcmp(ts_node_type(method), "decorated_definition") == 0) { + method = ts_node_child_by_field_name(method, TS_FIELD("definition")); + } + if (ts_node_is_null(method) || strcmp(ts_node_type(method), "function_definition") != 0) { + continue; + } + TSNode name_node = resolve_method_name(method, ctx->language); + char *name = + ts_node_is_null(name_node) + ? NULL + : cbm_func_name_node_text(ctx->arena, name_node, ctx->source, ctx->language); + if (name && name[0]) { + py_push_def_body(s, method, cbm_arena_sprintf(ctx->arena, "%s.%s", class_qn, name)); + } + } +} + // ASCII case-insensitive equality vs a lowercase literal (CFML attribute names // are case-insensitive, e.g. NAME / Name / name). static bool ascii_ci_equals(const char *s, const char *lower_lit) { @@ -12211,7 +12261,16 @@ static void walk_defs(CBMExtractCtx *ctx, TSNode root, const CBMLangSpec *spec, } if (cbm_kind_in_set(node, spec->function_node_types)) { if (!is_template_class_node(node, ctx->language)) { + int defs_before = ctx->result->defs.count; extract_func_def(ctx, node, spec); + if (ctx->language == CBM_LANG_PYTHON) { + if (ctx->result->defs.count > defs_before) { + py_push_def_body(&s, node, + ctx->result->defs.items[ctx->result->defs.count - SKIP_ONE] + .qualified_name); + } + continue; + } // Most languages stop here. JS/TS (and Wolfram) descend into the // function body so NESTED named definitions are also captured — // e.g. arrow methods of an object literal returned from a factory @@ -12268,6 +12327,9 @@ static void walk_defs(CBMExtractCtx *ctx, TSNode root, const CBMLangSpec *spec, extract_class_def(ctx, node, spec); const char *new_enclosing = compute_class_qn(ctx, node, frame.enclosing_class_qn); push_class_body_children(node, spec, &s, new_enclosing, ctx); + if (ctx->language == CBM_LANG_PYTHON && new_enclosing != frame.enclosing_class_qn) { + py_push_method_bodies(ctx, node, new_enclosing, &s); + } continue; } diff --git a/internal/cbm/extract_unified.c b/internal/cbm/extract_unified.c index 96e6ac865..c7b10d7e2 100644 --- a/internal/cbm/extract_unified.c +++ b/internal/cbm/extract_unified.c @@ -916,6 +916,21 @@ static const char *objectscript_get_method_qn(CBMExtractCtx *ctx, TSNode node, return NULL; } +/* Python: the nearest named scope around the node -- a def or a class. A def + * or class nested in a def is qualified by that def (proj.mod.create_app.index), + * exactly as extract_defs.c mints it; enclosing_class_qn alone would skip the + * def and hand a def nested in a method its method's QN. Lambdas push no named + * frame, so they stay transparent. */ +static const char *py_nearest_scope_qn(const WalkState *state) { + for (int i = state->scope_top - SKIP_ONE; i >= 0; i--) { + const CBMWalkScope *f = &state->scopes[i]; + if ((f->kind == SCOPE_FUNC || f->kind == SCOPE_CLASS) && f->qn) { + return f->qn; + } + } + return NULL; +} + // Compute function QN for scope tracking (mirrors cbm_enclosing_func_qn logic). static const char *compute_func_qn(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec *spec, WalkState *state) { @@ -1107,8 +1122,10 @@ static const char *compute_func_qn(CBMExtractCtx *ctx, TSNode node, const CBMLan } } - if (state->enclosing_class_qn) { - return cbm_arena_sprintf(ctx->arena, "%s.%s", state->enclosing_class_qn, qn_name); + const char *outer_qn = + ctx->language == CBM_LANG_PYTHON ? py_nearest_scope_qn(state) : state->enclosing_class_qn; + if (outer_qn) { + return cbm_arena_sprintf(ctx->arena, "%s.%s", outer_qn, qn_name); } /* Java/Go: directory-based module so this enclosing-func QN matches the def * QN and the LSP caller_qn (the lsp_resolve join keys on exact equality). */ @@ -1162,9 +1179,15 @@ static const char *compute_class_qn(CBMExtractCtx *ctx, TSNode node, const WalkS /* Nested class: prefix with the enclosing class QN (Outer.Inner) so this * scope QN matches the def-side class QN (extract_defs.c compute_class_qn / - * extract_class_def), which the lsp_resolve join requires for nested types. */ - if (state && state->enclosing_class_qn) { - return cbm_arena_sprintf(ctx->arena, "%s.%s", state->enclosing_class_qn, name); + * extract_class_def), which the lsp_resolve join requires for nested types. + * Python: a class inside a def is qualified by the def (py_nearest_scope_qn). */ + const char *outer_qn = NULL; + if (state) { + outer_qn = ctx->language == CBM_LANG_PYTHON ? py_nearest_scope_qn(state) + : state->enclosing_class_qn; + } + if (outer_qn) { + return cbm_arena_sprintf(ctx->arena, "%s.%s", outer_qn, name); } /* Java/Go: directory-based module (see compute_func_qn). */ diff --git a/internal/cbm/lsp/py_lsp.c b/internal/cbm/lsp/py_lsp.c index 6af7dae40..cbad720fa 100644 --- a/internal/cbm/lsp/py_lsp.c +++ b/internal/cbm/lsp/py_lsp.c @@ -246,6 +246,9 @@ static void py_register_lambda(PyLSPContext *ctx, const char *name, TSNode lambd static void py_register_dict_literal(PyLSPContext *ctx, const char *name, TSNode dict_node); static TSNode py_lookup_lambda(PyLSPContext *ctx, const char *name); static const char *py_lookup_dict_dispatch(PyLSPContext *ctx, const char *var, const char *key); +static void py_process_function(PyLSPContext *ctx, TSNode func_node, const char *container_qn); +typedef enum { PY_LOCAL_DEFS_BIND, PY_LOCAL_DEFS_PROCESS } PyLocalDefsMode; +static void py_visit_local_defs(PyLSPContext *ctx, TSNode block, PyLocalDefsMode mode, int depth); /* ── Scope mutation wrappers ───────────────────────────────────────────── * @@ -4362,6 +4365,90 @@ static void py_bind_parameters(PyLSPContext *ctx, TSNode params) { } } +/* Bind a def or class statement of the current function body as a local of + * that function (#1548): its graph QN extends the function's + * (enclosing_func_qn.name), the QN extraction gives it. An undecorated def + * registered with that QN becomes an exact callable alias; anything else + * (decorated, overloaded, not registered) is a hard local shadow, so no call + * falls through to a same-named module symbol. A class is bound by its type + * when registered, else shadowed. A parameter of the same name is already a + * local and keeps its own binding. */ +static void py_bind_local_def(PyLSPContext *ctx, TSNode def, bool is_class) { + TSNode name_node = ts_node_child_by_field_name(def, "name", 4); + char *name = ts_node_is_null(name_node) ? NULL : py_node_text(ctx, name_node); + if (!name || !name[0] || !ctx->enclosing_func_qn) { + py_disable_callable_value_proof(ctx); + return; + } + if (cbm_scope_lookup_local(ctx->current_scope, name)) + return; + const char *qn = cbm_arena_sprintf(ctx->arena, "%s.%s", ctx->enclosing_func_qn, name); + if (!qn) { + py_disable_callable_value_proof(ctx); + py_scope_bind(ctx, name, cbm_type_unknown()); + return; + } + if (is_class) { + py_scope_bind(ctx, name, + cbm_registry_lookup_type(ctx->registry, qn) ? cbm_type_named(ctx->arena, qn) + : cbm_type_unknown()); + return; + } + const CBMRegisteredFunc *func = cbm_registry_lookup_func(ctx->registry, qn); + if (py_func_is_exact_callable_value(func)) { + py_scope_bind_callable(ctx, name, cbm_type_unknown(), func->qualified_name); + } else { + py_scope_bind(ctx, name, cbm_type_unknown()); + } +} + +/* Statements a function body nests further statements in. A def inside any of + * them is still a local of the function; a nested def, class or lambda body + * is a scope of its own and is never entered. */ +static bool py_is_statement_block(const char *kind) { + static const char *const kinds[] = { + "block", "if_statement", "elif_clause", "else_clause", + "for_statement", "while_statement", "try_statement", "except_clause", + "except_group_clause", "finally_clause", "with_statement", "match_statement", + "case_clause", NULL}; + for (int i = 0; kinds[i]; i++) { + if (strcmp(kind, kinds[i]) == 0) + return true; + } + return false; +} + +/* Visit the def/class statements of a function body, including those inside + * its compound statements: BIND binds each as a local (py_bind_local_def), + * PROCESS walks each nested def's own body (py_process_function). */ +static void py_visit_local_defs(PyLSPContext *ctx, TSNode block, PyLocalDefsMode mode, int depth) { + if (!ctx || ctx->eval_failure || ts_node_is_null(block) || depth >= cbm_lsp_max_walk_depth()) + return; + PyKids kids = py_kids(ctx->arena, block); + for (uint32_t i = 0; i < kids.n && !ctx->eval_failure; i++) { + TSNode c = py_kid(&kids, i); + const char *ck = ts_node_type(c); + TSNode def = c; + if (strcmp(ck, "decorated_definition") == 0) { + def = ts_node_child_by_field_name(c, "definition", 10); + if (ts_node_is_null(def)) + continue; + ck = ts_node_type(def); + } + if (strcmp(ck, "function_definition") == 0) { + if (mode == PY_LOCAL_DEFS_BIND) + py_bind_local_def(ctx, def, false); + else + py_process_function(ctx, def, ctx->enclosing_func_qn); + } else if (strcmp(ck, "class_definition") == 0) { + if (mode == PY_LOCAL_DEFS_BIND) + py_bind_local_def(ctx, def, true); + } else if (py_is_statement_block(ck)) { + py_visit_local_defs(ctx, c, mode, depth + 1); + } + } +} + static void py_process_function(PyLSPContext *ctx, TSNode func_node, const char *container_qn) { if (!ctx || ctx->eval_failure) return; @@ -4391,22 +4478,13 @@ static void py_process_function(PyLSPContext *ctx, TSNode func_node, const char TSNode body = ts_node_child_by_field_name(func_node, "body", 4); if (!ts_node_is_null(body)) { + /* Python resolves a bare name in the function's own scope first: every + * def/class statement of the body binds a local for the whole body + * (LEGB), whatever a module-level symbol of the same spelling says. */ + py_visit_local_defs(ctx, body, PY_LOCAL_DEFS_BIND, 0); py_resolve_calls_in(ctx, body); - // Also descend into nested function/class definitions in the body. - PyKids bk = py_kids(ctx->arena, body); - for (uint32_t i = 0; i < bk.n; i++) { - TSNode c = py_kid(&bk, i); - const char *ck = ts_node_type(c); - if (strcmp(ck, "function_definition") == 0) { - py_process_function(ctx, c, ctx->enclosing_func_qn); - } else if (strcmp(ck, "decorated_definition") == 0) { - TSNode def = ts_node_child_by_field_name(c, "definition", 10); - if (!ts_node_is_null(def) && - strcmp(ts_node_type(def), "function_definition") == 0) { - py_process_function(ctx, def, ctx->enclosing_func_qn); - } - } - } + // Also descend into nested function definitions in the body. + py_visit_local_defs(ctx, body, PY_LOCAL_DEFS_PROCESS, 0); } py_scope_restore(ctx, saved); @@ -4687,6 +4765,28 @@ static char *py_module_prefix(CBMArena *scratch, const char *module_qn, size_t l return prefix; } +/* A class declared inside a def -- django's per-test `class MyModelAdmin`, an + * app factory's local class, and anything nested in one -- is a local of that + * def, not a global of the module: bound here by its short name, every + * same-named local class of the file collapsed onto whichever registered last. + * Its QN extends a registered function's (proj.mod.Tests.test_x.MyModelAdmin), + * which tells it apart from a class nested in a class (proj.mod.Outer.Inner). + * Only prefixes longer than the module are probed. */ +static bool py_type_is_function_local(const PyLSPContext *ctx, const char *qn, size_t module_len) { + char scope[512]; + size_t len = strlen(qn); + if (len >= sizeof(scope)) + return false; + memcpy(scope, qn, len + 1); + for (char *dot = strrchr(scope, '.'); dot && (size_t)(dot - scope) > module_len; + dot = strrchr(scope, '.')) { + *dot = '\0'; + if (cbm_registry_lookup_func(ctx->registry, scope)) + return true; + } + return false; +} + /* A project registry can contain another file's class in the same logical * Python module. Keep those cross-file globals available, but do not prebind * classes declared by this source: their binding epoch belongs in the ordered @@ -4734,6 +4834,8 @@ static void py_bind_external_module_classes(PyLSPContext *ctx, const PyKids *rk) continue; if (package_scope && strchr(qn + prefix_len + 1, '.')) continue; /* submodule/nested classes are not package globals */ + if (py_type_is_function_local(ctx, qn, prefix_len)) + continue; /* a def-local class is no module global (#1548) */ py_scope_bind(ctx, name, cbm_type_named(ctx->arena, qn)); } if (prefix) diff --git a/src/pipeline/pass_calls.c b/src/pipeline/pass_calls.c index 1eb3bddff..77fa14f4a 100644 --- a/src/pipeline/pass_calls.c +++ b/src/pipeline/pass_calls.c @@ -587,9 +587,19 @@ static void handle_route_registration(cbm_pipeline_ctx_t *ctx, const CBMCall *ca esc_fa); cbm_gbuf_insert_edge(ctx->gbuf, source_node->id, route_id, "CALLS", props); if (call->second_arg_name != NULL && call->second_arg_name[0] != '\0') { - cbm_resolution_t hres = - cbm_registry_resolve_handler(ctx->registry, call->second_arg_name, module_qn, imp_keys, - imp_vals, imp_count, ctx->gbuf, lang); + /* Python: an app factory registers its local defs + * (`app.add_url_rule("/", "index", index)` inside create_app); only the + * lexical lookup sees a function's locals. MUST match pass_parallel.c. */ + cbm_resolution_t hres = {0}; + if (lang == CBM_LANG_PYTHON) { + hres = cbm_registry_resolve_lexical(ctx->registry, call->second_arg_name, + call->enclosing_func_qn); + } + if (!hres.qualified_name) { + hres = cbm_registry_resolve_handler(ctx->registry, call->second_arg_name, + module_qn, imp_keys, imp_vals, imp_count, + ctx->gbuf, lang); + } if (hres.qualified_name != NULL && hres.qualified_name[0] != '\0') { const cbm_gbuf_node_t *handler = cbm_gbuf_find_by_qn(ctx->gbuf, hres.qualified_name); if (handler != NULL) { @@ -967,8 +977,10 @@ static int resolve_single_call(cbm_pipeline_ctx_t *ctx, CBMCall *call, const CBM lang, lsp->strategy, lsp->callee_qn, ctx->project_name); cbm_resolution_t res = {0}; if (!rust_external && !php_vendor_call) { - res = cbm_registry_resolve_call_lang(ctx->registry, call->callee_name, module_qn, imp_keys, - imp_vals, imp_count, lang); + /* Python LEGB fallback first (MUST match pass_parallel.c). */ + res = cbm_registry_resolve_call_scoped(ctx->registry, call->callee_name, module_qn, + imp_keys, imp_vals, imp_count, lang, + call->enclosing_func_qn); /* Cross-language veto: a name-only guess never binds another * language's symbol (a JS app.get must not become Python's * builtins.dict.get). The vetoed answer becomes EMPTY here, before diff --git a/src/pipeline/pass_parallel.c b/src/pipeline/pass_parallel.c index 2c0a9023a..501320dd6 100644 --- a/src/pipeline/pass_parallel.c +++ b/src/pipeline/pass_parallel.c @@ -2410,8 +2410,17 @@ static void emit_route_registration(cbm_gbuf_t *gbuf, const cbm_gbuf_node_t *sou esc_rp); cbm_gbuf_insert_edge(gbuf, source->id, rid, "CALLS", props); if (handler_ref && handler_ref[0] != '\0') { - cbm_resolution_t hres = cbm_registry_resolve_handler(registry, handler_ref, module_qn, ik, - iv, ic, main_gbuf, lang); + /* Python: a handler an app factory registers is usually its local def + * (`app.add_url_rule("/", "index", index)` inside create_app); only the + * lexical lookup sees a function's locals. MUST match pass_calls.c. */ + cbm_resolution_t hres = {0}; + if (lang == CBM_LANG_PYTHON) { + hres = cbm_registry_resolve_lexical(registry, handler_ref, call->enclosing_func_qn); + } + if (!hres.qualified_name) { + hres = cbm_registry_resolve_handler(registry, handler_ref, module_qn, ik, iv, ic, + main_gbuf, lang); + } if (hres.qualified_name && hres.qualified_name[0] != '\0') { const cbm_gbuf_node_t *h = cbm_gbuf_find_by_qn(main_gbuf, hres.qualified_name); if (h) { @@ -3267,8 +3276,10 @@ static void resolve_file_calls(resolve_ctx_t *rc, resolve_worker_state_t *ws, CB lang, lsp->strategy, lsp->callee_qn, rc->project_name); if ((!res.qualified_name || !res.qualified_name[0]) && !call->requires_lsp_resolution && !rust_external && !php_vendor_call) { - res = cbm_registry_resolve_call_lang(rc->registry, call->callee_name, module_qn, - imp_keys, imp_vals, imp_count, lang); + /* Python LEGB fallback first (MUST match pass_calls.c). */ + res = cbm_registry_resolve_call_scoped(rc->registry, call->callee_name, module_qn, + imp_keys, imp_vals, imp_count, lang, + call->enclosing_func_qn); } /* `p->open(fd)`: the registry does not split a callee on the arrow. * The object's type decides, as for the dot call handled further diff --git a/src/pipeline/pass_pkgmap.c b/src/pipeline/pass_pkgmap.c index badc27954..e30a0c231 100644 --- a/src/pipeline/pass_pkgmap.c +++ b/src/pipeline/pass_pkgmap.c @@ -1819,6 +1819,34 @@ bool cbm_import_symbol_fallback_allowed(CBMLanguage lang) { return lang != CBM_LANG_GO; } +/* A Python def or class declared inside a function body -- a closure, a + * decorator's `wrapper`, a test's local class -- is a local of that function: + * no import can name it. Its QN extends a Function/Method node's + * (proj.tests.mock_inputs.inner.wrapper.mock_getpass.getpass), which is how it + * is recognised; the symbol-name fallback bound `import getpass` to exactly + * that def. Mirrors cbm_registry_add_lang, which keeps the same + * symbols out of the call resolver's short-name index. */ +static bool import_candidate_is_python_local(const cbm_gbuf_t *gbuf, const cbm_gbuf_node_t *cand) { + const char *qn = cand->qualified_name; + const char *slash = cand->file_path ? strrchr(cand->file_path, '/') : NULL; + char scope[CBM_SZ_512]; + size_t len = qn ? strlen(qn) : 0; + if (!cand->file_path || len == 0 || len >= sizeof(scope) || + cbm_language_for_filename(slash ? slash + SKIP_ONE : cand->file_path) != CBM_LANG_PYTHON) { + return false; + } + memcpy(scope, qn, len + SKIP_ONE); + for (char *dot = strrchr(scope, '.'); dot; dot = strrchr(scope, '.')) { + *dot = '\0'; + const cbm_gbuf_node_t *outer = cbm_gbuf_find_by_qn(gbuf, scope); + if (outer && outer->label && + (strcmp(outer->label, "Function") == 0 || strcmp(outer->label, "Method") == 0)) { + return true; + } + } + return false; +} + static const char *path_leaf(const char *path) { const char *leaf = path; for (const char *p = path; p && *p; p++) { @@ -2516,6 +2544,10 @@ static const cbm_gbuf_node_t *resolve_import_strategies( !import_targetable_label(cand->label)) { continue; } + /* A Python def-local (#1548) is no importable name. */ + if (import_candidate_is_python_local(ctx->gbuf, cand)) { + continue; + } if (source_file_qn && cand->qualified_name && strcmp(cand->qualified_name, source_file_qn) == 0) { continue; /* self */ diff --git a/src/pipeline/pipeline.h b/src/pipeline/pipeline.h index b0ba5bc15..e03c0dbe6 100644 --- a/src/pipeline/pipeline.h +++ b/src/pipeline/pipeline.h @@ -253,7 +253,11 @@ void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified /* As cbm_registry_add, recording `lang`: the DETECTED language of the file * that defines the symbol (CBM_LANG_COUNT = unknown), which the cross-language - * veto (cbm_registry_name_guess_vetoed) checks against the caller. */ + * veto (cbm_registry_name_guess_vetoed) checks against the caller. A Python + * symbol declared inside a function body (see cbm_registry_qn_is_function_local) + * is registered for exact lookups only, never offered to the short-name + * strategies; every registration site seeds through this, so full and + * incremental indexes agree. */ void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, const char *label, CBMLanguage lang); @@ -268,6 +272,17 @@ bool cbm_lang_resolution_compatible(CBMLanguage caller, CBMLanguage target); * CBM_LANG_COUNT for anything else. */ CBMLanguage cbm_registry_synthetic_path_language(const char *file_path); +/* True when a proper prefix of `qualified_name` is a registered Function or + * Method, i.e. the symbol was declared inside a function body. */ +bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name); + +/* Python bare-name lexical lookup: the def or class named `callee_name` bound + * in the nearest function scope enclosing `scope_qn` (the caller's QN), + * skipping class bodies, stopping at the module. Strategy "lexical_scope"; + * qualified_name is NULL when no such local exists. */ +cbm_resolution_t cbm_registry_resolve_lexical(const cbm_registry_t *r, const char *callee_name, + const char *scope_qn); + /* Resolve a callee name using prioritized strategies. * import_map: NULL-terminated array of {local_name, resolved_qn} pairs, or NULL. * Returns result with qualified_name="" if unresolved. diff --git a/src/pipeline/pipeline_incremental.c b/src/pipeline/pipeline_incremental.c index 99df053e4..4d0d667f5 100644 --- a/src/pipeline/pipeline_incremental.c +++ b/src/pipeline/pipeline_incremental.c @@ -1327,7 +1327,10 @@ static bool incr_label_is_registry_symbol(const char *label) { /* Callback for cbm_gbuf_foreach_node: seed the registry with the existing * project's definition symbols so the resolver can match cross-file symbols * during incremental. Mirrors the full-index registry contents exactly so an - * incremental re-resolve picks the same nodes a full reindex would. */ + * incremental re-resolve picks the same nodes a full reindex would -- including + * a Python function's locals, which cbm_registry_add_lang keeps out of the + * short-name index there too (nodes load ORDER BY id and a file's defs are + * inserted parent first, so the enclosing function is seeded before its locals). */ /* Each symbol is seeded with its file's DETECTED language, exactly what the * full index recorded (pass_definitions.c / pass_parallel.c), so the cross- * language veto judges an incremental re-resolve the same way. Stored nodes carry no diff --git a/src/pipeline/pipeline_internal.h b/src/pipeline/pipeline_internal.h index fbae8044f..0965e0398 100644 --- a/src/pipeline/pipeline_internal.h +++ b/src/pipeline/pipeline_internal.h @@ -260,6 +260,16 @@ cbm_resolution_t cbm_registry_resolve_call_lang(const cbm_registry_t *r, const c const char *module_qn, const char **import_map_keys, const char **import_map_vals, int import_map_count, CBMLanguage lang); +/* The registry step of BOTH call resolvers once the LSP has not answered: + * for Python, the def or class `callee_name` binds in the nearest def enclosing + * `scope_qn` (cbm_registry_resolve_lexical, "lexical_scope"); otherwise, or + * when there is no such local, cbm_registry_resolve_call_lang. */ +cbm_resolution_t cbm_registry_resolve_call_scoped(const cbm_registry_t *r, const char *callee_name, + const char *module_qn, + const char **import_map_keys, + const char **import_map_vals, + int import_map_count, CBMLanguage lang, + const char *scope_qn); #ifdef CBM_ENABLE_TEST_SEAMS void cbm_pxc_test_import_probe_fail_after(int successful_allocations); #endif diff --git a/src/pipeline/registry.c b/src/pipeline/registry.c index f53d1dd12..8a1122ddf 100644 --- a/src/pipeline/registry.c +++ b/src/pipeline/registry.c @@ -62,6 +62,8 @@ const char *cbm_confidence_band(double score) { #define CONF_IMPORT_MAP_SUFFIX 0.85 /* Strategy 2: same_module — same file/package → high confidence */ #define CONF_SAME_MODULE 0.90 +/* Python lexical scope — a def bound in an enclosing function (LEGB) */ +#define CONF_LEXICAL_SCOPE 0.95 /* Strategy 3: unique_name — only one candidate project-wide */ #define CONF_UNIQUE_NAME 0.75 /* Strategy 4: suffix_match — multiple candidates, filtered */ @@ -1181,20 +1183,33 @@ static void index_under_name(cbm_registry_t *r, const char *key, const char *own } } -void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified_name, - const char *label) { - cbm_registry_add_lang(r, name, qualified_name, label, CBM_LANG_COUNT); +static bool registry_label_is_callable(const char *label) { + return label && (strcmp(label, "Function") == 0 || strcmp(label, "Method") == 0); } -void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, - const char *label, CBMLanguage lang) { - if (!r || !qualified_name || !label) { - return; +bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name) { + char scope[CBM_SZ_512]; + size_t len = qualified_name ? strlen(qualified_name) : 0; + if (!r || len == 0 || len >= sizeof(scope)) { + return false; } + memcpy(scope, qualified_name, len + SKIP_ONE); + for (char *dot = strrchr(scope, '.'); dot; dot = strrchr(scope, '.')) { + *dot = '\0'; + if (registry_label_is_callable(cbm_ht_get(r->exact, scope))) { + return true; + } + } + return false; +} +/* Record `qualified_name` in the exact map. Returns the registry's owned copy + * of the QN, or NULL when it is already registered or cannot be recorded. */ +static const char *registry_add_exact(cbm_registry_t *r, const char *qualified_name, + const char *label) { /* Check for duplicate */ if (cbm_ht_get(r->exact, qualified_name)) { - return; + return NULL; } /* Intern the label (bounded set; linear scan is fine at this size). */ @@ -1211,14 +1226,45 @@ void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qual r->label_pool_n++; } if (!interned) { - return; /* pool exhausted (cannot happen with sane label sets) */ + return NULL; /* pool exhausted (cannot happen with sane label sets) */ } /* Store in exact map: QN → interned label. The key is the registry's ONE - * owned copy of the QN; by_name below borrows it (same lifetime) instead + * owned copy of the QN; by_name borrows it (same lifetime) instead * of a second strdup — this pair of copies was ~280 MB on the kernel. */ cbm_ht_set(r->exact, strdup(qualified_name), (void *)interned); - const char *owned_qn = cbm_ht_get_key(r->exact, qualified_name); + return cbm_ht_get_key(r->exact, qualified_name); +} + +void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified_name, + const char *label) { + cbm_registry_add_lang(r, name, qualified_name, label, CBM_LANG_COUNT); +} + +/* Python nests defs: a def or class declared in a function body -- a closure, + * an app factory's route handler, a local class and its methods -- is a local + * of that function, and its QN extends the function's + * (proj.mod.create_app.index). No code outside the function can name it, so it + * stays out of the short-name index: the weak name strategies (unique_name, + * suffix_match) must not offer django's per-test `class MyModelAdmin` or a + * decorator's `def wrapper` to a caller in another function or file. The exact + * map still holds it, for LSP targets and cbm_registry_resolve_lexical. One + * file's defs register parent first, so the enclosing function is already in + * the exact map when its locals arrive. Gated to Python: probing every + * prefix of every symbol would cost a kernel index seconds for nothing. */ +void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, + const char *label, CBMLanguage lang) { + if (!r || !qualified_name || !label) { + return; + } + if (lang == CBM_LANG_PYTHON && cbm_registry_qn_is_function_local(r, qualified_name)) { + (void)registry_add_exact(r, qualified_name, label); + return; + } + const char *owned_qn = registry_add_exact(r, qualified_name, label); + if (!owned_qn) { + return; + } /* Index the symbol under the QN's last dot segment, and additionally under * the name its caller passed when that segment carries a '#' fence. @@ -2123,6 +2169,69 @@ cbm_resolution_t cbm_registry_resolve_lineage(const cbm_registry_t *r, const cha import_map_count, module_qn ? strlen(module_qn) : 0, false); } +/* Python looks a bare name up in the calling def's own scope, then in each + * enclosing def's scope, and only then in the module and the builtins; a class + * body is never among them. So a bare `index()` in create_app, or `visit(...)` + * in a def nested in method Tree.visit, names the def bound in the nearest + * enclosing def -- create_app.index, Tree.visit.visit -- before anything a + * module-wide strategy would find. Walks scope_qn outward while its prefixes + * are registered defs or classes (skipping the classes) and stops at the + * module, which the other strategies own. Empty unless such a local exists. + * Uncached: the answer depends on the caller, not only on the name. */ +cbm_resolution_t cbm_registry_resolve_lexical(const cbm_registry_t *r, const char *callee_name, + const char *scope_qn) { + char scope[CBM_SZ_512]; + char candidate[CBM_SZ_512]; + size_t len = scope_qn ? strlen(scope_qn) : 0; + if (!r || !callee_name || !callee_name[0] || strchr(callee_name, '.') || + strstr(callee_name, "::") || len == 0 || len >= sizeof(scope)) { + return empty_result(); + } + memcpy(scope, scope_qn, len + SKIP_ONE); + for (;;) { + const char *label = cbm_ht_get(r->exact, scope); + if (!label) { + break; + } + if (registry_label_is_callable(label)) { + int n = snprintf(candidate, sizeof(candidate), "%s.%s", scope, callee_name); + const char *stored_key = + n > 0 && (size_t)n < sizeof(candidate) ? cbm_ht_get_key(r->exact, candidate) : NULL; + if (stored_key) { + return (cbm_resolution_t){stored_key, "lexical_scope", CONF_LEXICAL_SCOPE, + REG_RESOLVED}; + } + } + char *dot = strrchr(scope, '.'); + if (!dot) { + break; + } + *dot = '\0'; + } + return empty_result(); +} + +/* The registry step both call resolvers (pass_calls.c, pass_parallel.c) run + * once the LSP has not answered: a Python bare name bound in an enclosing def + * first (the LSP binds those locals itself; this covers the sites it did not + * resolve), then the language-aware chain. One entry point, so the two + * resolvers cannot diverge on the rule. */ +cbm_resolution_t cbm_registry_resolve_call_scoped(const cbm_registry_t *r, const char *callee_name, + const char *module_qn, + const char **import_map_keys, + const char **import_map_vals, + int import_map_count, CBMLanguage lang, + const char *scope_qn) { + if (lang == CBM_LANG_PYTHON) { + cbm_resolution_t lexical = cbm_registry_resolve_lexical(r, callee_name, scope_qn); + if (lexical.qualified_name && lexical.qualified_name[0]) { + return lexical; + } + } + return cbm_registry_resolve_call_lang(r, callee_name, module_qn, import_map_keys, + import_map_vals, import_map_count, lang); +} + /* ── Fuzzy Resolve ──────────────────────────────────────────────── */ /* Filter candidates by import reachability. Returns count of reachable. */ diff --git a/tests/test_extraction.c b/tests/test_extraction.c index e39d40a8a..5f46a416d 100644 --- a/tests/test_extraction.c +++ b/tests/test_extraction.c @@ -4723,6 +4723,298 @@ TEST(python_init_nested_module_qn) { PASS(); } +/* ── #1548: Python defs nested in a function body ─────────────────── */ + +/* How many defs carry exactly this QN. */ +static int pyn_def_count(const CBMFileResult *r, const char *qn) { + int n = 0; + for (int i = 0; i < r->defs.count; i++) { + const char *q = r->defs.items[i].qualified_name; + if (q && strcmp(q, qn) == 0) { + n++; + } + } + return n; +} + +/* The one def with this QN, or NULL when there is none (or more than one). */ +static const CBMDefinition *pyn_def(const CBMFileResult *r, const char *qn) { + if (pyn_def_count(r, qn) != 1) { + return NULL; + } + for (int i = 0; i < r->defs.count; i++) { + const char *q = r->defs.items[i].qualified_name; + if (q && strcmp(q, qn) == 0) { + return &r->defs.items[i]; + } + } + return NULL; +} + +/* Calls of `callee` whose enclosing function is exactly `caller_qn`. */ +static int pyn_calls(const CBMFileResult *r, const char *callee, const char *caller_qn) { + int n = 0; + for (int i = 0; i < r->calls.count; i++) { + const CBMCall *c = &r->calls.items[i]; + if (c->callee_name && strcmp(c->callee_name, callee) == 0 && c->enclosing_func_qn && + strcmp(c->enclosing_func_qn, caller_qn) == 0) { + n++; + } + } + return n; +} + +/* No def QN is minted twice: a nested def must not also surface as a + * module-level one, and walking a def body must not re-extract anything. */ +static bool pyn_defs_unique(const CBMFileResult *r) { + for (int i = 0; i < r->defs.count; i++) { + const char *q = r->defs.items[i].qualified_name; + if (q && pyn_def_count(r, q) != 1) { + return false; + } + } + return true; +} + +/* The issue's reproducer: `def nested()` inside `def factory()` was never + * extracted. It is now a Function qualified by the def it sits in, the same + * QN the Python LSP gives its calls. Module-level defs, and defs in a + * module-level `if` block, keep their QNs. */ +TEST(extract_python_nested_def_qualified_by_enclosing_def_issue1548) { + CBMFileResult *r = extract("def target():\n" + " return 0\n" + "\n" + "\n" + "def module_level():\n" + " return target()\n" + "\n" + "\n" + "if True:\n" + " def inside_if():\n" + " return target()\n" + "\n" + "\n" + "def factory():\n" + " def nested():\n" + " return target()\n" + " return nested\n" + "\n" + "\n" + "def outer():\n" + " def middle():\n" + " def inner():\n" + " return target()\n" + " return inner\n" + " return middle\n", + CBM_LANG_PYTHON, "t", "app.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + const CBMDefinition *nested = pyn_def(r, "t.app.factory.nested"); + ASSERT_NOT_NULL(nested); + ASSERT_STR_EQ(nested->label, "Function"); + ASSERT_EQ(nested->start_line, 15); + ASSERT_NOT_NULL(pyn_def(r, "t.app.outer.middle")); + ASSERT_NOT_NULL(pyn_def(r, "t.app.outer.middle.inner")); + /* Never minted flat, where it would collide with a module-level name. */ + ASSERT_EQ(pyn_def_count(r, "t.app.nested"), 0); + ASSERT_EQ(pyn_def_count(r, "t.app.inner"), 0); + /* Controls: unchanged. */ + ASSERT_NOT_NULL(pyn_def(r, "t.app.target")); + ASSERT_NOT_NULL(pyn_def(r, "t.app.module_level")); + ASSERT_NOT_NULL(pyn_def(r, "t.app.inside_if")); + ASSERT_NOT_NULL(pyn_def(r, "t.app.factory")); + ASSERT_TRUE(pyn_defs_unique(r)); + cbm_free_result(r); + PASS(); +} + +/* A call inside a nested def belongs to that def. It used to carry the flat + * `t.app.nested`, a QN no node has, so the edge was pinned on the file node: + * inbound traces of target() lost the caller. It is not the outer def's + * either. */ +TEST(extract_python_nested_def_calls_attributed_to_it_issue1548) { + CBMFileResult *r = extract("def target():\n" + " return 0\n" + "\n" + "def factory():\n" + " def nested():\n" + " return target()\n" + " return nested\n" + "\n" + "def outer():\n" + " def middle():\n" + " def inner():\n" + " return target()\n" + " return inner\n" + " return middle\n", + CBM_LANG_PYTHON, "t", "app.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + ASSERT_EQ(pyn_calls(r, "target", "t.app.factory.nested"), 1); + ASSERT_EQ(pyn_calls(r, "target", "t.app.outer.middle.inner"), 1); + ASSERT_EQ(pyn_calls(r, "target", "t.app.factory"), 0); + ASSERT_EQ(pyn_calls(r, "target", "t.app.nested"), 0); + ASSERT_EQ(pyn_calls(r, "target", "t.app.inner"), 0); + cbm_free_result(r); + PASS(); +} + +/* The app-factory shape: route handlers are decorated defs inside + * create_app. Each is extracted with its QN and carries the route its + * decorator declares, which is what mints the Route node and its HANDLES + * edge (pipeline_python_app_factory_routes_issue1548 checks the graph). */ +TEST(extract_python_decorated_nested_route_handler_issue1548) { + CBMFileResult *r = extract("from flask import Flask\n" + "\n" + "def create_app():\n" + " app = Flask(__name__)\n" + "\n" + " @app.route(\"/users/\")\n" + " def show_user(uid):\n" + " return render(uid)\n" + "\n" + " @app.get(\"/health\")\n" + " async def health():\n" + " return render(\"ok\")\n" + "\n" + " return app\n", + CBM_LANG_PYTHON, "t", "web.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + const CBMDefinition *show = pyn_def(r, "t.web.create_app.show_user"); + ASSERT_NOT_NULL(show); + ASSERT_STR_EQ(show->route_path, "/users/"); + ASSERT_NOT_NULL(show->decorators); + const CBMDefinition *health = pyn_def(r, "t.web.create_app.health"); + ASSERT_NOT_NULL(health); + ASSERT_STR_EQ(health->route_path, "/health"); + ASSERT_STR_EQ(health->route_method, "GET"); + ASSERT_EQ(pyn_calls(r, "render", "t.web.create_app.show_user"), 1); + ASSERT_EQ(pyn_calls(r, "render", "t.web.create_app.health"), 1); + ASSERT_TRUE(pyn_defs_unique(r)); + cbm_free_result(r); + PASS(); +} + +/* A class declared in a def is qualified by the def; its methods are Methods + * of it, and a call in such a method is the method's. */ +TEST(extract_python_nested_class_issue1548) { + CBMFileResult *r = extract("def create_app():\n" + " class Settings:\n" + " debug = False\n" + "\n" + " def value(self):\n" + " return load(self)\n" + "\n" + " return Settings().value()\n", + CBM_LANG_PYTHON, "t", "web.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + const CBMDefinition *cls = pyn_def(r, "t.web.create_app.Settings"); + ASSERT_NOT_NULL(cls); + ASSERT_STR_EQ(cls->label, "Class"); + const CBMDefinition *value = pyn_def(r, "t.web.create_app.Settings.value"); + ASSERT_NOT_NULL(value); + ASSERT_STR_EQ(value->label, "Method"); + ASSERT_STR_EQ(value->parent_class, "t.web.create_app.Settings"); + ASSERT_EQ(pyn_calls(r, "load", "t.web.create_app.Settings.value"), 1); + ASSERT_EQ(pyn_def_count(r, "t.web.Settings"), 0); + ASSERT_TRUE(pyn_defs_unique(r)); + cbm_free_result(r); + PASS(); +} + +/* A def nested in a method, named like the method (django's + * AbstractBaseUser.check_password holds a `def setter`; Tree.visit here holds + * a `def visit`). It used to share the method's QN, so its calls -- its + * recursion included -- were the method's. It is its own Function now: the + * recursive call and render() are its, only visit(node) is the method's, and + * the recursion seed flags the nested def. */ +TEST(extract_python_nested_def_named_like_its_method_issue1548) { + CBMFileResult *r = extract("class Tree:\n" + " def visit(self, node):\n" + " def visit(n):\n" + " if n:\n" + " visit(n.child)\n" + " render(n)\n" + "\n" + " visit(node)\n", + CBM_LANG_PYTHON, "t", "m.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + const CBMDefinition *method = pyn_def(r, "t.m.Tree.visit"); + ASSERT_NOT_NULL(method); + ASSERT_STR_EQ(method->label, "Method"); + const CBMDefinition *nested = pyn_def(r, "t.m.Tree.visit.visit"); + ASSERT_NOT_NULL(nested); + ASSERT_STR_EQ(nested->label, "Function"); + ASSERT_EQ(nested->start_line, 3); + ASSERT_EQ(pyn_calls(r, "visit", "t.m.Tree.visit.visit"), 1); + ASSERT_EQ(pyn_calls(r, "render", "t.m.Tree.visit.visit"), 1); + ASSERT_EQ(pyn_calls(r, "visit", "t.m.Tree.visit"), 1); + ASSERT_EQ(pyn_calls(r, "render", "t.m.Tree.visit"), 0); + ASSERT_TRUE(nested->is_recursive); + ASSERT_TRUE(pyn_defs_unique(r)); + cbm_free_result(r); + PASS(); +} + +/* Controls: a lambda and the comprehensions are expressions, not defs. They + * mint nothing, and the calls inside them stay the enclosing def's. */ +TEST(extract_python_lambda_and_comprehension_stay_in_enclosing_def_issue1548) { + CBMFileResult *r = extract("def work(items):\n" + " double = lambda x: scale(x)\n" + " squares = [scale(i) for i in items]\n" + " table = {k: scale(k) for k in items}\n" + " return double(squares), table\n", + CBM_LANG_PYTHON, "t", "c.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + ASSERT_NOT_NULL(pyn_def(r, "t.c.work")); + for (int i = 0; i < r->defs.count; i++) { + const char *q = r->defs.items[i].qualified_name; + ASSERT_FALSE(q && strncmp(q, "t.c.work.", strlen("t.c.work.")) == 0); + } + ASSERT_EQ(pyn_calls(r, "scale", "t.c.work"), 3); + ASSERT_TRUE(pyn_defs_unique(r)); + cbm_free_result(r); + PASS(); +} + +/* Records other than calls take their enclosing function from the same + * unified-walk scope frame as the calls. The nested def's env read carries its + * QN; the outer def's reads, before and after the nested def, keep the outer + * QN once its frame is popped. */ +TEST(extract_python_nested_def_env_access_attributed_to_it_issue1548) { + CBMFileResult *r = extract("import os\n" + "\n" + "def outer():\n" + " token = os.environ[\"OUTER_TOKEN\"]\n" + "\n" + " def inner():\n" + " return os.environ[\"INNER_TOKEN\"]\n" + "\n" + " return inner, token, os.environ[\"OUTER_AGAIN\"]\n", + CBM_LANG_PYTHON, "t", "e.py"); + ASSERT_NOT_NULL(r); + ASSERT_FALSE(r->has_error); + int seen = 0; + for (int i = 0; i < r->env_accesses.count; i++) { + const CBMEnvAccess *ea = &r->env_accesses.items[i]; + ASSERT_NOT_NULL(ea->enclosing_func_qn); + if (strcmp(ea->env_key, "INNER_TOKEN") == 0) { + ASSERT_STR_EQ(ea->enclosing_func_qn, "t.e.outer.inner"); + seen++; + } else { + ASSERT_STR_EQ(ea->enclosing_func_qn, "t.e.outer"); + seen++; + } + } + ASSERT_EQ(seen, 3); + cbm_free_result(r); + PASS(); +} + TEST(js_index_module_qn_not_collide_with_folder) { /* Same bug for JS/TS index.ts files */ CBMFileResult *r = extract("export function App() { return null; }\n", CBM_LANG_TYPESCRIPT, @@ -12551,6 +12843,13 @@ SUITE(extraction) { /* __init__.py / index.ts Module QN collision regression */ RUN_TEST(python_init_module_qn_not_collide_with_folder); RUN_TEST(python_init_nested_module_qn); + RUN_TEST(extract_python_nested_def_qualified_by_enclosing_def_issue1548); + RUN_TEST(extract_python_nested_def_calls_attributed_to_it_issue1548); + RUN_TEST(extract_python_decorated_nested_route_handler_issue1548); + RUN_TEST(extract_python_nested_class_issue1548); + RUN_TEST(extract_python_nested_def_named_like_its_method_issue1548); + RUN_TEST(extract_python_lambda_and_comprehension_stay_in_enclosing_def_issue1548); + RUN_TEST(extract_python_nested_def_env_access_attributed_to_it_issue1548); RUN_TEST(js_index_module_qn_not_collide_with_folder); RUN_TEST(python_regular_module_qn_unchanged); RUN_TEST(extract_java_method_annotations_issue382); diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index b52513ace..36ef5667e 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14188,6 +14188,378 @@ static void teardown_lang_repo(void) { g_lang_tmpdir[0] = '\0'; } +/* ── #1548: Python defs nested in a function body ─────────────────── */ + +/* The QN of `rel` in `project` ("web.create_app.health"). Route QNs + * ("__route__GET__/health") carry no project and are taken as they are. */ +static void pyn_qn(char *buf, size_t cap, const char *project, const char *rel) { + if (strncmp(rel, "__route__", strlen("__route__")) == 0) { + snprintf(buf, cap, "%s", rel); + } else { + snprintf(buf, cap, "%s.%s", project, rel); + } +} + +/* True iff an edge of `type` runs from node `src_rel` to node `tgt_rel`. */ +static bool pyn_edge(cbm_store_t *s, const char *project, const char *src_rel, const char *tgt_rel, + const char *type) { + char sq[512]; + char tq[512]; + pyn_qn(sq, sizeof(sq), project, src_rel); + pyn_qn(tq, sizeof(tq), project, tgt_rel); + cbm_node_t src = {0}; + cbm_node_t tgt = {0}; + bool found = false; + if (cbm_store_find_node_by_qn(s, project, sq, &src) == CBM_STORE_OK && + cbm_store_find_node_by_qn(s, project, tq, &tgt) == CBM_STORE_OK) { + cbm_edge_t *edges = NULL; + int ec = 0; + cbm_store_find_edges_by_source_type(s, src.id, type, &edges, &ec); + for (int i = 0; i < ec && !found; i++) { + found = edges[i].target_id == tgt.id; + } + if (edges) { + cbm_store_free_edges(edges, ec); + } + } + cbm_node_free_fields(&src); + cbm_node_free_fields(&tgt); + return found; +} + +/* The fixtures of the #1548 pipeline tests. */ +static const char pyn_web_py[] = "from flask import Flask\n" + "\n" + "def load_user(uid):\n" + " return uid\n" + "\n" + "def render(name):\n" + " return name\n" + "\n" + "def create_app():\n" + " app = Flask(__name__)\n" + "\n" + " @app.route(\"/users/\")\n" + " def show_user(uid):\n" + " return render(load_user(uid))\n" + "\n" + " @app.get(\"/health\")\n" + " def health():\n" + " return render(\"ok\")\n" + "\n" + " def ping():\n" + " return render(\"pong\")\n" + "\n" + " app.add_url_rule(\"/ping\", \"ping\", ping)\n" + " return app\n"; +/* A nested def named like its method, and a local helper whose name exists + * nowhere else -- b.py calls that name without binding it. */ +static const char pyn_a_py[] = "class Tree:\n" + " def visit(self, node):\n" + " def visit(n):\n" + " if n:\n" + " visit(n.child)\n" + "\n" + " visit(node)\n" + "\n" + "\n" + "def outer():\n" + " def helper_only_here():\n" + " return 1\n" + "\n" + " return helper_only_here()\n"; +static const char pyn_b_py[] = "def elsewhere():\n" + " return helper_only_here()\n"; +/* Two defs, each with its own local class of the same name. */ +static const char pyn_c_py[] = "def make_forms():\n" + " class LocalForm:\n" + " def clean(self):\n" + " return 1\n" + "\n" + " form = LocalForm()\n" + " return form.clean()\n" + "\n" + "\n" + "def make_other_forms():\n" + " class LocalForm:\n" + " def clean(self):\n" + " return 2\n" + "\n" + " form = LocalForm()\n" + " return form.clean()\n"; +/* A local def named like a stdlib function another module imports. */ +static const char pyn_x_py[] = "def outer():\n" + " def getpass(prompt):\n" + " return prompt\n" + "\n" + " return getpass(\"?\")\n"; +static const char pyn_y_py[] = "from getpass import getpass\n" + "\n" + "\n" + "def ask():\n" + " return getpass(\"password: \")\n"; + +/* The issue's headline: an app factory's route handlers are defs inside + * create_app. They were never extracted, so the HTTP layer vanished -- no + * HANDLES, and load_user()'s only caller was pinned on the file node. */ +TEST(pipeline_python_app_factory_routes_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_factory_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "web.py", pyn_web_py); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/factory.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE( + pyn_edge(s, project, "web.create_app.show_user", "__route__ANY__/users/{}", "HANDLES")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.health", "__route__GET__/health", "HANDLES")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.show_user", "web.load_user", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.health", "web.render", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "web.py.__file__", "web.load_user", "CALLS")); + /* A call-registered handler: add_url_rule names create_app's local ping. */ + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.ping", "__route__ANY__/ping", "HANDLES")); + + cbm_store_close(s); + cbm_pipeline_free(p); + th_rmtree(tmp); + PASS(); +} + +/* A def nested in a function is a local of it (Python LEGB). A bare call + * resolves to the local bound in the nearest enclosing function: visit(node) + * in Tree.visit reaches the nested visit, whose own recursion stays a + * self-call instead of landing on the method. And no caller outside the + * function can reach the local by name -- b.py's unbound helper_only_here() + * must not be matched to it by a short-name strategy. */ +TEST(pipeline_python_nested_def_is_function_local_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_local_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "a.py", pyn_a_py); + write_temp_file(tmp, "b.py", pyn_b_py); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/local.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE(pyn_edge(s, project, "a.Tree.visit", "a.Tree.visit.visit", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "a.Tree.visit.visit", "a.Tree.visit", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "a.outer", "a.outer.helper_only_here", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "b.elsewhere", "a.outer.helper_only_here", "CALLS")); + + cbm_store_close(s); + cbm_pipeline_free(p); + th_rmtree(tmp); + PASS(); +} + +/* Two defs each declare `class LocalForm`. Each constructor call reaches its + * own def's class, and a method call on the instance never reaches the other + * def's class: the Python LSP must not bind a local class module-wide by its + * short name (every same-named local class collapsed onto one). */ +TEST(pipeline_python_local_classes_stay_in_their_def_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_localcls_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "c.py", pyn_c_py); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/localcls.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE(pyn_edge(s, project, "c.make_forms", "c.make_forms.LocalForm", "CALLS")); + ASSERT_TRUE( + pyn_edge(s, project, "c.make_other_forms", "c.make_other_forms.LocalForm", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "c.make_forms", "c.make_other_forms.LocalForm", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "c.make_other_forms", "c.make_forms.LocalForm", "CALLS")); + ASSERT_FALSE( + pyn_edge(s, project, "c.make_forms", "c.make_other_forms.LocalForm.clean", "CALLS")); + ASSERT_FALSE( + pyn_edge(s, project, "c.make_other_forms", "c.make_forms.LocalForm.clean", "CALLS")); + + cbm_store_close(s); + cbm_pipeline_free(p); + th_rmtree(tmp); + PASS(); +} + +/* No import can name a function's local. The symbol-name import fallback + * bound `from getpass import getpass` (the stdlib module, not in the project) + * to x.outer's local `def getpass`, and ask()'s call followed the import. */ +TEST(pipeline_python_import_never_binds_a_local_def_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_localimp_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "x.py", pyn_x_py); + write_temp_file(tmp, "y.py", pyn_y_py); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/localimp.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE(pyn_edge(s, project, "x.outer", "x.outer.getpass", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "y.py.__file__", "x.outer.getpass", "IMPORTS")); + ASSERT_FALSE(pyn_edge(s, project, "y.ask", "x.outer.getpass", "CALLS")); + + cbm_store_close(s); + cbm_pipeline_free(p); + th_rmtree(tmp); + PASS(); +} + +/* The same four fixtures through the parallel path: >= 50 files and + * CBM_WORKERS=4 force pass_parallel.c and the cross-file Python LSP, which + * register, resolve and bind through their own code. */ +TEST(pipeline_python_nested_defs_parallel_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_nested_par_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "web.py", pyn_web_py); + write_temp_file(tmp, "a.py", pyn_a_py); + write_temp_file(tmp, "b.py", pyn_b_py); + write_temp_file(tmp, "c.py", pyn_c_py); + write_temp_file(tmp, "x.py", pyn_x_py); + write_temp_file(tmp, "y.py", pyn_y_py); + for (int i = 0; i < 52; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "filler%d.py", i); + snprintf(body, sizeof(body), "def filler%d():\n return %d\n", i, i); + write_temp_file(tmp, name, body); + } + + char *old_workers = getenv("CBM_WORKERS"); + char *saved = old_workers ? strdup(old_workers) : NULL; + cbm_setenv("CBM_WORKERS", "4", 1); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/nested_par.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE( + pyn_edge(s, project, "web.create_app.show_user", "__route__ANY__/users/{}", "HANDLES")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.health", "__route__GET__/health", "HANDLES")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.show_user", "web.load_user", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "web.create_app.ping", "__route__ANY__/ping", "HANDLES")); + ASSERT_TRUE(pyn_edge(s, project, "a.Tree.visit", "a.Tree.visit.visit", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "a.Tree.visit.visit", "a.Tree.visit", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "a.outer", "a.outer.helper_only_here", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "b.elsewhere", "a.outer.helper_only_here", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "c.make_forms", "c.make_forms.LocalForm", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "c.make_forms", "c.make_other_forms.LocalForm", "CALLS")); + ASSERT_FALSE( + pyn_edge(s, project, "c.make_forms", "c.make_other_forms.LocalForm.clean", "CALLS")); + ASSERT_FALSE( + pyn_edge(s, project, "c.make_other_forms", "c.make_forms.LocalForm.clean", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "x.outer", "x.outer.getpass", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "y.py.__file__", "x.outer.getpass", "IMPORTS")); + ASSERT_FALSE(pyn_edge(s, project, "y.ask", "x.outer.getpass", "CALLS")); + + cbm_store_close(s); + cbm_pipeline_free(p); + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + th_rmtree(tmp); + PASS(); +} + +/* An incremental re-index seeds its registry from the stored graph, not from + * extraction. It must keep a function's locals out of the short-name index + * exactly as the full index does: re-resolving the touched b.py must not bind + * its unbound helper_only_here() to a.outer's local. */ +TEST(pipeline_python_nested_def_local_after_incremental_issue1548) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_local_incr_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "a.py", pyn_a_py); + write_temp_file(tmp, "b.py", pyn_b_py); + /* An extensionless script is Python by its shebang: its locals are locals + * in the incremental seed too, not only in the full index. */ + write_temp_file(tmp, "tool", + "#!/usr/bin/env python3\n" + "def run():\n" + " def helper_in_tool():\n" + " return 1\n" + "\n" + " return helper_in_tool()\n"); + write_temp_file(tmp, "d.py", + "def elsewhere_too():\n" + " return helper_in_tool()\n"); + + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/local_incr.db", tmp); + cbm_pipeline_t *p1 = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p1); + ASSERT_EQ(cbm_pipeline_run(p1), 0); + cbm_pipeline_free(p1); + + char b_path[512]; + snprintf(b_path, sizeof(b_path), "%s/b.py", tmp); + ASSERT_EQ(th_append_file(b_path, "\n# touched: re-resolve b.py incrementally\n"), 0); + snprintf(b_path, sizeof(b_path), "%s/d.py", tmp); + ASSERT_EQ(th_append_file(b_path, "\n# touched: re-resolve d.py incrementally\n"), 0); + + cbm_pipeline_t *p2 = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p2); + ASSERT_EQ(cbm_pipeline_run(p2), 0); + const char *project = cbm_pipeline_project_name(p2); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + ASSERT_TRUE(pyn_edge(s, project, "a.outer", "a.outer.helper_only_here", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "b.elsewhere", "a.outer.helper_only_here", "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, "tool.run", "tool.run.helper_in_tool", "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, "d.elsewhere_too", "tool.run.helper_in_tool", "CALLS")); + + cbm_store_close(s); + cbm_pipeline_free(p2); + th_rmtree(tmp); + PASS(); +} + TEST(pipeline_python_project) { /* Port of TestPipelinePythonProject */ const char *files[] = {"main.py", "utils.py"}; @@ -23681,6 +24053,12 @@ SUITE(pipeline) { RUN_TEST(pipeline_c_variants_on_sequential_and_parallel_paths_c1); RUN_TEST(pipeline_cpp_overloads_are_listed_as_variants_c1); RUN_TEST(pipeline_semantic_version_3_index_is_rebuilt_in_full_c1); + RUN_TEST(pipeline_python_app_factory_routes_issue1548); + RUN_TEST(pipeline_python_nested_def_is_function_local_issue1548); + RUN_TEST(pipeline_python_local_classes_stay_in_their_def_issue1548); + RUN_TEST(pipeline_python_import_never_binds_a_local_def_issue1548); + RUN_TEST(pipeline_python_nested_defs_parallel_issue1548); + RUN_TEST(pipeline_python_nested_def_local_after_incremental_issue1548); RUN_TEST(pipeline_header_include_target_is_independent_of_registration_order); RUN_TEST(pipeline_imports_multi_symbol_edges); RUN_TEST(pipeline_go_cross_package_call); diff --git a/tests/test_registry.c b/tests/test_registry.c index 45fd6db14..055de4d78 100644 --- a/tests/test_registry.c +++ b/tests/test_registry.c @@ -1962,6 +1962,96 @@ TEST(registry_python_scope_qualified_terminal_bucket_uncapped) { bool seeded = cbm_registry_size(r) == 302; cbm_registry_free(r); ASSERT_TRUE(shape && seeded && resolved); +/* #1548: a Python def or class declared in a function body is a local of it. + * It is registered for exact lookups but kept out of the short-name index, so + * no weak strategy offers it to a caller elsewhere. A class nested in a class + * (proj.m.Outer.Inner) is an attribute, not a local, and stays nameable; so + * does the same QN shape from another language. */ +TEST(registry_python_function_local_is_exact_only_issue1548) { + cbm_registry_t *r = cbm_registry_new(); + cbm_registry_add_lang(r, "create_app", "proj.web.create_app", "Function", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "index", "proj.web.create_app.index", "Function", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "Settings", "proj.web.create_app.Settings", "Class", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "value", "proj.web.create_app.Settings.value", "Method", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "Outer", "proj.web.Outer", "Class", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "Inner", "proj.web.Outer.Inner", "Class", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "f", "proj.ts.f", "Function", CBM_LANG_TYPESCRIPT); + cbm_registry_add_lang(r, "g", "proj.ts.f.g", "Function", CBM_LANG_TYPESCRIPT); + + static const char *const local[] = {"proj.web.create_app.index", "proj.web.create_app.Settings", + "proj.web.create_app.Settings.value"}; + for (int i = 0; i < 3; i++) { + ASSERT_TRUE(cbm_registry_exists(r, local[i])); + ASSERT_TRUE(cbm_registry_qn_is_function_local(r, local[i])); + const char *simple = strrchr(local[i], '.') + 1; + const char **out = NULL; + int count = -1; + cbm_registry_find_by_name(r, simple, &out, &count); + ASSERT_EQ(count, 0); + } + static const char *const nameable[][2] = {{"create_app", "proj.web.create_app"}, + {"Inner", "proj.web.Outer.Inner"}, + {"g", "proj.ts.f.g"}}; + for (int i = 0; i < 3; i++) { + const char **out = NULL; + int count = 0; + cbm_registry_find_by_name(r, nameable[i][0], &out, &count); + ASSERT_EQ(count, 1); + ASSERT_STR_EQ(out[0], nameable[i][1]); + } + ASSERT_FALSE(cbm_registry_qn_is_function_local(r, "proj.web.Outer.Inner")); + /* A weak strategy cannot reach the local; before, index() anywhere in + * the project resolved to it by unique_name. */ + cbm_resolution_t res = cbm_registry_resolve(r, "index", "proj.other", NULL, NULL, 0); + ASSERT_NULL(res.qualified_name); + cbm_registry_free(r); + PASS(); +} + +/* Python LEGB for bare names: the caller's own def first, then each enclosing + * def outward; a class body is not a scope its methods see; the module is left + * to the other strategies. A dotted callee is not a bare name. */ +TEST(registry_resolve_lexical_walks_enclosing_defs_issue1548) { + cbm_registry_t *r = cbm_registry_new(); + cbm_registry_add_lang(r, "Tree", "proj.m.Tree", "Class", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "visit", "proj.m.Tree.visit", "Method", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "visit", "proj.m.Tree.visit.visit", "Function", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "helper", "proj.m.Tree.helper", "Method", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "factory", "proj.m.factory", "Function", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "helper", "proj.m.factory.helper", "Function", + CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "Local", "proj.m.factory.Local", "Class", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "run", "proj.m.factory.Local.run", "Method", CBM_LANG_PYTHON); + cbm_registry_add_lang(r, "top", "proj.m.top", "Function", CBM_LANG_PYTHON); + + /* In method Tree.visit, `visit` is its nested def; so it is inside that + * nested def (its own recursion). */ + cbm_resolution_t res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit"); + ASSERT_STR_EQ(res.qualified_name, "proj.m.Tree.visit.visit"); + ASSERT_STR_EQ(res.strategy, "lexical_scope"); + res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit.visit"); + ASSERT_STR_EQ(res.qualified_name, "proj.m.Tree.visit.visit"); + /* The class body is skipped: Tree.helper is not a bare name in a method. */ + res = cbm_registry_resolve_lexical(r, "helper", "proj.m.Tree.visit.visit"); + ASSERT_NULL(res.qualified_name); + /* A method of a local class sees the def enclosing its class. */ + res = cbm_registry_resolve_lexical(r, "helper", "proj.m.factory.Local.run"); + ASSERT_STR_EQ(res.qualified_name, "proj.m.factory.helper"); + res = cbm_registry_resolve_lexical(r, "Local", "proj.m.factory"); + ASSERT_STR_EQ(res.qualified_name, "proj.m.factory.Local"); + /* Module-level names and dotted callees are not this strategy's. */ + res = cbm_registry_resolve_lexical(r, "top", "proj.m.factory"); + ASSERT_NULL(res.qualified_name); + res = cbm_registry_resolve_lexical(r, "self.helper", "proj.m.factory"); + ASSERT_NULL(res.qualified_name); + res = cbm_registry_resolve_lexical(r, "helper", "proj.m"); + ASSERT_NULL(res.qualified_name); + cbm_registry_free(r); PASS(); } @@ -2069,4 +2159,6 @@ SUITE(registry) { RUN_TEST(python_import_binding_contradicts_only_foreign_chains); RUN_TEST(python_external_base_contradicts_django_shapes); RUN_TEST(weak_call_guards_share_one_drop_list); + RUN_TEST(registry_python_function_local_is_exact_only_issue1548); + RUN_TEST(registry_resolve_lexical_walks_enclosing_defs_issue1548); } From 9cdae25c12f6190f362b38aadc9a68305afac969 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sat, 17 Oct 2026 22:52:43 +0200 Subject: [PATCH 02/15] fix(python): bind a function's nested defs in the LSP; lexical_scope after it Re-port onto current main (agreed with lane L7): - The Python LSP binds every def/class a function body defines as a local before walking the body (LEGB): an undecorated def as an exact callable alias to its nested QN, anything else as a hard shadow, so no call falls through to a same-named module symbol. Defs inside the body's compound statements count too, and their bodies are now walked. - Classes declared in a function are walked under the function's QN (py_process_class_in), so super()/self calls in their methods resolve. - cbm_registry_resolve_lexical moves AFTER the LSP override: both resolvers call one entry point, cbm_registry_resolve_call_scoped, at the same place (before the language-aware chain). It never uses the per-file resolve cache. - Main's cbm_registry_add_lang carries the def-local exclusion, so every registration site (full, parallel, incremental seed) applies it; the separate cbm_registry_add_for_language and the second incremental language derivation are dropped. - The pass_pkgmap.c def-local import skip is its own continue. Tests: local-vs-module shadowing in both orders, a decorated local, and a function-local class's super()/self calls, sequential and parallel. Signed-off-by: Martin Vogel --- internal/cbm/lsp/py_lsp.c | 16 +++- tests/test_pipeline.c | 155 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+), 2 deletions(-) diff --git a/internal/cbm/lsp/py_lsp.c b/internal/cbm/lsp/py_lsp.c index cbad720fa..cfc5499d4 100644 --- a/internal/cbm/lsp/py_lsp.c +++ b/internal/cbm/lsp/py_lsp.c @@ -247,6 +247,8 @@ static void py_register_dict_literal(PyLSPContext *ctx, const char *name, TSNode static TSNode py_lookup_lambda(PyLSPContext *ctx, const char *name); static const char *py_lookup_dict_dispatch(PyLSPContext *ctx, const char *var, const char *key); static void py_process_function(PyLSPContext *ctx, TSNode func_node, const char *container_qn); +static void py_process_class(PyLSPContext *ctx, TSNode class_node); +static void py_process_class_in(PyLSPContext *ctx, TSNode class_node, const char *container_qn); typedef enum { PY_LOCAL_DEFS_BIND, PY_LOCAL_DEFS_PROCESS } PyLocalDefsMode; static void py_visit_local_defs(PyLSPContext *ctx, TSNode block, PyLocalDefsMode mode, int depth); @@ -4443,6 +4445,8 @@ static void py_visit_local_defs(PyLSPContext *ctx, TSNode block, PyLocalDefsMode } else if (strcmp(ck, "class_definition") == 0) { if (mode == PY_LOCAL_DEFS_BIND) py_bind_local_def(ctx, def, true); + else + py_process_class_in(ctx, def, ctx->enclosing_func_qn); } else if (py_is_statement_block(ck)) { py_visit_local_defs(ctx, c, mode, depth + 1); } @@ -4614,7 +4618,10 @@ static void py_exec_class_definition(PyLSPContext *ctx, TSNode class_node) { py_scope_restore(ctx, saved_scope); } -static void py_process_class(PyLSPContext *ctx, TSNode class_node) { +/* Walk a class's methods with the class qualified under `container_qn`: the + * module for a module-level class, the enclosing def for a class declared in + * a function body (#1548: extraction gives it the def's QN as prefix). */ +static void py_process_class_in(PyLSPContext *ctx, TSNode class_node, const char *container_qn) { if (!ctx || ctx->eval_failure) return; TSNode name_node = ts_node_child_by_field_name(class_node, "name", 4); @@ -4625,7 +4632,8 @@ static void py_process_class(PyLSPContext *ctx, TSNode class_node) { return; const char *prev_class = ctx->enclosing_class_qn; - ctx->enclosing_class_qn = cbm_arena_sprintf(ctx->arena, "%s.%s", ctx->module_qn, cname); + ctx->enclosing_class_qn = cbm_arena_sprintf( + ctx->arena, "%s.%s", container_qn ? container_qn : ctx->module_qn, cname); TSNode body = ts_node_child_by_field_name(class_node, "body", 4); if (!ts_node_is_null(body)) { @@ -4699,6 +4707,10 @@ static void py_process_class(PyLSPContext *ctx, TSNode class_node) { ctx->enclosing_class_qn = prev_class; } +static void py_process_class(PyLSPContext *ctx, TSNode class_node) { + py_process_class_in(ctx, class_node, NULL); +} + /* Module bindings are replayed in source order before deferred function bodies * are analyzed. Class definitions always replace an earlier callable identity; * the registry decides whether the resulting class type is known precisely. */ diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index 36ef5667e..3141ab32d 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14560,6 +14560,159 @@ TEST(pipeline_python_nested_def_local_after_incremental_issue1548) { PASS(); } +/* As pyn_edge, and the edge's properties also contain `fragment`. */ +static bool pyn_edge_with(cbm_store_t *s, const char *project, const char *src_rel, + const char *tgt_rel, const char *type, const char *fragment) { + char sq[512]; + char tq[512]; + pyn_qn(sq, sizeof(sq), project, src_rel); + pyn_qn(tq, sizeof(tq), project, tgt_rel); + cbm_node_t src = {0}; + cbm_node_t tgt = {0}; + bool found = false; + if (cbm_store_find_node_by_qn(s, project, sq, &src) == CBM_STORE_OK && + cbm_store_find_node_by_qn(s, project, tq, &tgt) == CBM_STORE_OK) { + cbm_edge_t *edges = NULL; + int ec = 0; + cbm_store_find_edges_by_source_type(s, src.id, type, &edges, &ec); + for (int i = 0; i < ec && !found; i++) { + found = edges[i].target_id == tgt.id && edges[i].properties_json && + strstr(edges[i].properties_json, fragment); + } + if (edges) { + cbm_store_free_edges(edges, ec); + } + } + cbm_node_free_fields(&src); + cbm_node_free_fields(&tgt); + return found; +} + +/* A local def shadows a module def of the same name (Python LEGB), in both + * orders of the two callers within one file: the registry's per-file resolve + * cache is keyed by the callee name only, so a caller-dependent answer must + * never come from it. The undecorated local is bound by the Python LSP itself + * (an lsp_* edge); a decorated one is no exact callable value, so the LSP + * shadows it and the registry's lexical_scope step binds it. */ +static const char pyn_shadow_local_first_py[] = "def helper():\n" + " return 0\n" + "\n" + "def uses_local():\n" + " def helper():\n" + " return 1\n" + " return helper()\n" + "\n" + "def uses_module():\n" + " return helper()\n" + "\n" + "def uses_decorated():\n" + " @staticmethod\n" + " def helper():\n" + " return 2\n" + " return helper()\n"; +static const char pyn_shadow_module_first_py[] = "def helper():\n" + " return 0\n" + "\n" + "def uses_module():\n" + " return helper()\n" + "\n" + "def uses_local():\n" + " if True:\n" + " def helper():\n" + " return 1\n" + " return helper()\n"; + +/* A class declared in a def is walked by the LSP under the def's QN: super() + * and self calls in its methods resolve through the class's own bases. */ +static const char pyn_local_class_methods_py[] = "class Base:\n" + " def greet(self):\n" + " return 1\n" + "\n" + "def factory():\n" + " class Local(Base):\n" + " def greet(self):\n" + " return super().greet()\n" + "\n" + " def other(self):\n" + " return self.greet()\n" + " return Local\n"; + +static int pyn_assert_shadowing(cbm_store_t *s, const char *project) { + ASSERT_TRUE(pyn_edge_with(s, project, "sc.factory.Local.greet", "sc.Base.greet", "CALLS", + "\"strategy\":\"lsp_super\"")); + ASSERT_TRUE(pyn_edge_with(s, project, "sc.factory.Local.other", "sc.factory.Local.greet", + "CALLS", "\"strategy\":\"lsp_")); + const char *files[] = {"sa", "sb"}; + for (int f = 0; f < 2; f++) { + char local_caller[64], local_def[64], module_caller[64], module_def[64]; + snprintf(local_caller, sizeof(local_caller), "%s.uses_local", files[f]); + snprintf(local_def, sizeof(local_def), "%s.uses_local.helper", files[f]); + snprintf(module_caller, sizeof(module_caller), "%s.uses_module", files[f]); + snprintf(module_def, sizeof(module_def), "%s.helper", files[f]); + ASSERT_TRUE(pyn_edge_with(s, project, local_caller, local_def, "CALLS", + "\"strategy\":\"lsp_")); + ASSERT_FALSE(pyn_edge(s, project, local_caller, module_def, "CALLS")); + ASSERT_TRUE(pyn_edge(s, project, module_caller, module_def, "CALLS")); + ASSERT_FALSE(pyn_edge(s, project, module_caller, local_def, "CALLS")); + } + ASSERT_TRUE(pyn_edge_with(s, project, "sa.uses_decorated", "sa.uses_decorated.helper", + "CALLS", "\"strategy\":\"lexical_scope\"")); + ASSERT_FALSE(pyn_edge(s, project, "sa.uses_decorated", "sa.helper", "CALLS")); + return 0; +} + +static int pyn_run_shadowing(int fillers) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_shadow_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "sa.py", pyn_shadow_local_first_py); + write_temp_file(tmp, "sb.py", pyn_shadow_module_first_py); + write_temp_file(tmp, "sc.py", pyn_local_class_methods_py); + for (int i = 0; i < fillers; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "filler%d.py", i); + snprintf(body, sizeof(body), "def filler%d():\n return %d\n", i, i); + write_temp_file(tmp, name, body); + } + char *old_workers = getenv("CBM_WORKERS"); + char *saved = old_workers ? strdup(old_workers) : NULL; + if (fillers > 0) { + cbm_setenv("CBM_WORKERS", "4", 1); + } + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/shadow.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + int rc = p ? cbm_pipeline_run(p) : -1; + cbm_store_t *s = rc == 0 ? cbm_store_open_path(db_path) : NULL; + int verdict = s ? pyn_assert_shadowing(s, cbm_pipeline_project_name(p)) : 1; + if (s) { + cbm_store_close(s); + } + if (p) { + cbm_pipeline_free(p); + } + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + th_rmtree(tmp); + ASSERT_EQ(rc, 0); + return verdict; +} + +TEST(pipeline_python_local_def_shadows_module_def_issue1548) { + return pyn_run_shadowing(0); +} + +TEST(pipeline_python_local_def_shadows_module_def_parallel_issue1548) { + return pyn_run_shadowing(52); +} + TEST(pipeline_python_project) { /* Port of TestPipelinePythonProject */ const char *files[] = {"main.py", "utils.py"}; @@ -24059,6 +24212,8 @@ SUITE(pipeline) { RUN_TEST(pipeline_python_import_never_binds_a_local_def_issue1548); RUN_TEST(pipeline_python_nested_defs_parallel_issue1548); RUN_TEST(pipeline_python_nested_def_local_after_incremental_issue1548); + RUN_TEST(pipeline_python_local_def_shadows_module_def_issue1548); + RUN_TEST(pipeline_python_local_def_shadows_module_def_parallel_issue1548); RUN_TEST(pipeline_header_include_target_is_independent_of_registration_order); RUN_TEST(pipeline_imports_multi_symbol_edges); RUN_TEST(pipeline_go_cross_package_call); From 2ac9c33e187f600ab47f29b718a2a544f2be700a Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sat, 17 Oct 2026 23:22:56 +0200 Subject: [PATCH 03/15] fix(rust-lsp): a Rust call never binds another language's definition Both Rust registries were built from the pipeline's mixed-language def set. The shared Tier-2 build (cbm_rust_build_cross_registry) and the per-file conversion (pxc_lspdefs_to_rust) copied every def, so a Perl `sub new` or any C function became a Rust free function. The lsp_short_name_unique fallback then bound `Store::new()` to the Perl constructor at confidence 0.90 on both venues: its "crate scope" check compares only the first module-QN segment, which is the project name. (Found by the generic-engine lane on a 12-language fixture.) Register Rust defs only, at both conversion points, exactly as the Go builder filters to Go. Rust impl-relation records already carry CBM_LANG_RUST. As a side effect, the shared Rust build no longer registers the C defs of a mixed C+Rust tree. Tests: pipeline_rust_call_never_binds_another_language (sequential and parallel): no CALLS from the Rust caller into the Perl file, and the Rust associated function still binds. Signed-off-by: Martin Vogel --- internal/cbm/lsp/rust_lsp.c | 42 ++++++++------ src/pipeline/pass_lsp_cross.c | 50 ++++++++++------- tests/test_pipeline.c | 101 ++++++++++++++++++++++++++++++++++ 3 files changed, 157 insertions(+), 36 deletions(-) diff --git a/internal/cbm/lsp/rust_lsp.c b/internal/cbm/lsp/rust_lsp.c index 44680cb69..959f4d1cf 100644 --- a/internal/cbm/lsp/rust_lsp.c +++ b/internal/cbm/lsp/rust_lsp.c @@ -6323,7 +6323,10 @@ static void rust_resolve_against_registry(CBMArena *arena, const char *source, i * read-only, and shared across every Rust file's resolve (mirrors C/py/cs/ts). * Converts the pipeline's CBMLSPDef into CBMRustLSPDef inline (same field copy as * pass_lsp_cross.c's pxc_lspdefs_to_rust, including proven trait provenance). - * module_qn=NULL is byte-identical because all_defs always carry def_module_qn. */ + * module_qn=NULL is byte-identical because all_defs always carry def_module_qn. + * all_defs is mixed-language: only Rust defs are registered, exactly like the + * per-file conversion, so a Perl `new` or a C function can never become the + * target of a Rust call (the Go builder filters the same way). */ CBMTypeRegistry *cbm_rust_build_cross_registry(CBMArena *arena, CBMLSPDef *defs, int def_count) { if (!arena) return NULL; @@ -6331,29 +6334,34 @@ CBMTypeRegistry *cbm_rust_build_cross_registry(CBMArena *arena, CBMLSPDef *defs, if (!reg) return NULL; CBMRustLSPDef *rdefs = NULL; + int n = 0; if (def_count > 0) { rdefs = (CBMRustLSPDef *)cbm_arena_alloc(arena, (size_t)def_count * sizeof(CBMRustLSPDef)); if (!rdefs) return NULL; for (int i = 0; i < def_count; i++) { - rdefs[i].qualified_name = defs[i].qualified_name; - rdefs[i].short_name = defs[i].short_name; - rdefs[i].label = defs[i].label; - rdefs[i].receiver_type = defs[i].receiver_type; - rdefs[i].def_module_qn = defs[i].def_module_qn; - rdefs[i].return_types = defs[i].return_types; - rdefs[i].embedded_types = defs[i].embedded_types; - rdefs[i].field_defs = defs[i].field_defs; - rdefs[i].method_names_str = defs[i].method_names_str; - rdefs[i].signature_param_types = defs[i].signature_param_types; - rdefs[i].signature_param_count = defs[i].signature_param_count; - rdefs[i].trait_qn = defs[i].trait_qn; - rdefs[i].is_interface = defs[i].is_interface; - rdefs[i].is_rust_impl_relation = defs[i].is_rust_impl_relation; - rdefs[i].is_abstract = defs[i].is_abstract; + if (defs[i].lang != CBM_LANG_RUST) { + continue; + } + rdefs[n].qualified_name = defs[i].qualified_name; + rdefs[n].short_name = defs[i].short_name; + rdefs[n].label = defs[i].label; + rdefs[n].receiver_type = defs[i].receiver_type; + rdefs[n].def_module_qn = defs[i].def_module_qn; + rdefs[n].return_types = defs[i].return_types; + rdefs[n].embedded_types = defs[i].embedded_types; + rdefs[n].field_defs = defs[i].field_defs; + rdefs[n].method_names_str = defs[i].method_names_str; + rdefs[n].signature_param_types = defs[i].signature_param_types; + rdefs[n].signature_param_count = defs[i].signature_param_count; + rdefs[n].trait_qn = defs[i].trait_qn; + rdefs[n].is_interface = defs[i].is_interface; + rdefs[n].is_rust_impl_relation = defs[i].is_rust_impl_relation; + rdefs[n].is_abstract = defs[i].is_abstract; + n++; } } - rust_populate_cross_registry(reg, arena, rdefs, def_count, /*module_qn=*/NULL); + rust_populate_cross_registry(reg, arena, n > 0 ? rdefs : NULL, n, /*module_qn=*/NULL); reg->read_only = true; /* seal: shared Tier-2 registry is read-only during resolve */ return reg; } diff --git a/src/pipeline/pass_lsp_cross.c b/src/pipeline/pass_lsp_cross.c index cf94926c0..940c82cb2 100644 --- a/src/pipeline/pass_lsp_cross.c +++ b/src/pipeline/pass_lsp_cross.c @@ -1470,31 +1470,42 @@ const struct CBMCargoManifest *cbm_pxc_get_rust_manifest(void) { /* Convert a CBMLSPDef array (the pipeline's lingua franca, go_lsp.h:73) * into a CBMRustLSPDef array (rust_lsp.h) inside `arena`. The two structs * have similar fields but different layouts; CBMLSPDef adds - * language/namespace metadata, so a memcpy is unsafe — copy field-by-field. */ -static CBMRustLSPDef *pxc_lspdefs_to_rust(CBMArena *arena, const CBMLSPDef *defs, int def_count) { + * language/namespace metadata, so a memcpy is unsafe — copy field-by-field. + * Only Rust defs are kept: the def set is mixed-language, and a Perl or C + * function must never become a Rust call target (the Go registrar filters + * the same way). *out_count gets the number kept. */ +static CBMRustLSPDef *pxc_lspdefs_to_rust(CBMArena *arena, const CBMLSPDef *defs, int def_count, + int *out_count) { + *out_count = 0; if (!defs || def_count <= 0) return NULL; CBMRustLSPDef *out = (CBMRustLSPDef *)cbm_arena_alloc(arena, (size_t)def_count * sizeof(CBMRustLSPDef)); if (!out) return NULL; + int n = 0; for (int i = 0; i < def_count; i++) { - out[i].qualified_name = defs[i].qualified_name; - out[i].short_name = defs[i].short_name; - out[i].label = defs[i].label; - out[i].receiver_type = defs[i].receiver_type; - out[i].def_module_qn = defs[i].def_module_qn; - out[i].return_types = defs[i].return_types; - out[i].embedded_types = defs[i].embedded_types; - out[i].field_defs = defs[i].field_defs; - out[i].method_names_str = defs[i].method_names_str; - out[i].signature_param_types = defs[i].signature_param_types; - out[i].signature_param_count = defs[i].signature_param_count; - out[i].trait_qn = defs[i].trait_qn; - out[i].is_interface = defs[i].is_interface; - out[i].is_rust_impl_relation = defs[i].is_rust_impl_relation; - out[i].is_abstract = defs[i].is_abstract; + if (defs[i].lang != CBM_LANG_RUST) { + continue; + } + out[n].qualified_name = defs[i].qualified_name; + out[n].short_name = defs[i].short_name; + out[n].label = defs[i].label; + out[n].receiver_type = defs[i].receiver_type; + out[n].def_module_qn = defs[i].def_module_qn; + out[n].return_types = defs[i].return_types; + out[n].embedded_types = defs[i].embedded_types; + out[n].field_defs = defs[i].field_defs; + out[n].method_names_str = defs[i].method_names_str; + out[n].signature_param_types = defs[i].signature_param_types; + out[n].signature_param_count = defs[i].signature_param_count; + out[n].trait_qn = defs[i].trait_qn; + out[n].is_interface = defs[i].is_interface; + out[n].is_rust_impl_relation = defs[i].is_rust_impl_relation; + out[n].is_abstract = defs[i].is_abstract; + n++; } + *out_count = n; return out; } @@ -1581,9 +1592,10 @@ void cbm_pxc_run_one_with_php_bindings(CBMLanguage lang, CBMFileResult *r, const * but diverge after, so convert into the scratch arena. The * workspace manifest (set once by the sequential driver) lets * `crate_a::foo` route across the crate boundary (#56). */ - CBMRustLSPDef *rdefs = pxc_lspdefs_to_rust(&scratch, defs, def_count); + int rust_def_count = 0; + CBMRustLSPDef *rdefs = pxc_lspdefs_to_rust(&scratch, defs, def_count, &rust_def_count); cbm_run_rust_lsp_cross_with_manifest(&scratch, source, source_len, module_qn, rdefs, - def_count, imp_names, imp_qns, imp_count, tree, + rust_def_count, imp_names, imp_qns, imp_count, tree, g_pxc_rust_manifest, &out, &synthetic_calls); break; } diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index b52513ace..10d85d3ee 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -8757,6 +8757,105 @@ TEST(pipeline_go_rw_usage_never_cross_into_c) { PASS(); } +/* The Rust registries were built from the mixed-language def set: a Perl + * `sub new` became a Rust free function, and the project-wide + * lsp_short_name_unique fallback bound `Store::new()` to it at 0.90 (found by + * the generic-engine lane on its 12-language fixture). A Rust call never + * targets another language's definition. pad_files > 0 forces the parallel + * resolver, whose Rust registry is the shared all-defs build. */ +static int rust_registry_rust_only_case(int pad_files) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_rust_only_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "Cargo.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n"); + write_temp_file(tmp, "src/store.rs", + "pub struct Store {\n" + " n: u32,\n" + "}\n" + "\n" + "impl Store {\n" + " pub fn new() -> Store {\n" + " Store { n: 0 }\n" + " }\n" + "\n" + " pub fn get(&self, k: u32) -> u32 {\n" + " self.n + k\n" + " }\n" + "}\n"); + write_temp_file(tmp, "src/main.rs", + "mod store;\n" + "\n" + "use crate::store::Store;\n" + "\n" + "fn run() -> u32 {\n" + " let s = Store::new();\n" + " s.get(1)\n" + "}\n" + "\n" + "fn main() {\n" + " run();\n" + "}\n"); + write_temp_file(tmp, "perlsrc/Lib.pm", + "package Lib;\n" + "\n" + "sub new {\n" + " my ($class) = @_;\n" + " return bless {}, $class;\n" + "}\n" + "\n" + "1;\n"); + for (int i = 0; i < pad_files; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "src/filler%d.rs", i); + snprintf(body, sizeof(body), "pub fn filler%d() -> u32 {\n %d\n}\n", i, i); + write_temp_file(tmp, name, body); + } + + char *saved = NULL; + if (pad_files > 0) { + char *old_workers = getenv("CBM_WORKERS"); + saved = old_workers ? strdup(old_workers) : NULL; + cbm_setenv("CBM_WORKERS", "4", 1); + } + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/rust_only.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + ASSERT_NOT_NULL(p); + ASSERT_EQ(cbm_pipeline_run(p), 0); + const char *project = cbm_pipeline_project_name(p); + cbm_store_t *s = cbm_store_open_path(db_path); + ASSERT_NOT_NULL(s); + + /* Never onto the Perl constructor ... */ + ASSERT_EQ(named_edge_to_file_count(s, project, "CALLS", "run", "new", "perlsrc/Lib.pm"), 0); + /* ... and the Rust call still binds the Rust associated function. */ + ASSERT_GTE(named_edge_to_file_count(s, project, "CALLS", "run", "new", "src/store.rs"), 1); + + cbm_store_close(s); + cbm_pipeline_free(p); + if (pad_files > 0) { + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + } + th_rmtree(tmp); + PASS(); +} + +TEST(pipeline_rust_call_never_binds_another_language) { + return rust_registry_rust_only_case(0); +} + +TEST(pipeline_rust_call_never_binds_another_language_parallel) { + return rust_registry_rust_only_case(52); +} + TEST(pipeline_go_rw_usage_never_cross_into_c_parallel) { /* Parallel twin of the test above: USAGE and WRITES/READS each have a * second, independent resolver in pass_parallel.c (resolve_file_usages / @@ -23615,6 +23714,8 @@ SUITE(pipeline) { RUN_TEST(pipeline_html_embedded_member_call_stays_unbound); RUN_TEST(pipeline_go_rw_usage_never_cross_into_c); RUN_TEST(pipeline_go_rw_usage_never_cross_into_c_parallel); + RUN_TEST(pipeline_rust_call_never_binds_another_language); + RUN_TEST(pipeline_rust_call_never_binds_another_language_parallel); RUN_TEST(pipeline_rust_std_receiver_never_binds_project_method); RUN_TEST(pipeline_rust_std_receiver_never_binds_project_method_parallel); RUN_TEST(pipeline_go_bare_ref_never_binds_field); From bfe53f344b8a41c73aba8506c7e38ff9f89f3e92 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sat, 17 Oct 2026 23:34:24 +0200 Subject: [PATCH 04/15] style: clang-format the new lines Formatting only, with the repo's clang-format (Homebrew LLVM). Signed-off-by: Martin Vogel --- src/pipeline/pass_lsp_cross.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/pipeline/pass_lsp_cross.c b/src/pipeline/pass_lsp_cross.c index 940c82cb2..376b25b6b 100644 --- a/src/pipeline/pass_lsp_cross.c +++ b/src/pipeline/pass_lsp_cross.c @@ -1475,7 +1475,7 @@ const struct CBMCargoManifest *cbm_pxc_get_rust_manifest(void) { * function must never become a Rust call target (the Go registrar filters * the same way). *out_count gets the number kept. */ static CBMRustLSPDef *pxc_lspdefs_to_rust(CBMArena *arena, const CBMLSPDef *defs, int def_count, - int *out_count) { + int *out_count) { *out_count = 0; if (!defs || def_count <= 0) return NULL; From ec4f0eaae638540134ef1f08aabcf8aa5b352b92 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 02:59:00 +0200 Subject: [PATCH 05/15] fix(rust-lsp): a type-qualified call never takes the free-function fallback `Opts::write(true)` is an associated function. When the Rust LSP cannot place it, the registry can (import_map or qualified_suffix onto fsx.Opts.write). The two free-function short-name fallbacks (the cross-crate member lookup and the project-wide lsp_short_name_unique) ran first, though. They bound the call to the project's only free `write`, and that LSP row pre-empted the right answer. While the Rust registry also held other languages' definitions, a same-named JS or Perl function often made the free name ambiguous and hid this. With Rust-only registries the hijack replaced 516 qualified_suffix and 84 import_map edges on 7 Rust repos. A call whose qualifying segment names a type now skips both fallbacks. That covers an ASCII-uppercase initial, `Self`, and a qualified `::f` path. Signed-off-by: Martin Vogel --- internal/cbm/lsp/rust_lsp.c | 33 ++++++++- tests/test_pipeline.c | 135 ++++++++++++++++++++++++++++-------- 2 files changed, 137 insertions(+), 31 deletions(-) diff --git a/internal/cbm/lsp/rust_lsp.c b/internal/cbm/lsp/rust_lsp.c index 959f4d1cf..27c1feab6 100644 --- a/internal/cbm/lsp/rust_lsp.c +++ b/internal/cbm/lsp/rust_lsp.c @@ -4128,6 +4128,32 @@ static bool rust_qn_has_suffix(const char *qualified_name, const char *suffix) { (qn_len == suffix_len || qualified_name[qn_len - suffix_len - 1U] == '.'); } +/* True when the segment that qualifies a call path's tail names a TYPE: + * `Store::new`, `Self::helper`, `T::default`, `::f` (turbofish is + * already stripped, so a `>` before the last `::` closes a qualified path). + * Such a call is an associated function, never a free function, so the + * free-function short-name fallbacks must not bind it: with a project-wide + * unique free `new`/`write`, `Store::new()` landed on that function and the + * LSP row pre-empted the registry's correct `Store.new`. Module segments are + * snake_case, so an ASCII-uppercase initial is a type. `tail` must point just + * past the path's last `::`, or equal `path` when there is none. */ +static bool rust_path_qualifier_is_type(const char *path, const char *tail) { + if (!path || !tail || tail - path < 3) { + return false; + } + const char *seg_end = tail - 2; /* the last "::" */ + if (seg_end[-1] == '>') { + return true; + } + const char *seg = path; + for (const char *p = path; p + 1 < seg_end; p++) { + if (p[0] == ':' && p[1] == ':') { + seg = p + 2; + } + } + return seg < seg_end && seg[0] >= 'A' && seg[0] <= 'Z'; +} + /* Resolve only an exact identifier/scoped-path value. No branch, closure, or * other composite expression enters this helper. A suffix fallback is admitted * only for a qualified import path and only when it identifies one free func. */ @@ -4514,6 +4540,9 @@ static void rust_resolve_call_expression_inner(RustLSPContext *ctx, TSNode node) } else { tail = path; } + /* `Type::f` is an associated function: the two free-function + * fallbacks below never bind it (see rust_path_qualifier_is_type). */ + const bool assoc_call = rust_path_qualifier_is_type(path, tail); /* Cross-crate workspace-member resolution (#56): when the call * path's head is a declared Cargo workspace member (e.g. @@ -4524,7 +4553,7 @@ static void rust_resolve_call_expression_inner(RustLSPContext *ctx, TSNode node) * MEMBER crate by matching the registered QN's `..` * path segment plus the call tail. Requires a parsed manifest * (threaded through pass_lsp_cross.c); NULL manifest skips this. */ - if (ctx->cargo_manifest && tail && *tail) { + if (!assoc_call && ctx->cargo_manifest && tail && *tail) { const char *head_sep = strstr(path, "::"); if (head_sep && head_sep > path) { char *head = cbm_arena_strndup(ctx->arena, path, (size_t)(head_sep - path)); @@ -4568,7 +4597,7 @@ static void rust_resolve_call_expression_inner(RustLSPContext *ctx, TSNode node) * foo; use foo::bar; bar()` a chance to resolve when the * intermediate module wasn't tracked through an explicit * use-map entry. */ - if (tail && *tail && ctx->module_qn) { + if (!assoc_call && tail && *tail && ctx->module_qn) { /* Crate prefix is the first dotted segment of module_qn after * the project name, but for simplicity we just match on * "starts with first dot-segment". */ diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index d3b1239e4..ccf556bf5 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -8874,6 +8874,40 @@ TEST(pipeline_go_rw_usage_never_cross_into_c) { * the generic-engine lane on its 12-language fixture). A Rust call never * targets another language's definition. pad_files > 0 forces the parallel * resolver, whose Rust registry is the shared all-defs build. */ +/* Pads a Rust fixture with pad_files filler modules (pad_files > 0 forces the + * parallel resolver: CBM_WORKERS=4 for the run), runs the full pipeline into + * / and opens the store. NULL store = the run failed. */ +static cbm_store_t *rust_fixture_run(const char *tmp, int pad_files, const char *db_name, + cbm_pipeline_t **out_p) { + for (int i = 0; i < pad_files; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "src/filler%d.rs", i); + snprintf(body, sizeof(body), "pub fn filler%d() -> u32 {\n %d\n}\n", i, i); + write_temp_file(tmp, name, body); + } + char *saved = NULL; + if (pad_files > 0) { + char *old_workers = getenv("CBM_WORKERS"); + saved = old_workers ? strdup(old_workers) : NULL; + cbm_setenv("CBM_WORKERS", "4", 1); + } + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/%s", tmp, db_name); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + int rc = p ? cbm_pipeline_run(p) : -1; + if (pad_files > 0) { + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + } + *out_p = p; + return rc == 0 ? cbm_store_open_path(db_path) : NULL; +} + static int rust_registry_rust_only_case(int pad_files) { char tmp[256]; snprintf(tmp, sizeof(tmp), "/tmp/cbm_rust_only_XXXXXX"); @@ -8917,28 +8951,10 @@ static int rust_registry_rust_only_case(int pad_files) { "}\n" "\n" "1;\n"); - for (int i = 0; i < pad_files; i++) { - char name[64]; - char body[128]; - snprintf(name, sizeof(name), "src/filler%d.rs", i); - snprintf(body, sizeof(body), "pub fn filler%d() -> u32 {\n %d\n}\n", i, i); - write_temp_file(tmp, name, body); - } - - char *saved = NULL; - if (pad_files > 0) { - char *old_workers = getenv("CBM_WORKERS"); - saved = old_workers ? strdup(old_workers) : NULL; - cbm_setenv("CBM_WORKERS", "4", 1); - } - char db_path[512]; - snprintf(db_path, sizeof(db_path), "%s/rust_only.db", tmp); - cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); - ASSERT_NOT_NULL(p); - ASSERT_EQ(cbm_pipeline_run(p), 0); - const char *project = cbm_pipeline_project_name(p); - cbm_store_t *s = cbm_store_open_path(db_path); + cbm_pipeline_t *p = NULL; + cbm_store_t *s = rust_fixture_run(tmp, pad_files, "rust_only.db", &p); ASSERT_NOT_NULL(s); + const char *project = cbm_pipeline_project_name(p); /* Never onto the Perl constructor ... */ ASSERT_EQ(named_edge_to_file_count(s, project, "CALLS", "run", "new", "perlsrc/Lib.pm"), 0); @@ -8947,14 +8963,6 @@ static int rust_registry_rust_only_case(int pad_files) { cbm_store_close(s); cbm_pipeline_free(p); - if (pad_files > 0) { - if (saved) { - cbm_setenv("CBM_WORKERS", saved, 1); - free(saved); - } else { - cbm_unsetenv("CBM_WORKERS"); - } - } th_rmtree(tmp); PASS(); } @@ -8967,6 +8975,73 @@ TEST(pipeline_rust_call_never_binds_another_language_parallel) { return rust_registry_rust_only_case(52); } +/* `Opts::write(true)` is an associated function. The LSP leaves it to the + * registry (import_map -> fsx.Opts.write), but its free-function short-name + * fallback bound it to the project's only free `write` (cache.write) first, and + * that LSP row pre-empted the right answer. With the cross-language namesakes + * gone (above), this hijack replaced 600+ correct qualified_suffix/import_map + * edges on the Rust corpus. A type-qualified call never takes that fallback. */ +static int rust_assoc_call_free_namesake_case(int pad_files) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_rust_assoc_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "Cargo.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n"); + write_temp_file(tmp, "src/fsx.rs", + "pub struct Opts {\n" + " pub w: bool,\n" + "}\n" + "\n" + "impl Opts {\n" + " pub fn write(w: bool) -> Opts {\n" + " Opts { w }\n" + " }\n" + "}\n"); + write_temp_file(tmp, "src/cache.rs", + "pub fn write(p: &str) -> usize {\n" + " p.len()\n" + "}\n"); + write_temp_file(tmp, "src/main.rs", + "mod cache;\n" + "mod fsx;\n" + "\n" + "use crate::fsx::Opts;\n" + "\n" + "fn run() -> bool {\n" + " let o = Opts::write(true);\n" + " o.w\n" + "}\n" + "\n" + "fn main() {\n" + " run();\n" + " cache::write(\"x\");\n" + "}\n"); + cbm_pipeline_t *p = NULL; + cbm_store_t *s = rust_fixture_run(tmp, pad_files, "rust_assoc.db", &p); + ASSERT_NOT_NULL(s); + const char *project = cbm_pipeline_project_name(p); + + /* The associated function, never the free namesake ... */ + ASSERT_GTE(named_edge_to_file_count(s, project, "CALLS", "run", "write", "src/fsx.rs"), 1); + ASSERT_EQ(named_edge_to_file_count(s, project, "CALLS", "run", "write", "src/cache.rs"), 0); + /* ... while the module-qualified free call keeps its target. */ + ASSERT_GTE(named_edge_to_file_count(s, project, "CALLS", "main", "write", "src/cache.rs"), 1); + + cbm_store_close(s); + cbm_pipeline_free(p); + th_rmtree(tmp); + PASS(); +} + +TEST(pipeline_rust_assoc_call_never_binds_free_namesake) { + return rust_assoc_call_free_namesake_case(0); +} + +TEST(pipeline_rust_assoc_call_never_binds_free_namesake_parallel) { + return rust_assoc_call_free_namesake_case(52); +} + TEST(pipeline_go_rw_usage_never_cross_into_c_parallel) { /* Parallel twin of the test above: USAGE and WRITES/READS each have a * second, independent resolver in pass_parallel.c (resolve_file_usages / @@ -23829,6 +23904,8 @@ SUITE(pipeline) { RUN_TEST(pipeline_go_rw_usage_never_cross_into_c_parallel); RUN_TEST(pipeline_rust_call_never_binds_another_language); RUN_TEST(pipeline_rust_call_never_binds_another_language_parallel); + RUN_TEST(pipeline_rust_assoc_call_never_binds_free_namesake); + RUN_TEST(pipeline_rust_assoc_call_never_binds_free_namesake_parallel); RUN_TEST(pipeline_rust_std_receiver_never_binds_project_method); RUN_TEST(pipeline_rust_std_receiver_never_binds_project_method_parallel); RUN_TEST(pipeline_go_bare_ref_never_binds_field); From 6b1cd8aa0de7718c6ab10a0b2d84b15782f31f2c Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 03:02:50 +0200 Subject: [PATCH 06/15] style(python): clang-format the nested-def binding and lexical step Whitespace only; the token stream is unchanged. Signed-off-by: Martin Vogel --- internal/cbm/lsp/py_lsp.c | 23 ++++++++++++++++------- src/pipeline/pass_calls.c | 11 +++++------ 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/internal/cbm/lsp/py_lsp.c b/internal/cbm/lsp/py_lsp.c index cfc5499d4..18384d82e 100644 --- a/internal/cbm/lsp/py_lsp.c +++ b/internal/cbm/lsp/py_lsp.c @@ -4408,11 +4408,20 @@ static void py_bind_local_def(PyLSPContext *ctx, TSNode def, bool is_class) { * them is still a local of the function; a nested def, class or lambda body * is a scope of its own and is never entered. */ static bool py_is_statement_block(const char *kind) { - static const char *const kinds[] = { - "block", "if_statement", "elif_clause", "else_clause", - "for_statement", "while_statement", "try_statement", "except_clause", - "except_group_clause", "finally_clause", "with_statement", "match_statement", - "case_clause", NULL}; + static const char *const kinds[] = {"block", + "if_statement", + "elif_clause", + "else_clause", + "for_statement", + "while_statement", + "try_statement", + "except_clause", + "except_group_clause", + "finally_clause", + "with_statement", + "match_statement", + "case_clause", + NULL}; for (int i = 0; kinds[i]; i++) { if (strcmp(kind, kinds[i]) == 0) return true; @@ -4632,8 +4641,8 @@ static void py_process_class_in(PyLSPContext *ctx, TSNode class_node, const char return; const char *prev_class = ctx->enclosing_class_qn; - ctx->enclosing_class_qn = cbm_arena_sprintf( - ctx->arena, "%s.%s", container_qn ? container_qn : ctx->module_qn, cname); + ctx->enclosing_class_qn = + cbm_arena_sprintf(ctx->arena, "%s.%s", container_qn ? container_qn : ctx->module_qn, cname); TSNode body = ts_node_child_by_field_name(class_node, "body", 4); if (!ts_node_is_null(body)) { diff --git a/src/pipeline/pass_calls.c b/src/pipeline/pass_calls.c index 77fa14f4a..80b2e69b8 100644 --- a/src/pipeline/pass_calls.c +++ b/src/pipeline/pass_calls.c @@ -596,9 +596,8 @@ static void handle_route_registration(cbm_pipeline_ctx_t *ctx, const CBMCall *ca call->enclosing_func_qn); } if (!hres.qualified_name) { - hres = cbm_registry_resolve_handler(ctx->registry, call->second_arg_name, - module_qn, imp_keys, imp_vals, imp_count, - ctx->gbuf, lang); + hres = cbm_registry_resolve_handler(ctx->registry, call->second_arg_name, module_qn, + imp_keys, imp_vals, imp_count, ctx->gbuf, lang); } if (hres.qualified_name != NULL && hres.qualified_name[0] != '\0') { const cbm_gbuf_node_t *handler = cbm_gbuf_find_by_qn(ctx->gbuf, hres.qualified_name); @@ -978,9 +977,9 @@ static int resolve_single_call(cbm_pipeline_ctx_t *ctx, CBMCall *call, const CBM cbm_resolution_t res = {0}; if (!rust_external && !php_vendor_call) { /* Python LEGB fallback first (MUST match pass_parallel.c). */ - res = cbm_registry_resolve_call_scoped(ctx->registry, call->callee_name, module_qn, - imp_keys, imp_vals, imp_count, lang, - call->enclosing_func_qn); + res = + cbm_registry_resolve_call_scoped(ctx->registry, call->callee_name, module_qn, imp_keys, + imp_vals, imp_count, lang, call->enclosing_func_qn); /* Cross-language veto: a name-only guess never binds another * language's symbol (a JS app.get must not become Python's * builtins.dict.get). The vetoed answer becomes EMPTY here, before From 9b5139d73da34fc882beabeae403c3eee94c56b6 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 03:41:31 +0200 Subject: [PATCH 07/15] fix(python): bound the function-local walks at the symbol's own file A package __init__.py can define a function whose QN equals a submodule's (poetry: layouts/__init__.py `def layout` is the QN prefix of every symbol in layouts/layout.py). The prefix walks that recognise a def declared inside a function body accepted that other file's function as the enclosing def, so class Layout and its methods read as locals of layout(): they left the short-name index and the import symbol fallback, and the answer depended on which file registered first (discovery order on a full index, node-id order on an incremental one). Every walk now stops at the symbol's own module: - registration (pass_definitions.c, pass_parallel.c, and the incremental seed) passes the defining file's symbol prefix length, cbm_pipeline_fqn_symbol_prefix_len, computed without allocating, to cbm_registry_add_lang_in_module; cbm_registry_add_lang keeps its signature and does no local detection; - cbm_registry_resolve_lexical stops at the caller's module QN; - the import fallback's local test only counts an enclosing def in the same file. Also moves the #1548 registry tests out of the test they had been inserted into. Tests: registry_python_local_bound_at_own_file_issue1548 (both registration orders), fqn_symbol_prefix_len_matches_fqn_compute, pipeline_python_init_function_shares_submodule_qn{,_parallel}_issue1548 (src/ layout: the import and calls onto Layout survive, then again after an incremental re-index). Signed-off-by: Martin Vogel --- src/pipeline/fqn.c | 24 +++++ src/pipeline/pass_calls.c | 2 +- src/pipeline/pass_definitions.c | 13 ++- src/pipeline/pass_parallel.c | 18 +++- src/pipeline/pass_pkgmap.c | 10 +- src/pipeline/pipeline.h | 41 +++++--- src/pipeline/pipeline_incremental.c | 17 +++- src/pipeline/registry.c | 75 ++++++++++++--- tests/test_pipeline.c | 99 +++++++++++++++++++ tests/test_registry.c | 144 +++++++++++++++++++++------- 10 files changed, 366 insertions(+), 77 deletions(-) diff --git a/src/pipeline/fqn.c b/src/pipeline/fqn.c index f391ce1e9..f4603dc80 100644 --- a/src/pipeline/fqn.c +++ b/src/pipeline/fqn.c @@ -355,6 +355,30 @@ char *cbm_pipeline_resolve_relative_import(const char *source_rel, const char *m return resolve_js_relative(buf, sizeof(buf), module_path); } +size_t cbm_pipeline_fqn_symbol_prefix_len(const char *project, const char *rel_path) { + if (!project) { + return 0; + } + char path[FQN_PATH_BUF]; + int n = snprintf(path, sizeof(path), "%s", rel_path ? rel_path : ""); + if (n < 0 || (size_t)n >= sizeof(path)) { + return 0; + } + cbm_normalize_path_sep(path); + strip_file_extension(path); + const char *segments[CBM_SZ_256]; + int seg_count = 0; + segments[seg_count++] = project; + seg_count += tokenize_path(path, segments + seg_count, FQN_MAX_PATH_SEGS); + /* Any symbol name: the prefix cbm_pipeline_fqn_compute puts before it. */ + strip_init_or_index(segments, &seg_count, "symbol"); + size_t total = 0; + for (int i = 0; i < seg_count; i++) { + total += strlen(segments[i]) + (i > 0 ? FQN_SEP_LEN : 0); + } + return total; +} + char *cbm_pipeline_fqn_folder(const char *project, const char *rel_dir) { if (!project) { return strdup(""); diff --git a/src/pipeline/pass_calls.c b/src/pipeline/pass_calls.c index 80b2e69b8..dfbb8f7fb 100644 --- a/src/pipeline/pass_calls.c +++ b/src/pipeline/pass_calls.c @@ -593,7 +593,7 @@ static void handle_route_registration(cbm_pipeline_ctx_t *ctx, const CBMCall *ca cbm_resolution_t hres = {0}; if (lang == CBM_LANG_PYTHON) { hres = cbm_registry_resolve_lexical(ctx->registry, call->second_arg_name, - call->enclosing_func_qn); + call->enclosing_func_qn, module_qn); } if (!hres.qualified_name) { hres = cbm_registry_resolve_handler(ctx->registry, call->second_arg_name, module_qn, diff --git a/src/pipeline/pass_definitions.c b/src/pipeline/pass_definitions.c index aecd91d99..80bf0a8a9 100644 --- a/src/pipeline/pass_definitions.c +++ b/src/pipeline/pass_definitions.c @@ -416,7 +416,7 @@ static void pd_add_file_doc(const cbm_gbuf_node_t *file_node, const char *doc) { * `file_doc`, the file's own doc (passed with its first def only), goes on the * File node. */ static void process_def(cbm_pipeline_ctx_t *ctx, const CBMDefinition *def, const char *rel, - const char *file_doc, CBMLanguage lang) { + const char *file_doc, CBMLanguage lang, size_t module_prefix_len) { if (!def->qualified_name || !def->name) { return; } @@ -442,7 +442,8 @@ static void process_def(cbm_pipeline_ctx_t *ctx, const CBMDefinition *def, const * (SQL FROM/JOIN lineage). pass_parallel.c and pipeline_incremental.c seed * through the same predicate, so the three registries cannot diverge. */ if (node_id > 0 && cbm_label_is_registry_symbol(def->label)) { - cbm_registry_add_lang(ctx->registry, def->name, def->qualified_name, def->label, lang); + cbm_registry_add_lang_in_module(ctx->registry, def->name, def->qualified_name, def->label, + lang, module_prefix_len); } char *file_qn = cbm_pipeline_fqn_compute(ctx->project_name, rel, "__file__"); const cbm_gbuf_node_t *file_node = cbm_gbuf_find_by_qn(ctx->gbuf, file_qn); @@ -1019,10 +1020,14 @@ int cbm_pipeline_pass_definitions(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t "test_declarations"); } - /* Create nodes for each definition */ + /* Create nodes for each definition. A Python file's module QN bounds + * the registry's function-local detection to this file. */ const char *file_doc = result->module_doc; /* goes with the first def */ + size_t py_prefix_len = lang == CBM_LANG_PYTHON + ? cbm_pipeline_fqn_symbol_prefix_len(ctx->project_name, rel) + : 0; for (int d = 0; d < result->defs.count; d++) { - process_def(ctx, &result->defs.items[d], rel, file_doc, lang); + process_def(ctx, &result->defs.items[d], rel, file_doc, lang, py_prefix_len); file_doc = NULL; total_defs++; } diff --git a/src/pipeline/pass_parallel.c b/src/pipeline/pass_parallel.c index 501320dd6..f908de590 100644 --- a/src/pipeline/pass_parallel.c +++ b/src/pipeline/pass_parallel.c @@ -1773,7 +1773,8 @@ int cbm_parallel_extract(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t *files, * once per file by the caller: computing the file QN and finding its node for * every definition was 700 k allocations and lookups on the Go corpus. */ static int register_and_link_def(cbm_pipeline_ctx_t *ctx, const CBMDefinition *def, - int64_t file_node_id, CBMLanguage lang, int *reg_entries) { + int64_t file_node_id, CBMLanguage lang, size_t module_prefix_len, + int *reg_entries) { int edges = 0; if (!def->name || !def->qualified_name || !def->label) { return 0; @@ -1781,7 +1782,8 @@ static int register_and_link_def(cbm_pipeline_ctx_t *ctx, const CBMDefinition *d /* Registry membership is defined ONCE by cbm_label_is_registry_symbol * (helpers.c) — see pass_definitions.c for the per-label rationale. */ if (cbm_label_is_registry_symbol(def->label)) { - cbm_registry_add_lang(ctx->registry, def->name, def->qualified_name, def->label, lang); + cbm_registry_add_lang_in_module(ctx->registry, def->name, def->qualified_name, def->label, + lang, module_prefix_len); (*reg_entries)++; } char marked[CBM_SZ_4K]; @@ -1988,9 +1990,14 @@ int cbm_build_registry_from_cache(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t int64_t file_node_id = file_node ? file_node->id : 0; free(file_qn); pp_add_file_doc(file_node, result->module_doc); + /* Bounds the function-local detection to this file (pass_definitions.c). */ + size_t py_prefix_len = files[i].language == CBM_LANG_PYTHON + ? cbm_pipeline_fqn_symbol_prefix_len(ctx->project_name, rel) + : 0; for (int d = 0; d < result->defs.count; d++) { - defines_edges += register_and_link_def(ctx, &result->defs.items[d], file_node_id, - files[i].language, ®_entries); + defines_edges += + register_and_link_def(ctx, &result->defs.items[d], file_node_id, + files[i].language, py_prefix_len, ®_entries); } } @@ -2415,7 +2422,8 @@ static void emit_route_registration(cbm_gbuf_t *gbuf, const cbm_gbuf_node_t *sou * lexical lookup sees a function's locals. MUST match pass_calls.c. */ cbm_resolution_t hres = {0}; if (lang == CBM_LANG_PYTHON) { - hres = cbm_registry_resolve_lexical(registry, handler_ref, call->enclosing_func_qn); + hres = cbm_registry_resolve_lexical(registry, handler_ref, call->enclosing_func_qn, + module_qn); } if (!hres.qualified_name) { hres = cbm_registry_resolve_handler(registry, handler_ref, module_qn, ik, iv, ic, diff --git a/src/pipeline/pass_pkgmap.c b/src/pipeline/pass_pkgmap.c index e30a0c231..303b293b9 100644 --- a/src/pipeline/pass_pkgmap.c +++ b/src/pipeline/pass_pkgmap.c @@ -1824,8 +1824,11 @@ bool cbm_import_symbol_fallback_allowed(CBMLanguage lang) { * no import can name it. Its QN extends a Function/Method node's * (proj.tests.mock_inputs.inner.wrapper.mock_getpass.getpass), which is how it * is recognised; the symbol-name fallback bound `import getpass` to exactly - * that def. Mirrors cbm_registry_add_lang, which keeps the same - * symbols out of the call resolver's short-name index. */ + * that def. Only an enclosing def of the SAME file counts: a package + * __init__.py function can share a submodule's QN (poetry: layouts/__init__.py + * `def layout` vs layouts/layout.py `class Layout`). Mirrors + * cbm_registry_add_lang_in_module, which keeps the same symbols out of the call + * resolver's short-name index. */ static bool import_candidate_is_python_local(const cbm_gbuf_t *gbuf, const cbm_gbuf_node_t *cand) { const char *qn = cand->qualified_name; const char *slash = cand->file_path ? strrchr(cand->file_path, '/') : NULL; @@ -1839,7 +1842,8 @@ static bool import_candidate_is_python_local(const cbm_gbuf_t *gbuf, const cbm_g for (char *dot = strrchr(scope, '.'); dot; dot = strrchr(scope, '.')) { *dot = '\0'; const cbm_gbuf_node_t *outer = cbm_gbuf_find_by_qn(gbuf, scope); - if (outer && outer->label && + if (outer && outer->label && outer->file_path && + strcmp(outer->file_path, cand->file_path) == 0 && (strcmp(outer->label, "Function") == 0 || strcmp(outer->label, "Method") == 0)) { return true; } diff --git a/src/pipeline/pipeline.h b/src/pipeline/pipeline.h index e03c0dbe6..072674d8d 100644 --- a/src/pipeline/pipeline.h +++ b/src/pipeline/pipeline.h @@ -200,6 +200,12 @@ char *cbm_pipeline_fqn_compute(const char *project, const char *rel_path, const /* Module QN: project.dir.parts (no name). Caller must free(). */ char *cbm_pipeline_fqn_module(const char *project, const char *rel_path); +/* Length of the prefix cbm_pipeline_fqn_compute(project, rel_path, name) puts + * before a symbol `name` (the joining dot excluded): the QN prefix every + * definition of that file carries, __init__ / index already dropped. Computed + * without allocating; 0 when unavailable. */ +size_t cbm_pipeline_fqn_symbol_prefix_len(const char *project, const char *rel_path); + /* Language-aware module QN. When `module_is_dir` is true (Java/Go package * semantics) the module is derived from the CONTAINING DIRECTORY (the filename * stem is dropped), so it agrees with the extraction-side def QNs; when false @@ -253,14 +259,24 @@ void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified /* As cbm_registry_add, recording `lang`: the DETECTED language of the file * that defines the symbol (CBM_LANG_COUNT = unknown), which the cross-language - * veto (cbm_registry_name_guess_vetoed) checks against the caller. A Python - * symbol declared inside a function body (see cbm_registry_qn_is_function_local) - * is registered for exact lookups only, never offered to the short-name - * strategies; every registration site seeds through this, so full and - * incremental indexes agree. */ + * veto (cbm_registry_name_guess_vetoed) checks against the caller. Detects no + * function-local symbols: that needs the defining file's module QN, see + * cbm_registry_add_lang_in_module. */ void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, const char *label, CBMLanguage lang); +/* As cbm_registry_add_lang. A Python symbol declared inside a function body of + * its own file (cbm_registry_qn_is_function_local, bounded by + * `module_prefix_len`: cbm_pipeline_fqn_symbol_prefix_len of the defining file) + * is registered for exact lookups only, never offered to the short-name + * strategies. The three graph + * registration sites (pass_definitions.c, pass_parallel.c, + * pipeline_incremental.c) all register through this, so full and incremental + * indexes agree. 0 = no local detection. */ +void cbm_registry_add_lang_in_module(cbm_registry_t *r, const char *name, + const char *qualified_name, const char *label, + CBMLanguage lang, size_t module_prefix_len); + /* True when code in `caller` can call a symbol defined in `target` by bare * name: same language, same interop family (JS/TS + script hosts, C/C++/CUDA/ * ObjC, JVM, .NET, ...; the table lives in registry.c), or Swift with the C @@ -272,16 +288,19 @@ bool cbm_lang_resolution_compatible(CBMLanguage caller, CBMLanguage target); * CBM_LANG_COUNT for anything else. */ CBMLanguage cbm_registry_synthetic_path_language(const char *file_path); -/* True when a proper prefix of `qualified_name` is a registered Function or - * Method, i.e. the symbol was declared inside a function body. */ -bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name); +/* True when a proper prefix of `qualified_name` longer than + * `module_prefix_len` (cbm_pipeline_fqn_symbol_prefix_len of the defining file) + * is a registered Function or Method, i.e. the symbol was declared inside a + * function body of its own file. False when module_prefix_len is 0. */ +bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name, + size_t module_prefix_len); /* Python bare-name lexical lookup: the def or class named `callee_name` bound * in the nearest function scope enclosing `scope_qn` (the caller's QN), - * skipping class bodies, stopping at the module. Strategy "lexical_scope"; - * qualified_name is NULL when no such local exists. */ + * skipping class bodies, stopping at the caller's module (`module_qn`). + * Strategy "lexical_scope"; qualified_name is NULL when no such local exists. */ cbm_resolution_t cbm_registry_resolve_lexical(const cbm_registry_t *r, const char *callee_name, - const char *scope_qn); + const char *scope_qn, const char *module_qn); /* Resolve a callee name using prioritized strategies. * import_map: NULL-terminated array of {local_name, resolved_qn} pairs, or NULL. diff --git a/src/pipeline/pipeline_incremental.c b/src/pipeline/pipeline_incremental.c index 4d0d667f5..a860b5e32 100644 --- a/src/pipeline/pipeline_incremental.c +++ b/src/pipeline/pipeline_incremental.c @@ -1340,6 +1340,7 @@ static bool incr_label_is_registry_symbol(const char *label) { typedef struct { cbm_registry_t *registry; CBMHashTable *lang_by_path; /* rel_path -> (CBMLanguage + 1); keys borrowed */ + const char *project; /* bounds Python function-local detection per file */ } registry_seed_t; static void registry_visitor(const cbm_gbuf_node_t *node, void *userdata) { @@ -1354,14 +1355,20 @@ static void registry_visitor(const cbm_gbuf_node_t *node, void *userdata) { lang = (CBMLanguage)((uintptr_t)v - SKIP_ONE); } } - cbm_registry_add_lang(seed->registry, node->name, node->qualified_name, node->label, lang); + /* Same per-file bound as the full index (pass_definitions.c). */ + size_t prefix_len = lang == CBM_LANG_PYTHON + ? cbm_pipeline_fqn_symbol_prefix_len(seed->project, node->file_path) + : 0; + cbm_registry_add_lang_in_module(seed->registry, node->name, node->qualified_name, node->label, + lang, prefix_len); } /* Seed the registry from every registry symbol in the graph buffer, with * languages taken from the discovered files (see registry_visitor). */ static void registry_seed_from_gbuf(cbm_registry_t *registry, const cbm_gbuf_t *gbuf, - const cbm_file_info_t *files, int file_count) { - registry_seed_t seed = {.registry = registry, .lang_by_path = NULL}; + const char *project, const cbm_file_info_t *files, + int file_count) { + registry_seed_t seed = {.registry = registry, .lang_by_path = NULL, .project = project}; if (files && file_count > 0) { seed.lang_by_path = cbm_ht_create((size_t)file_count * PAIR_LEN); } @@ -3305,7 +3312,7 @@ static int run_closure_delta(cbm_pipeline_t *p, const char *db_path, const char if (!registry) { goto out; } - registry_seed_from_gbuf(registry, gbuf, files, file_count); + registry_seed_from_gbuf(registry, gbuf, project, files, file_count); cbm_log_info("delta.preseed_done", "registry", itoa_buf(cbm_registry_size(registry)), "elapsed_ms", itoa_buf((int)elapsed_ms(t))); @@ -4155,7 +4162,7 @@ int cbm_pipeline_run_incremental(cbm_pipeline_t *p, const char *db_path, cbm_fil /* Step 3-5: Registry + extract + resolve */ cbm_registry_t *registry = cbm_registry_new(); cbm_clock_gettime(CLOCK_MONOTONIC, &t); - registry_seed_from_gbuf(registry, existing, files, file_count); + registry_seed_from_gbuf(registry, existing, project, files, file_count); cbm_log_info("incremental.registry_seed", "symbols", itoa_buf(cbm_registry_size(registry)), "elapsed_ms", itoa_buf((int)elapsed_ms(t))); diff --git a/src/pipeline/registry.c b/src/pipeline/registry.c index 8a1122ddf..5dda868a4 100644 --- a/src/pipeline/registry.c +++ b/src/pipeline/registry.c @@ -1187,14 +1187,45 @@ static bool registry_label_is_callable(const char *label) { return label && (strcmp(label, "Function") == 0 || strcmp(label, "Method") == 0); } -bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name) { +/* Length of the QN prefix the defs of the caller's file carry: its module QN + * (cbm_pipeline_fqn_module), minus the trailing `__init__` / `index` segment + * that symbol QNs drop (pkg/__init__.py defines proj.pkg.f). 0 when there is + * no module QN or `qualified_name` does not extend that prefix. */ +static size_t python_def_prefix_len(const char *module_qn, const char *qualified_name) { + static const char *const dropped[] = {".__init__", ".index"}; + size_t n = module_qn ? strlen(module_qn) : 0; + for (size_t i = 0; i < sizeof(dropped) / sizeof(dropped[0]); i++) { + size_t k = strlen(dropped[i]); + if (n > k && strcmp(module_qn + n - k, dropped[i]) == 0) { + n -= k; + break; + } + } + if (n == 0 || !qualified_name || strncmp(qualified_name, module_qn, n) != 0 || + qualified_name[n] != '.') { + return 0; + } + return n; +} + +/* Whether `qualified_name` sits inside a def of its OWN file: only prefixes + * strictly longer than `module_prefix_len` (cbm_pipeline_fqn_symbol_prefix_len + * of the defining file) count. A package's __init__.py can define a function + * whose QN equals a submodule's (poetry: layouts/__init__.py `def layout` vs + * layouts/layout.py), so an unbounded walk read the submodule's classes as + * that function's locals -- and the answer depended on which file registered + * first. */ +bool cbm_registry_qn_is_function_local(const cbm_registry_t *r, const char *qualified_name, + size_t module_prefix_len) { char scope[CBM_SZ_512]; size_t len = qualified_name ? strlen(qualified_name) : 0; - if (!r || len == 0 || len >= sizeof(scope)) { + size_t floor = module_prefix_len; + if (!r || floor == 0 || len <= floor || len >= sizeof(scope) || qualified_name[floor] != '.') { return false; } memcpy(scope, qualified_name, len + SKIP_ONE); - for (char *dot = strrchr(scope, '.'); dot; dot = strrchr(scope, '.')) { + for (char *dot = strrchr(scope, '.'); dot && (size_t)(dot - scope) > floor; + dot = strrchr(scope, '.')) { *dot = '\0'; if (registry_label_is_callable(cbm_ht_get(r->exact, scope))) { return true; @@ -1238,7 +1269,12 @@ static const char *registry_add_exact(cbm_registry_t *r, const char *qualified_n void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified_name, const char *label) { - cbm_registry_add_lang(r, name, qualified_name, label, CBM_LANG_COUNT); + cbm_registry_add_lang_in_module(r, name, qualified_name, label, CBM_LANG_COUNT, 0); +} + +void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, + const char *label, CBMLanguage lang) { + cbm_registry_add_lang_in_module(r, name, qualified_name, label, lang, 0); } /* Python nests defs: a def or class declared in a function body -- a closure, @@ -1250,14 +1286,19 @@ void cbm_registry_add(cbm_registry_t *r, const char *name, const char *qualified * decorator's `def wrapper` to a caller in another function or file. The exact * map still holds it, for LSP targets and cbm_registry_resolve_lexical. One * file's defs register parent first, so the enclosing function is already in - * the exact map when its locals arrive. Gated to Python: probing every - * prefix of every symbol would cost a kernel index seconds for nothing. */ -void cbm_registry_add_lang(cbm_registry_t *r, const char *name, const char *qualified_name, - const char *label, CBMLanguage lang) { + * the exact map when its locals arrive; the walk never leaves the symbol's own + * file (`module_prefix_len`, cbm_pipeline_fqn_symbol_prefix_len of the defining + * file; 0 = no local detection). + * Gated to Python: probing every prefix of every symbol would cost a kernel + * index seconds for nothing. */ +void cbm_registry_add_lang_in_module(cbm_registry_t *r, const char *name, + const char *qualified_name, const char *label, + CBMLanguage lang, size_t module_prefix_len) { if (!r || !qualified_name || !label) { return; } - if (lang == CBM_LANG_PYTHON && cbm_registry_qn_is_function_local(r, qualified_name)) { + if (lang == CBM_LANG_PYTHON && + cbm_registry_qn_is_function_local(r, qualified_name, module_prefix_len)) { (void)registry_add_exact(r, qualified_name, label); return; } @@ -2176,19 +2217,22 @@ cbm_resolution_t cbm_registry_resolve_lineage(const cbm_registry_t *r, const cha * enclosing def -- create_app.index, Tree.visit.visit -- before anything a * module-wide strategy would find. Walks scope_qn outward while its prefixes * are registered defs or classes (skipping the classes) and stops at the - * module, which the other strategies own. Empty unless such a local exists. - * Uncached: the answer depends on the caller, not only on the name. */ + * caller's module (`module_qn`), which the other strategies own -- a + * same-QN function of a package __init__.py is never an enclosing scope. + * Empty unless such a local exists. Uncached: the answer depends on the + * caller, not only on the name. */ cbm_resolution_t cbm_registry_resolve_lexical(const cbm_registry_t *r, const char *callee_name, - const char *scope_qn) { + const char *scope_qn, const char *module_qn) { char scope[CBM_SZ_512]; char candidate[CBM_SZ_512]; size_t len = scope_qn ? strlen(scope_qn) : 0; + size_t floor = python_def_prefix_len(module_qn, scope_qn); if (!r || !callee_name || !callee_name[0] || strchr(callee_name, '.') || - strstr(callee_name, "::") || len == 0 || len >= sizeof(scope)) { + strstr(callee_name, "::") || len == 0 || len >= sizeof(scope) || floor == 0) { return empty_result(); } memcpy(scope, scope_qn, len + SKIP_ONE); - for (;;) { + while (strlen(scope) > floor) { const char *label = cbm_ht_get(r->exact, scope); if (!label) { break; @@ -2223,7 +2267,8 @@ cbm_resolution_t cbm_registry_resolve_call_scoped(const cbm_registry_t *r, const int import_map_count, CBMLanguage lang, const char *scope_qn) { if (lang == CBM_LANG_PYTHON) { - cbm_resolution_t lexical = cbm_registry_resolve_lexical(r, callee_name, scope_qn); + cbm_resolution_t lexical = + cbm_registry_resolve_lexical(r, callee_name, scope_qn, module_qn); if (lexical.qualified_name && lexical.qualified_name[0]) { return lexical; } diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index 3141ab32d..2cb51cfd6 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14713,6 +14713,103 @@ TEST(pipeline_python_local_def_shadows_module_def_parallel_issue1548) { return pyn_run_shadowing(52); } +/* L7 review of #1548: a package __init__.py function can share a submodule's QN + * (poetry: layouts/__init__.py `def layout` is the QN prefix of every symbol in + * layouts/layout.py). The function-local detection is bounded at the symbol's + * own file, so class Layout stays a module-level, importable class. src/ layout + * without a manifest: the module resolver misses `rowpkg.layout`, so the import + * relies on the symbol-name fallback, which skips function locals -- unbounded, + * Layout read as a local of layout() and its import and calls vanished. Both + * resolvers; the second run re-indexes incrementally after an edit of + * layout.py, so the incremental registry seed is held to the same bound. */ +static const char pyn_layout_init_py[] = "def layout(name):\n" + " return name\n"; +static const char pyn_layout_mod_py[] = "class Layout:\n" + " def __init__(self, name):\n" + " self.name = name\n" + "\n" + " def create(self):\n" + " return self.name\n"; +static const char pyn_layout_app_py[] = "from rowpkg.layout import Layout\n" + "\n" + "def test_build():\n" + " return Layout(\"demo\").create()\n"; + +static int pyn_run_init_collision(int fillers) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_initqn_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "src/rowpkg/__init__.py", pyn_layout_init_py); + write_temp_file(tmp, "src/rowpkg/layout.py", pyn_layout_mod_py); + write_temp_file(tmp, "tests/test_app.py", pyn_layout_app_py); + for (int i = 0; i < fillers; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "filler%d.py", i); + snprintf(body, sizeof(body), "def filler%d():\n return %d\n", i, i); + write_temp_file(tmp, name, body); + } + char *old_workers = getenv("CBM_WORKERS"); + char *saved = old_workers ? strdup(old_workers) : NULL; + if (fillers > 0) { + cbm_setenv("CBM_WORKERS", "4", 1); + } + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/initqn.db", tmp); + int rc[2] = {-1, -1}; + bool imports[2] = {false, false}; + bool calls[2] = {false, false}; + bool method[2] = {false, false}; + for (int run = 0; run < 2; run++) { + if (run == 1) { + char path[512]; + snprintf(path, sizeof(path), "%s/src/rowpkg/layout.py", tmp); + th_append_file(path, "\n# edited: the second run re-indexes incrementally\n"); + } + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + rc[run] = p ? cbm_pipeline_run(p) : -1; + cbm_store_t *s = rc[run] == 0 ? cbm_store_open_path(db_path) : NULL; + if (s) { + const char *project = cbm_pipeline_project_name(p); + imports[run] = pyn_edge(s, project, "tests.test_app.py.__file__", + "src.rowpkg.layout.Layout", "IMPORTS"); + calls[run] = + pyn_edge(s, project, "tests.test_app.test_build", "src.rowpkg.layout.Layout", + "CALLS"); + method[run] = pyn_edge(s, project, "tests.test_app.test_build", + "src.rowpkg.layout.Layout.create", "CALLS"); + cbm_store_close(s); + } + if (p) { + cbm_pipeline_free(p); + } + } + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + th_rmtree(tmp); + for (int run = 0; run < 2; run++) { + ASSERT_EQ(rc[run], 0); + ASSERT_TRUE(imports[run]); /* the class stays importable */ + ASSERT_TRUE(calls[run]); /* Layout("demo") */ + ASSERT_TRUE(method[run]); /* .create() through the constructed type */ + } + PASS(); +} + +TEST(pipeline_python_init_function_shares_submodule_qn_issue1548) { + return pyn_run_init_collision(0); +} + +TEST(pipeline_python_init_function_shares_submodule_qn_parallel_issue1548) { + return pyn_run_init_collision(52); +} + TEST(pipeline_python_project) { /* Port of TestPipelinePythonProject */ const char *files[] = {"main.py", "utils.py"}; @@ -24214,6 +24311,8 @@ SUITE(pipeline) { RUN_TEST(pipeline_python_nested_def_local_after_incremental_issue1548); RUN_TEST(pipeline_python_local_def_shadows_module_def_issue1548); RUN_TEST(pipeline_python_local_def_shadows_module_def_parallel_issue1548); + RUN_TEST(pipeline_python_init_function_shares_submodule_qn_issue1548); + RUN_TEST(pipeline_python_init_function_shares_submodule_qn_parallel_issue1548); RUN_TEST(pipeline_header_include_target_is_independent_of_registration_order); RUN_TEST(pipeline_imports_multi_symbol_edges); RUN_TEST(pipeline_go_cross_package_call); diff --git a/tests/test_registry.c b/tests/test_registry.c index 055de4d78..0f3e805f2 100644 --- a/tests/test_registry.c +++ b/tests/test_registry.c @@ -1962,6 +1962,9 @@ TEST(registry_python_scope_qualified_terminal_bucket_uncapped) { bool seeded = cbm_registry_size(r) == 302; cbm_registry_free(r); ASSERT_TRUE(shape && seeded && resolved); + PASS(); +} + /* #1548: a Python def or class declared in a function body is a local of it. * It is registered for exact lookups but kept out of the short-name index, so * no weak strategy offers it to a caller elsewhere. A class nested in a class @@ -1969,24 +1972,29 @@ TEST(registry_python_scope_qualified_terminal_bucket_uncapped) { * does the same QN shape from another language. */ TEST(registry_python_function_local_is_exact_only_issue1548) { cbm_registry_t *r = cbm_registry_new(); - cbm_registry_add_lang(r, "create_app", "proj.web.create_app", "Function", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "index", "proj.web.create_app.index", "Function", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "Settings", "proj.web.create_app.Settings", "Class", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "value", "proj.web.create_app.Settings.value", "Method", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "Outer", "proj.web.Outer", "Class", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "Inner", "proj.web.Outer.Inner", "Class", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "f", "proj.ts.f", "Function", CBM_LANG_TYPESCRIPT); - cbm_registry_add_lang(r, "g", "proj.ts.f.g", "Function", CBM_LANG_TYPESCRIPT); + const size_t web = strlen("proj.web"); /* symbol prefix of web.py */ + static const char *const defs[][3] = { + {"create_app", "proj.web.create_app", "Function"}, + {"index", "proj.web.create_app.index", "Function"}, + {"Settings", "proj.web.create_app.Settings", "Class"}, + {"value", "proj.web.create_app.Settings.value", "Method"}, + {"Outer", "proj.web.Outer", "Class"}, + {"Inner", "proj.web.Outer.Inner", "Class"}, + }; + for (size_t i = 0; i < sizeof(defs) / sizeof(defs[0]); i++) { + cbm_registry_add_lang_in_module(r, defs[i][0], defs[i][1], defs[i][2], CBM_LANG_PYTHON, + web); + } + cbm_registry_add_lang_in_module(r, "f", "proj.ts.f", "Function", CBM_LANG_TYPESCRIPT, + strlen("proj.ts")); + cbm_registry_add_lang_in_module(r, "g", "proj.ts.f.g", "Function", CBM_LANG_TYPESCRIPT, + strlen("proj.ts")); static const char *const local[] = {"proj.web.create_app.index", "proj.web.create_app.Settings", "proj.web.create_app.Settings.value"}; for (int i = 0; i < 3; i++) { ASSERT_TRUE(cbm_registry_exists(r, local[i])); - ASSERT_TRUE(cbm_registry_qn_is_function_local(r, local[i])); + ASSERT_TRUE(cbm_registry_qn_is_function_local(r, local[i], web)); const char *simple = strrchr(local[i], '.') + 1; const char **out = NULL; int count = -1; @@ -2003,7 +2011,9 @@ TEST(registry_python_function_local_is_exact_only_issue1548) { ASSERT_EQ(count, 1); ASSERT_STR_EQ(out[0], nameable[i][1]); } - ASSERT_FALSE(cbm_registry_qn_is_function_local(r, "proj.web.Outer.Inner")); + ASSERT_FALSE(cbm_registry_qn_is_function_local(r, "proj.web.Outer.Inner", web)); + /* No module prefix: no local detection. */ + ASSERT_FALSE(cbm_registry_qn_is_function_local(r, "proj.web.create_app.index", 0)); /* A weak strategy cannot reach the local; before, index() anywhere in * the project resolved to it by unique_name. */ cbm_resolution_t res = cbm_registry_resolve(r, "index", "proj.other", NULL, NULL, 0); @@ -2017,44 +2027,110 @@ TEST(registry_python_function_local_is_exact_only_issue1548) { * to the other strategies. A dotted callee is not a bare name. */ TEST(registry_resolve_lexical_walks_enclosing_defs_issue1548) { cbm_registry_t *r = cbm_registry_new(); - cbm_registry_add_lang(r, "Tree", "proj.m.Tree", "Class", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "visit", "proj.m.Tree.visit", "Method", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "visit", "proj.m.Tree.visit.visit", "Function", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "helper", "proj.m.Tree.helper", "Method", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "factory", "proj.m.factory", "Function", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "helper", "proj.m.factory.helper", "Function", - CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "Local", "proj.m.factory.Local", "Class", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "run", "proj.m.factory.Local.run", "Method", CBM_LANG_PYTHON); - cbm_registry_add_lang(r, "top", "proj.m.top", "Function", CBM_LANG_PYTHON); + static const char *const defs[][3] = { + {"Tree", "proj.m.Tree", "Class"}, + {"visit", "proj.m.Tree.visit", "Method"}, + {"visit", "proj.m.Tree.visit.visit", "Function"}, + {"helper", "proj.m.Tree.helper", "Method"}, + {"factory", "proj.m.factory", "Function"}, + {"helper", "proj.m.factory.helper", "Function"}, + {"Local", "proj.m.factory.Local", "Class"}, + {"run", "proj.m.factory.Local.run", "Method"}, + {"top", "proj.m.top", "Function"}, + }; + for (size_t i = 0; i < sizeof(defs) / sizeof(defs[0]); i++) { + cbm_registry_add_lang_in_module(r, defs[i][0], defs[i][1], defs[i][2], CBM_LANG_PYTHON, + strlen("proj.m")); + } /* In method Tree.visit, `visit` is its nested def; so it is inside that * nested def (its own recursion). */ - cbm_resolution_t res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit"); + cbm_resolution_t res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit", "proj.m"); ASSERT_STR_EQ(res.qualified_name, "proj.m.Tree.visit.visit"); ASSERT_STR_EQ(res.strategy, "lexical_scope"); - res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit.visit"); + res = cbm_registry_resolve_lexical(r, "visit", "proj.m.Tree.visit.visit", "proj.m"); ASSERT_STR_EQ(res.qualified_name, "proj.m.Tree.visit.visit"); /* The class body is skipped: Tree.helper is not a bare name in a method. */ - res = cbm_registry_resolve_lexical(r, "helper", "proj.m.Tree.visit.visit"); + res = cbm_registry_resolve_lexical(r, "helper", "proj.m.Tree.visit.visit", "proj.m"); ASSERT_NULL(res.qualified_name); /* A method of a local class sees the def enclosing its class. */ - res = cbm_registry_resolve_lexical(r, "helper", "proj.m.factory.Local.run"); + res = cbm_registry_resolve_lexical(r, "helper", "proj.m.factory.Local.run", "proj.m"); ASSERT_STR_EQ(res.qualified_name, "proj.m.factory.helper"); - res = cbm_registry_resolve_lexical(r, "Local", "proj.m.factory"); + res = cbm_registry_resolve_lexical(r, "Local", "proj.m.factory", "proj.m"); ASSERT_STR_EQ(res.qualified_name, "proj.m.factory.Local"); /* Module-level names and dotted callees are not this strategy's. */ - res = cbm_registry_resolve_lexical(r, "top", "proj.m.factory"); + res = cbm_registry_resolve_lexical(r, "top", "proj.m.factory", "proj.m"); ASSERT_NULL(res.qualified_name); - res = cbm_registry_resolve_lexical(r, "self.helper", "proj.m.factory"); + res = cbm_registry_resolve_lexical(r, "self.helper", "proj.m.factory", "proj.m"); ASSERT_NULL(res.qualified_name); - res = cbm_registry_resolve_lexical(r, "helper", "proj.m"); + res = cbm_registry_resolve_lexical(r, "helper", "proj.m", "proj.m"); ASSERT_NULL(res.qualified_name); cbm_registry_free(r); PASS(); } +/* A package __init__.py function can share a submodule's QN: poetry's + * layouts/__init__.py `def layout` is proj.pkg.layout, the QN prefix of every + * symbol in layouts/layout.py. Bounded at its own file, class Layout and its + * methods are no locals of that function, in either registration order (full + * index: discovery order; incremental: node-id order), and the lexical walk of + * a caller in layout.py never enters the __init__ function. */ +TEST(registry_python_local_bound_at_own_file_issue1548) { + const size_t init_prefix = strlen("proj.pkg"); /* pkg/__init__.py */ + const size_t mod_prefix = strlen("proj.pkg.layout"); /* pkg/layout.py */ + for (int order = 0; order < 2; order++) { + cbm_registry_t *r = cbm_registry_new(); + if (order == 0) { + cbm_registry_add_lang_in_module(r, "layout", "proj.pkg.layout", "Function", + CBM_LANG_PYTHON, init_prefix); + } + cbm_registry_add_lang_in_module(r, "Layout", "proj.pkg.layout.Layout", "Class", + CBM_LANG_PYTHON, mod_prefix); + cbm_registry_add_lang_in_module(r, "create", "proj.pkg.layout.Layout.create", "Method", + CBM_LANG_PYTHON, mod_prefix); + cbm_registry_add_lang_in_module(r, "make", "proj.pkg.layout.make", "Function", + CBM_LANG_PYTHON, mod_prefix); + if (order == 1) { + cbm_registry_add_lang_in_module(r, "layout", "proj.pkg.layout", "Function", + CBM_LANG_PYTHON, init_prefix); + } + static const char *const nameable[][2] = {{"Layout", "proj.pkg.layout.Layout"}, + {"create", "proj.pkg.layout.Layout.create"}, + {"make", "proj.pkg.layout.make"}}; + for (size_t i = 0; i < sizeof(nameable) / sizeof(nameable[0]); i++) { + const char **out = NULL; + int count = 0; + cbm_registry_find_by_name(r, nameable[i][0], &out, &count); + ASSERT_EQ(count, 1); + ASSERT_STR_EQ(out[0], nameable[i][1]); + ASSERT_FALSE(cbm_registry_qn_is_function_local(r, nameable[i][1], mod_prefix)); + } + /* `Layout()` in layout.py's make(): no lexical local; module strategies own it. */ + cbm_resolution_t res = + cbm_registry_resolve_lexical(r, "Layout", "proj.pkg.layout.make", "proj.pkg.layout"); + ASSERT_NULL(res.qualified_name); + cbm_registry_free(r); + } + PASS(); +} + +/* The prefix every def of a file carries, without allocating; the same as + * cbm_pipeline_fqn_compute(project, path, name) minus ".name". */ +TEST(fqn_symbol_prefix_len_matches_fqn_compute) { + static const char *const paths[] = {"pkg/layout.py", "pkg/__init__.py", "a.py", + "deep/x/y/mod.py", "web/index.py", "noext"}; + for (size_t i = 0; i < sizeof(paths) / sizeof(paths[0]); i++) { + char *qn = cbm_pipeline_fqn_compute("proj", paths[i], "sym"); + ASSERT_NOT_NULL(qn); + size_t want = strlen(qn) - strlen(".sym"); + free(qn); + ASSERT_EQ(cbm_pipeline_fqn_symbol_prefix_len("proj", paths[i]), want); + } + ASSERT_EQ(cbm_pipeline_fqn_symbol_prefix_len("proj", "pkg/__init__.py"), strlen("proj.pkg")); + ASSERT_EQ(cbm_pipeline_fqn_symbol_prefix_len(NULL, "a.py"), 0); + PASS(); +} + SUITE(registry) { RUN_TEST(python_symbol_scope_len_preserves_file_identity); RUN_TEST(registry_python_package_scope_is_explicit); @@ -2161,4 +2237,6 @@ SUITE(registry) { RUN_TEST(weak_call_guards_share_one_drop_list); RUN_TEST(registry_python_function_local_is_exact_only_issue1548); RUN_TEST(registry_resolve_lexical_walks_enclosing_defs_issue1548); + RUN_TEST(registry_python_local_bound_at_own_file_issue1548); + RUN_TEST(fqn_symbol_prefix_len_matches_fqn_compute); } From 0bab94ce1cf465e446f4ffb3a803269a9ec68854 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 04:08:24 +0200 Subject: [PATCH 08/15] test(python): a def under a branch is bound by the lexical step, not the LSP At a compound statement's join the Python LSP invalidates what the branch bound, so the local def under if True: is resolved by the registry's lexical_scope step; pin that producer instead of an lsp_ strategy. Found at the branch's first build. Signed-off-by: Martin Vogel --- tests/test_pipeline.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index 2cb51cfd6..9f64812cb 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14642,15 +14642,21 @@ static int pyn_assert_shadowing(cbm_store_t *s, const char *project) { "\"strategy\":\"lsp_super\"")); ASSERT_TRUE(pyn_edge_with(s, project, "sc.factory.Local.other", "sc.factory.Local.greet", "CALLS", "\"strategy\":\"lsp_")); + /* sa's local def is a plain statement of the body: the LSP binds it as an + * exact callable. sb's sits under `if True:`; at that compound statement's + * join the LSP invalidates what the branch bound (the def may not exist on + * every path), so it emits nothing and the registry's lexical step binds + * the local -- still never the module def. */ const char *files[] = {"sa", "sb"}; + const char *local_strategy[] = {"\"strategy\":\"lsp_", "\"strategy\":\"lexical_scope\""}; for (int f = 0; f < 2; f++) { char local_caller[64], local_def[64], module_caller[64], module_def[64]; snprintf(local_caller, sizeof(local_caller), "%s.uses_local", files[f]); snprintf(local_def, sizeof(local_def), "%s.uses_local.helper", files[f]); snprintf(module_caller, sizeof(module_caller), "%s.uses_module", files[f]); snprintf(module_def, sizeof(module_def), "%s.helper", files[f]); - ASSERT_TRUE(pyn_edge_with(s, project, local_caller, local_def, "CALLS", - "\"strategy\":\"lsp_")); + ASSERT_TRUE( + pyn_edge_with(s, project, local_caller, local_def, "CALLS", local_strategy[f])); ASSERT_FALSE(pyn_edge(s, project, local_caller, module_def, "CALLS")); ASSERT_TRUE(pyn_edge(s, project, module_caller, module_def, "CALLS")); ASSERT_FALSE(pyn_edge(s, project, module_caller, local_def, "CALLS")); From 5bfa7b53bfc9cd8d648e2a09e45f2d8caf427df2 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 04:11:16 +0200 Subject: [PATCH 09/15] test(python): note that H1 may move the branch-def case to an lsp strategy Signed-off-by: Martin Vogel --- tests/test_pipeline.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index 9f64812cb..25e467b99 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14646,7 +14646,9 @@ static int pyn_assert_shadowing(cbm_store_t *s, const char *project) { * exact callable. sb's sits under `if True:`; at that compound statement's * join the LSP invalidates what the branch bound (the def may not exist on * every path), so it emits nothing and the registry's lexical step binds - * the local -- still never the module def. */ + * the local -- still never the module def. H1 (the LSP walks compound + * statements) may legitimately move sb to an lsp_* strategy; that PR + * must update this expectation deliberately. */ const char *files[] = {"sa", "sb"}; const char *local_strategy[] = {"\"strategy\":\"lsp_", "\"strategy\":\"lexical_scope\""}; for (int f = 0; f < 2; f++) { From d4beec3c5811d8165f34905158c10a22e0c9d403 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 04:48:27 +0200 Subject: [PATCH 10/15] test(python): local classes are extracted defs now (#1548) pylsp_nested_and_local_classbody_calls_join_extracted_caller asserted that a class declared in a function body is NOT a def, which is the behaviour #1548 fixes. Their class-body calls still join the enclosing function; that part is unchanged. Signed-off-by: Martin Vogel --- tests/test_py_lsp.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/test_py_lsp.c b/tests/test_py_lsp.c index 3ee2d8ad5..7a1c460d5 100644 --- a/tests/test_py_lsp.c +++ b/tests/test_py_lsp.c @@ -3629,12 +3629,16 @@ TEST(pylsp_nested_and_local_classbody_calls_join_extracted_caller) { {"field(35)", "test.main.Controller.build", "test.main.field"}, }; CBMFileResult *r = extract_py(source); - bool correct = pylsp_class_sites(r, source, sites, 7) && - !pylsp_module_has_def(r, "test.main.build.Local") && - !pylsp_module_has_def(r, "test.main.Controller.build.Local"); + /* #1548: a class declared in a function body is extracted, as a local of + * that function; its class-body calls still join the enclosing function + * (the sites above), never the local class. */ + bool sites_ok = pylsp_class_sites(r, source, sites, 7); + bool locals_extracted = r && pylsp_module_has_def(r, "test.main.build.Local") && + pylsp_module_has_def(r, "test.main.Controller.build.Local"); if (r) cbm_free_result(r); - ASSERT_TRUE(correct); + ASSERT_TRUE(sites_ok); + ASSERT_TRUE(locals_extracted); PASS(); } From eec3418fef18066f457b46e3a243ab3c66732430 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 06:18:09 +0200 Subject: [PATCH 11/15] fix(python): a test's nested helper keeps TESTS for its calls (#1548) Since #1548 a def declared inside a Python test function is extracted, so its calls start at the helper instead of the test FILE node, whose name passed the test-name check. 4,641 of 6,315 TESTS edges lost on the corpus were this shape. TESTS for a call from such a helper is now attributed to the nearest enclosing test function of the same file; a helper nested in a non-test function stays without TESTS, like a module-level helper. Signed-off-by: Martin Vogel --- src/pipeline/pass_tests.c | 51 +++++++++++++++++++- tests/test_pipeline.c | 97 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+), 2 deletions(-) diff --git a/src/pipeline/pass_tests.c b/src/pipeline/pass_tests.c index 9cade02a0..a763016fb 100644 --- a/src/pipeline/pass_tests.c +++ b/src/pipeline/pass_tests.c @@ -28,6 +28,7 @@ enum { #include "pipeline/pipeline.h" #include "pipeline/pipeline_internal.h" #include "graph_buffer/graph_buffer.h" +#include "discover/discover.h" #include "foundation/log.h" #include @@ -208,6 +209,48 @@ static char *test_to_prod_path(const char *test_path) { return NULL; } +/* #1548: a def declared inside a Python test function (a closure, a local + * helper, a method of a test-local class) is that test's own code, extracted + * as a node of its own. Its calls exercise the target on the test's behalf, so + * TESTS is attributed to the nearest enclosing test function of the same file + * (QN prefix walk). Before #1548 those calls hung on the test FILE node, whose + * name passes the test-name check. NULL when no enclosing test function exists + * (a helper nested in a non-test function stays without TESTS, like a + * module-level helper). */ +static const cbm_gbuf_node_t *enclosing_python_test_func(const cbm_gbuf_t *gb, + const cbm_gbuf_node_t *src) { + if (!src->qualified_name || !src->file_path || !src->label || + (strcmp(src->label, "Function") != 0 && strcmp(src->label, "Method") != 0)) { + return NULL; + } + const char *slash = strrchr(src->file_path, '/'); + if (cbm_language_for_filename(slash ? slash + SKIP_ONE : src->file_path) != CBM_LANG_PYTHON) { + return NULL; + } + char scope[CBM_SZ_512]; + size_t len = strlen(src->qualified_name); + if (len >= sizeof(scope)) { + return NULL; + } + memcpy(scope, src->qualified_name, len + SKIP_ONE); + for (char *dot = strrchr(scope, '.'); dot; dot = strrchr(scope, '.')) { + *dot = '\0'; + const cbm_gbuf_node_t *outer = cbm_gbuf_find_by_qn(gb, scope); + if (!outer) { + continue; + } + if (!outer->file_path || strcmp(outer->file_path, src->file_path) != 0) { + return NULL; /* left the file: no enclosing test function */ + } + if (outer->label && + (strcmp(outer->label, "Function") == 0 || strcmp(outer->label, "Method") == 0) && + cbm_is_test_func_name(outer->name)) { + return outer; + } + } + return NULL; +} + /* Create TESTS edges from test functions to production functions via CALLS edges. */ static int create_tests_edges(cbm_pipeline_ctx_t *ctx) { int count = 0; @@ -238,11 +281,15 @@ static int create_tests_edges(cbm_pipeline_ctx_t *ctx) { continue; } + const cbm_gbuf_node_t *test_src = src; if (!cbm_is_test_func_name(src->name)) { - continue; + test_src = enclosing_python_test_func(ctx->gbuf, src); + if (!test_src) { + continue; + } } - cbm_gbuf_insert_edge(ctx->gbuf, src->id, tgt->id, "TESTS", "{}"); + cbm_gbuf_insert_edge(ctx->gbuf, test_src->id, tgt->id, "TESTS", "{}"); count++; } return count; diff --git a/tests/test_pipeline.c b/tests/test_pipeline.c index 25e467b99..8403a3aed 100644 --- a/tests/test_pipeline.c +++ b/tests/test_pipeline.c @@ -14818,6 +14818,101 @@ TEST(pipeline_python_init_function_shares_submodule_qn_parallel_issue1548) { return pyn_run_init_collision(52); } +/* #1548 keeps TESTS for calls made through a test's own nested helper: the + * call now starts at the extracted helper def (not at the test FILE node, as + * before #1548), so TESTS is attributed to the enclosing test function. A + * helper nested in a NON-test function stays without TESTS, like a module-level + * helper. Both resolvers. */ +static const char pyn_tests_prod_py[] = "def target(value):\n" + " return value\n" + "\n" + "def other_target(value):\n" + " return value\n"; +static const char pyn_tests_test_py[] = "from pkg.prod import target, other_target\n" + "\n" + "def test_through_nested_helper():\n" + " def check(v):\n" + " return target(v)\n" + " assert check(1) == 1\n" + "\n" + "class TestGroup:\n" + " def test_method_helper(self):\n" + " def probe():\n" + " return target(2)\n" + " assert probe() == 2\n" + "\n" + "def make_fixture():\n" + " def build():\n" + " return other_target(3)\n" + " return build\n"; + +static int pyn_run_nested_helper_tests(int fillers) { + char tmp[256]; + snprintf(tmp, sizeof(tmp), "/tmp/cbm_py_ntests_XXXXXX"); + if (!cbm_mkdtemp(tmp)) { + FAIL("tmpdir"); + } + write_temp_file(tmp, "pkg/__init__.py", ""); + write_temp_file(tmp, "pkg/prod.py", pyn_tests_prod_py); + write_temp_file(tmp, "tests/test_rows.py", pyn_tests_test_py); + for (int i = 0; i < fillers; i++) { + char name[64]; + char body[128]; + snprintf(name, sizeof(name), "filler%d.py", i); + snprintf(body, sizeof(body), "def filler%d():\n return %d\n", i, i); + write_temp_file(tmp, name, body); + } + char *old_workers = getenv("CBM_WORKERS"); + char *saved = old_workers ? strdup(old_workers) : NULL; + if (fillers > 0) { + cbm_setenv("CBM_WORKERS", "4", 1); + } + char db_path[512]; + snprintf(db_path, sizeof(db_path), "%s/ntests.db", tmp); + cbm_pipeline_t *p = cbm_pipeline_new(tmp, db_path, CBM_MODE_FULL); + int rc = p ? cbm_pipeline_run(p) : -1; + cbm_store_t *s = rc == 0 ? cbm_store_open_path(db_path) : NULL; + bool fn_tests = false, method_tests = false, helper_calls = false, fixture_tests = true; + if (s) { + const char *project = cbm_pipeline_project_name(p); + helper_calls = pyn_edge(s, project, "tests.test_rows.test_through_nested_helper.check", + "pkg.prod.target", "CALLS"); + fn_tests = pyn_edge(s, project, "tests.test_rows.test_through_nested_helper", + "pkg.prod.target", "TESTS"); + method_tests = pyn_edge(s, project, "tests.test_rows.TestGroup.test_method_helper", + "pkg.prod.target", "TESTS"); + fixture_tests = pyn_edge(s, project, "tests.test_rows.make_fixture", + "pkg.prod.other_target", "TESTS") || + pyn_edge(s, project, "tests.test_rows.make_fixture.build", + "pkg.prod.other_target", "TESTS"); + cbm_store_close(s); + } + if (p) { + cbm_pipeline_free(p); + } + if (saved) { + cbm_setenv("CBM_WORKERS", saved, 1); + free(saved); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + th_rmtree(tmp); + ASSERT_EQ(rc, 0); + ASSERT_TRUE(helper_calls); /* the call starts at the nested helper (#1548) */ + ASSERT_TRUE(fn_tests); /* ... and the enclosing test function still TESTS the target */ + ASSERT_TRUE(method_tests); /* the same inside a test method */ + ASSERT_FALSE(fixture_tests); /* a helper nested in a non-test function: no TESTS */ + PASS(); +} + +TEST(pipeline_python_nested_helper_keeps_tests_issue1548) { + return pyn_run_nested_helper_tests(0); +} + +TEST(pipeline_python_nested_helper_keeps_tests_parallel_issue1548) { + return pyn_run_nested_helper_tests(52); +} + TEST(pipeline_python_project) { /* Port of TestPipelinePythonProject */ const char *files[] = {"main.py", "utils.py"}; @@ -24321,6 +24416,8 @@ SUITE(pipeline) { RUN_TEST(pipeline_python_local_def_shadows_module_def_parallel_issue1548); RUN_TEST(pipeline_python_init_function_shares_submodule_qn_issue1548); RUN_TEST(pipeline_python_init_function_shares_submodule_qn_parallel_issue1548); + RUN_TEST(pipeline_python_nested_helper_keeps_tests_issue1548); + RUN_TEST(pipeline_python_nested_helper_keeps_tests_parallel_issue1548); RUN_TEST(pipeline_header_include_target_is_independent_of_registration_order); RUN_TEST(pipeline_imports_multi_symbol_edges); RUN_TEST(pipeline_go_cross_package_call); From 8469fed9602db7ff1d115abe5fbc0d887405b4d8 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 06:55:57 +0200 Subject: [PATCH 12/15] test(rust-lsp): tag the shared-registry fixtures as Rust defs The shared Tier-2 build now registers Rust definitions only. Two fixtures that feed cbm_rust_build_cross_registry left CBMLSPDef.lang zero, which the pipeline never does (pass_lsp_cross.c tags every def with its file's language). The other shared-registry fixtures in this file already set CBM_LANG_RUST. Set it on these two as well. Signed-off-by: Martin Vogel --- tests/test_rust_lsp.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/test_rust_lsp.c b/tests/test_rust_lsp.c index 3c5483915..e5ab4cef3 100644 --- a/tests/test_rust_lsp.c +++ b/tests/test_rust_lsp.c @@ -591,6 +591,9 @@ TEST(rustlsp_shared_registry_resolves_like_per_file) { defs[1].receiver_type = "test.demo.Database"; defs[1].def_module_qn = "test.demo"; defs[1].return_types = "alloc.string.String"; + /* The shared build registers Rust defs only, as the pipeline tags them. */ + defs[0].lang = CBM_LANG_RUST; + defs[1].lang = CBM_LANG_RUST; const char *imp_names[] = {"demo"}; const char *imp_qns[] = {"test::demo"}; @@ -695,6 +698,7 @@ TEST(rustlsp_shared_registry_macro_hidden_call_has_carrier) { defs[0].label = "Function"; defs[0].def_module_qn = "test.lib"; defs[0].return_types = "str"; + defs[0].lang = CBM_LANG_RUST; const char *imp_names[] = {"lib"}; const char *imp_qns[] = {"test::lib"}; From 090580b3c95ba2ec58e7e09eb2858438dc827111 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 16:36:35 +0200 Subject: [PATCH 13/15] refactor(python): scope the nested-def counter to the Python branch cppcheck 2.20 variableScope on defs_before; same behaviour. Signed-off-by: Martin Vogel --- internal/cbm/extract_defs.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/internal/cbm/extract_defs.c b/internal/cbm/extract_defs.c index 2b24de63f..6491a677f 100644 --- a/internal/cbm/extract_defs.c +++ b/internal/cbm/extract_defs.c @@ -12261,9 +12261,9 @@ static void walk_defs(CBMExtractCtx *ctx, TSNode root, const CBMLangSpec *spec, } if (cbm_kind_in_set(node, spec->function_node_types)) { if (!is_template_class_node(node, ctx->language)) { - int defs_before = ctx->result->defs.count; - extract_func_def(ctx, node, spec); if (ctx->language == CBM_LANG_PYTHON) { + int defs_before = ctx->result->defs.count; + extract_func_def(ctx, node, spec); if (ctx->result->defs.count > defs_before) { py_push_def_body(&s, node, ctx->result->defs.items[ctx->result->defs.count - SKIP_ONE] @@ -12271,6 +12271,7 @@ static void walk_defs(CBMExtractCtx *ctx, TSNode root, const CBMLangSpec *spec, } continue; } + extract_func_def(ctx, node, spec); // Most languages stop here. JS/TS (and Wolfram) descend into the // function body so NESTED named definitions are also captured — // e.g. arrow methods of an object literal returned from a factory From 67ff40d5cbe88d3fe933b846ab228340e8bb2f7d Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 16:42:30 +0200 Subject: [PATCH 14/15] fix(kotlin): resolve calls through typed class properties (#1555) The cross-file Kotlin walk resolved a call on a typed method parameter by type, but a call on a class property - a primary-constructor val, a body property or `this.prop` - fell to the name guess. The cross registry held each own-file class from its extraction defs, whose properties carry no type (constructor vals none at all), and the walk's typed AST registration of the same class was added behind that entry under the same QN, so every lookup returned the field-less one. Constructor injection is the common Kotlin dependency shape, so `repository.inserir(...)` on an injected port bound to whichever same-named method the guess found, including an implementation of another port (#1555). The walk now registers the file's own declarations into an overlay chained to the cross registry and checked first, indexed once the top-level declarations are collected. The sole-definer fallback walks the whole chain so its candidate set is unchanged. Typing more receivers exposed resolution gaps that the untyped receivers had hidden; each would otherwise turn a typed call into a wrong name guess: - a member of the receiver class wins over a same-named extension of the same file (Kotlin resolution order; the overlay holds both); - `x.hashCode()` / `toString()` / `equals()` on a receiver typed by another file: the per-file pass emits the kotlin.Any placeholder, the cross pass the declared override, and two targets at one occurrence are ambiguous to the call join. The per-file placeholder is dropped when the cross pass resolves the same occurrence; - the same members on a stdlib or JDK type (`count.toString()` on an Int) resolve to the kotlin.Any builtin instead of a seeded stdlib member that has no graph node; - a property READ on a typed receiver whose QN the graph spells differently (a nested class's property) never becomes a CALL_REFERENCE to a same-named function: the reference join falls back by name to non-callable targets only, which binds the property's own node; - a typed value without an `invoke` operator (`private val handshake: Handshake?`) no longer shadows the same-named member function; - inside a lambda whose typed receiver becomes `this` (`repo.let { .. }`), and inside a member extension, the enclosing class's members stay callable bare (the implicit-receiver chain); a call naming a class stays a constructor call. Fixes #1555 Signed-off-by: Martin Vogel --- Makefile.cbm | 2 +- internal/cbm/lsp/kotlin_lsp.c | 170 +++++++++++++-- internal/cbm/lsp/kotlin_lsp.h | 7 + src/pipeline/lsp_resolve.h | 18 ++ src/pipeline/pass_lsp_cross.c | 63 ++++++ src/pipeline/pass_parallel.c | 11 + src/pipeline/pass_usages.c | 11 + tests/test_kotlin_typed_calls.c | 359 ++++++++++++++++++++++++++++++++ tests/test_main.c | 2 + 9 files changed, 628 insertions(+), 15 deletions(-) create mode 100644 tests/test_kotlin_typed_calls.c diff --git a/Makefile.cbm b/Makefile.cbm index b4b8c8827..6d19f430a 100644 --- a/Makefile.cbm +++ b/Makefile.cbm @@ -846,7 +846,7 @@ TEST_PY_LSP_SCALE_SRCS = tests/test_py_lsp_scale.c TEST_TS_LSP_SRCS = tests/test_ts_lsp.c TEST_JAVA_LSP_SRCS = tests/test_java_lsp.c tests/test_java_lsp_coverage.c -TEST_KOTLIN_LSP_SRCS = tests/test_kotlin_lsp.c +TEST_KOTLIN_LSP_SRCS = tests/test_kotlin_lsp.c tests/test_kotlin_typed_calls.c TEST_RUST_LSP_SRCS = tests/test_rust_lsp.c TEST_INTEGRATION_SRCS = tests/test_integration.c tests/test_incremental.c tests/test_lang_contract.c tests/test_edge_imports.c tests/test_edge_structural.c tests/test_lsp_resolution_probe.c tests/test_node_creation_probe.c tests/test_edge_types_probe.c tests/test_convergence_probe.c tests/test_matrix_known_classes.c tests/test_matrix_new_constructs.c tests/test_grammar_probe_a.c tests/test_grammar_probe_b.c tests/test_grammar_probe_c.c tests/test_grammar_probe_d.c tests/test_grammar_probe_e.c tests/test_grammar_probe_f.c tests/test_grammar_probe_g.c tests/test_conditional_variants.c tests/test_spawns.c diff --git a/internal/cbm/lsp/kotlin_lsp.c b/internal/cbm/lsp/kotlin_lsp.c index e860e8857..be86587ca 100644 --- a/internal/cbm/lsp/kotlin_lsp.c +++ b/internal/cbm/lsp/kotlin_lsp.c @@ -395,9 +395,47 @@ static const CBMType *kt_unwrap_nullable(const CBMType *t) { return t; } +/* A kotlin.Any member (toString / equals / hashCode) resolved on a stdlib or + * JDK type -- `count.toString()` on `val count: Int` binds kotlin.Int.toString + * from the stdlib seed. That target has no graph node, so the call join fell + * back to a name guess and bound the call to whichever project class declares + * a same-named override (#1555: detekt `defaultValue.toString()` -> + * Entity.toString). The member is the universal kotlin.Any one, an external + * builtin node: emit that, no project edge. `kotlin.` and `java.` are reserved + * packages (the compilers reject project declarations in them), so the prefix + * proves the owner is not a project type; project QNs in this resolver are + * package-spelled and keep their declared override. */ +static bool kt_is_any_member(const char *name) { + return strcmp(name, "toString") == 0 || strcmp(name, "equals") == 0 || + strcmp(name, "hashCode") == 0; +} + +static const char *kt_external_any_member(KotlinLSPContext *ctx, const char *callee_qn) { + const char *dot = strrchr(callee_qn, '.'); + if (!dot || !kt_is_any_member(dot + 1)) { + return NULL; + } + if (strncmp(callee_qn, "kotlin.Any.", strlen("kotlin.Any.")) == 0) { + return NULL; /* already the builtin */ + } + if (strncmp(callee_qn, "kotlin.", strlen("kotlin.")) != 0 && + strncmp(callee_qn, "java.", strlen("java.")) != 0) { + return NULL; /* a project override: a real node */ + } + return kt_join_dot(ctx->arena, "kotlin.Any", dot + 1); +} + /* Append a typed resolution result to the output array. */ static void kt_emit_resolved_kind(KotlinLSPContext *ctx, const char *callee_qn, const char *strategy, float confidence, CBMResolvedKind kind) { + if (kind == CBM_RESOLVED_INVOCATION && callee_qn) { + const char *any_qn = kt_external_any_member(ctx, callee_qn); + if (any_qn) { + callee_qn = any_qn; + strategy = "lsp_kt_any"; + confidence = confidence < KT_CONF_PARTIAL ? confidence : KT_CONF_PARTIAL; + } + } if (ctx->debug) { fprintf(stderr, "[kotlin_lsp] EMIT %s -> %s [%s %.2f] (resolved_calls=%p enc=%s)\n", ctx->enclosing_func_qn ? ctx->enclosing_func_qn : "(null)", @@ -1127,20 +1165,26 @@ const char *kotlin_resolve_function_name(KotlinLSPContext *ctx, const char *name * package/import/bare/default-import lookups miss, so it never overrides a * more specific match; in the per-file pass the registry holds just this * file's defs, so the candidate is the file's own sole top-level fun. */ - if (ctx->registry && ctx->registry->funcs) { + /* The cross-file walk runs on an own-file overlay chained to the cross + * registry (#1555), so the candidates span the whole chain; a function + * registered in both layers under one QN is one candidate. */ + { const char *only_qn = NULL; int matches = 0; - for (int i = 0; i < ctx->registry->func_count && matches < 2; i++) { - const CBMRegisteredFunc *f = &ctx->registry->funcs[i]; - if (!f->qualified_name || !f->short_name) { - continue; - } - if (f->receiver_type) { /* method / extension — not a bare top-level fun */ - continue; - } - if (strcmp(f->short_name, name) == 0) { - only_qn = f->qualified_name; - matches++; + for (const CBMTypeRegistry *r = ctx->registry; r && matches < 2; r = r->fallback) { + for (int i = 0; r->funcs && i < r->func_count && matches < 2; i++) { + const CBMRegisteredFunc *f = &r->funcs[i]; + if (!f->qualified_name || !f->short_name) { + continue; + } + if (f->receiver_type) { /* method / extension — not a bare top-level fun */ + continue; + } + if (strcmp(f->short_name, name) == 0 && + !(only_qn && strcmp(only_qn, f->qualified_name) == 0)) { + only_qn = f->qualified_name; + matches++; + } } } if (matches == 1 && only_qn) { @@ -1335,6 +1379,22 @@ const CBMRegisteredFunc *kotlin_lookup_method(KotlinLSPContext *ctx, const char const CBMRegisteredFunc *rf = cbm_registry_lookup_method_aliased(ctx->registry, cur_qn, method_name); if (rf) { + /* A member wins over an extension of the same name (Kotlin + * resolution). The file's own extension functions carry the + * extended class as receiver_type, so the receiver lookup can + * return `fun Builder.add(..)` for a call meant for the member + * `Builder.add`; the member's QN nests under the class. */ + size_t cur_len = strlen(cur_qn); + bool is_member = rf->qualified_name && + strncmp(rf->qualified_name, cur_qn, cur_len) == 0 && + rf->qualified_name[cur_len] == '.'; + if (!is_member) { + const CBMRegisteredFunc *member = cbm_registry_lookup_func( + ctx->registry, kt_join_dot(ctx->arena, cur_qn, method_name)); + if (member) { + return member; + } + } return rf; } @@ -2907,6 +2967,37 @@ out: return result ? result : cbm_type_unknown(); } +/* `handshake()` inside a class whose constructor declares `private val + * handshake: Handshake?` next to `override fun handshake()` (okhttp + * RealConnection): the property is in the method scope (class fields are bound + * there), but a value of a known type without an `invoke` operator is no call + * candidate, so Kotlin resolves the call to the member function. Once #1555 + * typed the class's constructor vals, the lexical-shadow rule below failed + * such calls closed and they fell to a name guess. An unknown or callable + * binding still shadows, as before. */ +static bool kt_value_binding_yields_to_member(KotlinLSPContext *ctx, const CBMType *bound, + const char *name) { + if (!bound || cbm_type_is_unknown(bound) || bound->kind == CBM_TYPE_CALLABLE || + bound->kind == CBM_TYPE_FUNC || !ctx->this_type || cbm_type_is_unknown(ctx->this_type)) { + return false; + } + const CBMType *value = kt_unwrap_nullable(bound); + const char *value_qn = kt_type_qn_of(value); + if (value_qn && kotlin_lookup_method(ctx, value_qn, "invoke")) { + return false; /* an invokable value: the call may be `value.invoke()` */ + } + const char *this_qn = kt_type_qn_of(ctx->this_type); + if (!this_qn) { + return false; + } + const CBMRegisteredFunc *rf = kotlin_lookup_method(ctx, this_qn, name); + if (!rf || !rf->qualified_name) { + return false; + } + size_t len = strlen(this_qn); + return strncmp(rf->qualified_name, this_qn, len) == 0 && rf->qualified_name[len] == '.'; +} + static const CBMType *kt_eval_constructor_or_func_call(KotlinLSPContext *ctx, TSNode call_node, const char *callee_text) { /* callee_text is a bare identifier — it may be: @@ -2944,7 +3035,10 @@ static const CBMType *kt_eval_constructor_or_func_call(KotlinLSPContext *ctx, TS bound->data.func.return_types[0]) { return bound->data.func.return_types[0]; } - return cbm_type_unknown(); + if (!kt_value_binding_yields_to_member(ctx, bound, callee_text)) { + return cbm_type_unknown(); + } + /* A known non-invokable value yields to the member function: step 1. */ } /* 1. `this`-method dispatch: if we're inside a class method or DSL @@ -2967,6 +3061,37 @@ static const CBMType *kt_eval_constructor_or_func_call(KotlinLSPContext *ctx, TS } } + /* 1b. Implicit receivers form a chain, innermost first: inside a lambda + * whose receiver (or `it` type, which kt_process_lambda also installs as + * `this`) is another type, and inside a member extension whose extension + * receiver is another type, the enclosing class's members stay callable + * bare. Once #1555 typed more receivers, `repo.let { ping() }` resolved + * against Repo only and fell to a name guess. A call naming a class is a + * constructor call, left to step 2 (a secondary constructor is registered + * as a same-named member and would otherwise be taken for a method). */ + const char *ctor_cls = kotlin_resolve_class_name(ctx, callee_text); + bool is_ctor_call = + ctor_cls && ctx->registry && cbm_registry_lookup_type(ctx->registry, ctor_cls) != NULL; + if (ctx->enclosing_class_qn && !is_ctor_call) { + const char *this_qn = (ctx->this_type && !cbm_type_is_unknown(ctx->this_type)) + ? kt_type_qn_of(ctx->this_type) + : NULL; + if (!this_qn || strcmp(this_qn, ctx->enclosing_class_qn) != 0) { + const CBMRegisteredFunc *rf = + kotlin_lookup_method(ctx, ctx->enclosing_class_qn, callee_text); + if (rf && rf->qualified_name) { + kt_emit_resolved_at(ctx, rf->qualified_name, "lsp_kt_this", KT_CONF_THIS, + call_node); + if (rf->signature && rf->signature->kind == CBM_TYPE_FUNC && + rf->signature->data.func.return_types && + rf->signature->data.func.return_types[0]) { + return rf->signature->data.func.return_types[0]; + } + return cbm_type_unknown(); + } + } + } + /* 2. Class? */ const char *cls_qn = kotlin_resolve_class_name(ctx, callee_text); if (cls_qn && ctx->registry) { @@ -4533,6 +4658,11 @@ void kotlin_lsp_process_file(KotlinLSPContext *ctx, TSNode root) { /* 2. Collect top-level definitions for intra-file resolution. */ kt_collect_top_level_decls(ctx, root); + /* Cross-file: index the own-file overlay now that its declarations are + * in; later additions stay visible through the post-finalize tail. */ + if (ctx->overlay_index_arena && ctx->registry) { + cbm_registry_finalize_into((CBMTypeRegistry *)ctx->registry, ctx->overlay_index_arena); + } /* 3. Walk for call resolution. */ kt_walk_top_level_for_resolution(ctx, root, NULL); @@ -5436,12 +5566,24 @@ void cbm_run_kotlin_lsp_cross(CBMArena *arena, const char *source, int source_le project_name = module_qn; } + /* Own-file overlay (#1555). The cross registry above already holds this + * file's classes, built from extraction defs whose properties carry no + * type (primary-constructor vals none at all). The walk re-registers each + * class from the AST with typed fields; registered into `reg` under the + * same QN that entry was shadowed by the earlier one, so every + * `repository.inserir()` on an injected property fell to the name guess. + * In an overlay checked first, the AST entry wins. */ + CBMTypeRegistry overlay; + cbm_registry_init(&overlay, arena); + overlay.fallback = ® + KotlinLSPContext ctx; /* Match the per-file pass: until a source package declaration overrides * it, the path-derived module is the callable-QN prefix. This preserves * exact caller identity for default-package functions across files. */ - kotlin_lsp_init(&ctx, arena, source, source_len, ®, module_qn ? module_qn : "", + kotlin_lsp_init(&ctx, arena, source, source_len, &overlay, module_qn ? module_qn : "", module_qn ? module_qn : "", project_name, /*rel_path=*/NULL, out); + ctx.overlay_index_arena = &idx_arena; /* Unique short type name -> registered QN, for receiver annotations that * name a type defined in another file without an import. Two types diff --git a/internal/cbm/lsp/kotlin_lsp.h b/internal/cbm/lsp/kotlin_lsp.h index 98e048e6f..6359b5aa6 100644 --- a/internal/cbm/lsp/kotlin_lsp.h +++ b/internal/cbm/lsp/kotlin_lsp.h @@ -147,6 +147,13 @@ typedef struct KotlinLSPContext { * each field's REAL def QN here (kotlin class properties are minted with * module-level QNs, so composing . would miss the node). */ const void *cross_field_map; + /* Cross-file only (NULL per-file): when set, `registry` is a per-file + * overlay chained to the cross registry, and kotlin_lsp_process_file + * indexes the overlay into this scratch arena once the file's own + * declarations are collected. The overlay holds the AST registrations + * (typed primary-constructor and body properties), so they are found + * before the extraction-built entries of the same QN (#1555). */ + CBMArena *overlay_index_arena; } KotlinLSPContext; /* Initialize a KotlinLSPContext for processing one file. */ diff --git a/src/pipeline/lsp_resolve.h b/src/pipeline/lsp_resolve.h index d67918735..647b3f801 100644 --- a/src/pipeline/lsp_resolve.h +++ b/src/pipeline/lsp_resolve.h @@ -223,6 +223,24 @@ static inline bool cbm_pipeline_node_is_callable_target(const cbm_gbuf_node_t *n strcmp(node->label, "Constructor") == 0 || strcmp(node->label, "Class") == 0; } +/* The Kotlin resolver claims a property READ on a typed receiver with an exact + * row (lsp_kt_property_ref) so that the raw-name fallback cannot borrow a + * same-named function. That row names a property, never a function value. Its + * QN is not always materialized under that spelling (a property of a nested + * class); the graph lookup then finds nothing, or its unique-callable step + * picks a same-named function elsewhere (#1555: okhttp `server.webSocket` -> a + * test function `webSocket`, a fabricated CALL_REFERENCE). Such a row is not + * admissible: the caller falls back by name, restricted to non-callable + * targets, which binds the property's own node as before the receiver was + * typed and can never borrow a function. */ +static inline bool cbm_pipeline_reference_target_admissible(const CBMResolvedCall *ref, + const cbm_gbuf_node_t *tgt) { + if (!ref || !ref->strategy || strcmp(ref->strategy, "lsp_kt_property_ref") != 0) { + return true; + } + return tgt && !cbm_pipeline_node_is_callable_target(tgt); +} + static inline int cbm_pipeline_qn_class_method_tail_eq(const char *qn, const char *tail) { const char *qt = cbm_pipeline_qn_class_method_tail(qn); return qt && tail && strcmp(qt, tail) == 0; diff --git a/src/pipeline/pass_lsp_cross.c b/src/pipeline/pass_lsp_cross.c index cf94926c0..4f47e1b6b 100644 --- a/src/pipeline/pass_lsp_cross.c +++ b/src/pipeline/pass_lsp_cross.c @@ -1285,6 +1285,68 @@ static void pxc_supersede_occurrence(CBMArena *dst_arena, CBMResolvedCallArray * dst_calls->count = kept; } +/* The Kotlin universal-member placeholder: `x.hashCode()` / `toString()` / + * `equals()` on a receiver the resolver could not type is emitted on kotlin.Any + * (strategy lsp_kt_any, reduced confidence) -- the "no narrower receiver known" + * answer. */ +static bool pxc_is_kt_any_placeholder(const CBMResolvedCall *rc) { + return rc->strategy && strcmp(rc->strategy, "lsp_kt_any") == 0; +} + +static char *pxc_occurrence_key(CBMArena *keys, const CBMResolvedCall *rc) { + return cbm_arena_sprintf(keys, "%u\x1f%s\x1f%u:%u\x1f%u", (unsigned)rc->kind, rc->caller_qn, + rc->site_start_byte, rc->site_end_byte, (unsigned)rc->source_origin); +} + +/* #1555: the per-file pass sees only the file's own declarations, so a receiver + * typed by another file (`val type: ProtoType`) is unknown there and + * `type.hashCode()` gets the kotlin.Any placeholder. The cross pass types the + * receiver and answers with the declared override (`ProtoType.hashCode`), or + * with kotlin.Any again when no class on the receiver's chain declares one. Two + * distinct targets at one occurrence are a tombstone in the call join + * (pass_parallel.c LSP_IDX_AMBIGUOUS), so the occurrence fell to the name-based + * registry guess (wire: `type.hashCode` -> AllTypes.hashCode). A placeholder is + * therefore dropped when a cross row resolves the same occurrence; the cross row + * is then the only evidence. One hash pass per side keeps this linear. */ +static void pxc_drop_superseded_kt_any(CBMArena *keys, CBMResolvedCallArray *dst_calls, + const CBMResolvedCallArray *src_out) { + bool has_placeholder = false; + for (int i = 0; i < dst_calls->count && !has_placeholder; i++) { + has_placeholder = pxc_is_kt_any_placeholder(&dst_calls->items[i]); + } + if (!has_placeholder) { + return; + } + CBMHashTable *resolved = cbm_ht_create((uint32_t)src_out->count + 1); + for (int j = 0; j < src_out->count; j++) { + const CBMResolvedCall *src = &src_out->items[j]; + if (!src->caller_qn || !src->callee_qn || pxc_is_kt_any_placeholder(src) || + src->site_end_byte <= src->site_start_byte) { + continue; + } + char *k = pxc_occurrence_key(keys, src); + if (k) { + cbm_ht_set(resolved, k, (void *)(uintptr_t)1); + } + } + int kept = 0; + for (int i = 0; i < dst_calls->count; i++) { + const CBMResolvedCall *cur = &dst_calls->items[i]; + if (pxc_is_kt_any_placeholder(cur) && cur->caller_qn) { + char *k = pxc_occurrence_key(keys, cur); + if (k && cbm_ht_get(resolved, k)) { + continue; + } + } + if (kept != i) { + dst_calls->items[kept] = *cur; + } + kept++; + } + dst_calls->count = kept; + cbm_ht_free(resolved); +} + static void pxc_append_results(CBMArena *dst_arena, CBMResolvedCallArray *dst_calls, const CBMResolvedCallArray *src_out) { if (!dst_calls || !src_out) @@ -1312,6 +1374,7 @@ static void pxc_append_results(CBMArena *dst_arena, CBMResolvedCallArray *dst_ca } pxc_supersede_occurrence(dst_arena, dst_calls, src); } + pxc_drop_superseded_kt_any(&keys, dst_calls, src_out); for (int i = 0; i < dst_calls->count; i++) { const CBMResolvedCall *rc = &dst_calls->items[i]; diff --git a/src/pipeline/pass_parallel.c b/src/pipeline/pass_parallel.c index 2c0a9023a..4e398c948 100644 --- a/src/pipeline/pass_parallel.c +++ b/src/pipeline/pass_parallel.c @@ -3616,6 +3616,7 @@ static void resolve_file_usages(resolve_ctx_t *rc, resolve_worker_state_t *ws, const cbm_gbuf_node_t *tgt = NULL; bool precise_call_reference = false; const CBMResolvedCall *semantic_reference = NULL; + bool property_read = false; if (cbm_pipeline_usage_semantic_reference_candidate(usage)) { bool allow_tail = cbm_pipeline_lsp_allow_tail_match(lang); semantic_reference = cbm_pipeline_find_lsp_reference_indexed_in_graph( @@ -3628,6 +3629,13 @@ static void resolve_file_usages(resolve_ctx_t *rc, resolve_worker_state_t *ws, if (semantic_reference) { tgt = cbm_pipeline_lsp_target_node(rc->main_gbuf, rc->project_name, semantic_reference->callee_qn, allow_tail); + if (!cbm_pipeline_reference_target_admissible(semantic_reference, tgt)) { + /* a property read: the name fallback below may still bind the + * property's own node, never a callable */ + tgt = NULL; + semantic_reference = NULL; + property_read = true; + } precise_call_reference = cbm_pipeline_node_is_callable_target(tgt); } } @@ -3683,6 +3691,9 @@ static void resolve_file_usages(resolve_ctx_t *rc, resolve_worker_state_t *ws, cbm_pipeline_node_is_callable_target(tgt))) { continue; } + if (property_read && cbm_pipeline_node_is_callable_target(tgt)) { + continue; + } } if (!tgt || src->id == tgt->id) { continue; diff --git a/src/pipeline/pass_usages.c b/src/pipeline/pass_usages.c index d47f53de6..972822522 100644 --- a/src/pipeline/pass_usages.c +++ b/src/pipeline/pass_usages.c @@ -213,6 +213,7 @@ static int resolve_usage_edges(cbm_pipeline_ctx_t *ctx, const CBMFileResult *res const cbm_gbuf_node_t *tgt = NULL; bool precise_call_reference = false; const CBMResolvedCall *semantic_reference = NULL; + bool property_read = false; if (cbm_pipeline_usage_semantic_reference_candidate(usage)) { bool allow_tail = cbm_pipeline_lsp_allow_tail_match(lang); semantic_reference = cbm_pipeline_find_lsp_reference_indexed_in_graph( @@ -225,6 +226,13 @@ static int resolve_usage_edges(cbm_pipeline_ctx_t *ctx, const CBMFileResult *res if (semantic_reference) { tgt = cbm_pipeline_lsp_target_node(ctx->gbuf, ctx->project_name, semantic_reference->callee_qn, allow_tail); + if (!cbm_pipeline_reference_target_admissible(semantic_reference, tgt)) { + /* a property read: the name fallback below may still bind the + * property's own node, never a callable */ + tgt = NULL; + semantic_reference = NULL; + property_read = true; + } precise_call_reference = cbm_pipeline_node_is_callable_target(tgt); } } @@ -285,6 +293,9 @@ static int resolve_usage_edges(cbm_pipeline_ctx_t *ctx, const CBMFileResult *res cbm_pipeline_node_is_callable_target(tgt))) { continue; } + if (property_read && cbm_pipeline_node_is_callable_target(tgt)) { + continue; + } } if (!tgt || src->id == tgt->id) { continue; diff --git a/tests/test_kotlin_typed_calls.c b/tests/test_kotlin_typed_calls.c new file mode 100644 index 000000000..5b0aca528 --- /dev/null +++ b/tests/test_kotlin_typed_calls.c @@ -0,0 +1,359 @@ +/* + * test_kotlin_typed_calls.c — Kotlin member calls through a class PROPERTY + * (primary-constructor `val`, body property, `this.prop`) bind to the + * property's declared type, on both resolver venues (#1555). + * + * The cross-file Kotlin walk resolved a typed method PARAMETER by type, but + * a property-typed receiver fell to the name guess: the cross registry held + * the class from extraction defs (properties untyped, constructor vals + * absent), and the walk's typed AST registration of the same class was + * shadowed by that earlier entry. Each fixture call has a same-named decoy + * method on another type, so a name guess is visible as a wrong target: + * - Service.log: `audit.inserir()` on `audit: Audit`, next to + * `repository: Repo` whose interface also declares inserir (the + * sequential venue's guess picks Repo.inserir); + * - two bounded contexts whose use cases both inject `repository` + * (the parallel venue's receiver-name guess bound the feriado use case + * to CompromissoRepository.inserir, the #1555 report). + * One fixture, padded past MIN_FILES_FOR_PARALLEL, is indexed through the + * sequential resolver (CBM_INDEX_SINGLE_THREAD=1) and the parallel one. + */ +#include "../src/foundation/compat.h" +#include "test_framework.h" +#include "test_helpers.h" +#include "pipeline/pipeline.h" +#include "sqlite3.h" + +#include +#include +#include + +static const char *const KTC_FILES[][2] = { + {"src/main/kotlin/com/acme/a/Repo.kt", + "package com.acme.a\n\ninterface Repo {\n fun inserir(x: String): Long\n}\n"}, + {"src/main/kotlin/com/acme/b/Audit.kt", + "package com.acme.b\n\nclass Audit {\n fun inserir(x: String): Long = 0L\n}\n"}, + {"src/main/kotlin/com/acme/app/Service.kt", + "package com.acme.app\n\nimport com.acme.a.Repo\nimport com.acme.b.Audit\n\n" + "class Service(private val repository: Repo, private val audit: Audit) {\n" + " fun save(x: String): Long = repository.inserir(x)\n" + " fun log(x: String): Long = audit.inserir(x)\n}\n\n" + "class Holder {\n private val audit: Audit = Audit()\n" + " fun hold(x: String): Long = this.audit.inserir(x)\n}\n"}, + {"src/main/kotlin/com/acme/agenda/domain/CompromissoRepository.kt", + "package com.acme.agenda.domain\n\ninterface CompromissoRepository {\n" + " fun inserir(titulo: String): Long\n}\n"}, + {"src/main/kotlin/com/acme/feriado/domain/FeriadoRepository.kt", + "package com.acme.feriado.domain\n\ninterface FeriadoRepository {\n" + " fun inserir(nome: String): Long\n}\n"}, + {"src/main/kotlin/com/acme/agenda/app/CriarCompromissoUseCase.kt", + "package com.acme.agenda.app\n\nimport com.acme.agenda.domain.CompromissoRepository\n\n" + "class CriarCompromissoUseCase(private val repository: CompromissoRepository) {\n" + " fun executar(titulo: String): Long = repository.inserir(titulo)\n}\n"}, + {"src/main/kotlin/com/acme/feriado/app/CriarFeriadoUseCase.kt", + "package com.acme.feriado.app\n\nimport com.acme.feriado.domain.FeriadoRepository\n\n" + "class CriarFeriadoUseCase(private val repository: FeriadoRepository) {\n" + " fun executar(nome: String): Long = repository.inserir(nome)\n}\n"}, + /* Guard: inside an extension function, a bare call names the receiver's + * MEMBER before a same-named extension of the same file (Kotlin + * resolution); main gets this right and the own-file overlay must too. */ + {"src/main/kotlin/com/acme/b/Builders.kt", + "package com.acme.b\n\nclass Builder {\n fun add(x: Int): Boolean = true\n" + " fun put(k: String, v: Int): Int = v\n}\n\n" + "fun Builder.add(s: String): Boolean = add(s.length)\n\n" + "fun Builder.putAll(k: String) { put(k, 1) }\n\n" + "fun Builder.put(k: String): Int = put(k, 0)\n"}, + /* kotlin.Any members on a receiver typed by ANOTHER file. The per-file + * pass cannot type `type`, so it emits the kotlin.Any placeholder; the + * cross pass names the declared override. Both rows at one occurrence + * were a tombstone in the call join and the call fell to a name guess + * (AllTypes is the decoy). A class with no override keeps kotlin.Any, and + * so does a stdlib receiver type (`count: Int`), whose seeded + * kotlin.Int.toString names no node and left the call to the guess. */ + {"src/main/kotlin/com/acme/c/Proto.kt", + "package com.acme.c\n\nclass ProtoType {\n private val s: String = \"\"\n" + " override fun hashCode() = s.length\n\n override fun toString() = s\n}\n\n" + "class Plain(val n: Int)\n"}, + {"src/main/kotlin/com/acme/d/Member.kt", + "package com.acme.d\n\nimport com.acme.c.Plain\nimport com.acme.c.ProtoType\n\n" + "class Member(val type: ProtoType, val plain: Plain, val count: Int) {\n" + " override fun hashCode() = type.hashCode() * 37\n\n" + " fun label(): String = type.toString()\n\n" + " fun plainHash(): Int = plain.hashCode()\n\n" + " fun countText(): String = count.toString()\n}\n"}, + {"src/main/kotlin/com/acme/e/AllTypes.kt", + "package com.acme.e\n\nclass AllTypes {\n override fun hashCode(): Int = 1\n\n" + " override fun toString(): String = \"all\"\n}\n"}, + /* A property READ on a now-typed receiver (a nested class's property, whose + * QN the graph spells differently): it must keep the property's own node + * (USAGE) and never become a CALL_REFERENCE to a same-named function in + * another file (okhttp `server.webSocket` -> a test fun `webSocket`). */ + {"src/main/kotlin/com/acme/w/SocketTest.kt", + "package com.acme.w\n\nclass SocketTest {\n" + " private val server: Streams = Streams()\n\n" + " fun loop(): Boolean {\n return server.webSocket != null\n }\n\n" + " class Streams {\n var webSocket: String? = null\n }\n}\n"}, + {"src/main/kotlin/com/acme/w2/ModernTest.kt", + "package com.acme.w2\n\nclass ModernTest {\n fun webSocket(): Int = 1\n}\n"}, + /* A constructor val next to a same-named member function: a bare + * `handshake()` calls the function (the String? value has no invoke), as + * main resolves it; typing the val must not shadow the member (okhttp + * RealConnection). Other.handshake is the decoy a guess would hit. */ + {"src/main/kotlin/com/acme/h/Conn.kt", + "package com.acme.h\n\ninterface Connection {\n fun handshake(): String?\n}\n\n" + "class RealConnection(private val handshake: String?) : Connection {\n" + " override fun handshake(): String? = handshake\n\n" + " fun isEligible(): Boolean = check(handshake()!!.length)\n\n" + " private fun check(n: Int): Boolean = n > 0\n}\n"}, + {"src/main/kotlin/com/acme/h2/Other.kt", + "package com.acme.h2\n\nclass Other {\n fun handshake(): String? = null\n}\n"}, + /* Inside `repo.let { .. }` / `repo.run { .. }` the now-typed receiver Repo + * becomes the lambda's implicit receiver; the enclosing class's member + * stays callable bare (Kotlin's implicit-receiver chain). */ + {"src/main/kotlin/com/acme/l/Svc.kt", + "package com.acme.l\n\nimport com.acme.a.Repo\n\n" + "class Svc(private val repo: Repo) {\n fun ping(): Int = 2\n\n" + " fun viaLet(): Int = repo.let { ping() }\n\n" + " fun viaRun(): Int = repo.run { ping() }\n}\n"}, + {"src/main/kotlin/com/acme/l2/Pinger.kt", + "package com.acme.l2\n\nclass Pinger {\n fun ping(): Int = 3\n}\n"}, + /* Guard for the receiver chain: inside a companion object, `Ledger(x)` is + * a constructor call (here the secondary constructor), never a member + * method of the enclosing class. */ + {"src/main/kotlin/com/acme/k/Ledger.kt", + "package com.acme.k\n\nclass Ledger private constructor(val a: String, val b: String) {\n" + " constructor(a: String) : this(a, \"gen\")\n\n" + " companion object {\n fun of(x: String) = Ledger(x)\n }\n}\n"}, +}; + +static int ktc_write_fixture(const char *dir) { + for (size_t i = 0; i < sizeof(KTC_FILES) / sizeof(KTC_FILES[0]); i++) { + if (th_write_file(TH_PATH(dir, KTC_FILES[i][0]), KTC_FILES[i][1]) != 0) { + return -1; + } + } + /* Past MIN_FILES_FOR_PARALLEL (50): the parallel venue is selected. */ + for (int i = 0; i < 55; i++) { + char rel[96]; + char body[128]; + snprintf(rel, sizeof(rel), "src/main/kotlin/com/acme/pad/Pad%d.kt", i); + snprintf(body, sizeof(body), "package com.acme.pad\n\nclass Pad%d {\n fun v%d(): Int = %d\n}\n", + i, i, i); + if (th_write_file(TH_PATH(dir, rel), body) != 0) { + return -1; + } + } + return 0; +} + +/* Target QNs of the CALLS edges whose source QN ends in `.` and whose + * callee text ends in `.`, one per line into out; returns the count. */ +static int ktc_targets(sqlite3 *db, const char *src_tail, const char *leaf, char *out, + size_t cap) { + out[0] = '\0'; + static const char sql[] = + "SELECT t.qualified_name FROM edges e JOIN nodes s ON s.id = e.source_id " + "JOIN nodes t ON t.id = e.target_id WHERE e.type = 'CALLS' " + "AND s.qualified_name LIKE '%.' || ?1 " + "AND (json_extract(e.properties, '$.callee') = ?2 " + "OR json_extract(e.properties, '$.callee') LIKE '%.' || ?2) ORDER BY 1"; + sqlite3_stmt *st = NULL; + if (sqlite3_prepare_v2(db, sql, -1, &st, NULL) != SQLITE_OK) { + return -1; + } + sqlite3_bind_text(st, 1, src_tail, -1, SQLITE_STATIC); + sqlite3_bind_text(st, 2, leaf, -1, SQLITE_STATIC); + int n = 0; + while (sqlite3_step(st) == SQLITE_ROW) { + const char *qn = (const char *)sqlite3_column_text(st, 0); + size_t used = strlen(out); + snprintf(out + used, cap - used, "%s\n", qn ? qn : ""); + n++; + } + sqlite3_finalize(st); + return n; +} + +/* Exactly one such edge, and its target QN ends in `.`. */ +static int ktc_expect(sqlite3 *db, const char *venue, const char *src_tail, const char *leaf, + const char *want_tail) { + char got[2048]; + int n = ktc_targets(db, src_tail, leaf, got, sizeof(got)); + size_t gl = strlen(got); + size_t wl = strlen(want_tail); + if (n == 1 && gl > wl + 1 && got[gl - wl - 2] == '.' && + strncmp(got + gl - wl - 1, want_tail, wl) == 0) { + return 1; + } + printf("\n [%s] %s -> .%s: want one edge to ...%s, have %d:\n%s", venue, src_tail, leaf, + want_tail, n, got); + return 0; +} + +/* ktc_expect, and the edge comes from the resolver (an lsp_* strategy), not a + * name guess that happens to land on the same target. */ +static int ktc_expect_lsp(sqlite3 *db, const char *venue, const char *src_tail, const char *leaf, + const char *want_tail) { + if (!ktc_expect(db, venue, src_tail, leaf, want_tail)) { + return 0; + } + static const char sql[] = + "SELECT coalesce(json_extract(e.properties, '$.strategy'), '') FROM edges e " + "JOIN nodes s ON s.id = e.source_id WHERE e.type = 'CALLS' " + "AND s.qualified_name LIKE '%.' || ?1 " + "AND (json_extract(e.properties, '$.callee') = ?2 " + "OR json_extract(e.properties, '$.callee') LIKE '%.' || ?2)"; + sqlite3_stmt *st = NULL; + if (sqlite3_prepare_v2(db, sql, -1, &st, NULL) != SQLITE_OK) { + return 0; + } + sqlite3_bind_text(st, 1, src_tail, -1, SQLITE_STATIC); + sqlite3_bind_text(st, 2, leaf, -1, SQLITE_STATIC); + char strategy[64] = ""; + if (sqlite3_step(st) == SQLITE_ROW) { + const char *s = (const char *)sqlite3_column_text(st, 0); + snprintf(strategy, sizeof(strategy), "%s", s ? s : ""); + } + sqlite3_finalize(st); + if (strncmp(strategy, "lsp_", 4) == 0) { + return 1; + } + printf("\n [%s] %s -> .%s: right target, but strategy '%s' (want lsp_*)\n", venue, + src_tail, leaf, strategy); + return 0; +} + +/* The reference edges (USAGE / CALL_REFERENCE) from the method whose QN ends in + * `.` to nodes named `name`: exactly one, a USAGE onto a non-callable. */ +static int ktc_expect_property_usage(sqlite3 *db, const char *venue, const char *src_tail, + const char *name) { + static const char sql[] = + "SELECT e.type || ' ' || t.label || ' ' || t.qualified_name FROM edges e " + "JOIN nodes s ON s.id = e.source_id JOIN nodes t ON t.id = e.target_id " + "WHERE e.type IN ('USAGE', 'CALL_REFERENCE', 'CALLS') " + "AND s.qualified_name LIKE '%.' || ?1 AND t.name = ?2 ORDER BY 1"; + sqlite3_stmt *st = NULL; + if (sqlite3_prepare_v2(db, sql, -1, &st, NULL) != SQLITE_OK) { + return 0; + } + sqlite3_bind_text(st, 1, src_tail, -1, SQLITE_STATIC); + sqlite3_bind_text(st, 2, name, -1, SQLITE_STATIC); + char got[1024] = ""; + int n = 0; + bool ok = false; + while (sqlite3_step(st) == SQLITE_ROW) { + const char *row = (const char *)sqlite3_column_text(st, 0); + size_t used = strlen(got); + snprintf(got + used, sizeof(got) - used, "%s\n", row ? row : ""); + ok = row && strncmp(row, "USAGE ", 6) == 0 && strstr(row, " Function ") == NULL && + strstr(row, " Method ") == NULL; + n++; + } + sqlite3_finalize(st); + if (n == 1 && ok) { + return 1; + } + printf("\n [%s] %s -> %s: want one USAGE onto the property, have %d:\n%s", venue, src_tail, + name, n, got); + return 0; +} + +static int ktc_index(const char *dir, const char *db_name, bool single_thread, char *db_out, + size_t cap) { + snprintf(db_out, cap, "%s/%s", dir, db_name); + char *old_single = getenv("CBM_INDEX_SINGLE_THREAD"); + char *saved_single = old_single ? strdup(old_single) : NULL; + char *old_workers = getenv("CBM_WORKERS"); + char *saved_workers = old_workers ? strdup(old_workers) : NULL; + if (single_thread) { + cbm_setenv("CBM_INDEX_SINGLE_THREAD", "1", 1); + } else { + cbm_unsetenv("CBM_INDEX_SINGLE_THREAD"); + cbm_setenv("CBM_WORKERS", "4", 1); + } + int rc = -1; + cbm_pipeline_t *p = cbm_pipeline_new(dir, db_out, CBM_MODE_FULL); + if (p) { + rc = cbm_pipeline_run(p); + cbm_pipeline_free(p); + } + if (saved_single) { + cbm_setenv("CBM_INDEX_SINGLE_THREAD", saved_single, 1); + free(saved_single); + } else { + cbm_unsetenv("CBM_INDEX_SINGLE_THREAD"); + } + if (saved_workers) { + cbm_setenv("CBM_WORKERS", saved_workers, 1); + free(saved_workers); + } else { + cbm_unsetenv("CBM_WORKERS"); + } + return rc; +} + +static int ktc_check_venue(const char *dir, const char *venue, bool single_thread) { + char db_path[512]; + char name[64]; + snprintf(name, sizeof(name), "ktc-%s.db", venue); + if (ktc_index(dir, name, single_thread, db_path, sizeof(db_path)) != 0) { + printf("\n [%s] index failed\n", venue); + return 0; + } + sqlite3 *db = NULL; + if (sqlite3_open_v2(db_path, &db, SQLITE_OPEN_READONLY, NULL) != SQLITE_OK) { + if (db) { + sqlite3_close(db); + } + return 0; + } + int ok = 1; + /* Constructor vals, each next to a same-named decoy on the other type. */ + ok &= ktc_expect(db, venue, "Service.save", "inserir", "Repo.inserir"); + ok &= ktc_expect(db, venue, "Service.log", "inserir", "Audit.inserir"); + /* Body property read through `this`. */ + ok &= ktc_expect(db, venue, "Holder.hold", "inserir", "Audit.inserir"); + /* #1555: two ports sharing `inserir`, both injected as `repository`. */ + ok &= ktc_expect(db, venue, "CriarFeriadoUseCase.executar", "inserir", + "FeriadoRepository.inserir"); + ok &= ktc_expect(db, venue, "CriarCompromissoUseCase.executar", "inserir", + "CompromissoRepository.inserir"); + /* Guard: a member wins over a same-named extension of the same file. */ + ok &= ktc_expect(db, venue, "Builders.putAll", "put", "Builder.put"); + ok &= ktc_expect(db, venue, "Builders.add", "add", "Builder.add"); + /* kotlin.Any members: the declared override, else kotlin.Any (external). */ + ok &= ktc_expect_lsp(db, venue, "Member.hashCode", "hashCode", "ProtoType.hashCode"); + ok &= ktc_expect_lsp(db, venue, "Member.label", "toString", "ProtoType.toString"); + ok &= ktc_expect_lsp(db, venue, "Member.plainHash", "hashCode", "Any.hashCode"); + ok &= ktc_expect_lsp(db, venue, "Member.countText", "toString", "Any.toString"); + /* A typed property read keeps the property node, never a same-named function. */ + ok &= ktc_expect_property_usage(db, venue, "SocketTest.loop", "webSocket"); + /* A typed constructor val never shadows the same-named member function. */ + ok &= ktc_expect_lsp(db, venue, "RealConnection.isEligible", "handshake", + "RealConnection.handshake"); + /* The enclosing class's member inside a lambda with a typed receiver. */ + ok &= ktc_expect_lsp(db, venue, "Svc.viaLet", "ping", "Svc.ping"); + ok &= ktc_expect_lsp(db, venue, "Svc.viaRun", "ping", "Svc.ping"); + ok &= ktc_expect_lsp(db, venue, "Ledger.kt.__file__", "Ledger", "Ledger.Ledger.Ledger"); + sqlite3_close(db); + return ok; +} + +TEST(kotlin_property_receiver_binds_declared_type_both_venues) { + char *dir = th_mktempdir("cbm_kt_typed_calls"); + ASSERT_NOT_NULL(dir); + if (ktc_write_fixture(dir) != 0) { + th_cleanup(dir); + FAIL("fixture"); + } + int seq = ktc_check_venue(dir, "seq", true); + int par = ktc_check_venue(dir, "par", false); + th_cleanup(dir); + ASSERT_TRUE(seq); + ASSERT_TRUE(par); + PASS(); +} + +SUITE(kotlin_typed_calls) { + RUN_TEST(kotlin_property_receiver_binds_declared_type_both_venues); +} diff --git a/tests/test_main.c b/tests/test_main.c index d6e3c2cb2..e0526cf85 100644 --- a/tests/test_main.c +++ b/tests/test_main.c @@ -1812,6 +1812,7 @@ extern void suite_ts_lsp(void); extern void suite_java_lsp(void); extern void suite_java_lsp_coverage(void); extern void suite_kotlin_lsp(void); +extern void suite_kotlin_typed_calls(void); extern void suite_rust_lsp(void); extern void suite_store_arch(void); extern void suite_architecture_projection(void); @@ -2394,6 +2395,7 @@ int main(int argc, char **argv) { RUN_SELECTED_SUITE(perl_lsp); RUN_SELECTED_SUITE(py_lsp); RUN_SELECTED_SUITE(kotlin_lsp); + RUN_SELECTED_SUITE(kotlin_typed_calls); RUN_SELECTED_SUITE(rust_lsp); RUN_SELECTED_SUITE_PERF(py_lsp_bench); RUN_SELECTED_SUITE(py_lsp_stress); From 5992ea61a332dd8a11cddd59c562c1249310bf8f Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sun, 18 Oct 2026 17:14:04 +0200 Subject: [PATCH 15/15] fix(pipeline): size definition properties for every field A definition's properties buffer was sized for its docstring, variants and variant_key only; every other field had to share the remaining 2 KB. Fields that do not fit are dropped whole, so a long signature, decorator list or parameter list silently squeezed out the MinHash fingerprint (fp), the structural profile (sp) and the body tokens (bt) appended behind them. Those definitions got no SIMILAR_TO edge and fewer SEMANTICALLY_RELATED edges, and any newly added property (route_path, route_method) pushed more of them out. pd_props_buf and pp_props_buf (twins, kept in lockstep) now count the fixed metrics prefix plus the whole serialized form of every field build_def_props appends, including the reserve the fp/sp/bt room checks use. Definitions that fit keep the stack buffer. Signed-off-by: Martin Vogel --- src/pipeline/pass_definitions.c | 77 +++++++++++++++++++++++------ src/pipeline/pass_parallel.c | 77 +++++++++++++++++++++++------ tests/test_parallel.c | 86 +++++++++++++++++++++++++++++++++ 3 files changed, 210 insertions(+), 30 deletions(-) diff --git a/src/pipeline/pass_definitions.c b/src/pipeline/pass_definitions.c index 0f95c7989..62af154be 100644 --- a/src/pipeline/pass_definitions.c +++ b/src/pipeline/pass_definitions.c @@ -309,8 +309,8 @@ static void build_def_props(char *buf, size_t bufsize, const CBMDefinition *def) } size_t pos = (size_t)n; append_json_string(buf, bufsize, &pos, "docstring", def->docstring); - /* Right after the docstring: the buffer is sized for exactly these three - * uncapped fields (pd_props_buf), so none can be squeezed out. */ + /* The buffer is sized for every field below (pd_props_buf), so none can be + * squeezed out. */ append_json_raw(buf, bufsize, &pos, "variants", def->variants); append_json_string(buf, bufsize, &pos, "variant_key", def->variant_key); append_json_string(buf, bufsize, &pos, "signature", def->signature); @@ -353,24 +353,71 @@ static void build_def_props(char *buf, size_t bufsize, const CBMDefinition *def) } } -/* A def's properties buffer: CBM_SZ_2K for every other field plus the whole - * serialized docstring, variants and variant_key fields, which have no length - * cap (a field that does not fit is dropped whole). Returns `stack` for a def - * with none of them, or when the larger buffer cannot be allocated. Twin of - * pass_parallel.c -- keep both in sync. */ -static char *pd_props_buf(const CBMDefinition *def, char *stack, size_t *size) { - size_t need = *size; - if (def->docstring && def->docstring[0]) { - need += strlen("docstring") + def_json_escaped_len(def->docstring) + PD_JSON_FIELD_OVERHEAD; +/* Upper bound of build_def_props' fixed prefix: the Function/Method metrics + * literal (about 330 B) with nine ints and seven true/false/test_role strings at + * their widest stays under 600 B. */ +enum { PD_PROPS_PREFIX_MAX = CBM_SZ_1K }; + +/* Serialized size of one optional field as append_json_string emits it + * (0 when the field is absent). */ +static size_t pd_str_field_len(const char *key, const char *val) { + return val && val[0] ? strlen(key) + def_json_escaped_len(val) + PD_JSON_FIELD_OVERHEAD : 0; +} + +/* Serialized size of a string array as append_json_str_array emits it. */ +static size_t pd_arr_field_len(const char *key, const char **arr) { + if (!arr || !arr[0]) { + return 0; } + size_t n = strlen(key) + PD_JSON_FIELD_OVERHEAD; + for (int i = 0; arr[i]; i++) { + n += def_json_escaped_len(arr[i]) + PD_ESC_SPACE + (i > 0 ? SKIP_ONE : 0); + } + return n; +} + +/* A def's properties buffer, sized so EVERY field build_def_props appends fits: + * the fixed metrics prefix (PD_PROPS_PREFIX_MAX) plus each optional field's + * whole serialized form. A field that does not fit is dropped whole, so a buffer + * sized for only some of them let a long signature, decorator list or parameter + * list squeeze out the MinHash fingerprint ("fp"), structural profile ("sp") and + * body tokens ("bt") behind them: no SIMILAR_TO edge and fewer + * SEMANTICALLY_RELATED edges for exactly the widest definitions (jellyfin's + * ImageController actions, 2026-10-18). fp/sp/bt are appended behind their own + * room checks, so both the check's reserve and the field are counted. Returns + * `stack` when everything fits it, or when the larger buffer cannot be + * allocated. Twin of pass_parallel.c -- keep both in sync. */ +static char *pd_props_buf(const CBMDefinition *def, char *stack, size_t *size) { + size_t need = PD_PROPS_PREFIX_MAX + pd_str_field_len("docstring", def->docstring); if (def->variants && def->variants[0]) { need += strlen("variants") + strlen(def->variants) + PD_JSON_FIELD_OVERHEAD; } - if (def->variant_key && def->variant_key[0]) { - need += - strlen("variant_key") + def_json_escaped_len(def->variant_key) + PD_JSON_FIELD_OVERHEAD; + need += pd_str_field_len("variant_key", def->variant_key) + + pd_str_field_len("signature", def->signature) + + pd_str_field_len("return_type", def->return_type) + + pd_str_field_len("parent_class", def->parent_class) + + pd_arr_field_len("decorators", def->decorators) + + pd_arr_field_len("base_classes", def->base_classes) + + pd_arr_field_len("param_names", def->param_names) + + pd_arr_field_len("param_types", def->param_types) + + pd_str_field_len("route_path", def->route_path) + + pd_str_field_len("route_method", def->route_method) + + pd_str_field_len("http_client", def->http_client) + + pd_str_field_len("http_base_url", def->http_base_url); + for (const char **kv = def->extra_props; kv && kv[0] && kv[SKIP_ONE]; kv += PAIR_LEN) { + need += pd_str_field_len(kv[0], kv[SKIP_ONE]); } - if (need == *size) { + if (def->fingerprint && def->fingerprint_k > 0) { + need += CBM_MINHASH_HEX_LEN + CBM_MINHASH_JSON_OVERHEAD + SKIP_ONE; + } + if (def->structural_profile) { + need += CBM_AST_PROFILE_BUF + pd_str_field_len("sp", def->structural_profile) + SKIP_ONE; + } + if (def->body_tokens) { + need += CBM_SZ_512 + pd_str_field_len("bt", def->body_tokens) + SKIP_ONE; + } + need += PD_ESC_SPACE; /* the closing '}' + NUL */ + if (need <= *size) { return stack; } char *buf = cbm_alloc(CBM_MEM_CLASS_GBUF_STRING, need); diff --git a/src/pipeline/pass_parallel.c b/src/pipeline/pass_parallel.c index 2c0a9023a..ac2632ecf 100644 --- a/src/pipeline/pass_parallel.c +++ b/src/pipeline/pass_parallel.c @@ -539,8 +539,8 @@ static void build_def_props(char *buf, size_t bufsize, const CBMDefinition *def) } size_t pos = (size_t)n; append_json_string(buf, bufsize, &pos, "docstring", def->docstring); - /* Right after the docstring: the buffer is sized for exactly these three - * uncapped fields (pp_props_buf), so none can be squeezed out. */ + /* The buffer is sized for every field below (pp_props_buf), so none can be + * squeezed out. */ append_json_raw(buf, bufsize, &pos, "variants", def->variants); append_json_string(buf, bufsize, &pos, "variant_key", def->variant_key); append_json_string(buf, bufsize, &pos, "signature", def->signature); @@ -731,24 +731,71 @@ typedef struct { * to the response/logfile — this only throttles the stderr noise). */ enum { PP_OVERSIZED_WARN_MAX = 32 }; -/* A def's properties buffer: CBM_SZ_2K for every other field plus the whole - * serialized docstring, variants and variant_key fields, which have no length - * cap (a field that does not fit is dropped whole). Returns `stack` for a def - * with none of them, or when the larger buffer cannot be allocated. Twin of - * pass_definitions.c -- keep both in sync. */ -static char *pp_props_buf(const CBMDefinition *def, char *stack, size_t *size) { - size_t need = *size; - if (def->docstring && def->docstring[0]) { - need += strlen("docstring") + pp_json_escaped_len(def->docstring) + PP_JSON_FIELD_OVERHEAD; +/* Upper bound of build_def_props' fixed prefix: the Function/Method metrics + * literal (about 330 B) with nine ints and seven true/false/test_role strings at + * their widest stays under 600 B. */ +enum { PP_PROPS_PREFIX_MAX = CBM_SZ_1K }; + +/* Serialized size of one optional field as append_json_string emits it + * (0 when the field is absent). */ +static size_t pp_str_field_len(const char *key, const char *val) { + return val && val[0] ? strlen(key) + pp_json_escaped_len(val) + PP_JSON_FIELD_OVERHEAD : 0; +} + +/* Serialized size of a string array as append_json_str_array emits it. */ +static size_t pp_arr_field_len(const char *key, const char **arr) { + if (!arr || !arr[0]) { + return 0; } + size_t n = strlen(key) + PP_JSON_FIELD_OVERHEAD; + for (int i = 0; arr[i]; i++) { + n += pp_json_escaped_len(arr[i]) + PP_ESC_SPACE + (i > 0 ? SKIP_ONE : 0); + } + return n; +} + +/* A def's properties buffer, sized so EVERY field build_def_props appends fits: + * the fixed metrics prefix (PP_PROPS_PREFIX_MAX) plus each optional field's + * whole serialized form. A field that does not fit is dropped whole, so a buffer + * sized for only some of them let a long signature, decorator list or parameter + * list squeeze out the MinHash fingerprint ("fp"), structural profile ("sp") and + * body tokens ("bt") behind them: no SIMILAR_TO edge and fewer + * SEMANTICALLY_RELATED edges for exactly the widest definitions (jellyfin's + * ImageController actions, 2026-10-18). fp/sp/bt are appended behind their own + * room checks, so both the check's reserve and the field are counted. Returns + * `stack` when everything fits it, or when the larger buffer cannot be + * allocated. Twin of pass_definitions.c -- keep both in sync. */ +static char *pp_props_buf(const CBMDefinition *def, char *stack, size_t *size) { + size_t need = PP_PROPS_PREFIX_MAX + pp_str_field_len("docstring", def->docstring); if (def->variants && def->variants[0]) { need += strlen("variants") + strlen(def->variants) + PP_JSON_FIELD_OVERHEAD; } - if (def->variant_key && def->variant_key[0]) { - need += - strlen("variant_key") + pp_json_escaped_len(def->variant_key) + PP_JSON_FIELD_OVERHEAD; + need += pp_str_field_len("variant_key", def->variant_key) + + pp_str_field_len("signature", def->signature) + + pp_str_field_len("return_type", def->return_type) + + pp_str_field_len("parent_class", def->parent_class) + + pp_arr_field_len("decorators", def->decorators) + + pp_arr_field_len("base_classes", def->base_classes) + + pp_arr_field_len("param_names", def->param_names) + + pp_arr_field_len("param_types", def->param_types) + + pp_str_field_len("route_path", def->route_path) + + pp_str_field_len("route_method", def->route_method) + + pp_str_field_len("http_client", def->http_client) + + pp_str_field_len("http_base_url", def->http_base_url); + for (const char **kv = def->extra_props; kv && kv[0] && kv[SKIP_ONE]; kv += PAIR_LEN) { + need += pp_str_field_len(kv[0], kv[SKIP_ONE]); } - if (need == *size) { + if (def->fingerprint && def->fingerprint_k > 0) { + need += CBM_MINHASH_HEX_LEN + CBM_MINHASH_JSON_OVERHEAD + SKIP_ONE; + } + if (def->structural_profile) { + need += CBM_AST_PROFILE_BUF + pp_str_field_len("sp", def->structural_profile) + SKIP_ONE; + } + if (def->body_tokens) { + need += CBM_SZ_512 + pp_str_field_len("bt", def->body_tokens) + SKIP_ONE; + } + need += PP_ESC_SPACE; /* the closing '}' + NUL */ + if (need <= *size) { return stack; } char *buf = cbm_alloc(CBM_MEM_CLASS_GBUF_STRING, need); diff --git a/tests/test_parallel.c b/tests/test_parallel.c index a8c0618df..fc97332e6 100644 --- a/tests/test_parallel.c +++ b/tests/test_parallel.c @@ -847,6 +847,91 @@ TEST(parallel_args_json_no_overflow) { PASS(); } +/* F14: a definition's properties buffer was sized for its docstring and variants + * only. A wide signature (50 annotated parameters) filled the rest of the 2 KB, and + * the MinHash fingerprint "fp", structural profile "sp" and body tokens "bt" + * appended behind it were dropped whole: no SIMILAR_TO edge and fewer + * SEMANTICALLY_RELATED edges for exactly the widest definitions (jellyfin's + * ImageController actions, 2026-10-18). `small` has the same body and is the + * control that the fixture produces all three at all. Both resolvers. */ +static const char f14_body[] = + " total = 0\n" + " for item in range(10):\n" + " if item % 3 == 0:\n" + " total += item * 2\n" + " elif item % 5 == 1:\n" + " total -= len(str(item))\n" + " else:\n" + " total ^= hash((item, total))\n" + " values = [x * x for x in range(total % 7 + 3)]\n" + " mapping = {str(k): v for k, v in enumerate(values)}\n" + " while total > 100:\n" + " total //= 3\n" + " try:\n" + " result = sorted(mapping.items(), key=lambda kv: kv[1])\n" + " except ValueError as err:\n" + " result = [(\"error\", str(err))]\n" + " return result, total, mapping\n"; + +static bool f14_props_have(const cbm_gbuf_t *gb, const char *project, const char *name, + const char *venue) { + char *qn = cbm_pipeline_fqn_compute(project, "pkg/wide.py", name); + const cbm_gbuf_node_t *n = qn ? cbm_gbuf_find_by_qn(gb, qn) : NULL; + free(qn); + const char *p = n && n->properties_json ? n->properties_json : ""; + bool fp = strstr(p, "\"fp\":\"") != NULL; + bool sp = strstr(p, "\"sp\":\"") != NULL; + bool bt = strstr(p, "\"bt\":\"") != NULL; + bool sig = strstr(p, "\"signature\":\"") != NULL; + printf(" f14 %s %s: found=%d len=%zu fp=%d sp=%d bt=%d signature=%d\n", venue, name, n != NULL, + strlen(p), fp, sp, bt, sig); + return n && fp && sp && bt && sig; +} + +TEST(parallel_wide_definition_keeps_fp_sp_bt_f14) { + char params[CBM_SZ_4K]; + size_t pos = 0; + for (int i = 0; i < 50; i++) { + pos += (size_t)snprintf(params + pos, sizeof(params) - pos, "%sp%02d: Dict[str, int] = None", + i ? ", " : "", i); + } + char src[CBM_SZ_8K]; + snprintf(src, sizeof(src), "def small(a, b):\n%s\n\ndef wide(%s):\n%s", f14_body, params, + f14_body); + const char *project = "cbm_f14"; + char tmpdir[256]; + snprintf(tmpdir, sizeof(tmpdir), "/tmp/cbm_par_f14_XXXXXX"); + if (!cbm_mkdtemp(tmpdir)) { + FAIL("mkdtemp failed"); + } + char path[512]; + snprintf(path, sizeof(path), "%s/pkg/wide.py", tmpdir); + if (th_write_file(path, src) != 0) { + th_rmtree(tmpdir); + FAIL("failed to write the F14 fixture"); + } + cbm_file_info_t files[1] = {0}; + files[0].path = path; + files[0].rel_path = (char *)"pkg/wide.py"; + files[0].language = CBM_LANG_PYTHON; + cbm_gbuf_t *graphs[2] = {run_sequential(project, tmpdir, files, 1), + run_parallel(project, tmpdir, files, 1, 2)}; + ASSERT_NOT_NULL(graphs[0]); + ASSERT_NOT_NULL(graphs[1]); + bool all = true; + for (int g = 0; g < 2; g++) { + const char *venue = g == 0 ? "sequential" : "parallel"; + bool control = f14_props_have(graphs[g], project, "small", venue); + bool wide = f14_props_have(graphs[g], project, "wide", venue); + all = all && control && wide; + } + cbm_gbuf_free(graphs[0]); + cbm_gbuf_free(graphs[1]); + th_rmtree(tmpdir); + ASSERT_TRUE(all); + PASS(); +} + /* ── Graph buffer merge tests ─────────────────────────────────────── */ TEST(gbuf_shared_ids_unique) { @@ -5346,6 +5431,7 @@ SUITE(parallel) { RUN_TEST(parallel_post_extract_projection_spills_before_resolve); RUN_TEST(parallel_empty_files); RUN_TEST(parallel_args_json_no_overflow); + RUN_TEST(parallel_wide_definition_keeps_fp_sp_bt_f14); /* Cleanup shared state */ parity_teardown();