push_nested_class_nodes descended into field/template/declaration and conditional wrappers but not into Python's decorated_definition, so a decorated nested class (`@dataclass class Meta:` inside a class) was never extracted: no Class node, no Method nodes, and the calls in it fell to the FILE node.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A coverage map must cover every function it lists, and five of the seven
variants covered only function 1, so the map failed to parse and the test
stopped at its setup ("map is NULL") with the fix applied or not. The
changed function (0) is now covered by the setup row of a suite no runner
registers, so alpha's own rows stay quiet and each variant still reaches
alpha by exactly one path. RED with the selection fix reverted (alpha
lacks CONDITIONAL), green with it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A runner suite registered only under a preprocessor condition (the four
coverage_* suites under `#if defined(CBM_TEST_COVERAGE) && !defined(_WIN32)`
in tests/test_main.c) is absent from builds without that condition. The
selection marked such a suite CONDITIONAL only on the per-registration path,
and only when a registration of its own was conditional and contributed a
reason. Every suite-level path (setup unknown or hit, suite-body, changed and
rule triggers, inventory, rejected artifact) made the suite whole without it,
and the runner registration's own condition was never recorded.
PR CI on the x86 Linux gcc leg then failed with "the test selection names
suites this build does not register: coverage_setup_success
coverage_setup_uncertain" (#2679, #2680): the suite-body trigger and an
incomplete setup row selected both suites whole without CONDITIONAL, so
selection.py left them out of optional-suites.txt.
The model now records whether a runner registration sits under a condition
(conditional only when every site of the suite is), and the selection marks
any selected conditionally registered suite whole with CONDITIONAL after every
whole-suite path. Unconditionally registered suites keep the strict contract.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cross-file Kotlin walk resolved a call on a typed method parameter by
type, but a call on a class property - a primary-constructor val, a body
property or `this.prop` - fell to the name guess. The cross registry held
each own-file class from its extraction defs, whose properties carry no type
(constructor vals none at all), and the walk's typed AST registration of the
same class was added behind that entry under the same QN, so every lookup
returned the field-less one. Constructor injection is the common Kotlin
dependency shape, so `repository.inserir(...)` on an injected port bound to
whichever same-named method the guess found, including an implementation of
another port (#1555).
The walk now registers the file's own declarations into an overlay chained
to the cross registry and checked first, indexed once the top-level
declarations are collected. The sole-definer fallback walks the whole chain
so its candidate set is unchanged.
Typing more receivers exposed resolution gaps that the untyped receivers had
hidden; each would otherwise turn a typed call into a wrong name guess:
- a member of the receiver class wins over a same-named extension of the
same file (Kotlin resolution order; the overlay holds both);
- `x.hashCode()` / `toString()` / `equals()` on a receiver typed by another
file: the per-file pass emits the kotlin.Any placeholder, the cross pass
the declared override, and two targets at one occurrence are ambiguous to
the call join. The per-file placeholder is dropped when the cross pass
resolves the same occurrence;
- the same members on a stdlib or JDK type (`count.toString()` on an Int)
resolve to the kotlin.Any builtin instead of a seeded stdlib member that
has no graph node;
- a property READ on a typed receiver whose QN the graph spells differently
(a nested class's property) never becomes a CALL_REFERENCE to a
same-named function: the reference join falls back by name to
non-callable targets only, which binds the property's own node;
- a typed value without an `invoke` operator (`private val handshake:
Handshake?`) no longer shadows the same-named member function;
- inside a lambda whose typed receiver becomes `this` (`repo.let { .. }`),
and inside a member extension, the enclosing class's members stay
callable bare (the implicit-receiver chain); a call naming a class stays
a constructor call.
Fixes#1555
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Conflict in pass_definitions.c: process_def gained main's link_parents and this branch's module_prefix_len; both kept.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The shared Tier-2 build now registers Rust definitions only. Two
fixtures that feed cbm_rust_build_cross_registry left CBMLSPDef.lang
zero, which the pipeline never does (pass_lsp_cross.c tags every def
with its file's language). The other shared-registry fixtures in this
file already set CBM_LANG_RUST. Set it on these two as well.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Since #1548 a def declared inside a Python test function is extracted, so its calls start at the helper instead of the test FILE node, whose name passed the test-name check. 4,641 of 6,315 TESTS edges lost on the corpus were this shape. TESTS for a call from such a helper is now attributed to the nearest enclosing test function of the same file; a helper nested in a non-test function stays without TESTS, like a module-level helper.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pylsp_nested_and_local_classbody_calls_join_extracted_caller asserted that a class declared in a function body is NOT a def, which is the behaviour #1548 fixes. Their class-body calls still join the enclosing function; that part is unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
At a compound statement's join the Python LSP invalidates what the branch bound, so the local def under if True: is resolved by the registry's lexical_scope step; pin that producer instead of an lsp_ strategy. Found at the branch's first build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package __init__.py can define a function whose QN equals a submodule's
(poetry: layouts/__init__.py `def layout` is the QN prefix of every symbol in
layouts/layout.py). The prefix walks that recognise a def declared inside a
function body accepted that other file's function as the enclosing def, so
class Layout and its methods read as locals of layout(): they left the
short-name index and the import symbol fallback, and the answer depended on
which file registered first (discovery order on a full index, node-id order on
an incremental one).
Every walk now stops at the symbol's own module:
- registration (pass_definitions.c, pass_parallel.c, and the incremental seed)
passes the defining file's symbol prefix length,
cbm_pipeline_fqn_symbol_prefix_len, computed without allocating, to
cbm_registry_add_lang_in_module; cbm_registry_add_lang keeps its signature and
does no local detection;
- cbm_registry_resolve_lexical stops at the caller's module QN;
- the import fallback's local test only counts an enclosing def in the same file.
Also moves the #1548 registry tests out of the test they had been inserted into.
Tests: registry_python_local_bound_at_own_file_issue1548 (both registration
orders), fqn_symbol_prefix_len_matches_fqn_compute,
pipeline_python_init_function_shares_submodule_qn{,_parallel}_issue1548 (src/
layout: the import and calls onto Layout survive, then again after an
incremental re-index).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`Opts::write(true)` is an associated function. When the Rust LSP cannot
place it, the registry can (import_map or qualified_suffix onto
fsx.Opts.write). The two free-function short-name fallbacks (the
cross-crate member lookup and the project-wide lsp_short_name_unique)
ran first, though. They bound the call to the project's only free
`write`, and that LSP row pre-empted the right answer.
While the Rust registry also held other languages' definitions, a
same-named JS or Perl function often made the free name ambiguous and
hid this. With Rust-only registries the hijack replaced 516
qualified_suffix and 84 import_map edges on 7 Rust repos.
A call whose qualifying segment names a type now skips both fallbacks.
That covers an ASCII-uppercase initial, `Self`, and a qualified
`<T as Tr>::f` path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both Rust registries were built from the pipeline's mixed-language def
set. The shared Tier-2 build (cbm_rust_build_cross_registry) and the
per-file conversion (pxc_lspdefs_to_rust) copied every def, so a Perl
`sub new` or any C function became a Rust free function. The
lsp_short_name_unique fallback then bound `Store::new()` to the Perl
constructor at confidence 0.90 on both venues: its "crate scope" check
compares only the first module-QN segment, which is the project name.
(Found by the generic-engine lane on a 12-language fixture.)
Register Rust defs only, at both conversion points, exactly as the Go
builder filters to Go. Rust impl-relation records already carry
CBM_LANG_RUST. As a side effect, the shared Rust build no longer
registers the C defs of a mixed C+Rust tree.
Tests: pipeline_rust_call_never_binds_another_language (sequential and
parallel): no CALLS from the Rust caller into the Perl file, and the
Rust associated function still binds.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The sequential pipeline (50 files or fewer, one worker, CBM_WORKERS=1)
linked a method's DEFINES_METHOD edge while it was still registering
files. A method defined in a file the directory walk returns before its
type's file -- a Go method in another file of its package, an out-of-line
C++ method in foo.cpp before foo.h -- found no parent and lost the edge,
and with it the IMPLEMENTS and OVERRIDE edges derived from it. The
parallel registry build links after every node exists, so the same input
gave different graphs depending on repository size and worker count, and
on the order the filesystem enumerates a directory.
With the extraction cache, DEFINES_METHOD and diagram parent edges are now
linked in one pass after Phase 1, as the parallel registry build does.
Without the cache (allocation failure) each file still links its own.
The test pair puts each of two Go types in the file of the other's
method, so one method precedes its type whatever the walk order: RED on
main for the sequential venue (alpha_method == 0), and the venue-parity
case compares the padded input through both resolvers.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`import pkg.mod.nodes` binds the ROOT name `pkg`, so `class X(pkg.mod.nodes.Context)`
names module pkg.mod.nodes's Context. The cross-LSP base resolution asked
the registry, whose import_map step joined the imported MODULE (the
IMPORTS edge target, pkg.mod.nodes) with the whole remaining spelling
(mod.nodes.Context) and missed; the base kept its raw spelling, so the
Python LSP's inheritance view never saw it. Every member inherited through
a dotted base fell to a name guess or to nothing: self.m() became
unique_name, super().m() suffix_match, super().__init__() only resolved
through a synthesized QN.
pxc_resolve_base_qn now resolves such a spelling against the module the
import names: the import map's module when the root local is unique there,
else the dotted module path under the project's QN rule. Python only,
import-backed and exact (no short-name step), and only a type-like project
node is accepted. `import a.b as m` and `from a import b` bind the module
itself and keep resolving through the import map.
Tests: python_dotted_base_inheritance(_parallel): self.m(), super().m()
and super().__init__() through a dotted base resolve by the LSP; the
from-imported control keeps resolving; an untyped receiver stays without
an LSP edge.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Re-port onto current main (agreed with lane L7):
- The Python LSP binds every def/class a function body defines as a local
before walking the body (LEGB): an undecorated def as an exact callable
alias to its nested QN, anything else as a hard shadow, so no call falls
through to a same-named module symbol. Defs inside the body's compound
statements count too, and their bodies are now walked.
- Classes declared in a function are walked under the function's QN
(py_process_class_in), so super()/self calls in their methods resolve.
- cbm_registry_resolve_lexical moves AFTER the LSP override: both resolvers
call one entry point, cbm_registry_resolve_call_scoped, at the same place
(before the language-aware chain). It never uses the per-file resolve cache.
- Main's cbm_registry_add_lang carries the def-local exclusion, so every
registration site (full, parallel, incremental seed) applies it; the
separate cbm_registry_add_for_language and the second incremental language
derivation are dropped.
- The pass_pkgmap.c def-local import skip is its own continue.
Tests: local-vs-module shadowing in both orders, a decorated local, and a
function-local class's super()/self calls, sequential and parallel.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Why
The Python def walk stopped at every function_definition: it extracted
the def and never looked at its body. Every def or class declared inside
a function -- closures, decorator wrappers, the route handlers of a Flask
or FastAPI app factory (`@app.route` inside create_app), local classes and
their methods -- was absent from the graph. Blocks at module level (if /
try / for / with) were transparent, so only function bodies were blind.
The calls inside those defs were not re-attributed either: the call-scope
walk named a nested def by its class or module only (`pkg.mod.nested`, a
QN no node had), so the edges fell back to the file node, and inbound
traces of the called function silently lost the caller. A def nested in a
method got the method's own QN (`Tree.visit` for the `def visit` inside
method Tree.visit), so its calls, recursion included, landed on the
method. On an app-factory codebase this removes the whole HTTP layer.
Fix
- Extraction (extract_defs.c): after a Python def is extracted, its body
is walked with the def's QN as the enclosing scope; methods' bodies are
walked under the method QN. A nested def or class is qualified by the
def it sits in -- pkg.mod.create_app.index, pkg.mod.Tree.visit.visit,
pkg.mod.create_app.Settings.value -- the QN the Python LSP already gives
the calls inside it (py_process_function). Decorated nested defs keep
their decorators, so their route_path mints the Route and HANDLES.
Lambdas and comprehensions hold no statements and stay as they are.
- Call scope (extract_unified.c): for Python the nearest named scope, a
def or a class, prefixes a nested def or class, so every call, usage and
env read inside a nested def carries the nested def's QN.
- A def nested in a function is a local of it (Python LEGB), so it must
not become a global name: a Python symbol whose QN extends a
Function/Method node is registered for exact lookups only, never in the
short-name index (registry.c cbm_registry_add_for_language, used by the
full, parallel and incremental registration sites alike). Without this,
django's per-test `class MyModelAdmin` and every decorator's `def
wrapper` became unique_name / suffix_match targets for unrelated calls.
The incremental seed derives each stored file's language the way
discovery does (name, then shebang), so it keeps the same symbols out.
The import symbol-name fallback skips such locals too (pass_pkgmap.c),
and the Python LSP no longer binds a def-local class module-wide by its
short name (py_lsp.c).
- A bare call now resolves lexically first (registry.c
cbm_registry_resolve_lexical, strategy lexical_scope): the def or class
bound in the nearest enclosing function, skipping class bodies, stopping
at the module. The sequential and parallel call passes use it
identically, and so does the handler lookup of a call-registered route:
`app.add_url_rule("/ping", "ping", ping)` in create_app reaches the
local ping.
Proof (real code; every node and edge diffed, main 1f9b4db1 vs this)
- pallets/flask examples/tutorial (flaskr, create_app factory): the
nested `hello` handler is now create_app.hello with HANDLES to
/hello; 4 nested defs/classes added, 0 nodes and 0 edges removed.
- pallets/flask d73fa1c (tests declare handlers inside test functions):
Function 532 -> 1068, Method 318 -> 381, Class 130 -> 209. The +678
equals the 678 unique nested def/class QNs Python's own ast finds.
Handler HANDLES (decorators and add_url_rule) 32 -> 296. 0 nodes
removed, 0 duplicate QNs.
The 94 removed CALLS: 90 moved from the file node onto the nested def
that makes the call, 4 replaced by the lexically bound local.
- django fb11376 (local, 2,927 .py files): nodes 55,483 -> 60,052,
Function 1,992 -> 3,027, Method 28,850 -> 29,491, Class 8,187 ->
11,073. 0 nodes removed and 0 duplicate QNs. Every added
Function/Method/Class sits under a Function/Method of the same file,
and they match ast's 4,567 nested QNs except for 5 methods declared in
an `if` inside a local class body (class-body `if` is not walked for
any class). CALLS 62,430 -> 63,815. Every added CALLS edge has a nested
def at one end: 1,015 from nested defs, 1,219 lexical_scope into a
local, and none from outside the local's function. Of the 849 removed:
507 moved from the file node onto the nested def, 334 replaced by the
local the name binds, and 8 were false on main, all checked (e.g. the
recursive `get_actions` nested in call_command had been suffix-matched
to ModelAdmin.get_actions). HANDLES unchanged (135).
- Samples checked against the source:
AbstractBaseUser.check_password.setter -> set_password / save;
View.as_view.view -> setup / dispatch;
Apps.lazy_model_operation -> lazy_model_operation.apply_next_model
(line 426);
InstrumentedASGIRequest.META -> the local InstrumentedDict (line 279);
flask test_provide_automatic_options_arg_disable.index -> GET /.
Tests
- extraction: extract_python_nested_def_qualified_by_enclosing_def,
_calls_attributed_to_it, _decorated_nested_route_handler,
_nested_class, _nested_def_named_like_its_method,
_lambda_and_comprehension_stay_in_enclosing_def (control),
_nested_def_env_access_attributed_to_it (all _issue1548).
- pipeline: pipeline_python_app_factory_routes, _nested_def_is_function_
local, _local_classes_stay_in_their_def, _import_never_binds_a_local_def,
_nested_defs_parallel (>= 50 files, CBM_WORKERS=4),
_nested_def_local_after_incremental (all _issue1548).
- registry: registry_python_function_local_is_exact_only,
registry_resolve_lexical_walks_enclosing_defs (both _issue1548).
The extraction and pipeline tests were RED on main 3/3 for the asserted
reason and are green with the fix; the registry tests cover the new
registry API. Each production piece was reverted on its own, and each
revert turns at least one of these tests RED.
Fixes#1548
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
(cherry picked from commit 62fda43fd66200879db62a58903a32dab0dd1a20)
README.md, the site files and pkg/npm/README.md are read by test_cli, so
a change to one now runs the test legs (T1-tests). The September replay
held 15 README pushes to the docs tier the analysis assigned and failed
every test leg's contract step. Those rows are now a documented
deviation, read-by-tests, which like makefile-by-name may only raise a
tier: 767 pushes as expected, 53 documented fail-safe deviations, all 22
real failures still caught.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first commit of this branch is the Smart CI change as of the rebase
onto 9f85f104. A review of it found places where the engine could run
fewer tests than a change needs; each now runs everything or selects the
right tests, with a test that fails when its fix is reverted.
Engine:
- A `#define NAME "literal"` line counted as naming a role but selected
nobody, so the tests that start the blocking-git probe through their
marker macros would not have run. The scan now follows NAME, as a whole
word, to the code that uses it, through chains of up to 4 macros; a
define it cannot follow runs everything.
- main dispatches through the role table, so no edge leads from main to a
handler. A table entry the engine cannot read, or whose handler is not
exactly one function, was skipped; it now runs everything.
- In a runner file, only a test case names a role's literal. The runner's
own helper that reads the marker no longer counts as a spawner, so a
role only the runner names runs everything again.
- A handler reached first by a parse gap or by the spawn lift was never
lifted. Role lifts and parse gaps now run to a fixpoint, before the
runner check and again after the spawn lift.
- A reached sink that main calls (the blocking-git probe) counted as a way
into main and hid the real one: #2644 now names the start-up path main
-> cbm_index_supervisor_capture_build_fingerprint.
- No pointer past the end of a role line; the RUNNER_REACHED note clamps
its prefix.
CI:
- pkg/npm/README.md, which test_cli reads, runs the test legs instead of
the docs tier.
- The build rule lists the harness scripts the test legs run:
run-test-wave.py, path-safety.sh, clean.sh.
- The revert guard rejects the ancestor's coverage when a diff fails.
- The vetoed selection is uploaded (test-selection-vetoed).
Docs: SMART_CI.md states the new fail-safe cases, the version that lacks
the new config keys, the selection base, and that the narrowed leg builds
with gcc while the coverage comes from clang.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every release surface of the version-metadata contract states the
newest tag: the Chocolatey nuspec and installer (checksum re-pinned to
the released windows-amd64 archive), the Scoop manifest (both Windows
archive hashes), the Homebrew formula (all four archive hashes; Linux
installs the static -portable builds), the Go installer, the Nix flake,
server.json, and the npm and PyPI wrappers. Every hash is taken from the
release's checksums.txt.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.12.1 release smoke failed on every leg in Phase 13w, the
package-wrapper install gate added in #2664:
- macOS: the runner images have no Go on PATH, and
SMOKE_REQUIRE_WRAPPERS=1 rightly fails without one. pr-smoke and the
three release smoke jobs now set up Go from pkg/go/go.mod, with the
same pinned step the wrapper test job uses.
- Windows: `mkdir -m 700` changes the mode in a second step, which MSYS2
cannot do under the smoke profile's owner-only DACL. mktemp -d sets
the mode at creation.
- Linux: the runner's older Go downloaded the 1.26.1 toolchain into the
smoke profile's HOME. Go writes its module cache read-only, so the
harness could not delete its profile afterwards. All Go state now stays
in the wrapper's own work tree, and its cleanup restores write
permission before removing it.
npm and PyPI passed on every leg that reached them; the gate itself is
unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel