The grammar rejects from __future__ import * as Python does, so the file carries an error node and its whole namespace is a PARSE unknown: a stronger barrier than the UNSUPPORTED statement the previous commit guarded for. That guard could never run and is dropped; the control asserts the parse barrier the parser actually yields.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
py_lsp.c: #2697 moved py_lookup_attribute below the C3 code (own members, C3 order, then the depth-first walk) and declares it up front; the trivial definition this branch still carried next to its module-member lookup is dropped, the module-member block is kept, and the C3 section takes main's header.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The namespace facts proved only plain one-line strings safe, so every triple-quoted string statement (a module, class or attribute docstring) was an unknown EFFECT, and from __future__ import x was UNSUPPORTED. A barrier withdraws every name bound before it: sqlalchemy's sql/expression.py lost its 149 re-exports to the docstring after nullsfirst = nulls_first, and a docstring made an otherwise plain class unproven. A string literal now evaluates nothing in any quoting or prefix (r, b, u, rb), implicit concatenations included; an f-string with a replacement field stays an effect (__format__/__str__ may run code). from __future__ import x [as y] binds like a from-import of module __future__; a star import from __future__ is a syntax error and stays unsupported.
CBM_PY_NAMESPACE_FACTS_VERSION moves to 2. The version travels in the lsp_surface py payload and a payload of another version does not decode, so the first incremental run on an index of an older binary falls back to a full re-index (delta.rehydrate_failed -> CBM_PIPELINE_FORCE_FULL_REINDEX) instead of mixing old and new facts. The codec test now spells the current version and rejects a version-1 payload.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A string-literal statement has no effect and from __future__ import x binds x, but the namespace facts recorded both as unknown statements: a triple-quoted string as an EFFECT, a future import as UNSUPPORTED. sqlalchemy's sql/expression.py lost its 149 re-exports to a docstring after nullsfirst = nulls_first. The extraction test pins every string form (module, class and attribute docstrings, single and triple quotes, r/b/rb/u prefixes, implicit concatenation, an f-string without a replacement field) and the future import forms (alias, parentheses); controls: an f-string with a field (also inside a concatenation), a call, and a star import from __future__ stay barriers. The pipeline fixture adds the expression.py shape end to end and an f-string control package.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
flatten_trait_into_class looked up each trait's methods by scanning every
function of the registry, and every flattened method appends more of them.
A file with many trait-using classes paid O(classes x functions), and the
scan grew with its own output.
The class-field pass now keeps a receiver -> function-id index, synced
incrementally as functions are registered, and flattening reads only the
trait's own entries registered before the current class. If the index
cannot be built, the previous full scan runs unchanged. Only entries
registered before flattening started are visited, so the output is
identical.
Tests: the work-counter tests for #1265 (100 -> 200 trait-using classes,
ratio gate 2.5) and #765 (trait collision shapes stay bounded) use the
existing test-only work ceiling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
py_lsp.c: the module-member lookup of this branch and the C3 super() lookup of #2696 were added at the same place; both are kept.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A small incremental closure resolves each changed file through a per-file
selection of defs (its own and its imported modules') instead of the shared
registry. A package that re-exports a name keeps only the record there; the
definition lives in the module the record points at, possibly through more
re-exports, so a call through the package found nothing after an
incremental re-index of the caller alone. cbm_pxc_filter_defs_for_file now
follows every selected re-export or star record of a Python module to the
module it names (relative exactly, absolute under the longest matching
prefix, a member as a submodule) and selects that module's defs too, once
per record, so the per-file LSP folds the same chain the shared registry
does.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The re-export records of a package come from its namespace facts. An
incremental re-index that touches only the caller must still find them for
the unchanged packages (restored from their stored facts). Both drivers now
also run as a full index followed by an incremental re-index of app/main.py
alone, with the same expectations.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first D1 change fed the Go cross-file registrar the extractor's
return_types. Those are cleaned names for type-reference edges:
clean_type_name strips `[]` and `*`. A cross-file single result `[]T`
therefore reached the registrar as `T`. `xs := pkg.Generate(); xs[0].M()`
lost its typing and fell back to a name guess. On prometheus the base
binds `sample[0].T()` to chunks.Sample.T, but the first D1 change bound
it to tsdb.T.
pass_lsp_cross.c now leaves a single result's raw text alone, as before
D1; the registrar already parses `*T` and `[]T`. Only a result LIST
`(...)` is split, at its top-level commas, into raw element types:
- `(*T, error)` -> `*T|error`;
- the named `(t *T, err error)` -> `*T|error`;
- the grouped `(a, b int)` -> `int|int`.
`chan`, `func`, `map`, `struct` and `interface` never count as result
names. A list that does not split cleanly keeps its raw text.
Tests: the D1 fixture gains three call sites, each with two same-named
decoys:
- a single slice result indexed (`xs[0].TS()`);
- a tuple with a slice element;
- grouped named results.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Python surface-envelope test (pipeline round 0) checks that the
persisted row is the versioned codec envelope by looking for the
literal "v":2. With SURFACE_CODEC_VERSION at 3, the envelope reads
"v":3. The assertion keeps its meaning (the row is the versioned
envelope and the def is present) and follows the version.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cross-file pass handed every registrar the raw result text of a
def (pass_lsp_cross.c: `dst->return_types = src->return_type`). The Go
registrar splits that string on '|' (go_lsp.c split_pipe_types), so it
got `(*Encoder, error)` as one piece. That piece did not parse, and a
cross-file callee with more than one result, or with a named result
`(e *Encoder)`, had no usable return type. As a result,
`e, err := enc.NewEncoder()`, `h, _ := newHead()` and
`_, d := diag.Run()` all left the variable untyped, and calls on it fell
to name guesses. Same-file calls were fine, because they read the AST.
Go defs now pass the parsed result list (extract_go_multi_return),
'|'-joined: `*Encoder|error`, and `*Encoder` for the named result. Other
languages keep the single raw string.
The persisted LSP surface carries this string, so SURFACE_CODEC_VERSION
goes from 2 to 3: existing caches re-extract Go surfaces once. The
literal version-2 surfaces in the codec tests move to 3, so that they
keep testing the envelope rule they name rather than the version
mismatch.
Tests: pipeline_go_multi_return_receiver_is_typed (+_parallel). A tuple
result, a named result, a tuple-bound local reached through a field,
and a named slice type each must bind the right method with an lsp_*
strategy and none of its two same-named decoys.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter forbids a new raw allocator site. Growing the def
array for a Python file's namespace records needed a realloc because a
spilled result's facts were taken as not peekable; its header carries the
fact stream's status and size, though. cbm_pxc_collect_all_defs now adds
each Python module's record bound to the first count (one header peek per
spilled result, as the def count already took), writes the records into
that reserved space, and needs no growth: the realloc and its bookkeeping
are gone. An import-only package (zero defs) still gets its records.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter counts raw allocator calls per file and fails when a
file grows. The TYPE_CHECKING change freed the copied local name of an
annotation-only import with a new raw free in two import-map builders.
build_import_map now asks the predicate on the name's span in the edge
properties before copying it (cbm_imports_local_span_binds_runtime_name,
which cbm_imports_local_binds_runtime_name now wraps), and the cross-LSP
builder routes an annotation-only name through its existing no-value path.
Same behaviour, no new raw allocator site.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Review round 3 of #2677. The user decided on a rule sound by construction:
a change of maps the render does not run exactly leaves every later image
of that render UNKNOWN, with no exemption.
Changes of maps (helm_template.c):
- Under UNKNOWN control -- a skipped if / with / range body, the rest of a
loop an UNKNOWN break / continue may leave, an include or template not
run, a tpl not run -- any set / unset / merge (the must* forms too), any
tpl, and any include / template of a name not known, of no define or
template file of the chart, or of one that may change maps, leaves the
images after it UNKNOWN.
- An executed set / merge does too unless its target is a known map that
is poisoned exactly; a merge also for a target already poisoned, the
.Values root, or a map not known within it. The own provenance and the
freshness scan are gone.
- .Template.BasePath is modelled as Helm sets it: <chart path>/templates,
the top chart's name, then /charts/<alias or name> per subchart
(pass_k8s.c). An include naming one of the chart's template files by it
(the checksum idiom) is scanned with the same rule; a file reached again
while scanned, or no such file, may change maps.
Bounds (helm_template.c):
- A capture's keys are a hash set and its mark is built in one pass (one
sort for the prefix check), its steps charged.
- Each render works in an arena of its own; the caller keeps its results
only.
- The trip reason is recorded where a budget trips: an output trip is
logged as output.
Releases (pass_terraform.c, pass_k8s.c):
- Any set_sensitive leaves the release without an image of its own (the
name-overlap check is gone), and so does a postrender written as a
dynamic block or an attribute.
Cost on the 20 infra corpora against 5c5964ef: 20 Image edges lost, 0
added, no budget trip. 5 are changes the rule rightly cannot follow (a
second mustMerge into a poisoned values map; a skipped part that may
change maps), 15 come from checksum-included files that hold a tpl. Two
controls of rounds 1-3 now read UNKNOWN; their tests say so.
Tests: one per item, each failing on 5c5964ef by its assertion (the
BasePath ones also on the coarse rule without it); the wrong-image
scenarios of rounds 2 and 3 stay as tests.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings in the base branch with main 658639ec merged (#2685, Rust
generic-impl call scope), so this branch merges cleanly into main
again. No conflicts.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings in main 658639ec (#2685, Rust generic-impl call scope). One
conflict in tests/test_extraction.c: main's new F-R4 test and its
helper sit right before the flag-exempt test, whose comment this
branch rewrote (Go now sets is_method, so the guarded language is
Rust). Both are kept: main's block, then this branch's comment.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two PDF doc-link tests named a Go method by its flat QN. With #1913 a
Go method's QN carries its receiver, so the lookups found no node.
The resolver is unchanged: for a Go method, rel(parent_class).name now
equals the QN, so the method binds through its QN, like a member of
any other language.
- pdf_links_pipeline: page 1 names `server.Handler.Serve`; the edge
target pkg.server.Serve becomes pkg.server.Handler.Serve. Same
assertions: pdf_qn syntax, unique tier.
- pdf_qn_method_through_owner, page 2: `time.Provider.Now` binds
pkg.time.Provider.Now (was pkg.time.Now), syntax pdf_qn.
- pdf_qn_method_through_owner, page 1: `time.Now` must not bind the
method. The target moves to pkg.time.Provider.Now too. Against the
old flat QN, no node exists any more, so the zero count would have
passed vacuously. The page 2 assertion proves that the node exists.
The ownerless-QN rule in doc_links_pdf.c stays: a templ method keeps
the flat package QN while it carries parent_class (extract_defs.c only
receiver-qualifies CBM_LANG_GO). Its comment no longer says that Go
methods have no receiver.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A namespace fact stream is checked by cbm_py_namespace_facts_valid, which
accepts a shadow only for the value, decorated, unproven-class and
unsupported reasons. The withdrawn TYPE_CHECKING re-export carried the
compound reason, so the whole stream was rejected and the package published
no names at all (a lazy-module package lost its TYPE_CHECKING exports). It
now carries the unsupported reason: the builder cannot tell which source the
name has at runtime. python_namespace_facts_type_checking_alternatives pins
both shapes as complete, valid streams.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The re-export test checks a table of calls in one loop; a failure only
reported the assertion line. Each failing call is now named, with its
venue.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package namespace read in the type-checker view takes the consequence of
`if TYPE_CHECKING:` as executed (lazy-module packages publish their names
there). The compat shape imports a name there and again in `else: try:
from runtime.place import name`: the module's runtime export comes from the
else branch, so publishing the TYPE_CHECKING source as the re-export target
was a guess. Every name an elif/else import binds now becomes a shadow fact
(the re-export is withdrawn, as for a later assignment); a star import or an
import form the builder does not read there makes the whole namespace
unknown. The branch is walked iteratively.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_run_php_lsp never finalized its type registry, so every lookup during
single-file resolution scanned the stdlib plus every definition of the
file linearly: O(lookups x definitions), quadratic in the size of one
file. The cross-file path already indexes its registry.
The registry is now finalized with first-registered semantics (the first
registration of a duplicate QN wins), which keeps exactly the answers of
the linear scan, and re-indexed once after trait flattening adds methods.
The index lives in a scratch arena that dies with the call.
A test-only work ceiling (CBM_LSP_WORK_CEILING_HIT, seam builds only) lets
the work-counter test stop a reverted, unbounded loop deterministically
instead of hanging the runner; product builds have no ceiling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_pxc_collect_all_defs sizes the def array by every file's def count and
appends a Python file's namespace records behind its defs. The growth check
only made room for the records themselves, so the records consumed the slack
reserved for later files and their defs overflowed the array (ASan:
heap-buffer-overflow in pipeline_python_package_reexports_resolve_by_lsp).
The array now grows to keep every later file's defs reserved on top of the
records placed so far, geometrically, so the copying stays linear.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package `__init__.py` re-exports its submodules' names (`from .fields
import *`, `from .x import Orig as Renamed`), and callers reach them as
module attributes (`models.CharField()`) or through `from pkg import Name`.
The Python namespace facts already capture each module's top-level bindings
in order; nothing consumed them, so those calls fell to name guesses that
pick the first same-named definition under the package.
- Records: cbm_pxc_collect_all_defs replays each module's COMPLETE fact
stream into re-export, star-import and `__all__` records next to its
defs, including for an import-only `__init__.py` (zero defs). Records
carry the raw module spec; the Python registry resolves it (relative
exactly, absolute by unique scope tail, else unknown).
- Fold: the shared Python registry folds the records once, after finalize
and before the seal, into a sorted per-module export table (binary
search; no lookup in the registration loop). Order-deterministic; a
cycle contributes what it has so far.
- States (conservative): an incomplete fact stream publishes nothing; a
shadow, delete or whole-namespace unknown (a top-level try/if/with)
withdraws the names it covers; a star whose source cannot be enumerated
publishes nothing; a literal `__all__` restricts a star, else public
names only.
- Consult: the Python LSP looks a module attribute up in the export table
on a miss (calls, expression types, callable values) and when binding
`from pkg import Y` where Y is no def of pkg.
- Namespace facts read a module-level `if TYPE_CHECKING:` consequence as
executed: lazy-module packages publish their names there.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A lazy-module package publishes its names only under `if TYPE_CHECKING:`
(the else branch installs the lazy module): `lz.Lazy()` resolves to the
re-exported class. The compat shape imports a name under TYPE_CHECKING and
again in `else: try:`: the runtime binding comes from the else branch, so
`cmp.Shim()` must not reach the TYPE_CHECKING-side class.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package __init__.py re-exports its submodules' names (explicit, star,
`__all__`-restricted, renamed, chained through a subpackage, cyclic stars,
the django `from .fields import *` shape) and callers reach them as module
attributes (`pkg.Explicit()`) or through `from pkg import Name`. Each call
must resolve by an lsp_* strategy to the re-exported definition. Names a
star does not export (`__all__`-unlisted, private), a re-export the module
shadows later, and one before a whole-namespace barrier must not. Plain
from-imports of the defining module are the control. Both venues.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The compat shape imports a name under `if TYPE_CHECKING:` and again in the
else branch, usually inside try/except ImportError. The name is bound at
runtime there, so its TYPE_CHECKING import is no annotation-only import:
recording it flagged made every CALLS import map skip the name and lost the
calls and bases that bind through the runtime import (airflow mask_secret,
OpenLineageRedactor(SecretsMasker)). The imports of the elif/else branches
are collected (any depth, iteratively), every TYPE_CHECKING import whose
local name they bind is dropped with them, and a star import there drops the
whole block: those names keep the behaviour they had before #1277.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The airflow compat shape imports a name under `if TYPE_CHECKING:` and again
in the else branch, usually inside try/except ImportError: the name is bound
at runtime, so its TYPE_CHECKING import is not annotation-only and must stay
unrecorded, as the runtime import itself does. A star import in an elif or
else branch may bind any name. A pure annotation import stays flagged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The import an annotated field's type comes from usually sits under a
module-level `if TYPE_CHECKING:` (to avoid an import cycle).
parse_python_imports walked only the module's direct statements, so that
import was never recorded and the reporter's chain
trainer.strategies.process_batch() lost its type exactly where real code
writes it.
- Extraction records the imports of a module-level `if TYPE_CHECKING:` /
`if <module>.TYPE_CHECKING:` consequence block, flagged
CBMImport.type_checking_only. elif/else branches stay unrecorded, as
every other compound statement's imports do.
- One rule for every CALLS import-map builder:
cbm_import_binds_runtime_name() (false for a flagged import). The
sequential build_import_map applies it in its fast and slow paths; the
shared cbm_pxc_build_import_map (= _ex(..., false)) applies it in both
loops, so no CALLS import map ever binds an annotation-only name.
- The annotation consumers take cbm_pxc_build_import_map_ex(..., true):
the cross-file base/field-type resolution and the Python LSP dispatch.
The sequential dispatch always takes it; the parallel one builds that
map only for a file that records a flagged import and otherwise passes
its CALLS map, which is identical there by construction.
- The Python LSP binds those names into an annotation-only frame consulted
by annotation resolution after the ordinary scope chain: a runtime call
through such a name never resolves.
Tests: python_type_checking_imports_flagged_issue1277 (extraction),
python_type_checking_typed_field_issue1277(_parallel) (pipeline, both
venues; controls: no import_map and no LSP CALLS edge through the name).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The second half of #1277: the reporter's typed-field chain with both
annotation imports under `if TYPE_CHECKING:` (the usual shape when the
import would cycle), through `typing.TYPE_CHECKING` too. The call resolves
by the LSP as with runtime imports, and a runtime call through a
TYPE_CHECKING-only name gets no CALLS edge at all. The extraction test reads
the new CBMImport.type_checking_only flag, declared here and not yet set.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Claude Code defers every MCP tool behind ToolSearch by default: at session
start the model sees only tool names and server instructions, while Grep and
Read are always loaded. Codex does the same and shows the server instructions
as the namespace description of the deferred tools. Agents therefore reached
for grep before the graph tools were even loaded (#509, discussion #2025).
- tools/list: search_graph, trace_path and get_code_snippet carry
_meta["anthropic/alwaysLoad"] = true (a documented per-tool hint), so the
structural discovery path loads upfront while the other 14 tools stay
deferred. About 4.6K characters of schema; clients that do not know the key
ignore it, as MCP reserves _meta.
- Server instructions (default profile) now state, keywords first and as
facts, which structural questions the graph answers and where grep fits
better: 309 -> 999 characters, within Claude Code's 4,096-character cut.
- SessionStart/SubagentStart: the untrusted-metadata label now covers the
graph-derived project name only. Wrapping the routing sentences too told the
model to treat cbm's own guidance as data. The other sentences are unchanged;
sanitization is unchanged.
The initialize byte budget is raised deliberately from 512 B to 1.5 KiB (1,188
bytes now); tools/list stays within its 21 KiB budget (20,938 bytes).
Refs #509
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel