A config key linked to every code symbol whose name contained it. In
YAML, JSON and HCL files that guess was retired in WP0.9; TOML and INI
files kept it, and in 17 infrastructure repositories every one of their
72 key_symbol edges was a coincidence: `.gitleaks.toml useDefault` linked
to TestConfig_UseDefault and three other symbols, a toml-merge tool's test
data `config_path` to 46 `*ConfigPath` constants all over deckhouse,
`grafana.ini admin_user` to user-cluster constants, a Rust Cargo.toml
dependency to other services' go.mod requirements.
A .toml, .ini, .cfg or .conf key now links like an infrastructure key:
by exact name (camelCase and snake_case still normalize to one key) and
only to code inside the config file's component, its directory or the
one around a config/, conf/, settings/ or etc/ directory. .env and
.properties keys keep substring matching; the tests that covered it on a
TOML key now use a .properties key.
Measured on the 17 repositories: key_symbol 95 -> 23, the 23 being the
infrastructure edges WP0.9 reviewed, the 72 removed all TOML/INI and all
coincidences, no other edge or node changed; deterministic over three
runs; the incremental re-index equals a full index for INI, TOML, code
and tfvars edits.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
key_symbol linked a config key to every code symbol whose normalized name
equalled or merely CONTAINED it, anywhere in the repository, over at most
4,096 config keys x 8,192 code symbols -- the first ones of a QN-sorted
list, so which links existed depended on what else the repository held.
In 17 infrastructure repositories 4 of 40 sampled edges were right, all
exact and inside one directory tree: a stage's JSON schema "configured"
unrelated module variables, a chart's `apiVersion` every `*.apiVersion`
helper, `config_path` every Go `*ConfigPath`.
Per the decision for infrastructure files (YAML, JSON, HCL, tfvars):
- a pair with either side in such a file links by exact name only, and
only when the code file lies in the config file's component -- its
directory tree, or the tree around a conventional config folder
(config/, conf/, settings/, ...);
- an HCL symbol takes part only as a Terraform/OpenTofu `variable`, by its
declared name (`variable.<name>` matches `<name>`); outputs, modules,
resources and .tfvars assignments are not what a config key sets.
Other config formats keep exact + substring matching anywhere.
No count decides content any more: the collectors keep every candidate,
and an exact-name index plus a trigram index (a key's substring
candidates are the symbols holding its rarest trigram) replace the
all-pairs scan, so the cost follows the matches.
On real input, key_symbol edges against main: cloud-foundation-fabric
9,892 -> 1, prometheus-community/helm-charts 4,148 -> 0, deckhouse
2,631 -> 49, gardener 1,677 -> 0, kubermatic 739 -> 7, microservices-demo
376 -> 0, kubespray 156 -> 6, k8s.io 218 -> 0, opentelemetry-demo
71 -> 26. Every infra edge left was reviewed: kubespray's gcp
tfvars.json sets its module's variables (6/6), a project template's
YAML its variables, flags.json its generated flags, a values.yaml key
its Makefile variable; one miss -- a repository-root config sees the
whole repository. The rest are TOML/INI/Cargo substring matches, which
this change leaves as they were.
Tests: infra exact-only, component scope, config-folder scope, the
Terraform variable rule, and no config/code cap (4,200 keys; 9,000
candidates) fail on the old matcher. camel_case now matches a JSON key
exactly (its substring form is what infra files no longer do); the
order-independence test also asserts that all 9,000 candidates link.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
collect_config_entries and collect_code_entries fill fixed-capacity arrays
and stop at max_out. They walk gbuf label indexes in insertion order, which
under parallel extraction is worker-merge order and varies run to run, so on
any repository with more candidates than the cap the surviving set — and
therefore the emitted CONFIGURES edges — depended on thread scheduling.
Sort candidates by a pure content key (qualified_name, file_path, start_line,
name) before the cap applies, so the surviving set is a function of the inputs
alone. Tie-breaks are content-only: node ids are handed out in merge order and
belong in no canonical comparator.
Also log configlinker.truncated when a collector fills to capacity, so the cap
reports itself instead of dropping candidates silently.
Measured on a 4191-file TypeScript corpus (12940 candidates against the 8192
code cap): before, 3-61 CONFIGURES edges differed per run against a
single-threaded reference and no two multi-threaded runs agreed; after, the
single-threaded reference and four multi-threaded runs are byte-identical.
Corpora below the cap were already stable, and all divergence was CONFIGURES.
Signed-off-by: Andrew Studnicky <a.j.studnicky@gmail.com>
- cbm_mkdtemp: _mktemp + _mkdir on Windows
- cbm_setenv/cbm_unsetenv: _putenv_s on Windows
- cbm_pipe: _pipe on Windows
- Replace bare POSIX calls in 12 test files with compat wrappers
- All test files now include compat.h
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Martin Vogel
Andrew Studnicky
Martin Vogel