From eca1a004b4d3c5235f7bd50fa50872914cfb68fe Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Wed, 14 Oct 2026 22:09:37 +0200 Subject: [PATCH] feat(configlink): TOML and INI keys link by exact name in their component A config key linked to every code symbol whose name contained it. In YAML, JSON and HCL files that guess was retired in WP0.9; TOML and INI files kept it, and in 17 infrastructure repositories every one of their 72 key_symbol edges was a coincidence: `.gitleaks.toml useDefault` linked to TestConfig_UseDefault and three other symbols, a toml-merge tool's test data `config_path` to 46 `*ConfigPath` constants all over deckhouse, `grafana.ini admin_user` to user-cluster constants, a Rust Cargo.toml dependency to other services' go.mod requirements. A .toml, .ini, .cfg or .conf key now links like an infrastructure key: by exact name (camelCase and snake_case still normalize to one key) and only to code inside the config file's component, its directory or the one around a config/, conf/, settings/ or etc/ directory. .env and .properties keys keep substring matching; the tests that covered it on a TOML key now use a .properties key. Measured on the 17 repositories: key_symbol 95 -> 23, the 23 being the infrastructure edges WP0.9 reviewed, the 72 removed all TOML/INI and all coincidences, no other edge or node changed; deterministic over three runs; the incremental re-index equals a full index for INI, TOML, code and tfvars edits. Signed-off-by: Martin Vogel --- README.md | 1 + src/pipeline/pass_configlink.c | 19 ++++++---- tests/test_configlink.c | 68 +++++++++++++++++++++++++++++----- 3 files changed, 70 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 0f09953d4..27262a079 100644 --- a/README.md +++ b/README.md @@ -336,6 +336,7 @@ The install script placed beside the binary is **reported, not deleted**: uninst - `DEPLOYS`: what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`, a Terraform `kubernetes_*` resource the object it declares (a `Resource` like a manifest's, `ecosystem: terraform`, when its name is a literal), a kustomization inflating a chart (`helmCharts`; its values file an `IMPORTS`). A chart outside the repository is an external `Chart`, keyed by its publisher when its repository URL is stated (`__chart__/`, the same node in every project, the URL in `repository`), else one per project and name; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, `helmcharts`, ...) - `DEPENDS_ON`: a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name - `CONFIGURES`: code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile`; values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`): the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore` +- `CONFIGURES` with `strategy: key_symbol`: a code symbol to the config key it names (`config_key`). A YAML, JSON, HCL (Terraform, OpenTofu, tfvars), TOML or INI key links by exact name (`maxRetryCount` = `max_retry_count`) and only to code inside the config file's component: its directory, or the one around a `config/`, `configs/`, `conf/`, `settings/` or `etc/` directory. A `.env` or `.properties` key also links by substring (confidence 0.75) - `OVERRIDE` with `via: patch`: a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace, label selector); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build - `OVERRIDE` with `via: namePrefix` / `nameSuffix`: an object a kustomization renames, from its variant: a `Resource` named with the deployed name (`capi-` + `controller-manager`; the prefixes of nested kustomizations outermost first, suffixes innermost first) in the namespace that build gives it, `ecosystem: kustomize`, which the kustomization's file `DEFINES`. CRDs, Namespaces and API services keep their names, as kustomize does. The edge keeps `kustomization` and `line` - `USAGE` with `via: terraform`: a Terraform / OpenTofu reference (`var.X`, `local.X`, `module.X`, `data.T.N`, `T.N`) from the top-level block holding it to the block of its module (the `.tf` / `.tofu` files of its directory) that declares it (a `locals` entry: its `locals` block); an entry of the block's `depends_on` list is a `DEPENDS_ON` instead. The edge keeps `ref` (`var`, `local`, `module`, `data`, `resource`) and `line`; a name only mentioned in string text is no reference. A module call with a local source (`module "net" { source = "./modules/net" }`) `IMPORTS` (`local_name`: the module's name) the `Folder` of that directory (the root directory: the `Project`), and `module.net.id` (also `module.net[0].id`, `module.net[*].id`) `USAGE`s (`ref: output`) the `output "id"` there, beside `module.net` diff --git a/src/pipeline/pass_configlink.c b/src/pipeline/pass_configlink.c index 5c1b41654..d4cdf3582 100644 --- a/src/pipeline/pass_configlink.c +++ b/src/pipeline/pass_configlink.c @@ -116,15 +116,18 @@ static const cbm_gbuf_node_t **canonical_node_copy(const cbm_gbuf_node_t *const return sorted; } -/* Infrastructure files: YAML, JSON and HCL (Terraform, OpenTofu, tfvars). A - * key_symbol pair with either side in one links by exact name and only inside - * the config file's component (user decision 2026-10-10): in 17 infra - * repositories 4 of 40 sampled key_symbol edges were right, all of them exact - * and within one directory tree; the rest were substring and cross-component - * coincidences. Other config formats keep exact + substring matching. */ +/* Files whose key_symbol pairs link by exact name and only inside the config + * file's component: infrastructure files -- YAML, JSON, HCL (Terraform, + * OpenTofu, tfvars) -- (user decision 2026-10-10: in 17 infra repositories 4 + * of 40 sampled key_symbol edges were right, all of them exact and within one + * directory tree; the rest were substring and cross-component coincidences), + * and TOML and INI files (user decision 2026-10-14: all 72 of their substring + * or cross-component edges in those repositories were coincidences, such as + * `.gitleaks.toml useDefault` <- TestConfig_UseDefault). Other config formats + * (.env, .properties) keep exact + substring matching. */ static bool cl_is_infra_path(const char *path) { - static const char *const exts[] = {".yaml", ".yml", ".json", ".tf", - ".hcl", ".tofu", ".tfvars", NULL}; + static const char *const exts[] = {".yaml", ".yml", ".json", ".tf", ".hcl", ".tofu", + ".tfvars", ".toml", ".ini", ".cfg", ".conf", NULL}; const char *dot = path ? strrchr(path, '.') : NULL; const char *slash = path ? strrchr(path, '/') : NULL; if (!dot || (slash && dot < slash)) { diff --git a/tests/test_configlink.c b/tests/test_configlink.c index c7248a8bc..e665e99de 100644 --- a/tests/test_configlink.c +++ b/tests/test_configlink.c @@ -85,7 +85,8 @@ static void rm_rf(const char *path) { /* ── Strategy 1: Config Key → Code Symbol ───────────────────────── */ /* Go: TestConfigKeySymbol_ExactMatch - * config.toml has max_connections, main.go has getMaxConnections() */ + * config.toml has max_connections, main.go has maxConnections(). (A TOML key + * links by exact name only, see configlink_key_symbol_toml_ini_exact_scoped.) */ TEST(configlink_key_symbol_exact_match) { cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test"); @@ -94,9 +95,9 @@ TEST(configlink_key_symbol_exact_match) { "test.config.max_connections", "config.toml", 0, 0, NULL); ASSERT_GT(cfg_id, 0); - /* Code Function node: getMaxConnections from main.go */ - int64_t func_id = cbm_gbuf_upsert_node(gb, "Function", "getMaxConnections", - "test.main.getMaxConnections", "main.go", 0, 0, NULL); + /* Code Function node: maxConnections from main.go */ + int64_t func_id = cbm_gbuf_upsert_node(gb, "Function", "maxConnections", + "test.main.maxConnections", "main.go", 0, 0, NULL); ASSERT_GT(func_id, 0); /* Run configlink */ @@ -113,12 +114,14 @@ TEST(configlink_key_symbol_exact_match) { } /* Go: TestConfigKeySymbol_SubstringMatch - * config.toml has request_timeout, main.go has getRequestTimeoutSeconds() */ + * app.properties has request_timeout, main.go has getRequestTimeoutSeconds(). + * (Formats outside the exact + component rule -- .properties, .env -- keep + * substring matching.) */ TEST(configlink_key_symbol_substring_match) { cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test"); - cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", "test.config.request_timeout", - "config.toml", 0, 0, NULL); + cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", "test.app.request_timeout", + "app.properties", 0, 0, NULL); cbm_gbuf_upsert_node(gb, "Function", "getRequestTimeoutSeconds", "test.main.getRequestTimeoutSeconds", "main.go", 0, 0, NULL); @@ -281,6 +284,48 @@ TEST(configlink_key_symbol_terraform_variable) { PASS(); } +/* TOML and INI keys (.toml, .ini, .cfg, .conf) link like infrastructure keys: + * by exact name and only inside the config file's component. In 17 infra + * repositories all 72 TOML/INI key_symbol edges were substring or + * cross-component coincidences (`.gitleaks.toml useDefault` <- + * TestConfig_UseDefault, `grafana.ini admin_user` <- GenAdminUser). Each + * format's key below reaches only svc-a's exact `requestTimeout`: not svc-a's + * substring `getRequestTimeoutSeconds`, not svc-b's exact `requestTimeout`. */ +TEST(configlink_key_symbol_toml_ini_exact_scoped) { + static const char *const files[] = {"svc-a/config.toml", "svc-a/settings.ini", + "svc-a/setup.cfg", "svc-a/app.conf"}; + enum { FILES = 4 }; + cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test"); + int64_t cfg[FILES]; + for (int i = 0; i < FILES; i++) { + char qn[CBM_SZ_128]; + snprintf(qn, sizeof(qn), "test.cfg%d.request_timeout", i); + cfg[i] = cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", qn, files[i], 0, 0, NULL); + } + int64_t inside = cbm_gbuf_upsert_node(gb, "Function", "requestTimeout", + "test.svc_a.main.requestTimeout", "svc-a/main.go", 0, 0, + NULL); + cbm_gbuf_upsert_node(gb, "Function", "getRequestTimeoutSeconds", + "test.svc_a.client.getRequestTimeoutSeconds", "svc-a/client.go", 0, 0, + NULL); + cbm_gbuf_upsert_node(gb, "Function", "requestTimeout", "test.svc_b.main.requestTimeout", + "svc-b/main.go", 0, 0, NULL); + run_configlink(gb, "test", NULL); + const cbm_gbuf_edge_t **edges = NULL; + int count = 0; + cbm_gbuf_find_edges_by_type(gb, "CONFIGURES", &edges, &count); + ASSERT_EQ(count, FILES); + for (int i = 0; i < FILES; i++) { + bool found = false; + for (int e = 0; e < count; e++) { + found = found || (edges[e]->source_id == inside && edges[e]->target_id == cfg[i]); + } + ASSERT_TRUE(found); + } + cbm_gbuf_free(gb); + PASS(); +} + /* No count of config keys decides what links: the collector stopped at 4,096. */ TEST(configlink_key_symbol_no_config_cap) { enum { KEYS = 4200 }; @@ -469,9 +514,11 @@ enum { CL_DET_CANDIDATES = 9000 }; /* > the 8192 code-entry cap */ static cbm_gbuf_t *build_oversized_corpus(bool reverse) { cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test"); - /* One config key every candidate can match on: "max_connections". */ - cbm_gbuf_upsert_node(gb, "Variable", "max_connections", "test.config.max_connections", - "config.toml", 0, 0, NULL); + /* One config key every candidate can match on by substring: + * "max_connections", in a format that keeps substring matching (TOML and + * INI keys link by exact name only). */ + cbm_gbuf_upsert_node(gb, "Variable", "max_connections", "test.app.max_connections", + "app.properties", 0, 0, NULL); for (int k = 0; k < CL_DET_CANDIDATES; k++) { int i = reverse ? (CL_DET_CANDIDATES - 1 - k) : k; @@ -529,6 +576,7 @@ SUITE(configlink) { RUN_TEST(configlink_key_symbol_config_dir_component); RUN_TEST(configlink_key_symbol_no_config_cap); RUN_TEST(configlink_key_symbol_terraform_variable); + RUN_TEST(configlink_key_symbol_toml_ini_exact_scoped); /* Strategy 2: Dependency → Import */ RUN_TEST(configlink_dep_import_package_json);