Files
GeoLibre/scripts/scan-credentials.mjs
Qiusheng Wu db849458a6 fix(build): keep build-machine credentials out of redistributable builds (#2130)
* fix(build): keep build-machine credentials out of redistributable builds

An artifact we hand to someone else -- the Jupyter wheel above all -- must
carry no credential of ours. Three properties of this build made that easy
to get wrong:

1. vite.config.ts bridges bare shell vars into their VITE_ names
   (GOOGLE_MAPS_API_KEY, MAPBOX_TOKEN, CESIUM_TOKEN), so the build
   machine's shell is build input.
2. Something in the graph reads `import.meta.env` as a whole object, so
   Vite stops replacing per key and inlines the entire env record into
   every chunk that read reaches. Ours was
   packages/core/src/runtime-env.ts; the one we cannot fix is
   @clerk/shared's getEnvVariable.mjs, which does `import.meta.env[name]`
   with a computed name -- so the inlined record lands in ClerkGate-*.js.
3. hatch_build.py skips the JS build when static/app already exists and
   GEOLIBRE_FORCE_JS_BUILD is unset, so `python -m build` packages
   whatever an earlier `npm run build:embed` left staged, with no
   JavaScript running at all.

Changes:

- vite.config.ts gains BUILD_ENV_KEYS (allowlist of VITE_ names that may
  reach a bundle) and CREDENTIAL_ENV_KEYS (the subset that bills to
  whoever built). pruneBuildEnv() deletes unrecognized VITE_ vars from
  process.env and blanks credentials in redistributable builds, before
  Vite reads it -- so a whole-object read has nothing sensitive to
  inline, including Clerk's. The web deploy is unaffected and keeps its
  own referrer-restricted keys.
- runtime-env.ts reads the allowlisted __GEOLIBRE_BUILD_ENV__ define
  instead of import.meta.env, and exports getBuildEnvironment() for the
  five call sites that took a whole-object dependency.
- scripts/scan-credentials.mjs verifies build OUTPUT, wired into
  build-embed.mjs (before staging) and hatch_build.py (before packaging
  any wheel/sdist, needing no Node -- the path (3) above that no JS-side
  guard can cover). Both read scripts/credential-patterns.json so they
  cannot drift; it is force-included into the sdist so sdist -> wheel is
  gated too.
- CesiumJS's own public default Ion token is hash-allowlisted so the
  guard does not fire on every release.

Verified: an embed build on a machine with those shell vars set now emits
VITE_*:"" for every credential and no raw token shapes; the web build is
unchanged; a planted key makes `python -m build` refuse.

Stripped credentials still resolve at runtime through
getRuntimeEnvironment()'s Settings -> Environment variables overlay.

* Address review feedback

- pruneBuildEnv() now covers `.env*` files, not just `process.env`. Only 4
  of the 11 credential keys have a bare->prefixed shell bridge; the other 7
  are documented in getting-started.md as `.env.local` entries. A key set
  only in a file was never in `process.env` at prune time, so the loop
  skipped it and Vite's own loadEnv then resolved the raw value into
  `import.meta.env` -- reaching the @clerk/shared computed lookup in a
  redistributable build. Confirmed by planting a file-only
  VITE_STADIA_API_KEY: it was inlined verbatim, and only the output scan
  caught it. The sweep now unions process.env and FILE_ENV keys, and blanks
  rather than deletes file-backed ones, because process.env wins over a
  .env value only when the key is present (including as ""). The
  __GEOLIBRE_BUILD_ENV__ record falls back to FILE_ENV too, so a
  .env.local-configured var still reaches getBuildEnvironment().

- CREDENTIAL_ENV_KEYS is now read from scripts/credential-patterns.json
  instead of being a second hand-maintained copy, so the name this config
  strips and the name the scanners look for cannot drift.

- The Google API key pattern ends with a lookahead instead of `\b`. Its
  charset includes `-`, and `\b` needs a word character on the left, so a
  key ending in `-` did not match before a quote and both scanners missed
  it. Verified in both JavaScript RegExp and Python re; regression fixtures
  added on both sides.

- _load_patterns() raises instead of returning None. It previously made
  scan_for_credentials return [] and report "Credential scan clean.", which
  is indistinguishable in the build log from a genuine clean scan -- failing
  open, the opposite of the documented intent.

- Added python/tests/test_credential_scan.py (9 tests), including one that
  runs both scanners over the same fixture and asserts identical output.
  The Python scanner previously had no coverage. The hatchling import in
  hatch_build.py is now guarded so the scanner is importable standalone.
2026-08-27 10:06:36 -04:00

131 lines
5.1 KiB
JavaScript

// Scan a built asset directory for credentials that must not be redistributed.
//
// Why this exists: an artifact we redistribute must carry no credential of
// ours. `vite.config.ts` enforces that on the way in (BUILD_ENV_KEYS /
// CREDENTIAL_ENV_KEYS), but two things can defeat a config-level rule -- a build
// run by hand with an unexpected environment, and a third-party asset dropped
// into the tree with a key already inside it. Neither is visible from the
// config, so check the OUTPUT instead.
//
// Patterns live in scripts/credential-patterns.json because python/hatch_build.py
// runs the same scan without Node -- see that file before editing either reader.
//
// Usage:
// node scripts/scan-credentials.mjs <dir> # exits 1 on any finding
// import { scanForCredentials } from "./scan-credentials.mjs"
import { createHash } from "node:crypto";
import { readdirSync, readFileSync } from "node:fs";
import { dirname, extname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const scriptDir = dirname(fileURLToPath(import.meta.url));
const config = JSON.parse(readFileSync(resolve(scriptDir, "credential-patterns.json"), "utf8"));
const SCANNABLE = new Set(config.scannableExtensions);
const ALLOWED_VALUE_HASHES = new Set(Object.keys(config.allowedValueHashes ?? {}));
/**
* Whether a matched value is a known public vendor token rather than ours.
*
* @param {string} value - The matched value.
* @returns {boolean} True when the value is allowlisted.
*/
function isAllowed(value) {
return ALLOWED_VALUE_HASHES.has(createHash("sha256").update(value).digest("hex"));
}
// `VITE_FOO:"value"` or `VITE_FOO="value"` with a non-empty value. Vite emits
// the former inside an inlined env object literal -- the exact shape that
// a whole-object read is inlined. An empty string is what a correctly stripped
// build produces, so it must not trip this.
const assignedNameRe = new RegExp(
`\\b(${config.credentialEnvNames.join("|")})\\b\\s*[:=]\\s*["'\`]([^"'\`]+)["'\`]`,
"g",
);
/**
* Masks a secret so CI logs can point at it without republishing it.
*
* @param {string} value - The matched secret.
* @returns {string} A redacted, identifiable form.
*/
function mask(value) {
if (value.length <= 12) return `${value.slice(0, 2)}***`;
return `${value.slice(0, 6)}...${value.slice(-4)} (${value.length} chars)`;
}
/**
* Recursively lists every file under a directory.
*
* @param {string} dir - Directory to walk.
* @returns {string[]} Absolute paths of every file found.
*/
function walk(dir) {
const out = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const full = resolve(dir, entry.name);
if (entry.isDirectory()) out.push(...walk(full));
else out.push(full);
}
return out;
}
/**
* Scans a directory of built assets for redistributable credentials.
*
* @param {string} dir - The built asset directory (e.g. dist-embed).
* @returns {string[]} Sorted, unique findings; empty when the build is clean.
*/
export function scanForCredentials(dir) {
const findings = [];
for (const file of walk(dir)) {
if (!SCANNABLE.has(extname(file))) continue;
const text = readFileSync(file, "utf8");
const rel = file.slice(dir.length + 1);
for (const { name, regex } of config.patterns) {
for (const m of text.matchAll(new RegExp(regex, "g"))) {
if (isAllowed(m[0])) continue;
findings.push(`${rel}: ${name} ${mask(m[0])}`);
}
}
for (const m of text.matchAll(assignedNameRe)) {
if (isAllowed(m[2])) continue;
findings.push(`${rel}: ${m[1]} = ${mask(m[2])}`);
}
}
return [...new Set(findings)].sort();
}
/** Guidance printed alongside any finding, shared with hatch_build.py. */
export const REMEDIATION =
"This artifact is redistributed, so it must not carry your keys.\n" +
"Most likely your shell exports GOOGLE_MAPS_API_KEY / MAPBOX_TOKEN / CESIUM_TOKEN\n" +
"(vite.config.ts bridges bare names into their VITE_ forms) and the credential\n" +
"strip did not apply. Check that GEOLIBRE_EMBED=1 reached the build, and see\n" +
"CREDENTIAL_ENV_KEYS in apps/geolibre-desktop/vite.config.ts. Wheel users supply\n" +
"their own tokens at runtime via Settings → Environment variables.\n\n" +
"If this appeared right after a dependency bump, decode the value first: an\n" +
"upstream library may ship its own public token (CesiumJS does). Confirm the\n" +
"payload names the vendor, then update allowedValueHashes in\n" +
"scripts/credential-patterns.json -- see docs/maintenance.md.";
// CLI entry point.
if (process.argv[1] && resolve(process.argv[1]) === resolve(fileURLToPath(import.meta.url))) {
const target = process.argv[2];
if (!target) {
console.error("usage: node scripts/scan-credentials.mjs <dir>");
process.exit(2);
}
const findings = scanForCredentials(resolve(target));
if (findings.length > 0) {
console.error(
`[scan-credentials] ${findings.length} credential(s) found in ${target}:\n` +
findings.map((f) => ` - ${f}`).join("\n") +
`\n\n${REMEDIATION}`,
);
process.exit(1);
}
console.log(`[scan-credentials] Clean (${config.patterns.length} patterns) in ${target}`);
}