mirror of
https://github.com/opengeos/GeoLibre.git
synced 2026-11-04 10:43:36 +00:00
* fix(build): keep build-machine credentials out of redistributable builds An artifact we hand to someone else -- the Jupyter wheel above all -- must carry no credential of ours. Three properties of this build made that easy to get wrong: 1. vite.config.ts bridges bare shell vars into their VITE_ names (GOOGLE_MAPS_API_KEY, MAPBOX_TOKEN, CESIUM_TOKEN), so the build machine's shell is build input. 2. Something in the graph reads `import.meta.env` as a whole object, so Vite stops replacing per key and inlines the entire env record into every chunk that read reaches. Ours was packages/core/src/runtime-env.ts; the one we cannot fix is @clerk/shared's getEnvVariable.mjs, which does `import.meta.env[name]` with a computed name -- so the inlined record lands in ClerkGate-*.js. 3. hatch_build.py skips the JS build when static/app already exists and GEOLIBRE_FORCE_JS_BUILD is unset, so `python -m build` packages whatever an earlier `npm run build:embed` left staged, with no JavaScript running at all. Changes: - vite.config.ts gains BUILD_ENV_KEYS (allowlist of VITE_ names that may reach a bundle) and CREDENTIAL_ENV_KEYS (the subset that bills to whoever built). pruneBuildEnv() deletes unrecognized VITE_ vars from process.env and blanks credentials in redistributable builds, before Vite reads it -- so a whole-object read has nothing sensitive to inline, including Clerk's. The web deploy is unaffected and keeps its own referrer-restricted keys. - runtime-env.ts reads the allowlisted __GEOLIBRE_BUILD_ENV__ define instead of import.meta.env, and exports getBuildEnvironment() for the five call sites that took a whole-object dependency. - scripts/scan-credentials.mjs verifies build OUTPUT, wired into build-embed.mjs (before staging) and hatch_build.py (before packaging any wheel/sdist, needing no Node -- the path (3) above that no JS-side guard can cover). Both read scripts/credential-patterns.json so they cannot drift; it is force-included into the sdist so sdist -> wheel is gated too. - CesiumJS's own public default Ion token is hash-allowlisted so the guard does not fire on every release. Verified: an embed build on a machine with those shell vars set now emits VITE_*:"" for every credential and no raw token shapes; the web build is unchanged; a planted key makes `python -m build` refuse. Stripped credentials still resolve at runtime through getRuntimeEnvironment()'s Settings -> Environment variables overlay. * Address review feedback - pruneBuildEnv() now covers `.env*` files, not just `process.env`. Only 4 of the 11 credential keys have a bare->prefixed shell bridge; the other 7 are documented in getting-started.md as `.env.local` entries. A key set only in a file was never in `process.env` at prune time, so the loop skipped it and Vite's own loadEnv then resolved the raw value into `import.meta.env` -- reaching the @clerk/shared computed lookup in a redistributable build. Confirmed by planting a file-only VITE_STADIA_API_KEY: it was inlined verbatim, and only the output scan caught it. The sweep now unions process.env and FILE_ENV keys, and blanks rather than deletes file-backed ones, because process.env wins over a .env value only when the key is present (including as ""). The __GEOLIBRE_BUILD_ENV__ record falls back to FILE_ENV too, so a .env.local-configured var still reaches getBuildEnvironment(). - CREDENTIAL_ENV_KEYS is now read from scripts/credential-patterns.json instead of being a second hand-maintained copy, so the name this config strips and the name the scanners look for cannot drift. - The Google API key pattern ends with a lookahead instead of `\b`. Its charset includes `-`, and `\b` needs a word character on the left, so a key ending in `-` did not match before a quote and both scanners missed it. Verified in both JavaScript RegExp and Python re; regression fixtures added on both sides. - _load_patterns() raises instead of returning None. It previously made scan_for_credentials return [] and report "Credential scan clean.", which is indistinguishable in the build log from a genuine clean scan -- failing open, the opposite of the documented intent. - Added python/tests/test_credential_scan.py (9 tests), including one that runs both scanners over the same fixture and asserts identical output. The Python scanner previously had no coverage. The hatchling import in hatch_build.py is now guarded so the scanner is importable standalone.
131 lines
5.1 KiB
JavaScript
131 lines
5.1 KiB
JavaScript
// Scan a built asset directory for credentials that must not be redistributed.
|
|
//
|
|
// Why this exists: an artifact we redistribute must carry no credential of
|
|
// ours. `vite.config.ts` enforces that on the way in (BUILD_ENV_KEYS /
|
|
// CREDENTIAL_ENV_KEYS), but two things can defeat a config-level rule -- a build
|
|
// run by hand with an unexpected environment, and a third-party asset dropped
|
|
// into the tree with a key already inside it. Neither is visible from the
|
|
// config, so check the OUTPUT instead.
|
|
//
|
|
// Patterns live in scripts/credential-patterns.json because python/hatch_build.py
|
|
// runs the same scan without Node -- see that file before editing either reader.
|
|
//
|
|
// Usage:
|
|
// node scripts/scan-credentials.mjs <dir> # exits 1 on any finding
|
|
// import { scanForCredentials } from "./scan-credentials.mjs"
|
|
|
|
import { createHash } from "node:crypto";
|
|
import { readdirSync, readFileSync } from "node:fs";
|
|
import { dirname, extname, resolve } from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
|
|
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
|
const config = JSON.parse(readFileSync(resolve(scriptDir, "credential-patterns.json"), "utf8"));
|
|
|
|
const SCANNABLE = new Set(config.scannableExtensions);
|
|
const ALLOWED_VALUE_HASHES = new Set(Object.keys(config.allowedValueHashes ?? {}));
|
|
|
|
/**
|
|
* Whether a matched value is a known public vendor token rather than ours.
|
|
*
|
|
* @param {string} value - The matched value.
|
|
* @returns {boolean} True when the value is allowlisted.
|
|
*/
|
|
function isAllowed(value) {
|
|
return ALLOWED_VALUE_HASHES.has(createHash("sha256").update(value).digest("hex"));
|
|
}
|
|
|
|
// `VITE_FOO:"value"` or `VITE_FOO="value"` with a non-empty value. Vite emits
|
|
// the former inside an inlined env object literal -- the exact shape that
|
|
// a whole-object read is inlined. An empty string is what a correctly stripped
|
|
// build produces, so it must not trip this.
|
|
const assignedNameRe = new RegExp(
|
|
`\\b(${config.credentialEnvNames.join("|")})\\b\\s*[:=]\\s*["'\`]([^"'\`]+)["'\`]`,
|
|
"g",
|
|
);
|
|
|
|
/**
|
|
* Masks a secret so CI logs can point at it without republishing it.
|
|
*
|
|
* @param {string} value - The matched secret.
|
|
* @returns {string} A redacted, identifiable form.
|
|
*/
|
|
function mask(value) {
|
|
if (value.length <= 12) return `${value.slice(0, 2)}***`;
|
|
return `${value.slice(0, 6)}...${value.slice(-4)} (${value.length} chars)`;
|
|
}
|
|
|
|
/**
|
|
* Recursively lists every file under a directory.
|
|
*
|
|
* @param {string} dir - Directory to walk.
|
|
* @returns {string[]} Absolute paths of every file found.
|
|
*/
|
|
function walk(dir) {
|
|
const out = [];
|
|
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
|
const full = resolve(dir, entry.name);
|
|
if (entry.isDirectory()) out.push(...walk(full));
|
|
else out.push(full);
|
|
}
|
|
return out;
|
|
}
|
|
|
|
/**
|
|
* Scans a directory of built assets for redistributable credentials.
|
|
*
|
|
* @param {string} dir - The built asset directory (e.g. dist-embed).
|
|
* @returns {string[]} Sorted, unique findings; empty when the build is clean.
|
|
*/
|
|
export function scanForCredentials(dir) {
|
|
const findings = [];
|
|
for (const file of walk(dir)) {
|
|
if (!SCANNABLE.has(extname(file))) continue;
|
|
const text = readFileSync(file, "utf8");
|
|
const rel = file.slice(dir.length + 1);
|
|
for (const { name, regex } of config.patterns) {
|
|
for (const m of text.matchAll(new RegExp(regex, "g"))) {
|
|
if (isAllowed(m[0])) continue;
|
|
findings.push(`${rel}: ${name} ${mask(m[0])}`);
|
|
}
|
|
}
|
|
for (const m of text.matchAll(assignedNameRe)) {
|
|
if (isAllowed(m[2])) continue;
|
|
findings.push(`${rel}: ${m[1]} = ${mask(m[2])}`);
|
|
}
|
|
}
|
|
return [...new Set(findings)].sort();
|
|
}
|
|
|
|
/** Guidance printed alongside any finding, shared with hatch_build.py. */
|
|
export const REMEDIATION =
|
|
"This artifact is redistributed, so it must not carry your keys.\n" +
|
|
"Most likely your shell exports GOOGLE_MAPS_API_KEY / MAPBOX_TOKEN / CESIUM_TOKEN\n" +
|
|
"(vite.config.ts bridges bare names into their VITE_ forms) and the credential\n" +
|
|
"strip did not apply. Check that GEOLIBRE_EMBED=1 reached the build, and see\n" +
|
|
"CREDENTIAL_ENV_KEYS in apps/geolibre-desktop/vite.config.ts. Wheel users supply\n" +
|
|
"their own tokens at runtime via Settings → Environment variables.\n\n" +
|
|
"If this appeared right after a dependency bump, decode the value first: an\n" +
|
|
"upstream library may ship its own public token (CesiumJS does). Confirm the\n" +
|
|
"payload names the vendor, then update allowedValueHashes in\n" +
|
|
"scripts/credential-patterns.json -- see docs/maintenance.md.";
|
|
|
|
// CLI entry point.
|
|
if (process.argv[1] && resolve(process.argv[1]) === resolve(fileURLToPath(import.meta.url))) {
|
|
const target = process.argv[2];
|
|
if (!target) {
|
|
console.error("usage: node scripts/scan-credentials.mjs <dir>");
|
|
process.exit(2);
|
|
}
|
|
const findings = scanForCredentials(resolve(target));
|
|
if (findings.length > 0) {
|
|
console.error(
|
|
`[scan-credentials] ${findings.length} credential(s) found in ${target}:\n` +
|
|
findings.map((f) => ` - ${f}`).join("\n") +
|
|
`\n\n${REMEDIATION}`,
|
|
);
|
|
process.exit(1);
|
|
}
|
|
console.log(`[scan-credentials] Clean (${config.patterns.length} patterns) in ${target}`);
|
|
}
|