MyIP/api/AGENTS.md

api/AGENTS.md

Conventions for Express 5 handlers under api/ and the back-end code under server/ and common/. Universal rules live in ../AGENTS.md.

Overview

The Express app lives in backend-server.js at the repo root — every route is wired there and delegated to one handler module under api/. Code the handlers share is split by who imports it:

• server/ — backend-only (guards, logger, upstream clients, service-status poller, …), flat; server/datasets/ is the offline dataset engine and its readers (see "Offline datasets").
• common/ — only what the frontend or the build imports too (valid-ip.js, fetch-with-timeout.js, report-schema.js, …); it never imports from server/. It also holds the dataset directories (*-db/), which self-hosting docs and Docker volumes mount — they stay put.

Roughly one handler file per route: IP-geolocation sources (ipinfo-io / ipapi-com / ipapi-is / ip2location-io / ip-sb / ipcheck-ing / maxmind), tool backends (get-whois / dns-resolver / mac-checker / cf-radar / asn-history / asn-connectivity / asn-profile / ooni-blocking / globalping-probes / service-status / google-map / github-stars / invisibility-test / dns-leak-test / persona / ip-blocklist / bulk-lookup), user proxies (get-user-info / update-user-achievement), platform (configs / sentry-tunnel / share-report). Each file's header comment states its route and purpose — read those for specifics.

The exception to one-file-per-route is Cloudflare Radar: all Radar data rides /api/cfradar, dispatched by ?view= over RADAR_VIEWS in server/cf-radar.js, where each view declares its guards, edge-cache TTL and fetch function — new Radar data is a view function plus a registry row, never a new route. outages is served from a 30-minute in-memory poll (cold Radar queries outrun the request timeout), started at boot with a Radar key, outside the boot window, and filled on demand by a request that finds it empty; each source keeps its last good answer, partial until both have answered once. api/data/ holds contributor-editable static config consumed by handlers — currently dns-resolvers.js, the country-annotated resolver list behind dns-resolver (gated by tests/dns-resolvers-data.test.js).

ASN data

• /api/asn-profile (server/asn-profile.js) runs each ASN source's core under a deadline above the upstream timeouts it wraps (INNER_TIMEOUTS); a failed or late one is just absent. Always 200 { asn, status, incomplete, …sections } (ok / empty / error / disabled; an upstream fault is error, never empty; incomplete = ok but partial); 502 if all fail.
• Relationship counts follow the graph: asn-connectivity neighbour lists come from local CAIDA (server/as-relationships.js), dropping the providers the graph walk distrusts. The Radar asn view (ASN Info) carries no prefix or relationship counts — the ASN Profile has both.
• /api/bulk-lookup (POST, ≤ 200 IPs, own 16 KB body limit) takes only a clean list of distinct public IPs — the page extracts, dedupes and splits a larger run into requests of that size (common/bulk-lookup-limits.js); one bad or repeated entry rejects the request, nothing is cleaned server-side. Local data only: the source is a server/bulk-lookup-sources.js row picked by BULK_LOOKUP_SOURCE, never a metered upstream.
• /api/whois ?q=: IPs (RDAP, whoiser fallback), domains (whoiser, RDAP fallback), AS<n> (RDAP autnum); all carry a __raw block.
• Offline datasets (MaxMind, CAIDA, PeeringDB, IEEE MAC registries) are rows in server/datasets/datasets.js, run by one engine, server/datasets/dataset-updater.js (lock, atomic state, publish, boot download, daily schedule). A row holds only its own logic — find the newest remote version, fetch / distill it, validate, reload; a new dataset is a new row, never a new updater. Fetched at boot when its reader has nothing usable (missing, or a file it refuses) or a publish was cut short; refreshed daily at DATASET_UPDATE_CRON (default 04:30 local) unless DATASET_AUTO_UPDATE=false (a row's pre-engine flag, e.g. CAIDA_AUTO_UPDATE, still applies when that is unset), with a catch-up run at boot for a check a downtime skipped. Files changed on disk by anyone else (another process, a hand-placed .mmdb) are reloaded by the engine's watcher. MaxMind only with its credentials (City + ASN publish together); PeeringDB only with a Cloudflare key; its 116 MB dump is stream-distilled (never parsed whole) to a ~6 MB per-ASN index without contact data (server/datasets/peeringdb-distill.js). The IEEE row (MAC Lookup) needs no key: five registry CSVs published as downloaded, answered by the longest covering assignment (server/datasets/oui-db.js).

Conventions

• Handler shape. Single default export async (req, res) => …: read req.query / req.body, call upstream, write one response.
• Every upstream call uses fetchUpstream (common/fetch-with-timeout.js, 8s timeout — Radar's client raises it to 12 s, RADAR_TIMEOUT_MS; its own timeout is an AbortError reading Timed out after <ms>ms: <host>, host only — keys ride paths and queries), never a bare fetch() / https.get() — a hanging provider must time out. It injects a default User-Agent MyIP/v<version>/<VITE_SITE_URL> (server/upstream-ua.js; some WAFs block undici's node UA); caller-supplied ones, incl. the private-API { ...req.headers } pass-through, win.
• Error shape. res.status(500).json({ error: error.message }) on upstream failure, 400 on bad input. Terse — the frontend doesn't show them verbatim.
• Response shape. IP-geolocation handlers normalize to the canonical frontend shape (ip / country_code / latitude / asn / org / …); new sources match it. timezone is the exception (see "Response enrichment").
• ?lang is never validated in a handler. Pass the raw tag through: lookupMaxMind normalizes it onto the languages its data carries (SUPPORTED_LANGS in server/datasets/maxmind-service.js); the private-API proxies forward it for the upstream's own resolution. No allow-lists; /api/map's language only gets a tag-shape check, as each value is a year-cached Static Maps call.
• Logging. Shared logger only, logger.error({ err, ...ctx }, 'msg'); no console.*, no "received request" lines (pino-http covers those).
• Error monitoring (Sentry) is env-gated and invisible to handlers. sentry-instrument.js (via node --import, before express, for ESM auto-instrumentation) inits; backend-server.js adds setupExpressErrorHandler after the routes. No SENTRY_DSN_BACKEND → never loaded; SENTRY_ENVIRONMENT=development skips the init. Handlers never import Sentry: throws and 5xx traces are automatic; caught failures stay on the logger (its hook mirrors warn+ to Sentry Logs, error+ to Issues). Periodic jobs wrap their tick in server/sentry-cron.js; key / token query params are redacted (server/sentry-scrub.js).

Security & Boundaries

Guards live in middleware, not handlers

server/guards.js, attached in backend-server.js; handlers never repeat them:

• requireReferer — global on /api/* (ALLOWED_DOMAINS + localhost). For self-hosters it logs the allowlist (and entries that can never match) at boot, and the first rejection per Referer host at info — below the Sentry mirror, capped at 50 hosts.
• requirePublicIP() — per-route ?ip=: well-formed and publicly routable; reserved space (RFC 1918, loopback, CGNAT, link-local, documentation, …) never reaches a geo source. isUsablePublicIP (common/valid-ip.js) is the one definition, shared with the front-end IP forms.
• requireValidDomain() — ?domain=, lowercased in place (one cache key); a leading _ is allowed below the TLD (RFC 8552 _dmarc.…).
• requireValidPrefix() — ?prefix= (CIDR); the frontend quantizes to the BGP DFZ floor (/24 v4, /48 v6) for edge-cache reuse.
• requireValidASN() — ?asn= in AS1…AS4294967295, rewritten as bare digits. normalizeAsnQuery() — /api/whois ?q=: AS<n> (prefix required) is canonicalized; anything else passes through. Both use common/asn-input.js.
• requireValidCountry() — ?country= (alpha-2), uppercased, syntactic only. requireValidProviderId() — ?id= ∈ service-status slugs.
• requireValidRecordType() — whitelists and uppercases ?type= against DNS_RECORD_TYPES (common/dns-record-types.js), the list the DnsResolver picker and resolveDns also read; otherwise the DoH branch would forward any string to four third-party endpoints.
• requireValidReportId() — /api/report/:id route param (22-char base64url).

New param shape → new guard there, attached in backend-server.js. Rewrites go through setQueryParam() there: Express 5's req.query re-parses on every read, so a direct write is lost.

Response enrichment lives in middleware too

withTimeZone() (server/ip-timezone.js), on all seven geo routes, derives timezone (IANA name) from the latitude / longitude the handler returned (2xx only, the cacheable hook); no handler computes or forwards one. Own coordinates keep the zone consistent with the city beside it; only the name ships — the frontend renders the offset, as a 24h-cached offset breaks at every DST switch. A new geo source adds the middleware.

Offline data gates boot, not the listener

The backend listens before any missing offline dataset or the service-status snapshot is fetched; snapshots already on disk load first, so only a first boot waits on downloads. During that window requireOfflineData([...probes]) (server/offline-data.js) answers 503 on a route whose data is still missing — never edge-cached — and is a no-op after boot, when a missing dataset means a failed download served degraded. A route reading local data declares its probes (isMaxMindReady, isAsRelLoaded, …) in backend-server.js. Answers assembled from parts don't gate: an ASN Profile section whose data is still loading (isStillLoading) answers error, leaving the others to answer. The Radar outage poll stays out of that window (it would stretch the ASN Profile's no-degraded-caching span); a request that finds it empty fills it on demand.

Private-API header pass-through (intentional exception)

Handlers proxying our private IPCheck.ing API (ipcheck-ing, invisibility-test, update-user-achievement, get-user-info, dns-leak-test, persona, ip-blocklist, and asn-profile's reputation via server/asn-reputation.js) forward the caller's headers upstream (headers: { ...req.headers }) — the upstream needs caller context (Accept-Language, auth tokens). persona strips the framing headers (host / content-length / …) first, as it re-serializes the body. Do not replicate for third-party upstreams; those get only what's explicitly needed.

Firebase Auth handler proxy

frontend-server.js (production) and vite.config.js (dev) reverse-proxy /__/auth/* to <projectId>.firebaseapp.com (common/firebase-auth-proxy.js) when VITE_FIREBASE_PROJECT_ID is set, so VITE_FIREBASE_AUTH_DOMAIN can be the deployment's own host and the sign-in handler runs first-party — inside the installed PWA's webview and Safari's storage partition, where getRedirectResult finds the result. Every OAuth provider must accept https://<host>/__/auth/handler per environment. Not an /api route: no guards, caching or per-request logs.

Edge caching

Every /api/* response defaults to Cache-Control: no-store; slowly-changing public routes opt in via cacheable(maxAge) (server/cache-control.js) in backend-server.js, e.g. cacheable(24 * 60 * 60) (TTLs written that way, not raw seconds). maxAge may be a (req) => seconds resolver (cfradar: its view's TTL; falsy = no-store). { cacheIf } vetoes the full TTL for a 2xx body, { degradedMaxAge } gives a vetoed one a shorter TTL instead of no-store: /api/asn-profile caches a complete answer (no section error / incomplete; reputation is not per-user) for 7 days and a degraded one for 1 day — some upstreams fail for some ASes every time — except inside the boot window (isOfflineBootstrapping); cfradar caches no partial view answer (isCompleteRadarAnswer). The ASN family (profile, Radar asn / bgp-prefixes) shares 7 days. Only status < 400 is cached; handlers never touch Cache-Control. Auth'd / per-user endpoints must not be wrapped — their caching belongs to the upstream owning the auth context.

Testing

• Handlers get smoke tests in tests/api-handlers.test.js (method gating, param branches, "API key missing" early returns); a new or touched handler ships its block in the same change. A req.method !== 'GET' branch the route already gates stays when a test asserts on it.
• Never hit real upstreams: assert before the first fetchUpstream, or stub globalThis.fetch (restored in the shared afterEach).
• Middleware is covered by tests/guards.test.js (don't duplicate per handler); fetch timeout / abort by tests/fetch-with-timeout.test.js.