MyIP/api/AGENTS.md

api/AGENTS.md

Conventions for Express 5 handlers under api/ and the back-end code under server/ and common/. Universal rules live in ../AGENTS.md.

Overview

The Express app lives in backend-server.js at the repo root — every route is wired there and delegated to one handler module under api/. Code the handlers share is split by who imports it:

• server/ — backend-only (guards, logger, upstream clients, service-status poller, …), flat; server/datasets/ is the offline dataset engine and its readers (see "Offline datasets").
• common/ — only what the frontend or the build imports too (valid-ip.js, fetch-with-timeout.js, report-schema.js, …); it never imports from server/. It also holds the dataset directories (*-db/), which self-hosting docs and Docker volumes mount — they stay put.

Roughly one handler file per route: IP-geolocation sources (ipinfo-io / ipapi-com / ipapi-is / ip2location-io / ip-sb / ipcheck-ing / maxmind), tool backends (get-whois / dns-resolver / mac-checker / cf-radar / asn-history / asn-connectivity / asn-profile / ooni-blocking / globalping-probes / service-status / google-map / github-stars / invisibility-test / dns-leak-test / persona / ip-blocklist), user proxies (get-user-info / update-user-achievement), platform (configs / sentry-tunnel / share-report). Each file's header comment states its route and purpose — read those for specifics.

The exception to one-file-per-route is Cloudflare Radar: all Radar data rides /api/cfradar, dispatched by ?view= over RADAR_VIEWS in server/cf-radar.js, where each view declares its guards, edge-cache TTL and fetch function — new Radar data is a view function plus a registry row, never a new route. api/data/ holds contributor-editable static config consumed by handlers — currently dns-resolvers.js, the country-annotated resolver list behind dns-resolver (gated by tests/dns-resolvers-data.test.js).

ASN data

• /api/asn-profile (server/asn-profile.js) runs each ASN source's core under a deadline above the upstream timeouts it wraps (INNER_TIMEOUTS); a failed or late one is just absent. Always 200 { asn, status, incomplete, …sections } (ok / empty / error / disabled; an upstream fault is error, never empty; incomplete = ok but partial); 502 if all fail.
• Relationship counts follow the graph: asn-connectivity neighbour lists come from local CAIDA (server/as-relationships.js), dropping the providers the graph walk distrusts. The Radar asn view (ASN Info) carries no prefix or relationship counts — the ASN Profile has both.
• /api/whois ?q=: IPs (RDAP, whoiser fallback), domains (whoiser, RDAP fallback), AS<n> (RDAP autnum); all carry a __raw block.
• Offline datasets (MaxMind, CAIDA, PeeringDB, IEEE MAC registries) are rows in server/datasets/datasets.js, run by one engine, server/datasets/dataset-updater.js (lock, atomic state, publish, boot download, daily schedule). A row holds only its own logic — find the newest remote version, fetch / distill it, validate, reload; a new dataset is a new row, never a new updater. Fetched at boot when its reader has nothing usable (missing, or a file it refuses) or a publish was cut short; refreshed daily at DATASET_UPDATE_CRON (default 04:30 local) unless DATASET_AUTO_UPDATE=false (a row's pre-engine flag, e.g. CAIDA_AUTO_UPDATE, still applies when that is unset), with a catch-up run at boot for a check a downtime skipped. Files changed on disk by anyone else (another process, a hand-placed .mmdb) are reloaded by the engine's watcher. MaxMind only with its credentials (City + ASN publish together); PeeringDB only with a Cloudflare key; its 116 MB dump is stream-distilled (never parsed whole) to a ~6 MB per-ASN index without contact data (server/datasets/peeringdb-distill.js). The IEEE row (MAC Lookup) needs no key: five registry CSVs published as downloaded, answered by the longest covering assignment (server/datasets/oui-db.js).

Conventions

• Handler shape. Single default export async (req, res) => …: read req.query / req.body, call upstream, write one response.
• Every upstream call uses fetchUpstream (common/fetch-with-timeout.js, 8s timeout), never a bare fetch() / https.get() — a hanging provider must time out. It injects a default User-Agent MyIP/v<version>/<VITE_SITE_URL> (server/upstream-ua.js; some WAFs block undici's node UA); caller-supplied ones, incl. the private-API { ...req.headers } pass-through, win.
• Error shape. res.status(500).json({ error: error.message }) on upstream failure, 400 on bad input. Terse — the frontend doesn't show them verbatim.
• Response shape. IP-geolocation handlers normalize to the canonical frontend shape (ip / country_code / latitude / asn / org / …); new sources match it. timezone is the exception (see "Response enrichment").
• ?lang is never validated in a handler. Pass the raw tag through: lookupMaxMind normalizes it onto the languages its data carries (SUPPORTED_LANGS in server/datasets/maxmind-service.js); the private-API proxies forward it for the upstream's own resolution. No allow-lists; /api/map's language only gets a tag-shape check, as each value is a year-cached Static Maps call.
• Logging. Shared logger only, logger.error({ err, ...ctx }, 'msg'); no console.*, no "received request" lines (pino-http covers those).
• Error monitoring (Sentry) is env-gated and invisible to handlers. sentry-instrument.js (via node --import, before express, for ESM auto-instrumentation) inits; backend-server.js adds setupExpressErrorHandler after the routes. No SENTRY_DSN_BACKEND → never loaded; SENTRY_ENVIRONMENT=development skips the init. Handlers never import Sentry: throws and 5xx traces are automatic; caught failures stay on the logger (its hook mirrors warn+ to Sentry Logs, error+ to Issues). Periodic jobs wrap their tick in server/sentry-cron.js; key / token query params are redacted (server/sentry-scrub.js).

Security & Boundaries

Guards live in middleware, not handlers

server/guards.js, attached in backend-server.js; handlers never repeat them:

• requireReferer — global on /api/* (ALLOWED_DOMAINS + localhost).
• requirePublicIP() — per-route ?ip=: well-formed and publicly routable; reserved space (RFC 1918, loopback, CGNAT, link-local, documentation, …) never reaches a geo source. isUsablePublicIP (common/valid-ip.js) is the one definition, shared with the front-end IP forms.
• requireValidDomain() — ?domain=, lowercased in place (one cache key); a leading _ is allowed below the TLD (RFC 8552 _dmarc.…).
• requireValidPrefix() — ?prefix= (CIDR); the frontend quantizes to the BGP DFZ floor (/24 v4, /48 v6) for edge-cache reuse.
• requireValidASN() — ?asn= in AS1…AS4294967295, rewritten as bare digits. normalizeAsnQuery() — /api/whois ?q=: AS<n> (prefix required) is canonicalized; anything else passes through. Both use common/asn-input.js.
• requireValidCountry() — ?country= (alpha-2), uppercased, syntactic only. requireValidProviderId() — ?id= ∈ service-status slugs.
• requireValidRecordType() — whitelists and uppercases ?type= against DNS_RECORD_TYPES (common/dns-record-types.js), the list the DnsResolver picker and resolveDns also read; otherwise the DoH branch would forward any string to four third-party endpoints.
• requireValidReportId() — /api/report/:id route param (22-char base64url).

New param shape → new guard there, attached in backend-server.js. Rewrites go through setQueryParam() there: Express 5's req.query re-parses on every read, so a direct write is lost.

Response enrichment lives in middleware too

withTimeZone() (server/ip-timezone.js), on all seven geo routes, derives timezone (IANA name) from the latitude / longitude the handler returned (2xx only, the cacheable hook); no handler computes or forwards one. Own coordinates keep the zone consistent with the city beside it; only the name ships — the frontend renders the offset, as a 24h-cached offset breaks at every DST switch. A new geo source adds the middleware.

Offline data gates boot, not the listener

The backend listens before any missing offline dataset or the service-status snapshot is fetched; snapshots already on disk load first, so only a first boot waits on downloads. During that window requireOfflineData([...probes]) (server/offline-data.js) answers 503 on a route whose data is still missing — never edge-cached — and is a no-op after boot, when a missing dataset means a failed download served degraded. A route reading local data declares its probes (isMaxMindReady, isAsRelLoaded, …) in backend-server.js. Answers assembled from parts don't gate: an ASN Profile section whose data is still loading (isStillLoading) answers error, leaving the others to answer.

Private-API header pass-through (intentional exception)

Handlers proxying our private IPCheck.ing API (ipcheck-ing, invisibility-test, update-user-achievement, get-user-info, dns-leak-test, persona, ip-blocklist, and asn-profile's reputation via server/asn-reputation.js) forward the caller's headers upstream (headers: { ...req.headers }) — the upstream needs caller context (Accept-Language, auth tokens). persona strips the framing headers (host / content-length / …) first, as it re-serializes the body. Do not replicate for third-party upstreams; those get only what's explicitly needed.

Firebase Auth handler proxy

frontend-server.js (production) and vite.config.js (dev) reverse-proxy /__/auth/* to <projectId>.firebaseapp.com (common/firebase-auth-proxy.js) when VITE_FIREBASE_PROJECT_ID is set, so VITE_FIREBASE_AUTH_DOMAIN can be the deployment's own host and the sign-in handler runs first-party — inside the installed PWA's webview and Safari's storage partition, where getRedirectResult finds the result. Every OAuth provider must accept https://<host>/__/auth/handler per environment. Not an /api route: no guards, caching or per-request logs.

Edge caching

Every /api/* response defaults to Cache-Control: no-store; slowly-changing public routes opt in via cacheable(maxAge) (server/cache-control.js) in backend-server.js, e.g. cacheable(24 * 60 * 60) (TTLs written that way, not raw seconds). maxAge may be a (req) => seconds resolver (cfradar: its view's TTL; falsy = no-store). { cacheIf } vetoes the full TTL for a 2xx body, { degradedMaxAge } gives a vetoed one a shorter TTL instead of no-store: /api/asn-profile caches a complete answer (no section error / incomplete; reputation is not per-user) for 7 days and a degraded one for 1 day — some upstreams fail for some ASes every time — except inside the boot window (isOfflineBootstrapping); cfradar caches no partial view answer (isCompleteRadarAnswer). The ASN family (profile, Radar asn / bgp-prefixes) shares 7 days. Only status < 400 is cached; handlers never touch Cache-Control. Auth'd / per-user endpoints must not be wrapped — their caching belongs to the upstream owning the auth context.

Testing

• Handlers get smoke tests in tests/api-handlers.test.js (method gating, param branches, "API key missing" early returns); a new or touched handler ships its block in the same change. A req.method !== 'GET' branch the route already gates stays when a test asserts on it.
• Never hit real upstreams: assert before the first fetchUpstream, or stub globalThis.fetch (restored in the shared afterEach).
• Middleware is covered by tests/guards.test.js (don't duplicate per handler); fetch timeout / abort by tests/fetch-with-timeout.test.js.