Reconcile the complete Atlas and Lens stream with current main while retaining both histories. Preserve architecture functionality, current process and watcher safeguards, test infrastructure, and dependency security updates. Add coverage for Git-environment isolation combined with child-local overrides.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ui_server_index_start_stores_resolved_root built its expected root as
the resolved parent plus "/allowed". On Windows the server stores the
resolved form of the directory itself, which uses backslashes
throughout, so the comparison saw a mixed-separator string and failed
on the windows-latest CLANG64 job. The test now resolves the allowed
directory with cbm_canonical_path, as the server does, and compares
the 202 reply against the JSON-escaped form, as the status-listing
check already did; a backslash appears escaped in both bodies.
Production is unchanged; on POSIX the expected strings are identical
to before.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
GET /api/index-status rendered every job into a fixed 2 KB stack buffer
with a clamp. Now that a job carries its resolved root in a 4 KB field,
one long root alone exceeded that buffer and the listing came back as a
cut JSON body with status 200.
Build the body in a heap buffer sized from the table: per slot the entry
framing plus the root and the error message at cbm_json_escape's widest
expansion. The two strings are escaped straight into the body, the
append helpers keep their clamp as a second guard, and the buffer goes
through the memory core like the other variable-size responses.
The 202 reply of POST /api/index printed the stored root unescaped, so a
quote or backslash in the path ended its JSON string early. It is now
escaped the way the listing escapes it.
Tests: with every slot holding a root as long as the platform resolves
(just under the 4 KB field on Linux, under PATH_MAX on macOS), the
listing parses as JSON and carries each root whole, first with one job
and then with the table full; a root containing a quote comes back
parseable from the 202 reply, naming the resolved path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
POST /api/index resolves the root_path of the request into a 4 KB buffer,
but the index job kept the original request string in a 1 KB field. The
job, the worker log lines, the 202 reply and /api/index-status therefore
reported the request spelling instead of the resolved root, and a path
past 1023 bytes reached the daemon cut off.
Store the resolved path in the job, give the field the capacity of the
buffer it is copied from, and answer 400 for a path that would not fit
instead of storing a shortened one. The status listing's escape buffer
follows the new field size.
The job's downstream, application_background_index, re-resolves what it
receives, so a resolved input is a fixed point there and the daemon keys
the job by the same string the route worked with.
Tests drive the live route: a root outside a configured allowed root is
answered 403 and no job starts; a root spelled through a ".." segment is
stored and forwarded in resolved form; a root past 1 KB is either refused
or forwarded whole. The last case reaches the accepted branch on Linux
(PATH_MAX 4096); on macOS the path cannot resolve and the refusal branch
is asserted.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Separate agent configuration from chat visibility, keep chat compact, and
resume selection explanations after manual questions. Restore project
conversations locally for 24 hours with bounded storage and clear history.
Move daemon configuration into System and report persisted index metadata
and live watcher registration alongside its configuration control.
Validated with 303 focused frontend tests, TypeScript/Vite production build,
160 native watcher/HTTP tests, regression checks, and local browser checks.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add focused galaxy tracing, edge filters, organic cluster layouts and cached
graph exploration with compact architecture and impact summaries.
Make the browser-local agent explain the current selection proactively,
and add a Config panel with persisted daemon overrides, shared browser
preferences and a reference for external configuration inputs.
Validation: 182 focused frontend tests and TypeScript passed. Reused the
recorded production build and 485 passing native checks on unchanged
source. The pre-existing capped-layout loading-note test failure remains
documented separately; the full local CI ladder was waived.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolve the RUN_TEST list conflict in tests/test_daemon_application.c by keeping both sides' new tests.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add source-backed repository navigation, selection context and a shared change
analysis workspace for symbols, working-tree changes and Git comparisons.
Persist agent events and project logs through the existing daemon with SQLite
schema 4, bounded polling, retention and restart-safe deduplication.
Repair compact RPC decoding, path-search races, source pagination and identity,
large map responses, live error propagation and explicit local diagnosis flows.
Include the real-repository browser runners and focused regression coverage.
Validation: 2473 frontend tests, 203 acceptance checks, production build,
style/promise gates, 5 reference-boundary checks and 7 local hook tests pass.
The complete repository-map browser flow also passes after daemon restart.
Global C lint and cross-platform release validation remain open and are
recorded in the accompanying handoff report.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
Retain the existing frontend diagnostics and unreadable-file reporting
alongside the selected-code inspector, graph navigation, and local chat.
Keep both process telemetry and log-tail models, and preserve the HTTP
logging tests alongside the browser model download policy checks.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add Explore, Galaxy, Architecture, Agents, and System workspaces with a
persistent, resizable local chat dock. Attach exact source selections and
graph context, render Markdown, and offer pinned opt-in browser models.
Keep the galaxy visible below chat and focused on the active file or code
selection. Restore hierarchy root selection, resolve projection identities
by symbol, and retain loaded search matches after an empty index reply.
Expose process measurement units, embed browser runtime MJS/WASM assets,
and constrain binary-audit exceptions to reviewed dependency strings.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The browser console was the only place Atlas wrote to, and it dies with
the tab: a reader who saw "get_code_snippet returned no source" had
nothing to hand over, and an agent looking into it had nothing to read.
The page now keeps a second copy of everything it logs, on the server,
next to the daemon's own records.
Server (src/ui/http_server.c):
- POST /api/ui-log takes {page, session, entries[]} and appends one JSON
line per entry to <cache_dir>/logs/ui.log (received, page, session,
seq, ts, level, source, message, detail, stack, url, line, col).
Bounds: 64 KiB per post, 100 entries per post, 4096 characters per
text field cut on a UTF-8 boundary and marked, the level normalised to
a known word, every string escaped again on the way out. The file
rotates once to ui.log.1 at CBM_UI_LOG_ROTATE_BYTES (default 5 MiB).
Warnings and errors are mirrored into the ring GET /api/logs serves,
so both sides of one failure sit in one list; the chatter stays in
the file. The reply names the file and says whether it could be
written.
- GET /api/ui-log?lines=N tails the file (default 200, at most 1000,
from the last 256 KiB) and names the file and its rotated
predecessor; `partial` says when the lines do not start at the top.
- cbm_ui_log_file_path() is the one place the path is built.
Frontend (graph-ui):
- app/ui-log.ts: a bounded buffer (200 entries, oldest dropped first
and the loss named at the head of the next batch), batched posts
(25 entries or 1.5 s), backoff on a refused post, never writes to the
console itself. app/ui-log-transport.ts posts with fetch, and with a
beacon when the page is going away.
- app/ui-log-install.ts wraps the five console methods (they still
print; the copy is taken after), listens to window error and
unhandledrejection, and flushes on pagehide. Installed first in
main.tsx, so nothing can fail before the copy exists.
- provider/error-observer.ts is the seam for failures the panels catch
and show as a sentence: the /rpc transport and the /api client
announce every failed call there before throwing, with a level that
tells a tool the server does not offer (info) from a tool that said
no (warn) from a broken or unreachable server (error). The reader
announces a file the index does not know.
- The projects panel shows the tail under "This server" as sentences,
names the file to attach to a report, and reloads it with the rest.
- README (both) and INVENTAR.md name the file and the routes.
Tests: three httpd cases (JSON lines and tail, refused bodies and
dropped entries, rotation); vitest for the buffer, the wiring in jsdom,
the transport, the seam through both clients, the API method, the model
and the panel. Verified live against a real project: the reader's
failure appears in the file and in the panel within two seconds.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Re-stacks the CodeAtlasWeb branch on the updated base (feat/atlas-r1 at
a44732b6, which merges main 7b0f553c). Ten conflicts, all in the seam
between the previous frontend and its replacement:
- README.md: main's surface count (45) with this branch's Atlas line.
- graph-ui/package-lock.json, graph-ui/src/App.tsx: CodeAtlasWeb's own
files stay; the previous frontend's edits to the same paths do not
apply.
- Seven previous-frontend files main had modified (GraphTab,
NodeDetailPanel and its test, StatsTab test, useProjects, rationale
and its test) stay deleted, and two tests main added for that
frontend (App.test.tsx, useProjects.test.tsx) are not brought in;
their subjects no longer exist in this tree.
Nothing under graph-ui changes relative to this branch; the C side
takes main's lean-output work and the branch's merge resolution.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the Atlas branch up to current main (7b0f553c). Eight files
conflicted; how each was settled:
- src/mcp/mcp.c: the real ingest_traces description stays, in main's tool
row shape (main dropped the title column); index_repository keeps the
truthful abort branches (aborted_previous_preserved, persist_failed)
and gains main's format_migration flag in the success branch.
- src/ui/http_server.c and tests/test_httpd.c: the branch's twelve-tool
read allowlist for /rpc (#1663) over main's three, with main's %zu
and initialised counter.
- README.md: main's surface count (45), the branch's Atlas line.
- graph-ui (the previous frontend, replaced by the stacked #2068 but
kept building and green here): the branch's node-selection wiring
(onSelectNode) is kept consistently, so main's duplicate
handleSelectPath and its 3-argument call go; main's version chip is
added to the branch's header; useProjects keeps the branch's
render-first flow on top of main's paginated JSON helpers
(fetchAllProjects, fetchFullSchema), which the lean output contract
requires, and a project whose size the server did not send now gets
its schema too. main's GraphTab.selection.test.tsx, written against
main's GraphTab, is dropped in favour of the branch's own #1197
coverage in GraphTab.regions.test.tsx ("reports selected nodes to
the route for deep links"); main's useProjects pagination test waits
for the lazy schema enrichment instead of asserting before it runs.
Verified: mcp, httpd and ui suites 414 passed; the old frontend builds
and its 149 tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer glue for the imported frontend; no product behaviour of the
UI itself changes.
- Makefile.cbm: `npm ci` runs with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1.
The UI's dev dependencies include playwright for its local browser
proofs, and a plain install would download browsers that no build or
gate launches.
- graph-ui/vite.config.ts: the dev server listens on 5173 with the two
literal loopback origins, which is what
tests/test_ui_dev_proxy_security.sh pins.
- graph-ui/index.html: lang="en"; the catalog is English.
- CSP: the directive string moves to http_server.h as CBM_UI_CSP_VALUE
so a test can pin it, and connect-src now admits the two loopback
services the UI may talk to when the reader starts them, the
local-model sidecar on 127.0.0.1:4141 and the agent-event bridge on
127.0.0.1:4142. Under connect-src 'self' both fetches were silently
blocked in the embedded build. Every other host stays forbidden; a
new httpd test asserts that the only hosts in the policy are those
two loopback ports.
- scripts/security-ui.sh: two false positives that bit on the imported
source. The analytics pattern matched the English word "plausible"
(now plausible.io), and `\s` inside an ERE bracket is a literal
backslash and 's', which cut every localhost URL at its first 's' and
then failed the allowlist (now [:space:]).
Verified: httpd suite 64 passed; dev-proxy contract and UI security
audit pass; graph-ui builds and its 2024 vitest tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Expose the compiled server version through the existing UI config endpoint and render it beside the product name. Keep older or unavailable config responses compatible by hiding the label, with focused frontend and native endpoint coverage.
Fixes#1820
Signed-off-by: Rares Popa <2606875+rarepops@users.noreply.github.com>
Maintainer follow-up on top of the contributor's rebased commit,
implementing the Half-B strategy decision from the 2026-08-21 review:
- Linux: when the self-resolved path fails the executable check after an
atomic rename-over, hand back the /proc/self/exe magic link. The link
still executes the in-memory OLD build, which is the only spawn the
worker's build-fingerprint gate accepts. Deliberately NOT the saved
launch path: after a replacement that path holds a DIFFERENT build, so
preferring it swaps an ENOENT for a fingerprint refusal.
- macOS: no magic link exists; fail closed. The supervisor logs
index.supervisor.no_self_path and indexing resumes on daemon restart
instead of spawning a missing binary. A rename-over that leaves a VALID
binary still resolves (and the fingerprint gate refuses downstream, with
its own coverage).
- The prefer-saved-launch-path branch from the original head is dropped
per the same ruling.
Also fixes both Half-A review items:
- Aliasing in cbm_http_server_set_binary_path: `path` may alias
g_binary_path on a repeated call. The overlapping snprintf cannot
actually fire today — resolve's leading out[0]='\0' zeroes the shared
buffer first — but that same reset means the call silently DISCARDS the
stored path it was asked to re-resolve, and the in-place write is one
reset-reorder away from real UB. Resolve into a local first: correct
input, no aliasing trap.
- Coverage asymmetry: the probe and driver ran only on Linux while the
production edit covered all of POSIX. Both now run on macOS too, and the
Linux probe first proves the "(deleted)" state via readlink so its
assertions cannot pass vacuously on an intact image.
Revert-check: with main's http_server.c restored, the driver fails at
WEXITSTATUS 44 (resolver returned a dead path) on macOS; with the guard it
passes 63/63.
Co-authored-by: sunmingqiang <sunmingqiang@bytedance.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Prefer the saved executable launch path when it still points to an
executable, and reject a deleted /proc/self/exe path before spawning
index workers.
Add a Linux integration regression that forks, atomically replaces the
running test binary on disk, and verifies that subprocess resolution
uses the saved launch path instead of the now-deleted self path.
Signed-off-by: David Musk <davidmusk93@users.noreply.github.com>
Signed-off-by: sunmingqiang <sunmingqiang@bytedance.com>
The condition→action abstraction (Pennington 1987: the mental model
with the highest error rates, and the one code text serves worst),
served on demand. Design per the research synthesis: thin caller-only
spine (thin slicing), closed question menu, guard text ON the edge
in words (information scent), glyphs always paired with text
(Reacher's 22% missed-icon failures), complexity-gated (Scanlan),
declared elision, both directions (Green), honest footer.
- /api/why?id|qn&dir=up|down: callers (or callees) with the
SYNTACTIC guard chain around each call site — enclosing
if/else/switch-case/loop/ternary/catch conditions, outermost
first, else-arm negation detected — extracted by parsing the few
caller files live with the embedded tree-sitter grammars from the
call-site line the CALLS edges already carry. Zero indexing cost;
works on every existing index. Per-request parse cache; 2 MB file
cap; unparseable sources reported as guards_unavailable, never
guessed.
- /api/trace&guards=1: A-to-B paths gain per-hop guard chips -
'reachable in 5 hops' becomes 'reachable when ...' (the exact gap
Reacher's users hit: they hand-derived conditionals).
- Symbol page: 'When does this run?' tree - guard chips (amber
unless / blue when), loop marker, unguarded honestly labeled,
per-caller upward expansion, direction toggle, single-line answer
when every call site is unguarded, footer stating guards are
syntactic and dispatch/events are invisible.
- Trace results in Flows render the same chips per hop.
Tests: nested-if chain outermost-first + else-arm negation + loop
flag + downward direction + callsite helper on a real temp-dir C
file (C); /api/why route wiring; formatGuard/chain + the
complexity gate (vitest). Trap fixed en route: the extension table
keys WITH the dot ('.c'), not 'c'.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
GET /api/handout renders a self-contained, print-friendly HTML
document that explains the project to a newcomer: the region map
(cohesion as words, hubs), boundary spanners with a why-you-care
line, the main entry-to-effect journeys, and the churn x complexity
table. Linked from the Overview header ('Handout').
Honesty properties, tested: every graph-derived string is
HTML-escaped; no host paths leak into the artifact; caps are
declared inline ('20 of 33 regions shown', '10 of 60 journeys');
a project with no readable git history says so instead of skipping
silently; every fact is deterministic - the header states that no
model wrote any of it.
Exposing the same document as an MCP resource is recorded as a
follow-up (touches the MCP protocol surface; wants its own review).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
/api/blast buckets a file list by region (names included, 200-file
cap declared as truncated, every file accounted for: bucketed +
unmapped == files). The Changes tab feeds it the impacted symbols'
files (falling back to raw changed files) and renders the review
signal git cannot compute: 'your change reaches N files across K
regions' with per-region chips - disambiguated with the same
hub-suffix rule the galaxy uses, so two 'tests' communities don't
render as twins - and a seam warning at K>=3 suggesting a split or
citing the region names in the prompt.
Tests: blast bucketing + accounting + empty-input refusal (C),
/api/blast route wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The D6 trust sweep: every capped or scoped surface says what it hid
and what it means.
- Symbol connections: when the page cuts the list, the hidden
remainder is grouped BY FILE ('src/auth/session.c x 12') next to
the Show-more button - a bare '+N more' hides the answer to 'who
calls this' on exactly the nodes where it matters.
- Connection rows carry the neighbor's region; a row whose edge
crosses an architectural seam gets a quiet accent dot.
- The Data-flows panel no longer renders on projects with no
DATA_FLOWS edges at all: a section that can only ever say 'none'
is noise, not honesty.
- /api/bridges: boundary spanners ranked by DISTINCT foreign regions
a node calls into (cross-call count as tiebreak), test files
excluded and inbound direction ignored - orchestrators like main
and cbm_daemon_host_run surface; utility sinks and test harnesses
(which the naive both-directions metric ranked first) do not.
Overview shows the top 5 with a why-you-care line.
- Staleness banner was built and then REMOVED: no server signal
means 'files changed since indexing' today, and detect_changes
(a git-base diff) would have shown the wrong number. Recorded as
a follow-up instead of shipped with wrong semantics.
Tests: bridges need >=2 foreign regions (a dense gamma community
mirrors the beta fixture pattern), overflow tail accounts for
exactly what the page cut, per-row regions + data-flow presence
flag, /api/bridges route wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The D4 orientation package, mechanics adapted from the two reference
UIs studied in the screened snapshots:
- Search scent propagates up the hierarchy: a 'find a symbol' input on
the region sidebar lights matching regions in the 3D scene, pins hit
counts on the region rows and the minimap, and reports the total
with an outside-regions remainder. Served by /api/scent - a
case-insensitive name match bucketed per region via the cached
file->region map (File/Folder nodes excluded so a file never
double-counts its own symbols; sub-2-char queries refused).
- 2D minimap panel (bottom right, collapsible): every region as a dot
on a fixed top-down projection, sized by sqrt(members), colored by
region, open region ringed, scent counts above the dots, click to
open, and a Home button that fits the current scene.
- Region hover card: members/files, cohesion as a word (tightly /
moderately / loosely connected) with the score, and the hub.
- Node tooltip on graphite tokens (last pre-D1 hardcoded surface) and
in/out call degree - layout nodes now carry out_calls, which the
degree pass already computed and discarded.
- Honest scope note in the region detail panel: '+N callers outside
this region' whenever the full-graph in-degree exceeds the loaded
window.
Tests: scent bucketing + LIKE-escape refusal (C), /api/scent route
wiring, minimap projection + cohesion words (vitest), region-list
queries disambiguated from minimap titles.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The two questions the research says developers ask most and tooling
answers worst: why is this code the way it is (rationale via change
history and ownership), and does A reach B (LaToza reachability,
asked ~9x/day). Plus the user-requested data-flow lens.
- /api/trace: shortest A-to-B path over CALLS or DATA_FLOWS (BFS,
depth-capped at 12), endpoints by id or qualified name; the flow
cache gains a DATA_FLOWS CSR beside the CALLS one.
- Symbol bundle: file_history (recent commits, commits/year, top
author share from a bounded git log at metrics time, cached and
honest - available:false when no history was parsed) and
data_in/data_out with the stored edge detail.
- Flows tab: trace bar with symbol pickers and a control/data
toggle; the result path is clickable hop by hop.
- Symbol tab: 'Recent changes to this file' and 'Data flows'
panels with honest empty states.
- Tests: trace reachability both modes + endpoint errors, symbol
bundle data-flow/history shape, /api/trace route wiring, and a
fetchTrace client test.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The numbers view of a project, with the code behind every number one
click away and no composite health score — the cards are the score:
- /api/metrics (cached per project+indexed_at): complexity and function-
length histograms and top lists straight from pass_complexity's stored
cyclomatic/cognitive/lines numbers, edge certainty (CALLS vs
CALL_REFERENCE vs USAGE), dead-code candidates (same exclusions as the
galaxy classifier), TESTS linkage, SIMILAR_TO duplication, missed
files, documented-exports coverage.
- Per-file churn from one bounded git log pass (1 year, 10k commits, the
pass_githistory bounds) computed at metrics time — ZERO cost added to
indexing — plus churn × complexity, the classic where-the-bugs-live
list.
- A per-index history sidecar (JSONL in the cache dir, one snapshot per
indexed_at) feeds trend sparklines across reindexes.
- Dashboard tab: stat cards with warn tones, histograms, top lists
linking to symbol pages; churn honestly marked unavailable when the
project has no readable git history.
The wired-routes test covers /api/metrics; the ui suite pins totals,
certainty, histogram completeness and history presence on a fixture.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The human side of the Atlas program:
- Overview: the first read of a project — header numbers, region cards
with provenance and cohesion, hubs by fan-in, entry points, surprising
couplings with enumerated reasons, suggested questions with a why, and
the cross-package boundary table. Assembled client-side from the
read-only /rpc surface (get_architecture format:json) + the regions
payload; no new server code.
- Modules: a squarified treemap (in-house, no new dependency) of any
folder via /api/tree — sized by symbols, colored by dominant region,
hatched where the indexer missed files, breadcrumbed drill-down, file
panel listing symbols via search_graph.
- Symbol: one page per node via /api/symbol — docstring, flags, region,
editor/GitHub links, code on demand, callers/callees with true totals
and per-type confidence badges (resolved / reference / unproven),
tests, co-change partners, near-clones.
- Flows: the /api/flows journeys grouped across/within regions, each a
clickable indented call tree with truncation stated, mermaid export.
- Changes: detect_changes on the working tree grouped by risk, blast
radius citable into the prompt in one click.
- ⌘K: symbols (search_graph) + code text (search_code) in one box.
- Prompt composer: a basket of cited symbols/regions/flows/questions that
renders a prompt naming exact qualified names, file:line ranges and the
MCP calls that reproduce the context — the bridge back to the agent.
Routing carries every screen in the URL (tab, project, path, sym, flow,
node, region) so all of it deep-links. Fixed en route: the /api/flow
route pattern '/api/flow?*' never matched (req->path has no query
string) — now exact, pinned by a wired-routes test that distinguishes a
matched route's 400 from the static handler's 404.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph UI could call only list_projects and get_code_snippet through
/rpc; every other tool answered 403 'UI RPC method is not allowed'. That
starved the browser of schema, search, tracing, architecture and impact
data the server already computes, and broke the Projects tab, which needs
get_graph_schema (#1663).
Allow the read-only tools (get_graph_schema, search_graph, search_code,
trace_path/trace_call_path, get_architecture, query_graph, detect_changes,
check_index_coverage, index_status) and manage_adr in its read modes only
(default get, sections). A duplicated or non-string mode refuses, so a
write cannot ride in behind a read. Mutating tools keep their 403; the
loopback-only boundary is unchanged.
Fixes#1663.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ui_server_root_without_ready_assets_is_retryable asserted the pack architecture's
readiness semantics: the daemon warmed an external pack asynchronously, so a
request could legitimately arrive before the assets existed, and 503 +
Retry-After told the client to come back.
With the frontend linked into the image there is no warm-up window. Whether the
UI exists is decided at build time: a binary built without --with-ui has no
index.html and never will. Promising a retry for a condition that cannot change
would make every client poll forever, so this is a 404.
The replacement also asserts Retry-After is ABSENT, so the retryable behavior
cannot quietly come back with the header still attached.
macOS suites after this: cli 261 passed; ui/httpd/mcp/daemon* 469 passed with
this as the only failure, and httpd re-runs 62 passed / 0 failed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A pass over places where a bound was either miscalculated or missing.
UI response builders (src/ui/http_server.c) mixed the clamping helper
http_appendf with raw cursor indexing. Route the per-line separators and
quotes through the helper too, so the cursor cannot leave the buffer once
output saturates it, and terminate explicitly afterwards — the helper pins
the cursor and then writes nothing, which would otherwise leave the "%s"
reply with no terminator in range. The log buffer was budgeted at
LOG_LINE_MAX + 10 per line while JSON escaping can double every byte, so
size it for the escaped worst case. Job status entries now escape their path
and error fields, which were interpolated raw.
Cypher (src/cypher/cypher.c): the WHERE grammar descends once per nested '('
and once per NOT, so parse depth followed the query text rather than
anything bounded. Track depth on parser_t and refuse past 256 levels with a
parse error. Bounding parse depth also bounds the resulting tree, so the
recursive evaluator inherits the limit.
Gitignore (src/discover/gitignore.c): '**' retries the remainder at every
position and consecutive groups multiply, so match cost is exponential in
the number of groups. Thread a step budget through the matcher and give up
past 20000 steps, reporting no-match so a pathological pattern fails to
ignore rather than ignoring the wrong files. glob_match keeps its name so
recursion_whitelist.h still describes the functions that recurse; the new
non-recursive wrapper that seeds the budget is glob_match_bounded.
Call extraction (internal/cbm/extract_calls.c): extract_fp_callee recursed
once per applied argument, so stack use followed the parse-tree depth of the
indexed file. Rewritten as a left-spine loop. Reassigning node/nk before
continuing leaves the fall-through cases where the recursive form left them,
so behaviour is unchanged and the extraction suite is untouched.
CALLS props (src/pipeline/pass_calls.c): `cap - pos - PAIR_LEN` wraps once
pos reaches cap - PAIR_LEN and stops bounding the copy below. Guard
additively, matching the closing write in the same function.
Shell-bound paths (src/foundation/str_util.*): the three git shell-out sites
each wrap a repo path in cmd.exe-compatible double quotes, where %VAR%,
!VAR! and ^ remain active. Two carried private copies of the check that
rejects those three on Windows and the third used the bare validator.
Promote the stricter form to cbm_validate_shell_path_arg and route all three
through it, so the copies cannot drift apart again.
Tests cover the buffer bounds, the parse-depth refusal, matcher termination
and the validator. The bounds tests run in a forked child so a violation
surfaces as a signal instead of silently.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The build system now stamps CBM_SANITIZED_BUILD for any non-empty SANITIZE,
so timing budgets key off build-system truth instead of compiler-specific
__SANITIZE_ADDRESS__ probes that miss trap-UBSan and TSan; the two bench
suites use it.
The httpd interrupt test proved 'the interrupt worked' with a wall-clock
bound, which is a lottery under sanitizers. A send-deadline test hook pins
the deadline out of reach so the join itself is the causality proof, and
the stop watchdog becomes a true ten-second hang detector.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The UI handlers accumulate JSON with `pos += snprintf(buf + pos, size - pos,
...)`. snprintf returns the length it WOULD have written, so on truncation
pos ran past the end of the buffer and the next call's `size - pos` wrapped
to a huge value and wrote out of bounds — reachable through the directory
browser, whose readdir entries can fill the buffer before the trailing
parent/roots appends. Route every accumulator through a clamping
http_appendf() helper that pins pos at the buffer size on truncation. Adds a
forked-child regression test that browses a directory wide enough to fill the
buffer.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph-UI server binds to loopback only and reflected a localhost Origin
for CORS, but did not inspect the Host header — so a request routed to the
local port under a foreign name (a rebinding DNS record, a proxy) was served
normally, reaching the state-changing endpoints. Parse the Host header and
refuse any request whose Host is not a loopback address before dispatch; a
request with no Host (local HTTP/1.0 tooling) still proceeds. Adds a
live-socket regression test.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Closes the gap flagged in the distill review (test_security.c: "NOT proven
here: the full UI repro — listening socket + MSYS2 handle walk under a
single-threaded server"). Test-only; no production code changes.
Guard 1 — deterministic socket-isolation probe (any Windows, RED-able):
popen_isolates_listening_socket (tests/test_security.c) opens a real
inheritable AFD/listening socket — the exact handle class that deadlocked
git in #798 — then spawns THIS binary through cbm_popen (a cmd.exe
grandchild, git's spawn shape) in a new __cbm_sockprobe re-exec mode
(tests/test_main.c). The child reports via exit code whether that socket
handle is live in its address space: isolated spawn -> getsockopt fails ->
exit 0 (GREEN); raw-_popen regression leaks it transitively through
cmd.exe -> getsockopt succeeds -> exit 42 (RED). Verified RED with a local
spawn-inherits-all revert, GREEN with the isolated spawn.
Guard 2 — end-to-end liveness under live UI sockets (tests/test_httpd.c):
- ui_server_list_projects_responds_under_watchdog: POST /rpc list_projects
to the running single-threaded UI server with a client SO_RCVTIMEO
watchdog; a wedge -> no 200 -> hard FAIL, never an infinite CI hang.
- git_context_resolve_no_hang_under_live_ui_sockets: while the UI server
holds live listening/AFD handles in-process, run cbm_git_context_resolve
(the exact path list_projects takes) on a worker thread under a
WaitForSingleObject(30s) watchdog; a hang -> FAIL. SKIPs where git cannot
init a repo via system().
All three run in the existing security/httpd suites on the test-windows CI
leg; no new build wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018g9f9jr5S46BfKaDvDykBp
Signed-off-by: Flipper <jacobphilipp@ymail.com>
Distilled from #789 (safe features only). Adds backend dead-code classification
(status + in_calls in the layout JSON), a GET /api/repo-info endpoint for GitHub
deep-links, and the frontend dead-code filters, node code preview, and deep-links.
The render-cap revert was dropped (kept 2000 for DEFAULT_MAX_NODES/HARD_MAX_NODES
and GRAPH_RENDER_NODE_LIMIT); the sidebar regex-search refactor was omitted.
Security fixes over the original: repo-info strips credentials from any returned
remote_url; the legitimate https blob-URL construction is allow-listed so the
static gate passes; libgit2 is not re-initialized/shutdown per request (reuses the
process-wide init from cbm_alloc_init); deep-link path segments are URL-encoded.
Co-authored-by: Andy Zehady <azehady@ciroos.ai>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DELETE /api/project removed the project's .db file but never told the
watcher; the zombie watch kept polling the repo and the next change
silently resurrected the deleted DB (#803). Wire the watcher into the
UI HTTP server (non-owned ref; created before the server, freed after
it) and unwatch after a successful unlink and on ENOENT (clears an
already-orphaned watch, still 404). Invalid name stays a plain 404
with no unwatch; a non-ENOENT unlink failure stays 500 and keeps the
watch. The cbm_file_exists pre-check is replaced by unlink-errno
handling, removing the TOCTOU window.
Distilled from #830, rebased over the defer_state_free refactor
(#817): the OOM hardening of the deferred-free path moves into
defer_state_free itself so BOTH unwatch and stale-root prune get it.
On pending-free realloc failure the helper no longer frees the state
immediately (a potential use-after-free against an in-flight poll_once
snapshot) — it logs watcher.unwatch.oom and returns failure, and the
callers keep the entry registered; the prune path retries on the next
poll cycle.
Carries six httpd regression tests from #830 driving the live UI
server with a real watcher wired in.
Closes#803.
Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The CI lint gate (scripts/lint.sh --ci, clang-format-20) flagged 7 files whose
new lines were not run through the project formatter. Reformat only those lines
(line wrapping / continuation alignment) — no behavior change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The web server behind the graph UI is now a purpose-built in-house
module (src/ui/httpd.c): localhost-only listener, strict HTTP/1.1
parsing with fixed request caps, a per-connection receive deadline,
and Connection: close semantics. http_server.c keeps the routing and
handlers, rewritten against the new transport API; the public server
API and main.c are unchanged. The previously vendored third-party
server is removed entirely.
Comes with a new 28-test transport + routing suite (tests/test_httpd.c)
covering parsing edge cases (strict CRLF, Content-Length limits, raw
path matching, percent-decode rules) and live-socket behavior (CORS
policy, RPC dispatch, receive deadline, clean shutdown). The security
audit scripts now check the new file layout and treat any network call
in vendored code as a failure.
Martin Vogel
Bernhard Jackiewicz
Daichi Matsumoto
Rares Popa
FlipperandClaude Opus 4.8
Martin Vogel