chore: release v0.6.33 — ship pipeline auto-commit - #594
Merged
Conversation
…ging
Field report (0.6.31, Windows, Claude Code over stdio): a cold-index
uffs_search over MCP neither triggered the re-warm nor ever returned —
backgrounded by the host at 120 s, abandoned at its 1800 s idle
timeout, and unrecoverable even after the index was manually warmed
minutes later. The same query from the CLI warmed and answered in
~90 s; the same MCP query re-issued after warm answered in 1.96 s.
The gap: the readiness gate only reports the daemon's *initial*
Loading state. A shard that has tiered down (Parked after 30 min idle,
Cold after 24 h) reads Ready, so a query tool sailed through the gate
and blocked inside the daemon's re-warm with zero feedback — to an
agent host a silent 90 s block is indistinguishable from a hang.
The contract now, for every query tool (search, aggregate,
facet_values, info): before dispatching against drives that are not
Warm/Hot, the tool
1. starts the re-warm in a DETACHED request on its own daemon
connection — the daemon runs a received request to completion
regardless of what the host then does to the tool call, and the
per-letter single-flight dedup makes repeat triggers join the
in-flight load rather than duplicate it — and
2. returns immediately with a retry-shaped error mirroring the
startup gate: index warming, typically 30-120 s, poll uffs_status
(never gated) and retry.
The agent's loop becomes query → warming error → poll status → retry →
millisecond answer, and the warm proceeds no matter how impatient the
host is. Queries scoped to warm drives are never gated by some other
drive being cold; a pre-tiering daemon (no tier field) is treated as
warm, since it never demotes.
Push-style MCP progress notifications were considered and rejected:
hosts do not extend tool-call patience on progress events, so the call
would still read as hung — the pull model is the shape that works with
every host.
Where the original in-flight request actually evaporated (accepted,
then lost even though the daemon runs requests to completion) is still
open — but with this contract no query can sit in that window longer
than one drives-RPC round-trip.
The warming contract shipped in 979855c told agents to "poll uffs_status until every drive reports 'warm'". uffs_status reports no such thing — it returns the daemon's lifecycle state (Ready/Loading/ Refreshing), uptime, connections and PID, and never mentions drives or tiers. uffs_drives dropped `tier` too when mapping to its output. So the instruction was unfollowable, and worse: the two signals look alike, so an agent polling uffs_status on a fully parked index reads "Ready" and concludes the index is warm. That is not hypothetical. Observed on a box with all seven drives parked: the agent called uffs_status, saw `state: ready`, wrote "the daemon is not cold — that rules out the cold-index hang", and moved on to blame the query shape. It was reading the daemon's pulse and calling it the index's. uffs_status now answers the question it appeared to answer: * `index_ready` — true only when every drive is hot/warm; the field to poll while waiting out a warm, * `drives` — per-drive tier map ("C" -> "warm"), * both rendered in the text block, with the lifecycle `status` field documented as the daemon's, explicitly not the index's. The drives-RPC is best-effort: if it fails, the tool degrades to the old lifecycle-only view rather than failing the call. uffs_drives now carries `tier` in its structured output and annotates parked/cold rows inline — its `0 records` meant "body released", which read as an empty index. Same misreading in the CLI: the physical-drive table rendered parked shards as `indexed (0 records)`, which looks like a broken index on a machine holding 25M records. It now says `parked (re-warms on next query)` and takes the tier glyph. The warm-gate message points at `index_ready` by name and warns off the `status` field that caused the misdiagnosis.
Completes the status contract with the two corroborating signals a
field post-mortem asked for. A daemon with every drive parked reported
`{"state":"ready", uptime: 23900}` and nothing else, while the CLI in
the same moment showed `0 records`, `Index heap: 0 MB`, seven parked
shards. The agent read "ready" as "warm" and `uptime` as warm-duration,
then measured post-warm timings and cited them as proof nothing was
broken.
uffs_status now also returns `total_records` and `index_heap_mb`, and
labels uptime as PROCESS uptime, explicitly not how long the index has
been warm. Both numbers read 0 while parked and jump to ~25M / ~4.9 GB
warm, so "0 records, index heap 0 MB" states the parked case in terms
no one can misread as healthy — the tier map says it, and these two say
it again in different units.
Both probes are best-effort: a failed stats/drives RPC degrades the
view rather than failing the call.
Tests pin the conflation itself — parked tiers must name themselves
`parked` (never inherit the daemon's `Ready`), an absent tier reads
warm so old daemons are not gated forever, and record counts are
thousands-separated so `0` versus `24,988,343` is unmissable.
`just use-local` ran `taskkill /IM uffsmcp.exe /F` — by image name, unconditionally — to free the file so it could be replaced. That kills every uffsmcp on the box, including the stdio supervisors AI hosts spawn for interactive agent sessions. Those belong to other people's sessions, and one killed mid-request produces precisely the signature reported against the MCP bridge: the call is accepted, then silently lost, never returns, and stays lost afterwards. The supervisor exists to make this unnecessary. It watches its own image path and hot-swaps its worker when the bytes change, replaying the handshake so the host's connection never drops — the user manual has described that as a `use-local` rename-swap for some time. The installer never implemented the rename half; it just deleted, which Windows refuses for a running image, hence the kill. So: locked binaries are now RENAMED aside (`uffs.old0`, …), which Windows does permit — the running process keeps its handle to the old inode while the new binary takes the canonical path, and the supervisor upgrades itself underneath a live session. Sidecars are swept by the next install once the old process has exited. With replacement no longer needing a kill, teardown is scoped to lifecycle only: the daemon and the HTTP gateway are stopped gracefully and force-killed by their own reported PID if that fails. No image-name kill of uffsmcp remains. Note this does NOT explain the originally reported hang — that daemon had 6h uptime with no install running. It is a second, independent path to the same symptom, now closed so the two cannot be confused.
A hibernated drive is Cold, not Parked, and the physical-drive row labelled every demoted rung 'parked'. That hides which rung a drive is on precisely when you are looking — e.g. after 'uffs --daemon hibernate', which is how you force the state to test the re-warm path.
…ion) The PID-scoped teardown used `if let Some(pid) = … && …`, which needs edition 2024. install-bins.rs declares no edition, so rust-script compiles it on its default and `just use-local` failed outright: "let chains are only allowed in Rust 2024 or later". I verified the file with `--edition 2024` — the edition the workspace uses, not the one this file gets. Now checked against 2018, 2021 and 2024 so the script builds whatever rust-script picks.
Field validation of the warming contract passed end-to-end (cold → ready in 79 s, retry answered in 764 ms) with one gap: mid-warm the agent could see 6,070,621 records loaded but nothing said of how many. Progress was visible only because it polled, and even then it had a numerator without a denominator. Drive-count progress is not the answer — it lies. Four of seven drives warm was 57 % by drives and 24 % by records, because the three still cold (C, D, S) are the large ones. The honest denominator is records. A demoted shard drops its body and reports `records: 0`, so the count has to be remembered from when it was resident. `DriveStats` is the natural home: the registry already preserves its `Arc` across every tier transition, so the value survives demotion with no new plumbing through the `new_parked` / `new_cold` rebuild constructors. It is also persisted in `DriveStatsSnapshot`, so a restarted daemon can report progress on its first load rather than only after being warm once. `uffs_status` gains `records_when_warm` and `warming_progress_pct` (record-weighted, capped at 100), and the text block reads "Resident: 6,070,621 records of 24,990,080 expected (24% warmed)" mid-warm — suppressed at 100 % where it would be noise, and when no drive has ever been warm, where a percentage would be a guess. `DriveInfo.records_when_warm` is `Option` + `skip_serializing_if`, so an older daemon simply omits it and the field reads as "no denominator yet" rather than breaking the client.
A search that spent 21 s paging a cold index back in and 1 ms scanning it reported `duration_ms: 1`. True, and badly misleading: the client cannot see that it just paid twenty seconds, so the expensive case is invisible exactly when someone asks where the time went. Measured in the field — the daemon booked ~21 s per query while the MCP response showed 1 ms and 203 ms. `SearchResponse` gains `promotion_ms`: time inside `ensure_warm_for_dispatch`, kept separate from `duration_ms` rather than folded in, because they answer different questions. `duration_ms` is "how fast is the index"; `promotion_ms` is "what did I pay to have an index at all". Merging them would make a warm search look slow; leaving it out hides the slow one. Timed unconditionally, not only under `--profile` — one `Instant` pair on a path that takes tens of seconds whenever it does anything, and a diagnostic you have to know to ask for is one nobody asks for. The failure paths report it too: a 30 s timeout that spent most of its budget promoting is a different diagnosis from one that spent it scanning. Surfaced where it gets read: `uffs_search` appends "+ 21000ms index warm-up" to its result header and carries `promotion_ms` in structured output; `uffs --profile` prints an "Index warm-up" line. Both silent at zero, so the steady state is unchanged and the line appears only when it explains something. Two files crossed the 800-LOC ceiling on the way. Rather than claim exceptions: the three early-out paths in `search.rs` were building near-identical empty `SearchResponse` literals — that duplication is what made adding a field a four-site edit, so it is now one `empty_response` helper. `main.rs` gives up its `--profile` renderer to a `client_profile` sibling, which is a better home for a purely presentational block regardless of line count.
`uffs --status -v` reported "(none)" under MCP Stdio Sessions on a box that was demonstrably serving one — a supervisor still holding a renamed-aside `uffsmcp.old0` image, proven alive because the file could not be deleted. The section has been blind on Windows since it existed. Two causes, both fatal on their own: The scan shells out to `ps -eo pid,ppid,etime,args`. Windows has no `ps`, so the enumeration returned nothing. "(none)" and "I cannot see" rendered identically — the worst kind of status output, and on the one platform UFFS actually ships to. It also matched only command lines containing `--mcp` AND `run`. AI hosts spawn the supervisor as a bare `uffsmcp` with no arguments, so even on Unix the match would have missed every real host-launched session. It found `uffs --mcp run`, which is not how anyone runs it. Windows now enumerates via `Get-CimInstance Win32_Process` filtered on the image name (`uffsmcp%`), which catches both the bare supervisor and a rename-swapped `uffsmcp.old0` still serving after an install. `resolve_parent_name` gets the same treatment — it was `ps`-based too, so every Windows session would have rendered without the "(parent: …)" tag that says which host owns it, the most useful field in the list. A supervisor running from a renamed image is now flagged stale, with the note saying what that means: still functional, superseded binary, picked up on the host's next start. Previously "stale" could only mean "older mtime"; after rename-swap it also means "the file I was launched from is no longer at the canonical path".
…n list The Windows scan landed and immediately showed three entries where there is one session: the real supervisor (parent `claude.exe`), its worker child, and the HTTP gateway — which has its own section directly above and is not a stdio session at all. Filtering on the image name catches every `uffsmcp` process; the list has to tell them apart. The gateway is now excluded by PID. A `uffsmcp` whose parent is another `uffsmcp` is labelled "worker of PID N" rather than counted as a session — worth showing, because a worker younger than its supervisor is the visible proof that a hot-swap happened, which is precisely the question an installer-survival check is asking. Staleness now compares process age against the installed binary's mtime, as the Unix path always has. The image-name check I shipped an hour ago cannot work: Windows keeps reporting the original `Name` after a file is renamed, so a supervisor running from `uffsmcp.old0` still self-reports as `uffsmcp.exe`. Age is the signal that survives a rename-swap. Workers are exempt — they are spawned from the new binary by definition, and flagging them would invert the meaning.
`uffs --daemon status -v` rendered "Drives: 7 loaded · 0 records" on a hibernated index. Both numbers are true and together they read as an empty index rather than a demoted one — and during a re-warm the line gives no way to see progress at all, which is exactly when someone is watching it. Field report, testing the MCP progress fields: "the CLI shows Drives: 7 loaded · 24,990,739 records with no indication of whether that's resident or expected, and no warming state at all." Correct, and the denominator was already on the wire. When any drive is parked/cold the headline now reads Drives: 7 loaded · 9,453,038 of 24,990,686 records resident (37% warmed) and stays in its original short form when everything is warm, so the steady state is unchanged. Same record-weighted percentage the MCP `uffs_status` reports, from the same `records_when_warm` denominator — a drive-count percentage would have read 71% at that moment, because the two drives still cold held 62% of the index.
Two follow-ups from a field review of the warming contract, both aimed at the same failure: a caller reading the payload and reaching the wrong conclusion without having to reason. "ready" now appears exactly once. The daemon's lifecycle enum has a `Ready` variant meaning "the process is up", and serialising it put `"state": "ready"` directly beside `index_ready: false` — two fields answering what looks like the same question with different answers, and a reader skimming for a readiness signal finds the wrong one first. That is the documented misread that produced a confident "the daemon is not cold" against seven parked shards. The process state is now `daemon_process`, rendered as "running" / "loading (3/7 drives)" / "refreshing (C, D)" — what it actually means — so the only "ready" left in the payload is the field that answers the question. The plateau now explains itself. A re-warm lands one drive at a time, so the record count sits still for tens of seconds while a large drive loads; three identical polls read as "hung", and giving up is the natural response to what looks like no progress. The daemon already knew which letter was in flight — the single-flight promote map — it just never said. `DriveInfo.loading` surfaces it, `uffs_status` reports `currently_loading` plus a "Loading now: S (counts plateau until it lands)" line, and the CLI headline appends "· loading S". Both were ranked by the reviewer as the two most likely remaining causes of a premature bail. Their third — that the CLI blocks during a warm — is not reproducible: 24 timed `--daemon status -v` calls during an active warm returned in 0.05-0.34 s, and a mid-warm CLI capture showed 5 of 7 drives with the other two cold. The earlier report compared two samples taken in separate calls minutes apart.
The watchdog polled `uffs --status --json` every 5 s, which is the full system view: an SCM query, a named-pipe probe with a timeout, and a process enumeration that spawns a shell — plus another shell per stdio session found. On Windows that is roughly four PowerShell launches per call, ~69,000 a day, to answer two booleans. Most of that expense is mine, from three commits ago. Before the Windows stdio scan existed, the enumeration tried to spawn `ps`, failed instantly on Windows, and cost nothing. Fixing the *display* silently taxed the *supervisor*, because both read the same command — a good reminder that a diagnostic and a control loop should not share a code path without someone checking the price. Two changes: `--status --json --brief` reports only the per-service `running` booleans, from a socket connect and a PID-file read. No broker probe, no process scan. That is what a supervisor actually needs, and it is what the watchdog now calls. The interval backs off. After six uneventful ticks it doubles each time toward a 60 s ceiling, and any event at all — a service found down, a stop honoured, a service seen for the first time — snaps it straight back to 5 s. Worst-case detection latency goes from 5 s to 60 s during genuine calm, which for a crashed background service nobody perceives; the moment anything moves, full rate returns. An unreadable probe deliberately does NOT count as an event: it must not pin the loop at full rate indefinitely when the CLI is missing. Both matter for correctness rather than tidiness. A health check expensive enough to be affected by system load can fail *because* the machine is struggling — exactly when a supervisor must not.
…t HEAD `uffs --version` reported `93d3ed70c-dirty` against a demonstrably clean tree, and kept reporting it across rebuilds. `git status --porcelain` returned nothing; an empty commit cleared it instantly. The stamp is computed in a build script whose only rerun triggers were `.git/HEAD`, `.git/logs/HEAD` and `RUSTC` — all about which commit is checked out. Cleanliness is not a property of HEAD, so once the flag was baked it stayed baked until HEAD happened to move. A build made while the tree was dirty went on claiming dirty forever after it was cleaned. The reverse is the dangerous direction and was equally broken: a build from a clean tree keeps claiming clean after the tree is edited. That is precisely the false reassurance the suffix exists to prevent — this session leaned on `Commit: <sha>` repeatedly to confirm which build was running, and it was answering from cache. `.git/index` is added as a third trigger. It is rewritten by `add`, `commit`, `checkout`, and by the stat-refresh a plain `git status` performs, so it fires on essentially every real clean↔dirty transition. Not airtight, and the comment says so: editing a tracked file and building with no git command in between leaves the index untouched. No cargo trigger can be airtight, because "working tree cleanliness" is not a file. The alternatives are worse — forcing a re-run on every build recompiles this crate and every binary downstream of it, and dropping the suffix loses the signal altogether.
The comment I shipped with the `.git/index` trigger claimed it fires on "essentially every real clean↔dirty transition". A live test disproved that on the commonest case: appending to a tracked file, running `git status`, and rebuilding produced no `-dirty` suffix. Measured rather than assumed — `git status` leaves `.git/index`'s mtime untouched for an unstaged content edit. It rewrites the index only when refreshing racily-clean stat entries, not when merely observing a modified file. So the trigger covers `add` / `commit` / `checkout` / `reset` and nothing else. The suffix is therefore best-effort, and stale in the "claims clean while edited" direction — the less useful direction — until some git command touches the index. The comment now says that plainly, with the rejected alternatives and their real costs, so the next reader does not inherit my overclaim. No behaviour change.
`just ship-fresh` aborted in its first step. toolchain-sync rejected nightly-2026-08-14, stepped back to 2026-08-13, rejected that too, hit the floor and gave up: "Current pin no longer compiles for x86_64-unknown-linux-gnu. Manual intervention required." Nothing was wrong with either toolchain. Both passed clippy (all-features and no-default-features) and the Windows xwin gate; both failed only the Linux probe, with: error: unable to search for static library …/libaddr2line-….rlib: ProcessFdQuotaExceeded That is EMFILE from zig's linker, and the cause is the opposite of what it looks like: the file-descriptor limit was too HIGH, not too low. The shell ran at `ulimit -n 1048576` — 4x this host's `kern.maxfilesperproc` of 245760 — and zig fails when RLIMIT_NOFILE is absurdly large. It was not exhaustion either; the system had 10,473 of 491,520 files open. Verified rather than reasoned: the identical `cargo +nightly-2026-08-14 zigbuild --target x86_64-unknown-linux-gnu --workspace --all-features --locked` that failed at 1048576 exits 0 at 4096, first for the package that failed and then for the whole workspace. Both zig call sites — `toolchain-sync` and `lint-ci-linux-zig` — now lower the limit to 4096 first. Lowered only, never raised, and failure to set it is ignored, so a host whose hard limit is already lower is untouched. Worth noting what made this expensive to diagnose: the sync treats any non-zero zigbuild exit as "this nightly is broken" and steps back a day. An environmental failure therefore walks the whole 21-day window and reports a toolchain regression that does not exist. Distinguishing transient failures from real ones is a separate fix; this one removes the failure that was triggering it.
The guard I added one commit ago did nothing on the shell that needed it. It read: if [ "$(ulimit -n)" != "unlimited" ] && [ "$(ulimit -n)" -gt 4096 ] I excluded "unlimited" to keep a non-numeric value away from `-gt`, and in doing so skipped the clamp on precisely the shells whose limit is worst. `unlimited` is not "no problem", it is the maximum of the problem. Verified across the cases: 1048576 fires, unlimited SKIPPED, 4096 skips — the middle one being the bug. Now clamps on `unlimited` as well, and prints the result: fd limit: 1048576 → 4096 (zig linker guard) The print is the more important half. A guard that silently does nothing looks identical to a guard that worked, which is why the first version survived a commit, a push, and another full ship attempt before anyone could see it had not fired. One line of output turns "did this apply?" from an inference into an observation. Same fix in `lint-ci-linux-zig`, which carried the same inverted test.
I had this exactly backwards, twice, and shipped both without reproducing the failure. macOS sets a soft `RLIMIT_NOFILE` of 256. A zig link of this workspace passes ~240 object files plus ~50 rlibs and needs more than that, so it dies with `ProcessFdQuotaExceeded` — which means what it says: too FEW descriptors. toolchain-sync reads that as "this nightly is broken", steps back a day, repeats to the floor, and aborts the ship claiming the current pin no longer compiles for Linux. Every candidate passed clippy and the Windows gate; only the Linux probe ever failed, always on descriptors. My first fix CLAMPED the limit down, theorising from this host's `kern.maxfilesperproc`. The "evidence" was worthless: my shell sat at 1048576, I lowered it to 4096 — still 16x above the failing value — and concluded lowering was the cure. I never once tested at 256. The second commit then fixed the clamp's inverted guard, making a no-op correctly conditional. What the diagnostic line finally showed was `fd limit: 256 → 256`, which ended the theory in one line. Measured this time, before committing: soft 256 → ProcessFdQuotaExceeded, exit 101 soft 256, raised 4096 → exit 0 then end-to-end from a 256-fd shell: `just toolchain-sync` reports "✅ linux zigbuild ok" and promotes the pin to nightly-2026-08-14. Only the SOFT limit is raised; the hard limit is the ceiling and lowering it is irreversible for the shell. Where the hard limit is below the target, we take what it allows. rust-toolchain.toml moves 2026-08-13 → 2026-08-14 as a result: the bump the sync had been trying and failing to make for three ship attempts, validated by its own four gates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
just shipPhase 2 auto-commit for v0.6.33 — the[workspace.package].versionbump inCargo.toml. This PR routes that commit through branch-protection rules. Once it merges tomain, runjust release-tagto cut the signedv0.6.33tag, which firesrelease.ymland builds the cross-platform binaries + GitHub Release v0.6.33. (No auto-tag on merge — the tag step is manual on-demand, Path B.)Auto-merge
--auto --squashis queued — GitHub will merge as soon as the required status checks pass. Squash is required becausemain-protectionmandates signed commits, and GitHub's rebase-auto-merge cannot sign the rebased commit; the squash-merge commit is signed by GitHub's own key, which satisfiesrequired_signatures: true. The original author's signed commit remains verifiable in the PR branch history.After merge
The auto-commit lived only on
release/v0.6.33, so localmainnever drifted — sync it with a plaingit pull --ff-only origin main(noreset --hardneeded).