Skip to content

perf: keep agent publication on declaration plane - #520

Merged
schickling merged 2 commits into
mainfrom
schickling-assistant/2026-09-09-agent-publish-latency
Sep 10, 2026
Merged

schickling merged 2 commits into
mainfrom
schickling-assistant/2026-09-09-agent-publish-latency

Conversation

@schickling-assistant

Copy link
Copy Markdown
Contributor

Problem

st2 agent publish built its prospective validation shadow by recursively copying runtime state from canonical agent directories that had no adjacent declaration. On a representative catalog with 140,255 files, one idempotent publication took 14.56 seconds and issued 135,708 unlinkat calls while cleaning the temporary shadow.

Bundle capture also traversed .workspace descendants even though the catalog contract defines only the empty .workspace directory as declaration input.

Goal

Keep publication work proportional to the declaration plane, not runtime-state size, while preserving both full validation passes and the complete transaction safety contract.

Decisions

  • Classify every exact agents/<host>/<identity> directory as a runtime-state parent, independent of whether an adjacent declaration exists. The filter remains path-structural; directories named resources elsewhere are not excluded.
  • Capture a bundle-root .workspace as one empty real-directory fact. Never enumerate, hash, copy, or publish its descendants.
  • Reuse one immutable discovery result within each full validation. Structural and per-host validation still run, and prospective admission plus locked live re-admission remain separate.
  • Use Unix socket descendants as deterministic traversal oracles instead of adding a host-dependent CI timing assertion.

Rejected: skipping a validation pass based on caller behavior. st2 agent publish remains a standalone fail-closed transaction.

Verification

Representative release-build publication, same catalog and declaration:

before: 14.56 s

after, five runs without exporter shutdown variance:
0.753 s  0.543 s  0.544 s  0.561 s  0.801 s
median: 0.561 s

With OTLP export enabled, the same five runs had a 0.855-second median; one run reached 1.181 seconds. Syscalls fell from 135,708 to 649 unlinkat and from 27,052 to 738 copy_file_range calls.

Passed locally:

cargo test --locked --lib --test agent_publish -- --test-threads=1
  workspace libraries: 817 passed
  agent_publish: 19 passed, 9 ignored by existing #498 quarantine

cargo test --locked --test validate --test catalog_graph -- --test-threads=1
  69 passed

nix build .#checks.x86_64-linux.st2 --no-link --print-build-logs
  passed

Pre-flip deviation: this repository has no devenv.nix, so devenv tasks run check:all is unavailable. The repository-owned hermetic Nix package gate above passed.

Complexity

No new dependency or public API. The capture mode adds one internal distinction between agent bundles and prepared catalogs.

Concerns

Telemetry exporter shutdown adds visible variance near the one-second boundary. This PR removes declaration publication I/O; it does not change telemetry flushing.

Open PRs #517, #490, and #160 also touch src/catalog_transaction.rs, but their current diffs do not overlap these hunks.

Friction & bottlenecks

The repository-owned Nix package gate took 8 minutes 40 seconds. The catalog-state traversal itself was the measured bottleneck addressed by this PR.

Follow-ups

Re-measure new-agent end to end after the optimized st2 revision is consumed. Publication was the dominant phase, but account selection, supervisor preflight, telemetry, and provider readiness remain outside this PR.

References

Posted on behalf of @schickling
field value
agent_identity dev3.direct.omp.fuksjr5z
session dev3.fuksjr5z
agent_persona generalist
agent_supervisor unavailable
agent_tool OMP
agent_tool_version 18.1.7
agent_runtime OMP 18.1.7
tooling_profile dotfiles@8d937c9

agent-identity: dev3.direct.omp.fuksjr5z
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.7
agent-runtime: OMP 18.1.7
tooling-profile: dotfiles@8d937c9
agent-identity: dev3.direct.omp.fuksjr5z
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.7
agent-runtime: OMP 18.1.7
tooling-profile: dotfiles@8d937c9
@schickling-assistant
schickling-assistant marked this pull request as ready for review September 9, 2026 21:06
@schickling

Copy link
Copy Markdown
Contributor

@codex

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-10T09:37:25.689602Z 348e017 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 348e017beb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@schickling
schickling merged commit cc0898d into main Sep 10, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants