feat(kap-server): add flat entity message protocol (v3 WS + history API) - #3532
Conversation
|
commit: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e28327a85d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| for (const item of materializer.materialize(op)) { | ||
| target.send(this.buildV3Envelope(state, v3ItemFrame(item, seq))); |
There was a problem hiding this comment.
When one transcript op batch materializes into multiple items, this sends each as a separate frame carrying the same seq. If the connection drops after the client receives and persists that sequence from an early frame, reconnecting with transcript_since skips the entire batch, permanently losing the remaining messages; the client also has no marker indicating which frame is last. Send the materialized batch atomically or add an item index/count or completion marker so the watermark can be committed safely.
AGENTS.md reference: AGENTS.md:L28-L28
Useful? React with 👍 / 👎.
| id: `remove.${this.removeCounter}`, | ||
| session_id: env.sessionId, | ||
| agent_id: env.agentId, | ||
| ids: [...op.ids], |
There was a problem hiding this comment.
When context.undone removes a turn that has already streamed steps and frames, onContextUndone puts only top-level item IDs and anchored interaction IDs in op.ids, and this forwards that list unchanged. Because v3 exposes each step and frame as an independent message, applying this removal deletes the turn message but leaves its step_*, text, thinking, and tool messages orphaned, diverging from the REST snapshot where the whole turn subtree is gone. Include every flattened descendant ID in the removal.
Useful? React with 👍 / 👎.
ed38e86 to
319e6e4
Compare
ff5795b to
704b4eb
Compare
704b4eb to
b552e27
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b552e27182
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| readonly sinceSeq: number; | ||
| /** Injectable for tests. */ | ||
| readonly fetchImpl?: typeof fetch; | ||
| return { messages: parsed.data.messages, inFlight: parsed.data.in_flight }; |
There was a problem hiding this comment.
The history response includes has_more, but this adapter discards it, while ChatChannel and fetchFullHistory infer continuation by comparing the returned flat-message count with the requested size of 500. The server caps page_size to 200 turn anchors for default/before_turn pages, so a history with additional older turns normally returns a count other than exactly 500 even when has_more is true; lazy chat paging and full-history plan lookup then stop early and silently omit older history. Return and consume parsed.data.has_more instead of deriving it from message count.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L38-L38
Useful? React with 👍 / 👎.
| const end = Math.min(messages.length, index + 1 + pageSize); | ||
| return { messages: messages.slice(index + 1, end), hasMore: end < messages.length }; |
There was a problem hiding this comment.
When more than pageSize messages were persisted after the requested step, this arbitrary message slice can end before any later terminal step or partway through one. The API exposes only an after_step cursor: repeating the old cursor returns the same page, while advancing to a terminal step included in the page skips that step's remaining entities because the server searches for its last occurrence. A reconnect catch-up can therefore never retrieve the complete suffix; page at complete step boundaries or return a message-level continuation cursor.
Useful? React with 👍 / 👎.
| this.state = { ...this.state, entries: next }; | ||
| this.applyTruncations(messages); |
There was a problem hiding this comment.
When reconnect recovery falls back to a replace refresh after an undo or clear, this assignment replaces only entries and retains the old interaction/task/todo maps. If a pending interaction and its tool call were removed while disconnected, the rebuilt page contains neither entity; applyTruncations also cannot discover the removed tool because it is already absent from next, so the obsolete pending interaction remains rendered and subsequent attempts to answer it target a nonexistent request. Rebuild the state maps from the replacement page while preserving only demonstrably newer live entities.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
… and history has_more
…d when no task records exist
… filter empty thinking
b552e27 to
7f8bcbd
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f8bcbd048
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| default: | ||
| return []; |
There was a problem hiding this comment.
When /sessions/:id:delete publishes event.session.deleted, this switch falls through to default, so connected v3 clients never receive the session message with subtype deleted that the protocol schema supports. Any client maintaining a session list from global v3 events therefore retains the deleted session until it performs a separate full refresh.
Useful? React with 👍 / 👎.
| applyLive(message: ServerMessage): void { | ||
| switch (message.type) { | ||
| case 'assistant.delta': { | ||
| this.patchText(`assistant:${message.message_id}`, message.text); |
There was a problem hiding this comment.
When a WS delta lands while a REST refresh is in flight, this appends the text but leaves the entity's timestamp at its initial value. A REST snapshot produced before the latest delta can consequently appear newer in preferHeld, overwrite the accumulated text, and make later deltas append after a missing chunk until another authoritative entity frame arrives; propagate the delta timestamp into the patched entity (and do the same for the analogous thinking/tool patches).
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
| for (const [toolCallId, tool] of tools) { | ||
| if (tool.turnId === id) removedKeys.add(`tool:${toolCallId}`); |
There was a problem hiding this comment.
When an undone or cleared turn contains a tool with an approval or question, markRemoved removes the tool order key but not interaction records whose toolCallId points to it. The cold history response therefore still contains an orphan interaction, which a fresh inspector renders as an unanchored interaction at the bottom even though its turn no longer exists; cascade those interaction keys while the removed tool IDs are still available.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6c387a79fd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const key = timelineKeyOf(message); | ||
| const index = this.state.entries.findIndex((entry) => entry.key === key); | ||
| if (index < 0) { | ||
| this.state = { ...this.state, entries: [...this.state.entries, { key, message }] }; |
There was a problem hiding this comment.
When reconnecting after missing a completed turn while a newer turn is in flight, the server sends recovery immediately after the ack, while the ack starts an asynchronous REST catch-up. The newer recovery turn therefore usually reaches the store first, and this append path subsequently places the missed, older catch-up turn after it; groupTimeline preserves that first-encounter order, so the chat and audit views show turns out of chronology. Tail pages need to be merged before already-newer recovery entities or recovery traffic must be held until catch-up completes.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
| info.path === undefined && revisionPaths.length > 0 | ||
| ? { ...info, path: revisionPaths.at(-1) } |
There was a problem hiding this comment.
When history contains multiple plan cycles and an older ExitPlanMode call lacks a path in its interaction, display, and output, this assigns revisionPaths.at(-1), even when that revision belongs to a later plan and occurs after the call. Querying the older call—or listing all plans—then displays the later plan's document path; associate revisions by their payload identity or timeline position instead of applying the globally newest path to every pathless plan.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
…ment audit The matrix carried claims that do not match the code: native/glob.rs and native/bash.rs were named as the Glob and Bash implementations (they are a 55-line matcher helper and timeout constants), read_media.rs and core_tool_defs.rs were placed outside src/tools/, compaction/micro.rs was "354 lines + 8 tests" (326 + 7), server/remote_control.rs was 1339 lines (1317), agent-core-v2 was 1018 files (1544), the reverse RPC set was "10 of 11" (9, one of them dead), the lib suite was 2107 tests (2349), and the claim that packages/kosong/src/providers/* was deleted is false — two model metadata modules remain with live consumers. Two alignment claims were overstated and are now marked as partial: the injection board omitted the permission_mode reminder entirely, and the ACP board omitted the non-ACP stopReason values, the missing $/cancel_request handling, the uncalled terminal/kill, the silently dropped additionalDirectories, and the Bash rerouting that hardcodes the shell and passes cwd=None. Section 6 records the audit method, the delta ratchet, the shipped fixes and the open work items (permission-mode reminder, MoonshotAI#3734, the eight unverified behavior commits, MoonshotAI#3532, and the ACP findings).
Upstream's v3 protocol (kap-server MoonshotAI#3532) stops shipping per-occurrence events and addresses every message by the tuple `agent_id:type:entity_id` instead, so that the live WS stream and the history API serve the same shapes. This is the first slice of the port: the addressing contract the rest of the protocol and the projection layer are keyed by. `EntityAddressed` carries a message's own id fields; the free functions probe them in upstream's order (message_id, tool_call_id, interaction_id, task_id, todo_id, system_id, step_id, turn_id, agent_id) and format the key. Two details here are wire contract, not style. `agent_id` is the last member of that probe chain, so `entity_id` falls back to it rather than treating it as a separate namespace — without the fallback every agent-scoped message that carries no id of its own would collapse onto the same empty key. And an empty id field falls through to the next candidate: every schema declares ids with a minimum length of 1, so an empty id can only come from a producer that disagrees with them, and falling through is safer than handing the projection an identity nothing can be keyed by. The module docs also record where the protocol came from: upstream introduced it in 64505e3 ("flat entity message protocol (v3 WS + history API)", design revision 1094) while this fork still carried packages/kap-server, which was retired three days later — so upstream changes to it merge here invisibly, and scripts/check-upstream-v2-delta.mjs is what keeps that from going unnoticed.
Upstream's v3 protocol (kap-server MoonshotAI#3532, design revision 1094) replaces per- occurrence events with 26 server message variants and 2 client frames, each addressed by `agent_id:type:entity_id`. This is the contract half of the port: the Rust types, plus the frozen upstream snapshot they were transcribed from. The union is one tagged enum, so a message parses into the shape its `type` names, and every variant implements EntityAddressed with the id fields its schemas declare. Optionality follows upstream exactly, which is not the same as "make it an Option": v3 declares no nullable field at all, so optional fields are skipped when absent rather than written as null, and the two required `unknown` fields (config.config, user.meta) stay required. Three shapes needed care. Upstream's session metadata is a `catchall` object, so the Rust struct keeps a flattened extra map instead of dropping unknown keys. System payloads are typed for undo/clear and opaque elsewhere, and the untagged enum therefore tries the typed shape first — with the open shape first it would swallow every payload. And upstream mixes epoch milliseconds with ISO strings; those stay i64 and String, because nothing downstream can tell them apart once they are both numbers. v3-message-contract.json mirrors the contract field by field and records where it came from: the upstream commit, the design revision, and the module directory. It is the input scripts/scan-parity.mjs reads, so the mirror and the Rust types cannot drift apart silently.
…story API) (#3532) (#3920) This reverts 64505e3. The v1 WS + legacy REST + transcript surfaces are alive on main and remain the single protocol surface; kimi-inspect returns to the transcript-based data model (keeping #3747's removal of the prompt input, which adapted to the agent-core-v2 prompt-queue fold).
The revert removed the code but left four places still describing v3 as a live protocol axis, which would send the next reader looking for a surface that no longer exists: - the root AGENTS.md protocol legend now records the retirement (upstream 2.0.2, fork §8.11) and states that v1 is the only axis; - ROADMAP's legend entry likewise, and item 4 (MoonshotAI#3532) — whose body is a long list of "done" milestones — is struck through with a note that its text is pre-revert audit history, not a description of current code; - kimi-inspect's AGENTS.md audit paragraph describes op batches instead of REST history pages and entity messages; - ChatView's header describes the v1 subscribe/reset/ops flow instead of the v3 WS plus paged history route. Verified: bun run lint 0 errors; kimi-inspect typecheck clean and 106 tests pass.
* feat(kimi-code): carry turn trace id and copilot stats in rating surveys (MoonshotAI#3907) * feat: expose resolved base_url on models.dev catalog provider items (MoonshotAI#3909) * fix(agent-core-v2): restore thinking for the openrouter reasoning dialect (MoonshotAI#3910) Keep string reasoning fields when a reasoning_details array is present, stamp each think part by source, and replay those fields on the next request. * ci: release packages (MoonshotAI#3862) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * docs(changelog): sync 2.0.1 from apps/kimi-code/CHANGELOG.md (MoonshotAI#3912) * fix(agent-core-v2): pre-shrink compaction history to the effective model window (MoonshotAI#3911) * fix(vscode): ignore Enter during IME composition in question dialog (MoonshotAI#3915) * fix(kap-server): deliver session-level interaction events past agent filters (MoonshotAI#3901) * revert(kap-server): drop the flat entity message protocol (v3 WS + history API) (MoonshotAI#3532) (MoonshotAI#3920) This reverts 64505e3. The v1 WS + legacy REST + transcript surfaces are alive on main and remain the single protocol surface; kimi-inspect returns to the transcript-based data model (keeping MoonshotAI#3747's removal of the prompt input, which adapted to the agent-core-v2 prompt-queue fold). * feat(oauth): parse goods_version from the managed /me profile payload (MoonshotAI#3921) * fix: keep turn ids above the wire-wide max and fold cold transcripts over the active branch (MoonshotAI#3922) * fix(agent-core-v2): floor the human turn clock at the wire-wide max on engine journal reset * fix(kap-server): fold the cold transcript snapshot over the restorable branch chain * fix(agent-core-v2): advance the human turn clock when a turn starts * fix(kap-server): split reused wire turn ids in the live transcript projector * fix(kap-server): adopt the completed cold tip for mid-turn attach events Continuation deltas after a lazy transcript attach belong on the last cold turn, not a newly split export id. * fix(agent-core-v2): remove the project-root assertion for cwd from the system prompt (MoonshotAI#3929) Co-authored-by: 7Sageer <7sageer@djwcb.cn> * chore: remove the tdd skill (MoonshotAI#3932) * fix(agent-core-v2): don't record turn.steer when an unconsumed steer seeds the next turn (MoonshotAI#3933) * chore: sync web dist from code-app (MoonshotAI#3934) * chore: sync web dist from code-app code-app: 44d7281c7a63ee3c7a907f9406e490efdbec7411 * chore: collapse web dist sync changesets into one summary entry * ci: release packages (MoonshotAI#3913) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * feat: generate native Kimi ACP session titles (#15) --------- Co-authored-by: Grapedge <shiwang.lj@alibaba-inc.com> Co-authored-by: liruifengv <liruifeng1024@gmail.com> Co-authored-by: Haozhe <yanghaozhe@moonshot.ai> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: 7Hanrui <qihanrui@moonshot.ai> Co-authored-by: 7Sageer <7sageer@djwcb.cn> Co-authored-by: Zixuan Chen <remch183@outlook.com>
Related Issue
N/A — internal protocol redesign per the "New API draft" design doc (flat, self-contained message union to replace the dual-track event/transcript streams).
Problem
kap-server currently ships two divergent delivery pipelines: 51 agent frame types + 19
event.*frame types (legacy lane) and transcript reset/ops (v2 lane). The same fact is projected twice, kept consistent by a hand-maintained suppression table; transcript's store/ops/granularity concepts force every consumer through an apply/store middle layer before WS data becomes usable; approvals, tasks, busy and other state domains have 2–3 competing sources of truth; and the declared schema has systematically drifted from what the server actually emits.What changed
Implements the next-generation protocol as a pure addition — v1/v2 lanes, legacy REST, and the transcript package are untouched, so existing clients keep working:
packages/kap-server/src/protocol/messages/): the zod single source for a flat union of 22 entity messages + 4 control messages (snake_case, no envelope, no seq/epoch/volatile/offset). Browser-safe; exported via the./protocolsubpath for clients to import directly.services/projection/): a single direct projector mapping agent-core-v2 event-bus events + service emitters + queryable services to entity messages, with zero dependency on the transcript package. Holds in-flight turn/step accumulation and current state entities; heals against wire.jsonl at turn end; validates every outbound message against the schema./api/v3/ws:hello → subscribe → ack → recovery payload → live. Recovery in three sentences: persisted state comes from REST, in-flight steps replay, state entities resend in full — no cursors, no journal, no seq/epoch. Per-session ordered sequence (recovery and live queued atomically); bounded outbound queue with a dedicatedWS_SLOW_CONSUMERdisconnect; protocol-level ping/pong.GET /api/v1/sessions/{id}/history: cold rebuild from wire.jsonl into the same flat entity messages (same schema, same id rules as the live projection), withbefore_turn/after_stepcursors and anin_flightmarker.system(undo,clear)truncation); dependency on@moonshot-ai/transcriptremoved.Zero changes to
agent-core-v2, to the v1/v2 lanes, to legacy REST, or to the transcript package.Verification: kap-server/kimi-inspect typecheck green; 67 contract + 24 projection + 20 WS + 27 history + 116 kimi-inspect tests green; 39/39 end-to-end smoke checks against a real server (handshake, live streaming, approvals, todo, subagent modes A/B/C, undo/clear, all three recovery scenarios, REST↔WS convergence); full repo suite green except pre-existing agent-core-v2 zip environment failures.
Checklist
gen-changesetsskill, or this PR needs no changeset. (No changeset: internal server protocol surface + kimi-inspect; not user-perceivable.)gen-docsskill, or this PR needs no doc update.