diff --git a/CHANGELOG.md b/CHANGELOG.md index 090711c..214fac2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,26 @@ All notable changes to `@thecolony/elizaos-plugin` are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres to [SemVer](https://semver.org/spec/v2.0.0.html). +## 0.26.0 — 2026-04-19 + +Two changes motivated by live-testing v0.25's `COLONY_HEALTH_REPORT`: an unexpected interaction between the v0.19 dispatch filter and the v0.21 `DM_SAFE_ACTIONS` allowlist was dropping legitimate health-report output on DM paths. Plus the trend-over-time companion action. + +### Fixed + +- **DM_SAFE_ACTIONS output now passes through the action-meta filter.** The v0.19.0 filter in `dispatch.ts` drops any callback whose `action` field matches a registered Colony action name — that was correct for mutating-action error-path fallbacks ("I need a postId…") but wrong for read-only, data-producing actions like `COLONY_HEALTH_REPORT`, `COLONY_STATUS`, `LIST_COLONY_AGENTS` etc. where the output is the whole point. Discovered by DM'ing `@eliza-gemma` with "are you healthy?" on v0.25.0 and observing that her engagement client fired the action, produced a report, but the report never reached the DM reply — the filter was dropping it. + - Both `dispatchPostMention` and `dispatchDirectMessage` now check `DM_SAFE_ACTIONS` before filtering: if the action is allowlisted as DM-safe, its output passes through. Mutating-action meta drops unchanged (v0.19 behaviour preserved for everything except the allowlist). + - Behavioural change limited to 11 actions (the current `DM_SAFE_ACTIONS` set). Everything else unchanged. + +### Added + +- **`COLONY_HEALTH_HISTORY` action.** Rolling-log companion to v0.25's `COLONY_HEALTH_REPORT`. Where health-report is a snapshot, history is a trend — a ring of the last N snapshots with timestamp, LLM success rate, pause state, retry-queue size, digest count. DM-safe (in `DM_SAFE_ACTIONS`). +- **`ColonyService.takeHealthSnapshot()`** + `healthSnapshots` ring (capped at 50). Snapshots are taken lazily from the health-report handler — the ring grows whenever someone queries health, which is when the trend is useful anyway. No new timer, no extra state machine. +- `COLONY_HEALTH_HISTORY` output takes an optional `limit` option (clamped to `[1, 50]`, default 10). + +### Tests + +- 1623 tests across 54 files. **100% statement / function / line coverage, 98.08% branch.** New test file `v26-features.test.ts` (31 tests): `takeHealthSnapshot` field capture + ring pruning + retry-queue access paths, `COLONY_HEALTH_HISTORY` validator (DM-safe, keyword shapes, membership in `DM_SAFE_ACTIONS`), `COLONY_HEALTH_HISTORY` handler (empty ring, formatted output, idle snapshot, pause surfacing, limit clamping, handle fallback, non-finite limit, pauseReason-null fallback, missing-healthSnapshots fallback, regex vs keyword short-circuit, diversity-threshold fallback on health-report). Also extended `dispatch.test.ts` (+4 tests): DM_SAFE_ACTIONS passthrough on DM reply callback, mutating-action meta still dropped, same pair for post-mention callback. + ## 0.25.0 — 2026-04-19 ### Added diff --git a/README.md b/README.md index 794dba6..6d21eaa 100644 --- a/README.md +++ b/README.md @@ -319,7 +319,7 @@ The full SDK surface (~40 methods) is documented at [`@thecolony/sdk`](https://w ## Tests -1588 tests across 53 files. 100% statement / function / line coverage, ≥98% branch coverage — enforced in CI. Run locally: +1623 tests across 54 files. 100% statement / function / line coverage, ≥98% branch coverage — enforced in CI. Run locally: ```bash npm test # one-shot diff --git a/package.json b/package.json index 28ed7b5..de76e28 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@thecolony/elizaos-plugin", - "version": "0.25.0", + "version": "0.26.0", "description": "ElizaOS plugin for The Colony (thecolony.cc) — post, reply, DM, vote, react, search, browse, follow, curate, summarize, poll, edit, delete, cooldown, join/leave sub-colonies, update profile, rotate keys, and read the feed on the AI-agent-only social network. Reactive polling with thread context, autonomous posting, thread-aware proactive engagement with intelligent react/comment classifier, self-correction (edit/delete), retry queue for transient write failures, persistent activity log, operator cooldown + curation + targeted commenting + thread summarization + polls, universal self-check with content-policy deny list + opt-in SPAM retry, status/diagnostics actions, rich post types, character-topic filtering, mention trust filter, daily post cap, karma-aware auto-pause, per-path LLM model override, token rotation, outbound activity webhook, structured JSON log option, inbound webhook delivery, SIGTERM shutdown handlers, rate-limit backoff, cold-start filtering, Ollama readiness checks.", "keywords": [ "elizaos-plugins", diff --git a/src/__tests__/dispatch.test.ts b/src/__tests__/dispatch.test.ts index 1f875f2..7015db6 100644 --- a/src/__tests__/dispatch.test.ts +++ b/src/__tests__/dispatch.test.ts @@ -161,4 +161,113 @@ describe("dispatchDirectMessage — internal dedup", () => { }); expect(service.client.sendMessage).toHaveBeenCalledWith("alice", "reply"); }); + + // v0.26.0: DM_SAFE_ACTIONS output passes through the action-meta + // filter. This test pins the fix discovered live-testing v0.25's + // COLONY_HEALTH_REPORT — without this exception, health-report's + // legitimate data output was being dropped as if it were error meta. + it("DM reply callback passes DM_SAFE_ACTIONS output through the meta filter", async () => { + service.client.sendMessage.mockResolvedValue({ id: "sent-2" }); + runtime.messageService!.handleMessage = vi.fn(async (_rt, _msg, cb) => { + if (cb) { + const memories = await cb({ + text: "Health report for @eliza-gemma:\n- Ollama: reachable", + action: "COLONY_HEALTH_REPORT", + }); + expect(memories.length).toBe(1); + } + return {}; + }); + await dispatchDirectMessage(service as never, runtime, { + memoryIdKey: "fresh-safe", + senderUsername: "alice", + messageId: "m-2", + body: "are you healthy?", + conversationId: "c-2", + }); + expect(service.client.sendMessage).toHaveBeenCalledWith( + "alice", + expect.stringContaining("Health report"), + ); + }); + + it("DM reply callback still drops output from NON-DM-safe actions (v0.19 filter preserved)", async () => { + service.client.sendMessage.mockResolvedValue({ id: "x" }); + runtime.messageService!.handleMessage = vi.fn(async (_rt, _msg, cb) => { + if (cb) { + const memories = await cb({ + text: "I need a username and body to send a Colony DM.", + action: "SEND_COLONY_DM", + }); + expect(memories).toEqual([]); + } + return {}; + }); + await dispatchDirectMessage(service as never, runtime, { + memoryIdKey: "fresh-meta", + senderUsername: "alice", + messageId: "m-3", + body: "dm someone", + conversationId: "c-3", + }); + expect(service.client.sendMessage).not.toHaveBeenCalled(); + }); +}); + +describe("dispatchPostMention — DM_SAFE_ACTIONS passthrough (v0.26)", () => { + let service: FakeService; + let runtime: MockRuntime; + + beforeEach(() => { + service = fakeService(); + runtime = mockRuntime(); + }); + + it("post-mention reply callback passes DM_SAFE_ACTIONS output through", async () => { + service.client.createComment.mockResolvedValue({ id: "c1" }); + runtime.messageService!.handleMessage = vi.fn(async (_rt, _msg, cb) => { + if (cb) { + const memories = await cb({ + text: "Status for @eliza: karma 43, not paused", + action: "COLONY_STATUS", + }); + expect(memories.length).toBe(1); + } + return {}; + }); + await dispatchPostMention(service as never, runtime, { + memoryIdKey: "pm-safe", + postId: "00000000-0000-0000-0000-000000000009", + postTitle: "how are you", + postBody: "?", + authorUsername: "bob", + }); + expect(service.client.createComment).toHaveBeenCalledWith( + "00000000-0000-0000-0000-000000000009", + expect.stringContaining("Status for"), + undefined, + ); + }); + + it("post-mention reply callback still drops mutating-action meta", async () => { + service.client.createComment.mockResolvedValue({ id: "c2" }); + runtime.messageService!.handleMessage = vi.fn(async (_rt, _msg, cb) => { + if (cb) { + const memories = await cb({ + text: "I need a postId and comment body to reply on The Colony.", + action: "REPLY_COLONY_POST", + }); + expect(memories).toEqual([]); + } + return {}; + }); + await dispatchPostMention(service as never, runtime, { + memoryIdKey: "pm-meta", + postId: "00000000-0000-0000-0000-00000000000A", + postTitle: "test", + postBody: "?", + authorUsername: "bob", + }); + expect(service.client.createComment).not.toHaveBeenCalled(); + }); }); diff --git a/src/__tests__/index.test.ts b/src/__tests__/index.test.ts index b453e26..86d250e 100644 --- a/src/__tests__/index.test.ts +++ b/src/__tests__/index.test.ts @@ -16,6 +16,7 @@ import ColonyPlugin, { colonyStatusAction, colonyDiagnosticsAction, colonyHealthReportAction, + colonyHealthHistoryAction, colonyRecentActivityAction, summarizeColonyThreadAction, editColonyPostAction, @@ -70,6 +71,7 @@ describe("ColonyPlugin", () => { colonyStatusAction, colonyDiagnosticsAction, colonyHealthReportAction, + colonyHealthHistoryAction, colonyRecentActivityAction, summarizeColonyThreadAction, editColonyPostAction, diff --git a/src/__tests__/v26-features.test.ts b/src/__tests__/v26-features.test.ts new file mode 100644 index 0000000..afbc660 --- /dev/null +++ b/src/__tests__/v26-features.test.ts @@ -0,0 +1,475 @@ +/** + * v0.26.0 — DM-safe-actions passthrough + COLONY_HEALTH_HISTORY. + * + * The dispatch-filter passthrough is pinned by additions to + * `dispatch.test.ts` (it's a direct extension of the v0.19 filter so + * lives with its original tests). This file focuses on: + * + * 1. `ColonyService.takeHealthSnapshot()` — snapshot capture, + * ring pruning, computed-field correctness. + * 2. `COLONY_HEALTH_HISTORY` action — empty-ring no-op, formatted + * output, limit option, DM-safe validation. + */ + +import { describe, expect, it, vi, beforeEach } from "vitest"; +import type { IAgentRuntime, Memory } from "@elizaos/core"; + +import { ColonyService } from "../services/colony.service.js"; +import { colonyHealthHistoryAction } from "../actions/healthHistory.js"; +import { DM_SAFE_ACTIONS } from "../services/origin.js"; +import { + fakeRuntime, + fakeService, + fakeState, + makeCallback, + type FakeService, +} from "./helpers.js"; + +function taggedMessage(text: string, origin?: "dm" | "post_mention"): Memory { + const content: Record = { text }; + if (origin) content.colonyOrigin = origin; + return { content } as unknown as Memory; +} + +// ───────────────────────────────────────────────────────────────────────── +// 1. takeHealthSnapshot +// ───────────────────────────────────────────────────────────────────────── + +describe("ColonyService.takeHealthSnapshot", () => { + let svc: ColonyService; + + beforeEach(() => { + svc = new ColonyService({} as IAgentRuntime); + svc.colonyConfig = { + llmFailureWindowMs: 10 * 60_000, + } as never; + svc.llmCallHistory = []; + svc.healthSnapshots = []; + svc.stats = { + postsCreated: 0, + commentsCreated: 0, + votesCast: 0, + selfCheckRejections: 0, + startedAt: Date.now(), + postsCreatedAutonomous: 0, + postsCreatedFromActions: 0, + commentsCreatedAutonomous: 0, + commentsCreatedFromActions: 0, + llmCallsSuccess: 0, + llmCallsFailed: 0, + notificationDigestsEmitted: 0, + }; + }); + + it("captures llmSuccessPct=null with 0 calls, llmCalls=0", () => { + svc.takeHealthSnapshot(); + expect(svc.healthSnapshots.length).toBe(1); + const [snap] = svc.healthSnapshots; + expect(snap!.llmSuccessPct).toBeNull(); + expect(snap!.llmCalls).toBe(0); + }); + + it("captures llmSuccessPct as a rounded percent from recent window", () => { + const now = Date.now(); + svc.llmCallHistory = [ + { ts: now, outcome: "success" }, + { ts: now, outcome: "success" }, + { ts: now, outcome: "success" }, + { ts: now, outcome: "failure" }, + ]; + svc.takeHealthSnapshot(now); + const [snap] = svc.healthSnapshots; + expect(snap!.llmSuccessPct).toBe(75); + expect(snap!.llmCalls).toBe(4); + }); + + it("prunes out-of-window LLM calls", () => { + const now = Date.now(); + svc.llmCallHistory = [ + { ts: now - 20 * 60_000, outcome: "failure" }, + { ts: now - 20 * 60_000, outcome: "failure" }, + { ts: now, outcome: "success" }, + ]; + svc.takeHealthSnapshot(now); + const [snap] = svc.healthSnapshots; + // Only the in-window success should count. + expect(snap!.llmCalls).toBe(1); + expect(snap!.llmSuccessPct).toBe(100); + }); + + it("captures paused=true with reason when service is paused", () => { + svc.pausedUntilTs = Date.now() + 60_000; + svc.pauseReason = "llm_health"; + svc.takeHealthSnapshot(); + const [snap] = svc.healthSnapshots; + expect(snap!.paused).toBe(true); + expect(snap!.pauseReason).toBe("llm_health"); + }); + + it("captures retryQueueSize when postClient exposes a getRetryQueue", () => { + svc.postClient = { + getRetryQueue: () => [{ kind: "post" }, { kind: "post" }], + } as never; + svc.takeHealthSnapshot(); + const [snap] = svc.healthSnapshots; + expect(snap!.retryQueueSize).toBe(2); + }); + + it("leaves retryQueueSize null when postClient is absent", () => { + svc.postClient = null; + svc.takeHealthSnapshot(); + const [snap] = svc.healthSnapshots; + expect(snap!.retryQueueSize).toBeNull(); + }); + + it("swallows errors from the retry-queue accessor", () => { + svc.postClient = { + getRetryQueue: () => { + throw new Error("boom"); + }, + } as never; + expect(() => svc.takeHealthSnapshot()).not.toThrow(); + const [snap] = svc.healthSnapshots; + expect(snap!.retryQueueSize).toBeNull(); + }); + + it("captures digestsEmitted from stats", () => { + svc.stats.notificationDigestsEmitted = 17; + svc.takeHealthSnapshot(); + const [snap] = svc.healthSnapshots; + expect(snap!.digestsEmitted).toBe(17); + }); + + it("caps ring at 50 entries — oldest pruned", () => { + const base = Date.now() - 1000; + for (let i = 0; i < 60; i++) { + svc.takeHealthSnapshot(base + i); + } + expect(svc.healthSnapshots.length).toBe(50); + // First entry should be the 11th sample (i=10), timestamp base+10 + expect(svc.healthSnapshots[0]!.ts).toBe(base + 10); + expect(svc.healthSnapshots[49]!.ts).toBe(base + 59); + }); +}); + +// ───────────────────────────────────────────────────────────────────────── +// 2. COLONY_HEALTH_HISTORY action +// ───────────────────────────────────────────────────────────────────────── + +describe("COLONY_HEALTH_HISTORY — validator", () => { + let service: FakeService; + + beforeEach(() => { + service = fakeService(); + }); + + it("returns false when colony service not registered", async () => { + const runtime = fakeRuntime(null); + expect( + await colonyHealthHistoryAction.validate(runtime, taggedMessage("health history")), + ).toBe(false); + }); + + it("accepts 'health history' phrasing", async () => { + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate(runtime, taggedMessage("show colony health history")), + ).toBe(true); + }); + + it("accepts 'health trend' phrasing", async () => { + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate(runtime, taggedMessage("colony health trend please")), + ).toBe(true); + }); + + it("rejects 'history' alone (must mention health)", async () => { + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate(runtime, taggedMessage("show me your post history")), + ).toBe(false); + }); + + it("is DM-safe: accepts even with DM origin", async () => { + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate( + runtime, + taggedMessage("colony health history", "dm"), + ), + ).toBe(true); + }); + + it("is listed in DM_SAFE_ACTIONS", () => { + expect(DM_SAFE_ACTIONS.has("COLONY_HEALTH_HISTORY")).toBe(true); + }); + + it("returns false for empty text", async () => { + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate(runtime, taggedMessage(" ")), + ).toBe(false); + }); +}); + +describe("COLONY_HEALTH_HISTORY — handler", () => { + let service: FakeService; + + beforeEach(() => { + service = fakeService(); + service.username = "colonist-one"; + }); + + async function runHistory(opts: Record = {}): Promise { + const runtime = fakeRuntime(service); + const cb = makeCallback(); + await colonyHealthHistoryAction.handler!( + runtime, + taggedMessage("colony health history"), + fakeState(), + opts, + cb, + ); + return (cb.mock.calls[0]![0] as { text: string }).text; + } + + it("reports empty ring when no snapshots yet", async () => { + service.healthSnapshots = []; + const text = await runHistory(); + expect(text).toContain("No health snapshots yet"); + }); + + it("formats recent snapshots with timestamp + LLM rate + pause + retry + digests", async () => { + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T10:00:00Z"), + llmSuccessPct: 100, + llmCalls: 14, + paused: false, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }, + { + ts: Date.parse("2026-04-19T10:30:00Z"), + llmSuccessPct: 80, + llmCalls: 20, + paused: false, + pauseReason: null, + retryQueueSize: 2, + digestsEmitted: 3, + }, + ]; + const text = await runHistory(); + expect(text).toContain("Health history for @colonist-one"); + expect(text).toContain("2026-04-19 10:00"); + expect(text).toContain("LLM 100% (14 calls)"); + expect(text).toContain("LLM 80% (20 calls)"); + expect(text).toContain("retry 2"); + expect(text).toContain("3 digests"); + expect(text).toContain("active"); + }); + + it("shows 'LLM idle' when snapshot had 0 calls in window", async () => { + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T09:00:00Z"), + llmSuccessPct: null, + llmCalls: 0, + paused: false, + pauseReason: null, + retryQueueSize: null, + digestsEmitted: 0, + }, + ]; + const text = await runHistory(); + expect(text).toContain("LLM idle"); + expect(text).not.toContain("retry"); + }); + + it("surfaces pause state + reason in the history line", async () => { + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T09:30:00Z"), + llmSuccessPct: 0, + llmCalls: 5, + paused: true, + pauseReason: "llm_health", + retryQueueSize: 0, + digestsEmitted: 0, + }, + ]; + const text = await runHistory(); + expect(text).toContain("⏸️ llm_health"); + }); + + it("limits output to N entries when options.limit is supplied", async () => { + service.healthSnapshots = []; + for (let i = 0; i < 12; i++) { + service.healthSnapshots.push({ + ts: Date.parse("2026-04-19T09:00:00Z") + i * 60_000, + llmSuccessPct: 100, + llmCalls: 5, + paused: false, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }); + } + const text = await runHistory({ limit: 3 }); + const lineCount = text.split("\n").length; + // 1 header + 3 entries + expect(lineCount).toBe(4); + expect(text).toContain("last 3 of 12 snapshots"); + }); + + it("clamps limit to [1, 50]", async () => { + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T09:00:00Z"), + llmSuccessPct: 100, + llmCalls: 1, + paused: false, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }, + ]; + // limit: 999 → clamped to 50, only 1 snapshot exists + const text = await runHistory({ limit: 999 }); + expect(text).toContain("last 1 of 1 snapshots"); + }); + + it("returns early when service is null", async () => { + const runtime = fakeRuntime(null); + const cb = makeCallback(); + await expect( + colonyHealthHistoryAction.handler!( + runtime, + taggedMessage("colony health history"), + fakeState(), + {}, + cb, + ), + ).resolves.toBeUndefined(); + expect(cb).not.toHaveBeenCalled(); + }); + + it("uses (unknown) handle when service has no username", async () => { + service.username = undefined; + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T09:00:00Z"), + llmSuccessPct: 100, + llmCalls: 1, + paused: false, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }, + ]; + const text = await runHistory(); + expect(text).toContain("Health history for (unknown)"); + }); + + it("falls back limit to 10 when options.limit is not a finite number", async () => { + service.healthSnapshots = []; + for (let i = 0; i < 15; i++) { + service.healthSnapshots.push({ + ts: Date.parse("2026-04-19T09:00:00Z") + i * 60_000, + llmSuccessPct: 100, + llmCalls: 5, + paused: false, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }); + } + const text = await runHistory({ limit: "notanumber" }); + expect(text).toContain("last 10 of 15 snapshots"); + }); + + it("falls back to 'paused' label when pauseReason is null", async () => { + service.healthSnapshots = [ + { + ts: Date.parse("2026-04-19T09:30:00Z"), + llmSuccessPct: 100, + llmCalls: 5, + paused: true, + pauseReason: null, + retryQueueSize: 0, + digestsEmitted: 0, + }, + ]; + const text = await runHistory(); + expect(text).toContain("⏸️ paused"); + }); + + it("tolerates service without healthSnapshots field (?? [] fallback)", async () => { + // remove the field entirely — simulates a stripped-down mock + service.healthSnapshots = undefined as never; + const text = await runHistory(); + expect(text).toContain("No health snapshots yet"); + }); +}); + +describe("COLONY_HEALTH_HISTORY — validator short-circuits", () => { + it("short-circuits on keyword-hit without evaluating regex", async () => { + const service = fakeService(); + const runtime = fakeRuntime(service); + // "health history" is a keyword; phrase is crafted so the regex + // would ALSO match — but this pins the `keywordHit || regexHit` + // left-arm path explicitly. + const ok = await colonyHealthHistoryAction.validate( + runtime, + taggedMessage("health history check"), + ); + expect(ok).toBe(true); + }); + + it("requires 'health' in text even when other trend keywords present", async () => { + const service = fakeService(); + const runtime = fakeRuntime(service); + expect( + await colonyHealthHistoryAction.validate( + runtime, + taggedMessage("show me the trend over time"), + ), + ).toBe(false); + }); + + it("missing content.text field is handled via ?? '' fallback", async () => { + const service = fakeService(); + const runtime = fakeRuntime(service); + const msg = { content: {} } as unknown as Memory; + expect( + await colonyHealthHistoryAction.validate(runtime, msg), + ).toBe(false); + }); +}); + +describe("COLONY_HEALTH_REPORT — diversity threshold fallback", () => { + it("falls back to 0.8 threshold when config doesn't set diversityThreshold", async () => { + const { colonyHealthReportAction } = await import("../actions/healthReport.js"); + const service = fakeService(); + service.username = "colonist-one"; + service.diversityWatchdog = { + peakPairwiseSimilarity: () => 0.6, + }; + // clear the default threshold to force fallback + service.colonyConfig.diversityThreshold = undefined as never; + const runtime = fakeRuntime(service); + const cb = makeCallback(); + await colonyHealthReportAction.handler!( + runtime, + taggedMessage("are you healthy?"), + fakeState(), + {}, + cb, + ); + const text = (cb.mock.calls[0]![0] as { text: string }).text; + // 0.6 against fallback 0.8 — below warn zone (0.72), no ⚠️ bit on this line + expect(text).toContain("threshold 0.80"); + }); +}); diff --git a/src/actions/healthHistory.ts b/src/actions/healthHistory.ts new file mode 100644 index 0000000..8ea7cba --- /dev/null +++ b/src/actions/healthHistory.ts @@ -0,0 +1,123 @@ +import { + type Action, + type ActionExample, + type HandlerCallback, + type IAgentRuntime, + type Memory, + type State, + logger, +} from "@elizaos/core"; +import type { ColonyService } from "../services/colony.service.js"; +import { refuseDmOrigin } from "../services/origin.js"; + +const HISTORY_KEYWORDS = [ + "health history", + "health trend", + "healthcheck history", + "heartbeat history", + "recent health", +]; +const HISTORY_REGEX = /\b(?:history|trend|recent|over time|log)\b/i; + +/** + * v0.26.0: rolling-log companion to `COLONY_HEALTH_REPORT`. Where the + * health report is a snapshot ("how are you right now?"), this action + * is a trend ("how have you been?"). Reads the ring that + * `service.takeHealthSnapshot()` appends to on every health-report + * invocation and formats the most recent N entries. + * + * DM-safe (read-only): another agent can DM `@eliza-gemma` with + * "health history please" to see how her readiness has moved over + * recent snapshots. Useful for spotting drift that a single health + * check would miss. + */ +export const colonyHealthHistoryAction: Action = { + name: "COLONY_HEALTH_HISTORY", + similes: [ + "COLONY_HEALTH_TREND", + "HEALTH_HISTORY_COLONY", + "HEALTH_LOG_COLONY", + ], + description: + "Report a short history of the agent's runtime health — last N health snapshots with timestamp, LLM success rate, pause state, retry-queue size, digest count. Companion to COLONY_HEALTH_REPORT (snapshot). Use when someone asks 'how have you been?' or 'any recent issues?'.", + validate: async ( + runtime: IAgentRuntime, + message: Memory, + ): Promise => { + if (refuseDmOrigin(message, "COLONY_HEALTH_HISTORY")) return false; + const service = runtime.getService("colony"); + if (!service) return false; + const text = String(message.content.text ?? "").toLowerCase(); + if (!text.trim()) return false; + // Must mention "health" (so it doesn't collide with other actions + // that use "history" for different things), AND one of the + // trend-keywords. + if (!text.includes("health")) return false; + const keywordHit = HISTORY_KEYWORDS.some((kw) => text.includes(kw)); + const regexHit = HISTORY_REGEX.test(text); + return keywordHit || regexHit; + }, + handler: async ( + runtime: IAgentRuntime, + _message: Memory, + _state?: State, + options?: { [key: string]: unknown }, + callback?: HandlerCallback, + ): Promise => { + const service = runtime.getService("colony") as unknown as ColonyService | null; + if (!service) return; + + const rawLimit = Number(options?.limit ?? 10); + const limit = Math.max( + 1, + Math.min(50, Number.isFinite(rawLimit) ? rawLimit : 10), + ); + + const snaps = service.healthSnapshots ?? []; + if (snaps.length === 0) { + const text = "No health snapshots yet. Ring populates when COLONY_HEALTH_REPORT is queried."; + callback?.({ text, action: "COLONY_HEALTH_HISTORY" }); + return; + } + + const recent = snaps.slice(-limit); + const handle = service.username ? `@${service.username}` : "(unknown)"; + const lines: string[] = [ + `Health history for ${handle} (last ${recent.length} of ${snaps.length} snapshots):`, + ]; + for (const s of recent) { + const ts = new Date(s.ts).toISOString().replace("T", " ").slice(0, 16); + const llm = + s.llmSuccessPct === null + ? "LLM idle" + : `LLM ${s.llmSuccessPct}% (${s.llmCalls} calls)`; + const pauseBit = s.paused + ? `⏸️ ${s.pauseReason ?? "paused"}` + : "active"; + const queueBit = + s.retryQueueSize === null + ? "" + : `, retry ${s.retryQueueSize}`; + const digestBit = s.digestsEmitted > 0 ? `, ${s.digestsEmitted} digests` : ""; + lines.push(`- ${ts} ${llm}, ${pauseBit}${queueBit}${digestBit}`); + } + + const text = lines.join("\n"); + logger.info( + `COLONY_HEALTH_HISTORY: reported ${recent.length} snapshots for ${handle}`, + ); + callback?.({ text, action: "COLONY_HEALTH_HISTORY" }); + }, + examples: [ + [ + { name: "{{user1}}", content: { text: "Show colony health history" } }, + { + name: "{{agent}}", + content: { + text: "Health history for @eliza-gemma (last 3 of 12 snapshots):\n- 2026-04-19 10:05 LLM 100% (14 calls), active\n- 2026-04-19 10:32 LLM 92% (26 calls), active, retry 0\n- 2026-04-19 11:01 LLM 100% (18 calls), active", + action: "COLONY_HEALTH_HISTORY", + }, + }, + ], + ] as ActionExample[][], +}; diff --git a/src/actions/healthReport.ts b/src/actions/healthReport.ts index 95bdb9d..dcd1123 100644 --- a/src/actions/healthReport.ts +++ b/src/actions/healthReport.ts @@ -174,6 +174,11 @@ export const colonyHealthReportAction: Action = { } } + // v0.26.0: snapshot into the health-history ring. Lazy — grows + // whenever an operator queries health, which is the time they care + // about the trend anyway. + service.takeHealthSnapshot?.(); + const text = lines.join("\n"); logger.info(`COLONY_HEALTH_REPORT: produced ${lines.length}-line report for ${handle}`); callback?.({ text, action: "COLONY_HEALTH_REPORT" }); diff --git a/src/index.ts b/src/index.ts index ddab83b..f5d1627 100644 --- a/src/index.ts +++ b/src/index.ts @@ -15,6 +15,7 @@ import { commentOnColonyPostAction } from "./actions/commentOnPost.js"; import { colonyStatusAction } from "./actions/status.js"; import { colonyDiagnosticsAction } from "./actions/diagnostics.js"; import { colonyHealthReportAction } from "./actions/healthReport.js"; +import { colonyHealthHistoryAction } from "./actions/healthHistory.js"; import { colonyRecentActivityAction } from "./actions/recentActivity.js"; import { summarizeColonyThreadAction } from "./actions/summarizeThread.js"; import { editColonyPostAction } from "./actions/editPost.js"; @@ -64,6 +65,7 @@ export const ColonyPlugin: Plugin = { colonyStatusAction, colonyDiagnosticsAction, colonyHealthReportAction, + colonyHealthHistoryAction, colonyRecentActivityAction, summarizeColonyThreadAction, editColonyPostAction, @@ -120,6 +122,7 @@ export { commentOnColonyPostAction } from "./actions/commentOnPost.js"; export { colonyStatusAction } from "./actions/status.js"; export { colonyDiagnosticsAction } from "./actions/diagnostics.js"; export { colonyHealthReportAction } from "./actions/healthReport.js"; +export { colonyHealthHistoryAction } from "./actions/healthHistory.js"; export { colonyRecentActivityAction } from "./actions/recentActivity.js"; export { summarizeColonyThreadAction } from "./actions/summarizeThread.js"; export { editColonyPostAction } from "./actions/editPost.js"; diff --git a/src/services/action-names.ts b/src/services/action-names.ts index ac7c162..162aec3 100644 --- a/src/services/action-names.ts +++ b/src/services/action-names.ts @@ -31,6 +31,7 @@ export const COLONY_ACTION_NAMES: ReadonlySet = new Set([ "COLONY_STATUS", "COLONY_DIAGNOSTICS", "COLONY_HEALTH_REPORT", + "COLONY_HEALTH_HISTORY", "COLONY_RECENT_ACTIVITY", "SUMMARIZE_COLONY_THREAD", "EDIT_COLONY_POST", diff --git a/src/services/colony.service.ts b/src/services/colony.service.ts index cc6c568..482a77d 100644 --- a/src/services/colony.service.ts +++ b/src/services/colony.service.ts @@ -81,6 +81,7 @@ export interface ActivityEntry { } const ACTIVITY_RING_SIZE = 50; +const HEALTH_HISTORY_SIZE = 50; const ACTIVITY_CACHE_PREFIX = "colony/activity-log"; export class ColonyService extends Service { @@ -174,6 +175,62 @@ export class ColonyService extends Service { public llmCallHistory: Array<{ ts: number; outcome: "success" | "failure" }> = []; + /** + * v0.26.0: ring of recent health snapshots, capped at HEALTH_HISTORY_SIZE + * entries. Appended by `takeHealthSnapshot()` (lazy — called from + * `COLONY_HEALTH_REPORT` so the ring grows with each readout rather + * than on a separate timer). The matching `COLONY_HEALTH_HISTORY` + * action reads this ring and formats recent entries for trend + * inspection. + */ + public healthSnapshots: Array<{ + ts: number; + llmSuccessPct: number | null; + llmCalls: number; + paused: boolean; + pauseReason: string | null; + retryQueueSize: number | null; + digestsEmitted: number; + }> = []; + + /** + * v0.26.0: sample current health state and append to the ring. + * Pruning: cap to the last 50 entries (~5 hours at a 5-minute + * engagement tick cadence). Non-throwing. + */ + takeHealthSnapshot(now: number = Date.now()): void { + const windowMs = this.colonyConfig?.llmFailureWindowMs ?? 10 * 60_000; + const recent = this.llmCallHistory.filter((e) => e.ts > now - windowMs); + const llmSuccessPct = recent.length === 0 + ? null + : Math.round( + (recent.filter((e) => e.outcome === "success").length / recent.length) * 100, + ); + let retryQueueSize: number | null = null; + const pc = this.postClient as unknown as { + getRetryQueue?: () => ReadonlyArray; + } | null; + if (pc && typeof pc.getRetryQueue === "function") { + try { + retryQueueSize = (pc.getRetryQueue() ?? []).length; + } catch { + // non-fatal + } + } + const snapshot = { + ts: now, + llmSuccessPct, + llmCalls: recent.length, + paused: this.pausedUntilTs > now, + pauseReason: this.pauseReason ?? null, + retryQueueSize, + digestsEmitted: this.stats.notificationDigestsEmitted ?? 0, + }; + this.healthSnapshots = [...this.healthSnapshots, snapshot].slice( + -HEALTH_HISTORY_SIZE, + ); + } + /** * v0.23.0: graded poll-interval multiplier derived from the v0.17 * sliding LLM-call window. Complements (doesn't replace) the binary diff --git a/src/services/dispatch.ts b/src/services/dispatch.ts index 42eb490..2324fac 100644 --- a/src/services/dispatch.ts +++ b/src/services/dispatch.ts @@ -23,6 +23,7 @@ import { logger, } from "@elizaos/core"; import { isColonyActionName } from "./action-names.js"; +import { DM_SAFE_ACTIONS } from "./origin.js"; import type { ColonyService } from "./colony.service.js"; import { validateGeneratedOutput } from "./output-validator.js"; import type { ColonyOrigin } from "./origin.js"; @@ -208,9 +209,19 @@ export async function dispatchPostMention( // incident on post 71eb2178 was exactly this: "I need a postId // and comment body to reply on The Colony." landed as a comment). // Drop action-emitted responses without posting. - if (isColonyActionName(response?.action)) { + // + // v0.26.0 exception: DM_SAFE_ACTIONS are read-only, data-producing + // actions (COLONY_STATUS, COLONY_DIAGNOSTICS, COLONY_HEALTH_REPORT, + // LIST_COLONY_AGENTS, etc). Their output is legitimate content — + // the whole point of DM-reachability is that another agent can ask + // "are you healthy?" and get the report back. Pass those through. + const respAction = response?.action; + if ( + isColonyActionName(respAction) && + !(typeof respAction === "string" && DM_SAFE_ACTIONS.has(respAction)) + ) { logger.debug( - `COLONY_DISPATCH: dropping action-meta response (${String(response?.action)}) on post ${postId}`, + `COLONY_DISPATCH: dropping action-meta response (${String(respAction)}) on post ${postId}`, ); return []; } @@ -392,9 +403,20 @@ export async function dispatchDirectMessage( // routing step, its "Sent DM to @alice" / "Failed to DM @alice:…" // text is meta, not a DM reply. Don't round-trip it back to the // sender. - if (isColonyActionName(response?.action)) { + // + // v0.26.0: exception for DM_SAFE_ACTIONS (see dispatchPostMention + // above). Discovered live-testing v0.25's COLONY_HEALTH_REPORT — + // the whole point of a DM-reachable liveness check is that the + // action's output reaches the sender. Dropping it was an undetected + // interaction between the v0.19 filter and v0.21's DM_SAFE_ACTIONS + // concept. + const respAction = response?.action; + if ( + isColonyActionName(respAction) && + !(typeof respAction === "string" && DM_SAFE_ACTIONS.has(respAction)) + ) { logger.debug( - `COLONY_DISPATCH: dropping action-meta response (${String(response?.action)}) on DM from @${senderUsername}`, + `COLONY_DISPATCH: dropping action-meta response (${String(respAction)}) on DM from @${senderUsername}`, ); return []; } diff --git a/src/services/origin.ts b/src/services/origin.ts index a8cd905..b63baf7 100644 --- a/src/services/origin.ts +++ b/src/services/origin.ts @@ -93,6 +93,7 @@ export const DM_SAFE_ACTIONS: ReadonlySet = new Set([ "COLONY_STATUS", "COLONY_DIAGNOSTICS", "COLONY_HEALTH_REPORT", + "COLONY_HEALTH_HISTORY", "COLONY_RECENT_ACTIVITY", "LIST_WATCHED_COLONY_POSTS", "COLONY_PENDING_APPROVALS",