feat: implement agent heartbeat and automatic dead-agent cleanup

[dubemoyibe-star]

Agent Heartbeat Mechanism

Problem

Agents register in backend/src/db/index.ts (agent_registry / agents table) but there is no heartbeat mechanism. If an agent process crashes or disconnects, it remains in the registry as online indefinitely. The coordinator in backend/src/coordinator/index.ts may assign tasks to dead agents.

Solution

Implemented a full heartbeat lifecycle spanning the database layer, API routes, agent client, coordinator, and server startup/shutdown.

Files Changed

Database Layer

• backend/src/db/agents.ts
  • Added status: 'online' | 'offline' to AgentRecord interface.
  • Added status TEXT NOT NULL DEFAULT 'online' column to the agents table schema.
  • Updated upsert() to persist status.
  • Updated list() to accept an optional status filter.
  • Added markOffline(olderThan: string) to AgentDb — bulk-updates agents to offline when lastSeenAt is older than the cutoff.

Type Definitions

• backend/src/types/agent.ts
  • Added status: 'online' | 'offline' to AgentRegistration so the coordinator can filter stale agents.

Coordinator

• backend/src/coordinator/coordinator.ts
  • In agentsFor(), filtered out agents whose status !== 'online' before sorting by cost and dispatching.

API Routes

• backend/src/api/routes/agents.ts
  • Added POST /api/agents/:id/heartbeat endpoint.
  • On hit, looks up the agent, updates lastSeenAt to new Date().toISOString(), sets status = 'online', and returns 204 No Content.
  • Returns 404 if the agent does not exist.
  • Updated POST /api/agents/register to set status: 'online' on registration.

Agent-Side Heartbeat

• backend/src/agents/heartbeat.ts (new)

  • HeartbeatClient pings POST /api/agents/:id/heartbeat every 30s (configurable).
  • Exposes start() and stop() for lifecycle management.
  • Logs warnings on failure without throwing.

• backend/src/agents/base/BaseAgent.ts

  • Imported HeartbeatClient.
  • Instantiated a HeartbeatClient in the constructor when apiBaseUrl is provided.
  • Added startHeartbeat() and stopHeartbeat() methods.

Agent Registry

• backend/src/agents/registry.ts
  • In initialize(), after a successful instance.register(), calls instance.startHeartbeat().
  • Added shutdown() that iterates all agents and calls stopHeartbeat().

Cleanup Service

• backend/src/services/agentCleanup.ts (new)
  • AgentCleanupService runs on a configurable interval (default 60s).
  • Each tick marks agents offline if lastSeenAt < now - ttlMs (default TTL 90s).
  • Exposes start() and stop().

Server Integration & Graceful Shutdown

• backend/src/index.ts
  • Imported AgentCleanupService and globalAgentRegistry.
  • Instantiated and started AgentCleanupService after agent initialization.
  • Updated graceful shutdown handler to call cleanupService.stop(), globalAgentRegistry.shutdown(), and stopAgentSync() before closing the HTTP server.

Tests

• backend/tests/agents.test.ts
  • Updated in-memory schema to include status column.
  • Updated codingAgent fixture to include status: 'online'.
  • Added test: returns 204 and updates lastSeenAt on heartbeat.
  • Added test: returns 404 when sending heartbeat to an unknown agent.

Registry Sync

• backend/src/registry/sync.ts
  • Updated Soroban event sync to set status: 'online' when upserting agents.

Acceptance Criteria Verification

Criterion	Status
last_seen_at / status columns exist in agents table	Implemented
POST /agents/:id/heartbeat updates timestamp	Implemented at /api/agents/:id/heartbeat
Agent-side heartbeat sends pings every 30s	Implemented via HeartbeatClient
Cleanup service removes stale agents (configurable TTL)	Implemented via AgentCleanupService
Coordinator skips offline agents in task assignment	Implemented in agentsFor() filter
Graceful shutdown stops heartbeat and cleanup	Implemented in index.ts shutdown handler

Migration / Compatibility Notes

• Existing agents tables will not be altered on install if they already exist (CREATE TABLE IF NOT EXISTS). To add the status column to an existing production database, run:

  ALTER TABLE agents ADD COLUMN status TEXT NOT NULL DEFAULT 'online';

• The lastSeenAt column was already present in the schema and registration code; this PR standardizes its update via heartbeat.

Testing Notes

• TypeScript compilation succeeds (npm run build).
• Tests that do not depend on native better-sqlite3 binaries pass locally (8 suites, 94 tests).
• Tests requiring native SQLite binaries (better-sqlite3) are currently skipped in the Windows environment due to missing prebuilt bindings; they are expected to pass in CI environments where the native module is compiled.

Closes #107

[vercel]

@dubemoyibe-star is attempting to deploy a commit to the Jaja's projects Team on Vercel.

A member of the Team first needs to authorize it.

[devJaja]

Nice Implementation @dubemoyibe-star
resolve the conflicts

[devJaja]

Fix the CI checks failing @dubemoyibe-star

[dubemoyibe-star]

Fix the CI checks failing @dubemoyibe-star

I'm on it chief

[devJaja]

@dubemoyibe-star
Still waiting for the fixes for your Pull request to be merged