Skip to content

[Agent] Log shipper max batch (500) exceeds the API's 200-entries-per-request cap — full batches rejected wholesale #2397

Description

@ToddHebebrand

Description

The agent log shipper accumulates up to 500 entries per flush, but the API logs endpoint rejects any request with more than 200 entries. A full batch therefore 400s as a unit, and since 4xx responses are never retried, all 500 entries are dropped — precisely under bursty logging, when logs matter most.

Root Cause

  • agent/internal/logging/shipper.go:36defaultMaxBatchSize = 500 (buffer also 500).
  • apps/api/src/routes/agents/logs.ts:37logs: z.array(agentLogEntrySchema).max(200).
  • agent/internal/logging/shipper.go:322-333 — any 4xx drops the whole batch, no retry (correct policy for genuinely bad entries, wrong outcome for a size-shape mismatch).

Same burn-the-batch failure class as #2386 (oversized fields), which PR #2392 fixed with a pre-ship fields size cap — the batch-count dimension was flagged there and deliberately left for this issue.

Proposed Fix

Chunk flushes agent-side to ≤200 entries per request (the server cap must stay — self-hosted API versions vary). Keep buffer/batch accumulation at 500 if useful; just split the HTTP sends. Add a test that a 500-entry flush produces 3 requests and a mid-chunk 4xx only drops that chunk.

Affected Files

  • agent/internal/logging/shipper.go (primary)

Found during #2386 (PR #2392 review).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions