Filter 13: background_jobs

[leseb]

Purpose

Handle background response processing. Queue, workers, timeout, cancellation, polling.

Behavior

• Queue: bounded channel (capacity 100)
• Workers: 10 long-lived tokio tasks pulling from queue
• Per-request timeout: 5 minutes
• On enqueue:
  • Store response with status=queued in response store
  • Return immediately with queued response object
  • Worker picks up, runs full filter chain, updates store
  • Handle QueueFull → reject (ensure no orphaned queued responses in storage)
• Cancellation:
  • Fetch response, check status
  • If queued: update to cancelled
  • If in_progress: cancel tokio task + update to cancelled
  • If already terminal: conflict error
  • Idempotent for already-cancelled
• Polling: clients GET the response to check status
• HTTP handler: POST /v1/responses/{id}/cancel

Config

filter: background_jobs
queue_size: 100
num_workers: 10
timeout_seconds: 300

Praxis trait methods

• on_request — if background=true, enqueue and return immediately with queued response
• Separate: worker loop that processes queued requests through the full filter chain

Dependencies

• Response store
• Tokio runtime

Reference

• Proposal: Responses API Filters — Filter 13
• Part of Epic: Responses API and Agentic Loop Orchestration #93

[leseb]

Implementation note for the cancel handler: make sure POST /v1/responses/{id}/cancel is not rejected by responses_format before background_jobs sees it. If responses_format is earlier in the same chain with on_invalid: reject, an empty POST body can be classified as non_json; the background filter should either run before that classifier for cancel paths or explicitly bypass/defer body-format validation for the cancel endpoint.

[leseb]

Sub-issues

This issue has been decomposed into two scoped sub-issues:

1. #503 — Queue, workers, and enqueue path (foundation)
2. #504 — Cancellation endpoint (depends on background_jobs: queue, workers, and enqueue path #44)