Skip to content

test: Implement end-to-end integration tests for stream processing pipeline #196

Description

@Xhristin3

Problem Statement

The stream processing worker (xstreamroll-processing/) has excellent unit tests for individual components (StreamSession, SessionRegistry, EventFilter, LifecycleManager, Metrics, Logger) but no integration test covering the full pipeline: poll → filter → session → publish. The worker orchestration in worker.ts is completely untested.

Evidence

xstreamroll-processing/__tests__/ contains 7 test files, all unit tests. No test exercises the full flow from API polling to session creation to event publishing. The worker.ts file has no test file at all.

Impact

A regression in the polling loop, session routing, or event filtering would only be caught in manual testing or production. The worker's startup, shutdown, and error recovery behaviors are untested. The graceful shutdown coordinator's integration with the session drain is untested end-to-end.

Proposed Solution

Create an integration test that:

  1. Mocks the API server (using nock or a simple HTTP server)
  2. Starts the worker with the mock API URL
  3. Mocks /streams/pending to return test events
  4. Mocks /streams/processed to capture published events
  5. Verifies that events flow through the pipeline correctly
  6. Tests capacity back-pressure (return "capacity" when full)
  7. Tests event filtering (blocked events not published)
  8. Tests graceful shutdown (worker stops gracefully)

Technical Requirements

  • Must mock HTTP endpoints (not call real API)
  • Must test with realistic timing (poll interval configurable in test)
  • Must verify processed events match expected output
  • Must test error recovery (API down, returns)

Acceptance Criteria

  • Single event: polled → filtered → session → published correctly
  • Multiple events for same stream: routed to same session
  • Filtered events: never published
  • Capacity exceeded: event dropped (not crashed)
  • Graceful shutdown: remaining events flushed before exit
  • API errors: worker recovers and retries on next poll
  • Malformed events: dropped with warning (not crashed)

File Map

  • xstreamroll-processing/__tests__/integration/worker.integration.test.ts — new file
  • xstreamroll-processing/__tests__/integration/pipeline.integration.test.ts — new file
  • xstreamroll-processing/package.json — add nock dev dependency

Dependencies

  • Related: REPO-042 (fix shutdown hook)

Testing Strategy

  • Integration: Full pipeline test with mock HTTP server
  • Test concurrent event processing
  • Test worker startup with malformed API responses
  • Test shutdown during active processing

Security Considerations

No security impact. Verify mock server doesn't expose real ports.

Definition of Done

  • Code implemented and peer-reviewed
  • Tests written and passing (unit + integration where applicable)
  • Documentation updated if behavior changed
  • No new linting errors or type errors introduced
  • Security considerations addressed
  • PR linked to this issue and merged

Labels: testing, high impact
Priority: High
Difficulty: Advanced
Estimated Effort: 2d
Milestone: v1.0-alpha


Labels: testing,high impact
Priority: High | Difficulty: Advanced | Estimated Effort: 2d
Backlog ID: REPO-013

Metadata

Metadata

Assignees

Labels

GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardOfficial CampaignCampaign: Official Campaignhigh impacttestingTests and test coverage

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions