-
Notifications
You must be signed in to change notification settings - Fork 31
Open
Description
Description
There is a memory leak in RedisKeysSourceTask where the sourceOffsets list continuously grows without ever being cleared, leading to memory exhaustion in long-running connector deployments.
Steps to Reproduce
- Deploy a Redis Keys Source Connector monitoring high-volume key changes
- Let it run for an extended period
- Observe memory usage continuously increasing
- Memory leak occurs due to
sourceOffsets.add()calls incommitRecord()with no corresponding cleanup
Expected Behavior
Memory usage should remain stable for long-running connectors.
Actual Behavior
Memory continuously increases due to unbounded list growth, eventually leading to OutOfMemoryError in production deployments.
Root Cause
commitRecord()method adds offsets tosourceOffsetslist viaaddSourceOffset()commit()method is a no-op and never clears the list- The collected offsets are never used since Redis key monitoring doesn't require acknowledgment
Environment
- Component:
RedisKeysSourceTask - Impact: Production deployments with high-volume key monitoring
Fix
I've submitted a fix in PR [#53] that removes the unused sourceOffsets field and related code, eliminating the memory leak while preserving all functionality.
Metadata
Metadata
Assignees
Labels
No labels