-
Notifications
You must be signed in to change notification settings - Fork 28
Description
Problem Description
When using taskiq-redis with specific connection configurations, there's a potential for Redis timeout errors if the socket_timeout or socket_connect_timeout values in connection_kwargs are set to values smaller than xread_block // 1000.
This occurs because the blocking operation in Redis XREAD uses the xread_block parameter (in milliseconds), but the socket timeouts are specified in seconds. If the socket timeouts are too short relative to the blocking time, the connection may timeout before the XREAD operation completes.
Root Cause
- The Redis client uses
socket_timeoutandsocket_connect_timeoutto set maximum wait times for socket operations - XREAD operations with
blockparameter wait up toxread_blockmilliseconds for new data - If
socket_timeout < (xread_block / 1000), the socket may timeout while XREAD is still waiting for data
Example Scenario
from taskiq_redis import RedisStreamClusterBroker
# Problematic configuration
redis_broker = RedisStreamClusterBroker(
redis_url="redis://localhost:6379",
connection_kwargs={
"socket_timeout": 1, # 1 second
"socket_connect_timeout": 1, # 1 second
},
xread_block=5000, # 5 seconds blocking
)In this case, socket_timeout (1 second) is less than xread_block // 1000 (5 seconds), which will cause timeout errors.
Expected Behavior
The library should either:
- Automatically adjust socket timeouts to be compatible with
xread_blocksettings, OR - Provide clear warnings when misconfigured timeout values are detected
Actual Behavior
Timeout exceptions occur when:
socket_timeout < (xread_block / 1000)socket_connect_timeout < (xread_block / 1000)
Proposed Solution
Add validation that checks if either socket_timeout or socket_connect_timeout in connection_kwargs is less than xread_block // 1000 and issue a clear warning message.
Suggested warning message:
Warning: socket_timeout ({current_timeout}s) is less than xread_block // 1000 ({required_timeout}s). This may cause Redis timeout errors during blocking operations. Consider increasing socket_timeout to at least {required_timeout}s.
Implementation Details
Add a validation check during backend initialization that:
- Extracts
socket_timeoutandsocket_connect_timeoutfromconnection_kwargs - Compares them with
xread_block // 1000 - Issues warnings for any values that are too small
Environment
- taskiq-python Version: 0.11.18
- taskiq-redis Version: 1.1.0
- Python Version: 3.13.1
- redis-py Version: 6.4.0