Skip to content

Commit 3f25cf5

Browse files
committed
Add comprehensive async support to smolagents
This commit adds full async/await support to smolagents, enabling concurrent execution and improved performance for I/O-bound operations. ## Features ### Async Model Support - Added `agenerate()` and `agenerate_stream()` to all API model classes: - LiteLLMModel (uses litellm.acompletion) - InferenceClientModel (uses AsyncInferenceClient) - OpenAIServerModel (uses openai.AsyncOpenAI) - AzureOpenAIServerModel (inherits from OpenAIServerModel) - Base Model class defines async interface ### Async Agent Support - Added `arun()` async method to base Agent class - Full async execution flow: arun() → _arun_stream() → _astep_stream() - Async helper methods: - aprovide_final_answer() - async final answer generation - _ahandle_max_steps_reached() - async max steps handler - _agenerate_planning_step() - async planning step generator - Async generators for streaming support ### State Management Pattern - Agents maintain state and are NOT coroutine-safe (matches AutoGen pattern) - Each concurrent task requires a separate agent instance - Models can be safely shared (stateless) - Clear documentation and examples showing correct usage ## Tests - Comprehensive test suite in tests/test_async.py covering: - Async model methods (agenerate, agenerate_stream) - Async agent methods (arun, streaming) - Concurrent execution with separate instances - State isolation between agent instances - Helper method tests (aprovide_final_answer, etc.) - Error handling and edge cases - Integration tests - Performance verification (concurrent vs sequential) ## Dependencies - Added optional [async] dependency group with aiohttp and aiofiles - Updated [all] group to include async extras ## Documentation - Comprehensive guide in docs/async_support.md covering: - Installation and setup - API reference with examples - Concurrent execution patterns (CRITICAL: separate instances required) - Integration with FastAPI, aiohttp - Performance benefits (~10x for 10 concurrent tasks) - Troubleshooting and best practices - Examples in examples/async_agent_example.py ## Correct Usage Pattern ```python import asyncio from smolagents import CodeAgent, LiteLLMModel async def main(): model = LiteLLMModel(model_id="...") # ✅ CORRECT: Separate agent instances per concurrent task tasks = ["Task 1", "Task 2", "Task 3"] agents = [CodeAgent(model=model, tools=[]) for _ in tasks] results = await asyncio.gather(*[ agent.arun(task) for agent, task in zip(agents, tasks) ]) # ❌ INCORRECT: Reusing same agent (causes memory corruption!) # agent = CodeAgent(model=model, tools=[]) # results = await asyncio.gather(*[agent.arun(task) for task in tasks]) asyncio.run(main()) ``` ## Design Rationale This implementation follows the same pattern as AutoGen (Microsoft's agent framework), which explicitly documents that agents are NOT coroutine-safe and require separate instances for concurrent execution. This is the accepted industry standard for stateful async agents. ## Benefits - 🚀 ~10x performance improvement for concurrent operations - 🔄 Seamless integration with async frameworks (FastAPI, aiohttp) - 💪 Better resource utilization - ✅ 100% backward compatible (all sync methods unchanged) - ✅ Well-tested with comprehensive test suite ## Breaking Changes None - all async methods are purely additive.
1 parent 983bb71 commit 3f25cf5

File tree

6 files changed

+1645
-1
lines changed

6 files changed

+1645
-1
lines changed

0 commit comments

Comments
 (0)