-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Problem
When using CodeAgent with certain models, the code parsing fails with a SyntaxError because partial stop sequences (like </code without the closing >) are being included in the parsed code output.
Steps to reproduce
- Create a CodeAgent with a model that uses
</code>as the closing code block tag (the default) - Run a simple calculation task that requires the agent to execute Python code
- The model's generation stops at the stop sequence, but sometimes includes a partial match of the stop sequence (e.g.,
</codeinstead of</code>) - This partial tag gets parsed as part of the Python code, causing a SyntaxError
Example code:
import os
from smolagents import InferenceClientModel, CodeAgent
model = InferenceClientModel(
model_id="zai-org/GLM-4.6",
provider="novita",
token=os.getenv("HF_TOKEN"),
)
agent = CodeAgent(
model=model,
tools=[],
)
result = agent.run("What is 13483 times 7614734 divided by 349192 and squared?")
print(result)Actual behavior and error logs
─ Executing parsed code: ─────────────────────────────────────────────────────
result = (13483 * 7614734 / 349192) ** 2
final_answer(result)
</code
────────────────────────────────────────────────────────────────────────────
Code parsing failed on line 3 due to: SyntaxError
</code
^
Error: invalid syntax (<unknown>, line 3)
The partial stop sequence </code (without the closing >) is being included in the output and parsed as Python code.
Expected behavior
The code parser should receive clean output without any partial stop sequences. The agent should successfully execute the code without syntax errors.
Root cause
When the model's generation is stopped using stop sequences, sometimes the model stops partway through generating the stop sequence. For example, with stop sequence </code>, the model might generate </code and stop before completing the >. This partial tag is then included in the output text that gets sent to the code parser.
Environment:
- OS: macOS 24.6.0
- Python version: 3.12.10
- Package version: smolagents 1.23.0.dev0
Additional context
This issue appears to be more prevalent with certain models and providers. It may be related to how different model APIs handle stop sequences - some may include partial matches in the output when generation is terminated.
Checklist
- I have searched the existing issues and have not found a similar bug report.
- I have provided a minimal, reproducible example.
- I have provided the full traceback of the error.
- I have provided my environment details.
- I am willing to work on this issue and submit a pull request.