fix: agent session timeout caused by 30s per-action limit on slow user simulator responses

[yoavkatz]

Summary

Agent sessions time out when the user simulator (LLM) takes longer than 30 seconds to respond to an agent message. The per-action timeout in the MCP server is too short for simulator inference at higher token counts, causing sessions to abort mid-conversation before exchange_delivered_order_items (or any final action) is ever called.

Trace Reference

• Trace ID: tr-6635ef947049886b3b143e621b508cab
• Benchmark: tau2, Task: 0 (retail — exchange keyboard + thermostat)
• Agent: tool_calling, Model: openai/Qwen3.6-35B-A3B
• Session ID: 5788bd81-33f1-471b-9ec2-c75a9110bd02

Root Cause Chain

1. Agent sends a large message requiring complex user simulator reasoning

At step 11, the agent dumped all 12 available keyboard variants and 5 thermostat variants as a raw list to the user, asking them to pick item IDs. This forced the user simulator (Qwen3.6-35B-A3B at temperature=0.0) to reason through the options — generating 1,990 output tokens of chain-of-thought — which took 57 seconds.

2. 30-second per-action timeout fires in mcp.py

# mcp.py:376-381
step_thread = threading.Thread(target=execute_step, daemon=True)
step_thread.start()
step_thread.join(timeout=30.0)          # ← fires after 30s

if step_thread.is_alive():
    return {"error": "Action execution timed out after 30 seconds"}

The message tool's sess.step() is blocking on user simulator inference. The 30s timeout fires 27 seconds before the simulator finishes.

3. a2a_executor.py interprets the timeout error as session completion

# a2a_executor.py:483,489,494-499
raise ValueError("Failed to run mcp call" + result_text)   # result has no status:success
...
except Exception as e:
    ...
    if "timed out" in str(e).lower() or "timeout" in str(e).lower():
        final_result = "Session completed (timeout)"        # ← session aborted here
        break

The session ends with final_result = "Session completed (timeout)" and eval score = 0.

Timeline from tau2_session.log:

11:41:37  — Agent sends message with variant list (step 11)
11:42:04  — tau2 logs show agent message received  (+27s from agent's LLM call start)
11:42:34  — User simulator responds (+57s from agent message, 1990 output tokens)
11:43:04  — Session terminated (timeout already fired at 11:42:07)

Contributing Factor: Agent over-delegates reasoning to the user

The agent had all the information needed to resolve the exchange unambiguously:

• User said: "clicky switches, RGB preferred — if not available, no backlight; Google Home for thermostat"
• Agent had get_product_details results for both products

A better agent would have resolved to 7706410293 (clicky, no backlight, full size) and 7747408585 (Google Assistant) and presented a concise confirmation, rather than asking the user to choose from 17 raw item IDs. Shorter agent messages → shorter simulator responses → less pressure on the timeout.

Also Contributing: Agent scanned all 5 orders before the named one

The user explicitly said "my order #W2378156", but the agent fetched all 5 orders sequentially before reaching #W2378156:

Step 4: get_order_details #W6247578  (T-Shirt — irrelevant)
Step 5: get_order_details #W9711842  (Sunglasses — irrelevant)
Step 6: get_order_details #W4776164  (Espresso Machine — irrelevant)
Step 7: get_order_details #W6679257  (Digital Camera — irrelevant)
Step 8: get_order_details #W2378156  ← the one the user specified

This wasted 4 LLM + tool round trips (~10s) that reduced the available time budget.

Fix Options

Option A — Increase the per-action timeout (quick fix)

The 30-second limit in mcp.py:378 is too aggressive for message actions whose latency depends on user simulator inference time. A value of 90–120 seconds would have allowed this session to complete. The timeout could also be made configurable per action type or per benchmark.

Option B — Make the message action timeout configurable separately

message tool calls involve an LLM round trip on the benchmark side. Other actions (DB lookups, order details) are fast and can keep 30s. A distinct timeout for is_message actions would avoid cutting off legitimate slow simulator responses.

Option C — Agent-level fix: resolve preferences before messaging user (separate issue)

The agent should synthesize the user's stated preferences against retrieved product data and present a specific confirmation proposal rather than a raw variant dump. This reduces simulator response length and latency, and is the right agent behavior regardless of timeout values.

Files

• exgentic/src/exgentic/interfaces/cli/commands/mcp.py:376-381 — per-action step timeout
• exgentic/src/exgentic/adapters/agents/a2a_executor.py:494-499 — timeout → session abort