feat: inform LLM about runId background task in prompt_sandbox#257
Merged
sweetmantech merged 3 commits intotestfrom Mar 4, 2026
Conversation
When the sandbox is being set up for the first time, prompt_sandbox dispatches the command to a background task and returns a runId with empty output. The LLM was unaware of this and would try to interpret the empty result. Updated the tool description to explain that runId means a background task is running and the UI shows live progress. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (1)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe public tool description for Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ❌ 1❌ Failed checks (1 warning)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
lib/chat/tools/createPromptSandboxStreamingTool.ts (1)
55-60: Consider extracting the background task guidance to a named constant.The description is becoming verbose with multiple concatenated sentences. For consistency with
SANDBOX_PROMPT_NOTE(lines 5-11) and improved maintainability, consider extracting the runId guidance to a dedicated constant.♻️ Suggested refactor
export const SANDBOX_PROMPT_NOTE = "IMPORTANT: When you make changes to any files inside the orgs/ directory, " + "always commit and push those changes directly to main so they are preserved and shared across sessions.\n\n" + "IMPORTANT: When a prompt includes attached file URLs (e.g. from email attachments), " + "always download the files first using curl and save them locally before referencing them. " + "These URLs are temporary and expire after 1 hour. Never store the download URL directly in files — " + "download the content, save it to the appropriate location in the repo, and reference the local path instead."; +export const SANDBOX_BACKGROUND_TASK_NOTE = + "IMPORTANT: When the result contains a `runId`, it means the sandbox is being set up for the first time " + + "and the command was dispatched to a background task. The output will be empty because the task is still running. " + + "The UI automatically shows a live progress view for background tasks — do NOT summarize or interpret the empty output. " + + "Simply tell the user their request is being processed in the sandbox and the results will appear in the task progress view above. " + + "Do NOT automatically poll or check the task status — instead, let the user know they can ask you to check on it whenever they want."; + const promptSandboxSchema = z.object({Then use it in the description:
description: "Send a prompt to the agent running in the artist's persistent sandbox environment. " + "This is your primary tool — use it for release management (creating, updating, or reviewing releases), " + "file operations, data analysis, content generation, and any multi-step task. " + "The sandbox has skills for managing RELEASE.md documents, generating deliverables, and more. " + "Reuses the account's existing running sandbox or creates one from the latest snapshot. " + - "Streams output in real-time. " + - "IMPORTANT: When the result contains a `runId`, it means the sandbox is being set up for the first time " + - "and the command was dispatched to a background task. The output will be empty because the task is still running. " + - "The UI automatically shows a live progress view for background tasks — do NOT summarize or interpret the empty output. " + - "Simply tell the user their request is being processed in the sandbox and the results will appear in the task progress view above. " + - "Do NOT automatically poll or check the task status — instead, let the user know they can ask you to check on it whenever they want.", + "Streams output in real-time. " + + SANDBOX_BACKGROUND_TASK_NOTE,🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@lib/chat/tools/createPromptSandboxStreamingTool.ts` around lines 55 - 60, Extract the long runId/background-task guidance from the description into a named constant (e.g., RUNID_BACKGROUND_TASK_NOTE) and reuse it in createPromptSandboxStreamingTool's description string instead of concatenating those sentences inline; keep the text identical, place the new constant near SANDBOX_PROMPT_NOTE for consistency, export it if other modules need it, and replace the concatenated block in the description with a reference to RUNID_BACKGROUND_TASK_NOTE.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@lib/chat/tools/createPromptSandboxStreamingTool.ts`:
- Around line 56-59: Update the long instruction string in
createPromptSandboxStreamingTool (the multi-line message built around the
`runId` behavior) to remove the inaccurate "UI automatically shows a live
progress view" claim and replace it with the provided wording: state that when a
`runId` is present the sandbox is being set up, the output will be empty while
the background task runs, do NOT summarize the empty output, and tell the user
the request is being processed and results will appear once the task completes;
locate the string construction in createPromptSandboxStreamingTool.ts (the block
concatenating multiple quoted segments) and substitute the revised text exactly
as suggested.
---
Nitpick comments:
In `@lib/chat/tools/createPromptSandboxStreamingTool.ts`:
- Around line 55-60: Extract the long runId/background-task guidance from the
description into a named constant (e.g., RUNID_BACKGROUND_TASK_NOTE) and reuse
it in createPromptSandboxStreamingTool's description string instead of
concatenating those sentences inline; keep the text identical, place the new
constant near SANDBOX_PROMPT_NOTE for consistency, export it if other modules
need it, and replace the concatenated block in the description with a reference
to RUNID_BACKGROUND_TASK_NOTE.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 27d5b8f3-c60c-4a92-aae9-d2f41577e86b
📒 Files selected for processing (1)
lib/chat/tools/createPromptSandboxStreamingTool.ts
Comment on lines
+56
to
+59
| "IMPORTANT: When the result contains a `runId`, it means the sandbox is being set up for the first time " + | ||
| "and the command was dispatched to a background task. The output will be empty because the task is still running. " + | ||
| "The UI automatically shows a live progress view for background tasks — do NOT summarize or interpret the empty output. " + | ||
| "Simply tell the user their request is being processed in the sandbox and the results will appear in the task progress view above. " + |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for UI components that handle runId or task progress views
echo "=== Searching for runId handling in UI/app components ==="
rg -n 'runId' --type=ts --type=tsx | grep -E '(app/|components/)' | head -20
echo ""
echo "=== Searching for task progress view components ==="
rg -n -i 'progress.*view|task.*progress|background.*task' --type=ts --type=tsx | grep -E '(app/|components/)' | head -20
echo ""
echo "=== Searching for Trigger.dev run subscription patterns ==="
rg -n 'useRealtimeRun|subscribeToRun|task.*status' --type=ts --type=tsx | head -20
echo ""
echo "=== Checking app directory structure ==="
find app -type f -name "*.ts" -o -name "*.tsx" | head -20Repository: recoupable/api
Length of output: 975
🏁 Script executed:
#!/bin/bash
# Search for UI components in other common locations
echo "=== Checking for components directory ==="
find . -type d -name "components" | head -10
echo ""
echo "=== Checking for pages directory ==="
find . -type d -name "pages" | head -10
echo ""
echo "=== Search for runId in all TypeScript/JavaScript files ==="
rg 'runId' --type=ts | head -30
echo ""
echo "=== Check createPromptSandboxStreamingTool.ts for imports and context ==="
head -100 lib/chat/tools/createPromptSandboxStreamingTool.ts | grep -E '^import|^export|runId|progress'
echo ""
echo "=== Verify: does promptSandboxStreaming return runId? ==="
rg -A10 'return.*runId|runId.*:' lib/ | head -40Repository: recoupable/api
Length of output: 6782
Remove or revise the inaccurate claim about the UI progress view.
The description references "The UI automatically shows a live progress view for background tasks," but this repository is a backend API-only codebase with no frontend UI components. The claim will mislead the LLM and ultimately confuse users about where results will appear.
Revise to:
"IMPORTANT: When the result contains a `runId`, it means the sandbox is being set up for the first time " +
"and the command was dispatched to a background task. The output will be empty because the task is still running. " +
"Do NOT summarize or interpret the empty output. " +
"Simply tell the user their request is being processed in the sandbox and the results will appear once the task completes."
The runId behavior itself is correctly documented based on the backend implementation, but remove references to UI components that don't exist in this codebase.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@lib/chat/tools/createPromptSandboxStreamingTool.ts` around lines 56 - 59,
Update the long instruction string in createPromptSandboxStreamingTool (the
multi-line message built around the `runId` behavior) to remove the inaccurate
"UI automatically shows a live progress view" claim and replace it with the
provided wording: state that when a `runId` is present the sandbox is being set
up, the output will be empty while the background task runs, do NOT summarize
the empty output, and tell the user the request is being processed and results
will appear once the task completes; locate the string construction in
createPromptSandboxStreamingTool.ts (the block concatenating multiple quoted
segments) and substitute the revised text exactly as suggested.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
prompt_sandboxtool description to explainrunIdbehavior to the LLMrunIdwith empty outputTest plan
runIdis returned🤖 Generated with Claude Code
Summary by CodeRabbit