Skip to content

[examples, docs] feat: add end-to-end Agent RL training recipes#90

Merged
yyDing1 merged 12 commits into
mainfrom
train
Jul 24, 2026
Merged

[examples, docs] feat: add end-to-end Agent RL training recipes#90
yyDing1 merged 12 commits into
mainfrom
train

Conversation

@yyDing1

@yyDing1 yyDing1 commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

This PR adds end-to-end Agent RL training recipes for both white-box ReAct and black-box Claude Code Agents using verl's V1 colocate_async trainer.

Key changes

  • Add runnable training recipes for:
    • Qwen3-Coder-30B-A3B-Instruct with ReAct.
    • Qwen3.5-4B with Claude Code.
  • Add Task Configs for SWE-reBench training and SWE-Bench Verified validation.
  • Document data preparation, Ray Runtime Environment, distributed launch parameters, shared storage layout, and output monitoring.
  • Expose RLOO/GRPO, DPPO, clipping, rollout correction, and Router Replay settings through environment variables.
  • Add per-runner trajectory_selection with backward-compatible all and new longest policies.
  • Configure Claude Code training to keep the trajectory with the most model-generated tokens before reward assignment and TransferQueue writes.
  • Improve sandboxed Claude Code invocation with explicit model and permission-mode configuration.
  • Update the legacy MoE recipe to use separated data/runtime paths and Ray task dispatch.
  • Replace example runtime credentials with placeholders.

Compatibility

Existing runners retain trajectory_selection=all by default. ReAct behavior is unchanged, while the Claude Code recipe explicitly enables longest.

Validation

  • Framework trajectory tests: 20 passed.
  • Added coverage for trajectory selection and invalid configuration.
  • Updated Claude Code argument and environment tests.
  • Validated training script syntax and Python compilation.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@yyDing1 yyDing1 changed the title [example, docs] feat: add end-to-end Agent RL training recipes [examples, docs] feat: add end-to-end Agent RL training recipes Jul 23, 2026
@yyDing1
yyDing1 merged commit c466364 into main Jul 24, 2026
4 checks passed
@yyDing1
yyDing1 deleted the train branch July 24, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants