Skip to content

[core] fix: stop fabricating rollout log probabilities#92

Merged
wuxibin89 merged 1 commit into
verl-project:mainfrom
zackcxb:cxb/gateway-logprobs-sampling-cleanup
Jul 24, 2026
Merged

[core] fix: stop fabricating rollout log probabilities#92
wuxibin89 merged 1 commit into
verl-project:mainfrom
zackcxb:cxb/gateway-logprobs-sampling-cleanup

Conversation

@zackcxb

@zackcxb zackcxb commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Stop emitting fabricated zero rollout log probabilities when a trajectory does not provide them. VERL uses the presence of rollout_log_probs to enable rollout-logprob paths, so synthetic zeros can silently contaminate bypass or off-policy importance-sampling behavior.

This PR also removes the dead gateway-level base_sampling_params config layer. Session sampling parameters are already the runtime source of defaults, and merging an always-empty base dictionary is a no-op.

Checklist Before Starting

  • Searched for similar PRs and issues; relevant links are listed above.
  • Formatted the PR title as [core] fix: stop fabricating rollout log probabilities.

Test

python -m pytest \
  tests/uni_agent/gateway/test_gateway_actor_on_cpu.py \
  tests/uni_agent/framework/test_generate_sequences_on_cpu.py -q
# 61 passed, 4 warnings

pre-commit run --all-files --show-diff-on-failure --color=always
# ruff, ruff format, mypy, and compile-all passed

No recipe rollout was run; this focused core change is covered by the affected CPU unit tests.

API and Usage Example

GatewayActorConfig.base_sampling_params is removed. Put sampling defaults on each session instead:

config = GatewayActorConfig(tokenizer=tokenizer)
await gateway.create_session(
    "session-id",
    sampling_params={"temperature": 0.7, "top_p": 0.9},
)

Design & Code Changes

  • Emit rollout_log_probs only when trajectory.response_logprobs is present, matching the adjacent conditional routed_experts emission pattern.
  • Remove GatewayActorConfig.base_sampling_params and _GatewayActor._base_sampling_params.
  • Pass a copy of session.sampling_params directly to the OpenAI and Anthropic adapters; their existing base_sampling_params argument names remain unchanged.
  • Update Gateway tests to provide defaults through create_session(..., sampling_params=...).
  • Update the framework regression test to assert that missing rollout log probabilities omit the key while reward scores remain zero-filled.

Checklist Before Submitting

  • Read the Contribute Guide.
  • Ran pre-commit run --all-files --show-diff-on-failure --color=always.
  • Confirmed no repository docs or examples reference the removed field; included the migration example above.
  • Updated the affected CPU tests.
  • Confirmed the PR title matches the required format.
  • Replaced all template placeholder text with real content.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@wuxibin89
wuxibin89 merged commit 38c655f into verl-project:main Jul 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants