Skip to content

Honour --group-size when replaying a trace, and read traces from GCS - #3389

Merged
wenxindongwork merged 2 commits into
mainfrom
wxd-agentic-group-size
Aug 14, 2026
Merged

Honour --group-size when replaying a trace, and read traces from GCS#3389
wenxindongwork merged 2 commits into
mainfrom
wxd-agentic-group-size

Conversation

@wenxindongwork

@wenxindongwork wenxindongwork commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Two small usability fixes to the agentic benchmark, both hit while sweeping batch size against the MLPerf traces.

--group-size was ignored when replaying a trace

run_group took its stream count from len(specs), so --group-size did nothing whenever --trace-file was supplied. A run intending 8 streams silently replayed the trace's full 16, and the only visible sign was the stream count buried in the report — easy to miss, and it quietly invalidates a batch-size sweep.

Each group is now trimmed to --group-size at load time, so the startup summary and the run agree.

--trace-file now accepts gs://

python benchmark_agentic.py \
  --model Qwen/Qwen3.5-397B-A17B-FP8 \
  --trace-file gs://... \
  --global-prefix-len 6476 --num-groups 1 --group-size 8 --concurrency 1

--group-size was silently ignored whenever --trace-file was given, since
the stream count came straight from len(specs). Runs meant to sweep batch
size all replayed the trace's full group instead, which is easy to miss
because the only sign is the stream count in the report. Trim each group
to --group-size at load time, warn when it exceeds what the trace holds,
and default it to None so a trace still replays whole unless asked.

--trace-file also went through open(), so a gs:// URI failed. Accept one
directly, via google-cloud-storage when installed and the gsutil CLI
otherwise, so neither is a hard dependency.

Signed-off-by: wenxindongwork <wenxindong@google.com>
@github-actions

Copy link
Copy Markdown

Description

Start with a short description of what the PR does and how this is a change from
the past.

The rest of the description includes relevant details and context, examples:

  • why is this change being made,
  • the problem being solved and any relevant context,
  • why this is a good solution,
  • some information about the specific implementation,
  • shortcomings of the solution and possible future improvements.

If the change fixes a Github issue, please include a link, e.g.,:
FIXES: #123456

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure:

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have made or will make corresponding changes to any relevant documentation.

@wenxindongwork wenxindongwork added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 13, 2026
@wenxindongwork
wenxindongwork marked this pull request as ready for review August 13, 2026 21:16
Signed-off-by: wenxindongwork <wenxindong@google.com>
@wenxindongwork
wenxindongwork enabled auto-merge (squash) August 13, 2026 21:17

@sierraisland sierraisland left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenxindongwork
wenxindongwork merged commit 1270ddc into main Aug 14, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants