Skip to content

Extend the upload script for other local use cases #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

huydhn
Copy link
Contributor

@huydhn huydhn commented Jun 6, 2025

This is the script I'm using to upload vLLM benchmark results. It can be retrofit to upload other benchmark results too, for example torch.compile benchmark results on GB200. I will write a wiki on how to use this script, but the simple usage is:

UPLOADER_USERNAME=<REACT>
UPLOADER_PASSWORD=<REACT>
GPU_DEVICE=$(nvidia-smi -i 0 --query-gpu=name --format=csv,noheader | awk '{print $2}')

python upload_benchmark_results.py \
  --repo pytorch \
  --benchmark-name TorchInductor \
  --benchmark-results test-reports \
  --device "${GPU_DEVICE}" \
  --upload-url "https://qrr6jzjpvyyd77fkj6mqkes4mq0tpirr.lambda-url.us-east-1.on.aws" \
  --dry-run

cc @zhe-thoughts @ZainRizvi

Signed-off-by: Huy Do <[email protected]>
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:45 — with GitHub Actions Inactive
@huydhn huydhn requested a review from ZainRizvi June 6, 2025 02:47
@huydhn huydhn marked this pull request as ready for review June 6, 2025 02:47
Signed-off-by: Huy Do <[email protected]>
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:56 — with GitHub Actions Inactive
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:56 — with GitHub Actions Inactive
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:56 — with GitHub Actions Inactive
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:56 — with GitHub Actions Inactive
@huydhn huydhn temporarily deployed to pytorch-x-vllm June 6, 2025 02:56 — with GitHub Actions Inactive
huydhn added a commit to pytorch/test-infra that referenced this pull request Jun 6, 2025
Courtesy of Claude code. This is the initial version of an API that I'm
building to allow people to upload benchmark results. This works in
conjunction with
pytorch/pytorch-integration-testing#36

I will need to prepare a Terraform change for the lambda, but the API
works as follows:

```
import requests

with open("FILE_TO_BE_UPLOADED.json") as f:
    content = f.read()

json_data = {
    "username": "REDACT",
    "password": "REDACT",
    "content": content,
    "path": "v3/foobar/debug.json",
}

headers = {
    "content-type": "application/json"
}

url = "https://qrr6jzjpvyyd77fkj6mqkes4mq0tpirr.lambda-url.us-east-1.on.aws"
requests.post(url, json=json_data, headers=headers)
```

cc @zhe-thoughts @ZainRizvi

---------

Signed-off-by: Huy Do <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants