Skip to content

Add automotive benchmarking setting #2139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

pgmpablo157321
Copy link
Contributor

@pgmpablo157321 pgmpablo157321 commented Feb 26, 2025

  • Add server_constant_gen parameter to the test_settings
  • Add server_constant_gen parameter to the python API
  • Let server_constant_gen be able to be loaded from the config files (mlperf.conf and user.conf) using FromConfig
  • For the server scenario, the query scheduling distribution from poisson to constant when the parameter server_constant_gen is set to 1, i.e:
    • When server_constant_gen = 0 number of queries in a fixed time follows a poisson distribution (with mean qps) and time between queries follows an exponential distribution (with mean 1/qps).
    • When server_constant_gen = 1 time between queries is constant equal to 1/qps
  • Custom Sample Grouping
  • In order to run the a benchmark in a grouped setting:
    • Construct a QSL using ConstructGroupedQSL
      • Pass a vector that contains the group sizes in the order they appear in the dataset
    • Start running the test using StartTestWithGroupedQSL
    • Set the variable use_grouped_qsl in the mlperf.conf
    • Run your benchmark in the server scenario

@pgmpablo157321 pgmpablo157321 requested a review from a team as a code owner February 26, 2025 03:42
Copy link
Contributor

github-actions bot commented Feb 26, 2025

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@pgmpablo157321 pgmpablo157321 changed the title Add constant generation setting Add automotive benchmarking setting Mar 21, 2025
@pgmpablo157321 pgmpablo157321 force-pushed the automotive-loadgen branch 2 times, most recently from c3fc495 to db71d6d Compare March 24, 2025 13:53
@pgmpablo157321 pgmpablo157321 force-pushed the automotive-loadgen branch 8 times, most recently from a698dd0 to c87eeb2 Compare April 1, 2025 22:14
@pgmpablo157321 pgmpablo157321 force-pushed the automotive-loadgen branch 3 times, most recently from 56ceea4 to de61ef3 Compare April 3, 2025 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant