Skip to content

Capture cpu profile for keynote bench in nightly job#5163

Open
joshua-spacetime wants to merge 2 commits into
masterfrom
joshua/samply-keynote-bench
Open

Capture cpu profile for keynote bench in nightly job#5163
joshua-spacetime wants to merge 2 commits into
masterfrom
joshua/samply-keynote-bench

Conversation

@joshua-spacetime
Copy link
Copy Markdown
Contributor

Description of Changes

Also capture a cpu profile on a keynote bench job failure.

API and ABI breaking changes

N/A

Expected complexity level and risk

2

Testing

  • Profile artifacts are generated and stored by the CI job

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml
@cloutiertyler
Copy link
Copy Markdown
Contributor

I'd like @jdetter to review this one.

name: keynote-bench-nightly-profile
path: ${{ env.KEYNOTE_PROFILE_DIR }}
if-no-files-found: warn
retention-days: 90
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for @jdetter: I'm not sure if or how much of an impact this will have on our github actions spend. Happy to reconfigure this. I would just like some level of cpu profile history for diagnostic purposes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fine for now, if the spend here becomes too large then we can always move these to an S3 bucket or something

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How large are these artifacts?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~4MB each.

@joshua-spacetime joshua-spacetime force-pushed the joshua/samply-keynote-bench branch from 4293798 to e6e1c18 Compare June 3, 2026 20:03
Copy link
Copy Markdown
Contributor

@jdetter jdetter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking this from merging, just a few comments and a question

name: keynote-bench-nightly-profile
path: ${{ env.KEYNOTE_PROFILE_DIR }}
if-no-files-found: warn
retention-days: 90
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fine for now, if the spend here becomes too large then we can always move these to an S3 bucket or something

name: keynote-bench-nightly-profile
path: ${{ env.KEYNOTE_PROFILE_DIR }}
if-no-files-found: warn
retention-days: 90
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How large are these artifacts?

Comment thread .github/workflows/ci.yml
CARGO_TARGET_DIR: ${{ github.workspace }}/target
RUST_BACKTRACE: full
CARGO_TARGET_DIR: ${{ github.workspace }}/target
CARGO_PROFILE_RELEASE_DEBUG: line-tables-only
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may cause cargo cache misses with the new vm caching change but that's more of a me problem for later

Comment thread .github/workflows/ci.yml
--release
-p spacetimedb-cli
-p spacetimedb-standalone
--features spacetimedb-standalone/perfmap
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will also probably cause a cache miss - if this workflow ends up being too slow I can bake a new cache with all of these options enabled so we're more likely to get a compilation cache hit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I think --features spacetimedb-standalone/perfmap should be the default for all of our release builds.

on:
schedule:
# Nightly at 3 AM UTC.
- cron: '0 3 * * *'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer this to run more around 8am UTC - when we do maintenance on the runners we typically like to do it in the middle of the night when nobody is working to reduce the impact on productivity. If you keep this as 3AM UTC you will just run the risk of it getting cancelled sometimes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants