Skip to content

Commit

Permalink
[sharktank] Use pytest-timeout in ci-sharktank.yml. (#868)
Browse files Browse the repository at this point in the history
Some of these jobs have been taking the full [6 hour default
timeout](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes)
before failing. Example stalls:
*
https://github.com/nod-ai/shark-ai/actions/runs/12875057753/job/35895676842
*
https://github.com/nod-ai/shark-ai/actions/runs/12948380437/job/36116930181
*
https://github.com/nod-ai/shark-ai/actions/runs/12939601933/job/36092205910

Introducing timeouts in the test runner gives us a chance of seeing a
callstack when a test case is stalled and not burning CI time. Sample
output from using a shorter timeout:

Job name | Logs
-- | --
Data-dependent Tests |
https://github.com/nod-ai/shark-ai/actions/runs/12953817421/job/36134335605
Model Integration Tests |
https://github.com/nod-ai/shark-ai/actions/runs/12953817421/job/36134336455
  • Loading branch information
ScottTodd authored Jan 26, 2025
1 parent 3c32f5d commit 4a41607
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
6 changes: 4 additions & 2 deletions .github/workflows/ci-sharktank.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,8 @@ jobs:
sharktank/tests/models/t5/t5_test.py \
sharktank/tests/models/flux/flux_test.py \
sharktank/tests/models/vae/vae_test.py \
--durations=0
--durations=0 \
--timeout=600
test_integration:
Expand Down Expand Up @@ -186,7 +187,8 @@ jobs:
- name: Run punet tests
run: |
pytest -v sharktank/ -m punet_quick \
--durations=0
--durations=0 \
--timeout=600
# Depends on other jobs to provide an aggregate job status.
# TODO(#584): move test_with_data and test_integration to a pkgci integration test workflow?
Expand Down
1 change: 1 addition & 0 deletions sharktank/requirements-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@ parameterized
protobuf
pytest==8.0.0
pytest-html
pytest-timeout
pytest-xdist==3.5.0
safetensors>=0.4.5

0 comments on commit 4a41607

Please sign in to comment.