Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test notebooks with a separate CI workflow #151

Merged
merged 5 commits into from
Feb 16, 2025
Merged

Test notebooks with a separate CI workflow #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
25dcbcd
Move test-notebooks job to a new workflow
mdpiper Feb 14, 2025
dd0fbc2
Fix read_esri_ascii import
mdpiper Feb 14, 2025
0925015
Fix requested version
mdpiper Feb 14, 2025
1443ea7
Skip testing terrainbento and pymt notebooks
mdpiper Feb 14, 2025
4dc97cb
Clean up list of ignored notebooks
mdpiper Feb 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/test-notebooks.yml
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you decide to run this job as a separate workflow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like separating unit tests from checking notebooks. They feel like different things, even though we call both "tests" and they run through pytest. Also test.yml was longer than one screen.

Is there a penalty for splitting jobs across workflows instead of grouping them into one workflow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a subject for discussion on the Forum!

Copy link
Member

@mcflugen mcflugen Feb 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the workflows are independent of one another, it's fine separating them. If you had a third job (e.g. maybe building/deploying docs or deploying to PyPI) that required both the regular tests and the notebook tests to complete, then you would have to put them in a single workflow.

Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Test Notebooks

on: [push, pull_request]

jobs:

test-notebooks:

if:
github.event_name == 'push' ||
github.event.pull_request.head.repo.full_name != github.repository

name: Test the notebooks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: conda-incubator/setup-miniconda@v3
with:
python-version: "3.11"
miniforge-variant: Miniforge3
miniforge-version: latest
auto-update-conda: true

- name: Install nox
run: pip install nox

- name: Test
env:
MPLBACKEND: "Agg"
OPENTOPOGRAPHY_API_KEY: ${{ secrets.OPENTOPOGRAPHY_API_KEY }}
run: nox -s test-notebooks --python "3.11"
27 changes: 0 additions & 27 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,30 +37,3 @@ jobs:

- name: Test
run: nox -s test

test-notebooks:

if:
github.event_name == 'push' ||
github.event.pull_request.head.repo.full_name != github.repository

name: Test the notebooks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: conda-incubator/setup-miniconda@v3
with:
python-version: "3.11"
miniforge-variant: Miniforge3
miniforge-version: latest
auto-update-conda: true

- name: Install nox
run: pip install nox

- name: Test
env:
MPLBACKEND: "Agg"
OPENTOPOGRAPHY_API_KEY: ${{ secrets.OPENTOPOGRAPHY_API_KEY }}
run: nox -s test-notebooks --python "3.11"
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ repos:
files: \.py$

- repo: https://github.com/mcflugen/heartfelt-hooks
rev: v1.3.3
rev: v0.3.3
hooks:
- id: check-whitespace
# - id: check-mixed-case
Expand Down
2 changes: 1 addition & 1 deletion lessons/landlab/landlab/overland_flow.ipynb
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the old way of reading an ESRI ASCII file. We should update this notebook to use the new way. But in another pull request.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, OK. I couldn't remember which was the new way. I only changed it because it was failing in the CI test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a future pr, the new way is something like,

from landlab.io import esri_ascii

with open("data/square-test-basin.asc") as stream:
    grid = esri_ascii.load(filepath, name="topographic__elevation")
z = grid.at_node["topographic__elevation"]

Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
"source": [
"import numpy as np\n",
"from landlab.components.overland_flow import OverlandFlow\n",
"from landlab.io.esri_ascii import read_esri_ascii\n",
"from landlab.io import read_esri_ascii\n",
"from matplotlib.pyplot import figure"
]
},
Expand Down
6 changes: 3 additions & 3 deletions pytest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ addopts =
--strict-markers
--doctest-modules
-vvv
--ignore=lessons/landlab/landlab-terrainbento
--ignore=lessons/pymt
--ignore-glob=*/practice-your-skills.ipynb
--ignore-glob=*/Welcome_to_TerrainBento.ipynb
--ignore-glob=*/bedrockLandslides_on_DEMs.ipynb
--ignore-glob=*/bedrock_landslides_on_dems.ipynb
--ignore-glob=*/ku-eml-paleomip.ipynb
--ignore-glob=*/paleomip-processing.ipynb
--ignore-glob=*/PFdepth-dependent-diffusion.ipynb
--ignore-glob=*/create_a_component.ipynb
--ignore-glob=*/landlab-fault-scarp-for-espin.ipynb
--ignore-glob=*/intro-to-grids.ipynb
--ignore-glob=*/pymt/*.ipynb
--ignore-glob=*/python/yet_another_oop.ipynb
Loading