fix: check if miniconda is present before installation, closes #47 #48
Conversation
📝 WalkthroughWalkthroughAdded a pre-check step that detects an existing Miniconda installation and sets an output; the conda setup step now runs only when that output is Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (2)📚 Learning: 2024-12-09T10:52:48.855ZApplied to files:
📚 Learning: 2024-12-09T16:41:37.740ZApplied to files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
🔇 Additional comments (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
action.yml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: tedil
Repo: snakemake/snakemake-github-action PR: 39
File: action.yml:47-48
Timestamp: 2024-12-09T10:52:48.855Z
Learning: In the `action.yml` file for `snakemake-github-action`, since `environment.yaml.template` is part of the repository and is expected to exist, adding error handling to check for its presence is unnecessary.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
action.yml (1)
76-85: Consider a more robust check for a valid Miniconda installation.While checking for directory existence is an improvement over the previous
.condarcapproach, it doesn't verify that the installation is valid or functional. A partially failed installation could leave the directory in a corrupt state, causing subsequent steps to fail silently.For better robustness, consider checking for the conda executable itself:
if [ -x "/home/runner/miniconda3/bin/conda" ]; then echo "installed=true" >> $GITHUB_OUTPUT else echo "installed=false" >> $GITHUB_OUTPUT fiAlternatively, if the setup-miniconda action sets the
$CONDAenvironment variable on successful installation, you could check that. However, the directory-only check does address the core issue from #47 (preventing re-installation and the symlink error).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
action.yml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2024-12-09T10:52:48.855Z
Learnt from: tedil
Repo: snakemake/snakemake-github-action PR: 39
File: action.yml:47-48
Timestamp: 2024-12-09T10:52:48.855Z
Learning: In the `action.yml` file for `snakemake-github-action`, since `environment.yaml.template` is part of the repository and is expected to exist, adding error handling to check for its presence is unnecessary.
Applied to files:
action.yml
🔇 Additional comments (1)
action.yml (1)
86-95: No issues found —installation-diris a supported parameter in setup-miniconda@v3.The
installation-dirinput is officially supported and documented in the action's v3.x releases. The conditional logic and parameter usage in lines 86–95 are correct.
ezherman
left a comment
ezherman
left a comment
There was a problem hiding this comment.
Thanks @m-jahn! See my thoughts below on the changes & let me know what you think.
Also, qould it be sensible to add a second run step, and/or a preceding setup-miniconda step, to https://github.com/snakemake/snakemake-github-action/blob/master/.github/workflows/main.yml to ensure we test the feature keeps working going forward?
action.yml
Outdated
| id: check-miniconda | ||
| shell: bash -el {0} | ||
| run: | | ||
| if [ -d "/home/runner/miniconda3" ]; then |
There was a problem hiding this comment.
A self‑hosted runner may use a different home path. What about $HOME/miniconda3?
There was a problem hiding this comment.
Or to check more generally for a conda installation, e.g. which conda
There was a problem hiding this comment.
good points. I like the checking for $HOME, if this env var is available on the github runner, it's fine with me.
Just checking with which conda is dangerous though, as the test would succeed for any conda available, not necessarily our one with snakemake.
There was a problem hiding this comment.
OK so the problem is that $HOME is not available in statements like installation-dir: /home/runner/miniconda3, only in the run: directive of a step. So it would be more complicated to do this, then what we have now.
There was a problem hiding this comment.
Why do we need to specify installation-dir? I believe it already defaults to $HOME/miniconda3
| fi | ||
| - name: Setup conda | ||
| if: steps.check-miniconda.outputs.installed == 'false' |
There was a problem hiding this comment.
If the setup step is skipped, conda won't be set up with .snakemake.environment.yaml as the default environment. Will snakemake then not be available as a consequence? If so, I imagine we need an additional step that installs snakemake into the base environment, or creates a snakemake conda environment, if conda is already available.
There was a problem hiding this comment.
the idea is that this check is specific for the miniconda setup step. when you run this step twice, as we do in the standard snakemake testing workflow, the env is already present and checked out from the first step, including snakemake. the test in this repo succeeded, meaning it works as expected. that's why I would also not test for any conda installation, as this could be the wrong one, only for the specific miniconda3 one.
There was a problem hiding this comment.
Gotcha yes, I see the containerize step would trigger a second setup miniconda step so there isn't a need for a second run step.
Regarding testing for miniconda3: if the workflow includes setup-miniconda upstream, then I believe the current implementation would not install snakemake. Or am I missing something? For example, if setup-miniconda is used to enable pytest:
Testing:
runs-on: ubuntu-latest
needs:
- Linting
- Formatting
steps:
- uses: actions/checkout@v4
- uses: conda-incubator/setup-miniconda@v3
with:
miniforge-version: latest
use-mamba: true
environment-file: .test/envs/test.yaml
- name: Test workflow python functions with pytest
shell: bash -el {0}
run: |
pytest workflow/tests/*.py
- name: Test workflow
uses: snakemake/[email protected]
with:
directory: .test
snakefile: workflow/Snakefile
args: "--use-conda --show-failed-logs --cores 2 --conda-cleanup-pkgs cache"
There was a problem hiding this comment.
Right. This is an edge case. A custom workflow with another setup-miniconda@v3 step would skip installing snakemake and fail. Is this a risk we can take (honest question)? In the end, the github action needs to work mainly for the test workflows, and it does so by wrapping several steps into a single one. In the example you posted, one would need to work around this by replacing the snakemake action with some selected steps.
There was a problem hiding this comment.
It's actually an important use-case, in my opinion. An extension is of this is to follow the snakemake-github-action with integration tests, which would not be possible if the action needs to be run in a separate job. Why limit the action to only operate in jobs that only use the snakemake github action?
What could go wrong by checking for a conda installation more generally, and updating/installing the snakemake environment if i conda exists?
There was a problem hiding this comment.
An extension of this is to follow the snakemake-github-action with integration tests, which would not be possible if the action needs to be run in a separate job
You can do this if you first run the snakemake action that installs miniconda, then the tests. The other way around as the example above won't work.
What could go wrong by checking for a conda installation more generally, and updating/installing the snakemake environment if i conda exists?
Possible, but then the action steps will change quite a lot. One would need different checks, like A) is any conda available? If not install miniconda + snakemake env as before. If yes, is snakemake available? If not create only env and activate, not using the setup-miniconda action. And checking for conda does not make sure that mamba is available (don't know what consequences that would have). At least current conda should also use the libmamba solver so should be equally fast.
If you have a good solution in mind don't hesitate to make a commit!
action.yml
Outdated
| miniforge-version: latest | ||
| environment-file: .snakemake.environment.yaml | ||
| activate-environment: snakemake | ||
| installation-dir: /home/runner/miniconda3 |
There was a problem hiding this comment.
Here too, I believe $HOME/miniconda3 would be safer in case a self-hosted runner is used. Although is this parameter needed, which points to the default path, if we've already checked for a conda installation?
2 participants
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.