Improve reproducibility and reduce manual setup steps with Calkit #54
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Disclaimer: Calkit is an open source tool I have been working on to improve the reproducibility and developer experience for these kinds of projects. It can be installed with
pip install calkit-python
.Summary of changes
The README has been updated to demonstrate the fewer number of manual steps needed to reproduce the results.
uv.lock
file).env
fileThe steps to reproduce the results simply become:
calkit xenv -n docker-arm64 -- ISSUE_ID=1 bash /app/tests/run.sh
(see below forISSUE_ID
details)calkit check env-vars
calkit run
Current issues
ISSUE_ID
must be set to run the Docker container. Is this expected?run_swelancer.py
script errors out with:Error: Too many docker networks have been created. Most machines/laptops default to allowing 16 LocalCluster instances.
Once this is fixed, the outputs can be cached appropriately.For what it's worth, running the instructions in the current
main
branch fails similarly on my machine (Apple M1). I assume this is similar to #43, #44, #49.