Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve reproducibility and reduce manual setup steps with Calkit #54

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

petebachant
Copy link

Disclaimer: Calkit is an open source tool I have been working on to improve the reproducibility and developer experience for these kinds of projects. It can be installed with pip install calkit-python.

Summary of changes

The README has been updated to demonstrate the fewer number of manual steps needed to reproduce the results.

  • Automated environment management
    • The uv venv is created and updated as needed at runtime with the desired Python version (note this ignores the uv.lock file)
    • The Docker image is built/rebuilt as needed at runtime
  • Environmental variables are declared and checked, automatically producing a complete .env file
  • The main script is run as part of a DVC pipeline to track dependencies and outputs, which can be rerun every time the code changes to ensure the results don't change

The steps to reproduce the results simply become:

  • calkit xenv -n docker-arm64 -- ISSUE_ID=1 bash /app/tests/run.sh (see below for ISSUE_ID details)
  • calkit check env-vars
  • calkit run

Current issues

  • ISSUE_ID must be set to run the Docker container. Is this expected?
  • Running the run_swelancer.py script errors out with: Error: Too many docker networks have been created. Most machines/laptops default to allowing 16 LocalCluster instances. Once this is fixed, the outputs can be cached appropriately.

For what it's worth, running the instructions in the current main branch fails similarly on my machine (Apple M1). I assume this is similar to #43, #44, #49.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant