-
-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use devpi as PyPI cache #6340
Use devpi as PyPI cache #6340
Conversation
Adds a new service for docker-compose that spin up a devpi-server to cache all the Python packages used by the builders. If configured, `--index-url` and `--trusted-host` is added to all the `pip` commands. Fixes #3553
Is this meant to be used in prod, or only on dev? It could perhaps speed up builds if we had it in prod, but I don't know if we want to maintain it, and fastly is likely fast enough already to minimize this being a huge install-time issue. |
The original idea was to use when developing since if you are doing some work related to the build process, it takes a lot of time to download each package over and over again (even more now that we are cleaning up after build ends). Also, when working with limited internet access. Although, I'm not opposite to use this in production if there is a real difference between downloading from internal cache than from internet. |
Marking as WIP for now since the docker file could be moved to |
This PR is updated and ready for review. Some code was moved to the linked |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This PR needs to be updated with latest changes con |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This is ready for review. It should work in a transparent way on development environment. It can be disabled with |
No really sure this is a big improvement, if I'm having a slow connection I just set this setting to false
Also, this requires to spin another container, docker is already slow for some people. |
This requires to re-download all the packages again for every different project.
You can decide to not use this just by passing |
Any objection on merging this? |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
I'll try to tackle this next week and merge it. |
This didn't have too much traction and nobody has complained about slow connections in a while. Let's come back to this when we are allowed to travel and be behind a cell connection 😄 |
This is a working POC to use adevpi-server
on development. Anyways, do not merge! This is more a way to share the diff for now and be able to apply it when needed.Use
devpi-server
inside docker (when running under docker-compose) to speed up python package installation when triggering builds.Adds a new service for docker-compose that spin up a devpi-server to cache all the Python packages used by the builders.
If configured,
--index-url
and--trusted-host
is added to all thepip
commands.Closes #3553