Support blobs and trees in fetchGit #14052
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
with-tests
This makes it possible to fetch the source for specific folders / files, provided it's supported by the remote and you know the hash of the object. This is especially useful for monorepos if you just want to grab the source for a specific directory. When using this via fetchGit, shallow is required as there's no history to fetch. Since .gitattributes isn't available, submodules and lfs have no effect. When fetching blobs and trees, revCount and lastModified are omitted from the resulting attribute set as those attributes only really make sense for commits (or things that point at commits like annotated tags).
joshheinrichs-shopify
force-pushed
the
fetch-blobs-and-trees
branch
from
f0bd309 to
7add6a3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This adds support for fetching trees / blobs to fetchGit / fetchTree, making it possible to fetch the source for specific subfolders of a repo. This is especially useful in a monorepo setting as it enables fetching the source for a particular project, and also avoids needless rebuilds if that project hasn't changed.
Context
Robert suggested that this might be possible at NixCon. I did an some initial poking at this a few weeks ago, and Ericson provided some good suggestions on how to approach this for real[1]. I deviated slightly from his suggestions and put some awareness of blobs + trees into GitRepo.getRevCount and GitRepo.getLastModified as they're ultimately only called from GitInputScheme which currently doesn't have any visibility into the types of objects.
A few things of note:
I've opted to not add documentation as it's pretty succinct currently and I didn't want to bloat it with a relatively niche use case, but I can if that's desired. The existing documentation in fetchTree kind of implies that fetching tree objects could work[2].
[1] #13962
[2]
nix/src/libexpr/primops/fetchTree.cc
Line 337 in d9de675
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.