Optimize copy_file
#565
Merged
Optimize copy_file
#565
+13
−11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fmeum
requested review from
brandjon,
c-mita,
comius,
hvadehra and
tetromino
as code owners
Contributor
Author
fmeum
force-pushed
the
drop-exec-requirements
branch
3 times, most recently
from
8e6d9e9 to
3127c3f
Compare
Contributor
Author
|
FYI @kormide, I think that most of the exec requirements are actually harmful for |
fmeum
force-pushed
the
drop-exec-requirements
branch
from
3127c3f to
0ebea4d
Compare
|
I wonder if it would be helpful to have a mechanism to pass in |
ulfjack
approved these changes
Mar 20, 2025
Contributor
Author
You mean on a per-target level (per-mnemonic level is possible with |
fmeum
force-pushed
the
drop-exec-requirements
branch
from
4c70b00 to
e5875fd
Compare
* `ctx.actions.symlink` can be used on all platforms and falls back to a copy if a symlink is unsupported. It is also heavily optimized, avoiding the need to hash the output file again. * Bazel never guarantees that an input to an action is staged as a non-symlink, so whether an output is a symlink or a hard copy only matters for top-level outputs consumed outside Bazel and handled by tools that don't follow symlinks by default, which should be extremely rare and could be worked around by explicitly setting `allow_symlink` to `False` in user-controlled code. * For file copy actions that do not go through `ctx.actions.symlink`, caching is extremely cheap since the CAS entry of the input will be reused as the CAS entry of the output. Allowing remote execution and caching enables BwoB for the copy, which can avoid downloads of both the input and the output file. The same changes are not applied to `copy_directory` as source directories are not officially supported by Bazel and any kind of change could cause subtle incorrectness.
fmeum
force-pushed
the
drop-exec-requirements
branch
from
e5875fd to
c1fcab0
Compare
tjgq
approved these changes
Nov 4, 2025
tjgq
left a comment
tjgq
left a comment
There was a problem hiding this comment.
Approved. Rationale: this change is beneficial to many Google-owned (and other) Bazel projects that use skylib. It's also widely used in the Google monorepo.
Collaborator
|
So the idea of this PR is to
Seems reasonable. |
tetromino
approved these changes
Nov 4, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ctx.actions.symlinkcan be used on all platforms and falls back to a copy if a symlink is unsupported. It is also heavily optimized, avoiding the need to hash the output file again.allow_symlinktoFalsein user-controlled code.ctx.actions.symlink, caching is extremely cheap since the CAS entry of the input will be reused as the CAS entry of the output. Allowing remote execution and caching enables BwoB for the copy, which can avoid downloads of both the input and the output file.The same changes are not applied to
copy_directoryas source directories are not officially supported by Bazel before 9 and any kind of change could cause subtle incorrectness.Fixes #562