Local clones fail if `git-upload-pack` is available but not in `PATH`

[EliahKagan]

Current behavior 😯

Cloning asks the remote to run git-upload-pack, and the operation fails if that cannot be done. In most cases, this is what we expect. If the remote doesn't offer git-upload-pack and does not provide an alternative, then it is correct for the clone to fail, even if some fallback strategy were possible. With network transport, this is arguably always the case, even when connecting from the local machine to itself (even over the loopback interface), because special casing a machine or a network interface is probably unexpected and unintended.

However, for local clones, when passing a path or file:// URL, I don't think we intend this to fail.

Of course we can't use it if it's unavailable

It will sometimes fail such that we can't yet do anything about it, because gitoxide does not yet have its own implementation of git-upload-pack functionality. This issue is not about that case, which #734 tracks, noting:

Currently gitoxide relies on shelling this out to the git upload-pack binary which isn't necessarily available on all systems.

(However, I don't think we are really shelling it out. As shown below, the gix_command::prepare call that sets up the command to run does so without a shell. Also as described below, I think avoiding a shell here is the right thing to do. Note that this is for a local clone; I recognize that shells are generally used over SSH transport.)

This issue is about how, even if git-upload-pack cannot be found in PATH, it may still be available.

We should use git-upload-pack locally if we know where it is or how to call it

One way for it to be available is for a git-upload-pack binary to be found in a PATH search. But I think we consider (or should consider) it also to be available when git is available and we're already using it (or willing to use it) to find information such as its installation-scope configuration.

Often, when git is found in PATH, so is git-upload-pack. On Unix-like systems, both are usually added to bin directories in PATH (with the latter often being a symlink to the former, which distinguishes it by name, though perhaps that is an implementation detail).

I'm not sure that's guaranteed. However...

This is mostly (or entirely?) a Windows issue

The situation I've observed where not using git as a fallback to find git-upload-pack prevents local clones from working is on Windows. In some ways of installing Git for Windows, git is not in PATH at all, and neither is git-upload-pack. But in some other way of installing Git for Windows, git is in PATH but git-upload-pack is not.

In particular, when git is installed via scoop, the binaries in (git root)\bin get scoop shim installed in a PATH directory, but the binaries in the (git root)\cmd directory don't. This is the opposite of the usual situation when Git for Windows is installed with the InnoSetup installer and defaults are unchanged--in that case, cmd goes in PATH and bin doesn't.

Expected behavior 🤔

I find the above-described behavior of scoop fairly strange, but I don't know of any reason it should be considered a bug in the scoop package definition for git. Furthermore, people may be relying on it--if it gets changed, then presumably some users will set things up in a way that effectively restores it. Whether that's an argument for supporting this even if it is a bug, I am not sure.

But on Windows we support having git not be in PATH at all, which means we should be finding git-upload-pack through git. (That's also why this issue is a bug rather than a feature request.)

I think there are two reasonable ways to find git-upload-pack through git.

Way 1: We could let git use it implicitly

One way is to not find it ourselves, but instead run git with upload-pack as its first argument, and let git take care of it:

C:\Users\ek> git-upload-pack
git-upload-pack: The term 'git-upload-pack' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
C:\Users\ek> git upload-pack
usage: git-upload-pack [--[no-]strict] [--timeout=<n>] [--stateless-rpc]
                       [--advertise-refs] <directory>

    --[no-]stateless-rpc  quit after a single request/response exchange
    --[no-]advertise-refs ...
                          alias of --http-backend-info-refs
    --[no-]strict         do not try <directory>/.git/ if <directory> is no Git directory
    --[no-]timeout <n>    interrupt transfer after <n> seconds of inactivity

If we decide to do this, I don't know if we should default to it or only use it as a fallback. But when using network transport I believe we should continue not to do it at all (see below on why).

Way 2: We could find it much as we find other git-associated programs

The other way is to find git-upload-pack by looking in places it is likely to be. On Windows, we can look for (git root)\cmd\git-upload-pack.exe i.e. the Git for Windows git-upload-pack shim. If that's not found, we can look for something like (git root)\mingw64\bin\git-upload-pack.exe, keeping in mind that the correct subdirectory is not necessaruily mingw64; for example, it's clangarm64 on an ARM64 build of Git for Windows.

We already have most of the machinery in place to do that. We would have to add support for finding commands in cmd, and make some design decisions about the order in which directories are attempted, as well as how to expose the further functionality, as well as how much functionality to expose in the public interface of gix-path.

Another way to say this is that the current situation with git-upload-pack is an opportunity to figure out what an interface (beyond just finding sh.exe) to the functionality in gix_path::env::auxiliary should be like.

The code to expand to provide the functionality of finding git-upload-path is mainly this, and the expansion it would require might be as slight as adding"cmd" to BIN_DIR_FRAGMENTS, or might be a bit more involved:

gitoxide/gix-path/src/env/auxiliary.rs

Line 80 in d602d7e

const BIN_DIR_FRAGMENTS: &[&str] = &["bin", "usr/bin"];

gitoxide/gix-path/src/env/auxiliary.rs

Lines 92 to 108 in d602d7e

	fn find_git_associated_windows_executable(stem: &str) -> Option<OsString> {
	let git_root = git_for_windows_root()?;
	BIN_DIR_FRAGMENTS
	.iter()
	.map(|bin_dir_fragment| {
	// Perform explicit raw concatenation with `/` to avoid introducing any `\` separators.
	let mut raw_path = OsString::from(git_root);
	raw_path.push("/");
	raw_path.push(bin_dir_fragment);
	raw_path.push("/");
	raw_path.push(stem);
	raw_path.push(".exe");
	raw_path
	})
	.find(|raw_path| Path::new(raw_path).is_file())
	}

What not to do with the alternative command

Whether we are going to call git with upload-pack as its first argument, or call git-upload-pack by the path where we found it, I think the most important things are what we need to make sure not to do:

1. We would need to make sure not to use this path via network transport. This is more blatant in the case that we are finding a Windows path to git-upload-pack.exe. It would be bad to try to run a command starting with that path on a remote server, where it would be wrong and where some characters in the path could be treated specially on the remote. But I think we also must avoid this if we run git with upload-pack as its first argument. My understanding is that git-upload-pack is supposed to be used over network transport to allow remote servers to provide any appropriate customizations--so using git upload-pack instead would break things over SSH or HTTP transport to most software forges--and possibly for other reasons.
2. We should not use a shell to run it, unless correctness requires it for reasons other than implementation convenience, in which case special effort may need to be taken to avoid bugs. On Windows, with sh or bash provided by Git for Windows or another MSYS2 environment or similar, running commands through a shell incurs some complexities in how they are parsed. Because one reason to clone a repository is to neutralize untrusted configuration and hooks, such a repository may be very much untrusted, so security is of heightened importance.

What to do with the alternative command

In addition to avoiding those things, we would need to use the alternate command for local clones, i.e., clones of a path or file:// URL. I believe the code to modify for that is mainly:

gitoxide/gix-transport/src/client/blocking_io/file.rs

Lines 219 to 223 in d602d7e

	None => (
	gix_command::prepare(service.as_str()).stderr(Stdio::null()),
	None,
	Cow::Borrowed(OsStr::new(service.as_str())),
	),

Where the Service::as_str method is currently, and probably should remain:

gitoxide/gix-transport/src/lib.rs

Lines 48 to 52 in d602d7e

	pub fn as_str(&self) -> &'static str {
	match self {
	Service::ReceivePack => "git-receive-pack",
	Service::UploadPack => "git-upload-pack",
	}

Avoiding modifying Service::as_str is partly because the current semantics for it make sense, and partly because there are a bunch of other uses of this as_str method that need to continue to use git-upload-pack literally.

Git behavior

As demonstrated above in "Expected behavior" (under "Way 1"), git has access to its upload-pack functionality even when git-upload-pack is not in the PATH in the environment from which git is called.

However, that doesn't prove that it really works, so here's a demonstration that it does, on a Windows 10 system (same as the system used above) where git-upload-pack is not found in PATH but git upload-pack is able to print its help message.

First I set up a repository to attempt to clone:

C:\Users\ek\src> git init repo
Initialized empty Git repository in C:/Users/ek/src/repo/.git/
C:\Users\ek\src> ni repo/a | Out-Null
C:\Users\ek\src> git -C repo add a
C:\Users\ek\src> git -C repo commit -m 'Initial commit'
[main (root-commit) bcb761b] Initial commit
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 a

Then I attempt to clone it, which succeeds, both with a path and with a file:// URL:

C:\Users\ek\src> git clone repo repo-clone
Cloning into 'repo-clone'...
done.

C:\Users\ek\src> rm -r -fo repo-clone
C:\Users\ek\src> git clone file:///C:/Users/ek/src/repo repo-clone
Cloning into 'repo-clone'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (3/3), done.

Steps to reproduce 🕹

Simple reproducer

The above demonstration can be repeated with gix instead of git (still in an environment where git is available but git-upload-pack is not).

C:\Users\ek\src> rm -r -fo repo-clone
C:\Users\ek\src> gix clone repo repo-clone
Error: Failed to invoke program "git-upload-pack"

Caused by:
    program not found

C:\Users\ek\src> gix clone file:///C:/Users/ek/src/repo repo-clone
Error: Failed to invoke program "git-upload-pack"

Caused by:
    program not found

Then add the cmd directory that contains a Git for Windows shim for git-upload-pack.exe to PATH. On my test system I'm using for this, that's:

C:\Users\ek\src> $env:PATH = "C:\Users\ek\scoop\apps\git\current\cmd;$env:PATH"

Trying the gix clone commands again now works, due to git-upload-pack being present in PATH:

C:\Users\ek\src> gix clone repo repo-clone
 07:09:21 indexing done 3.0 objects in 0.00s (12.8K objects/s)
 07:09:21 decompressing done 228B in 0.00s (903.3kB/s)
 07:09:21     Resolving done 3.0 objects in 0.05s (59.0 objects/s)
 07:09:21      Decoding done 228B in 0.05s (4.5kB/s)
 07:09:21 writing index file done 1.2kB in 0.00s (6.5MB/s)
 07:09:21  create index file done 3.0 objects in 0.05s (56.0 objects/s)
 07:09:21          read pack done 215B in 0.07s (3.2kB/s)
 07:09:21           checkout done 1.0 files in 0.00s (3.9K files/s)
 07:09:21            writing done 0B in 0.00s (0B/s)
HEAD:refs/remotes/origin/HEAD (implicit)
        bcb761beca6859eab9b95c1b64fdb8394b819acb HEAD symref-target:refs/heads/main -> refs/remotes/origin/HEAD [new]
+refs/heads/*:refs/remotes/origin/*
        bcb761beca6859eab9b95c1b64fdb8394b819acb refs/heads/main -> refs/remotes/origin/main [new]

C:\Users\ek\src> rm -r -fo repo-clone
C:\Users\ek\src> gix clone file:///C:/Users/ek/src/repo repo-clone
 07:09:29 indexing done 3.0 objects in 0.00s (14.7K objects/s)
 07:09:29 decompressing done 228B in 0.00s (1.0MB/s)
 07:09:29     Resolving done 3.0 objects in 0.05s (59.0 objects/s)
 07:09:29      Decoding done 228B in 0.05s (4.5kB/s)
 07:09:29 writing index file done 1.2kB in 0.00s (6.3MB/s)
 07:09:29  create index file done 3.0 objects in 0.05s (55.0 objects/s)
 07:09:29          read pack done 215B in 0.08s (2.7kB/s)
 07:09:29           checkout done 1.0 files in 0.00s (4.2K files/s)
 07:09:29            writing done 0B in 0.00s (0B/s)
HEAD:refs/remotes/origin/HEAD (implicit)
        bcb761beca6859eab9b95c1b64fdb8394b819acb HEAD symref-target:refs/heads/main -> refs/remotes/origin/HEAD [new]
+refs/heads/*:refs/remotes/origin/*
        bcb761beca6859eab9b95c1b64fdb8394b819acb refs/heads/main -> refs/remotes/origin/main [new]

Reproducer via the test suite - intro

The rest of this "Steps to reproduce" section covers the use of the test suite, why I didn't find this before by running tests, and how I have now done so. It can safely be skipped if these subtopics are of less interest.

Reproducer via the test suite - background

We have a number of tests that aren't expected to work when git is not in PATH. I've always tested improvements to gitoxide that relate to systems where git is not in PATH--such as various changes to gix-path and gix-command for finding git and using it to find other pieces of its installation--mainly by running particular parts of the test suite, especially the tests for those two crates.

That we don't have the whole test suite compatible with systems where git has to be found by means other than a PATH search is one of the factors obscuring issues like this one. I considered listing this under "Expected behavior" too, but it should probably have its own issue if I don't manage to open a PR for it soon--I'll do one or the other.

The other main factor is that the main Windows system where git is installed via scoop is also a system with a "dirty" environment: it has components of other Git installations leaking into the environment, becuase I have a full standaline MSYS2 installation with some of its directories in PATH. This has the advantage of surfacing some oddities that arise in "dirty" environments so I can write code or choose implementation strategies that guard against them, as in #1862 (comment). But it has the disadvantage that some bugs, such as this one, that I should've found and reported a long time ago, have eluded me.

The system used here for all commands and output shown in this issue is a different one with Git installed only via scoop. (Except technically for the presence of other Git componetns in the MinGit installation bundled by Visual Studio 2026 for its own internal use, which do not "leak" into any of the environments shown or discussed here.)

Reproducer via the test suite - details

However, this can be reproducd using the test suite while avoiding most test failures that aren't related to the failure to find and use git-upload-pack in local clones by testing on a system where the Git for Windows (git root)\bin directory is in PATH while its (git root)\cmd directory, and other bin directories associated with Git for Windows, are not:

• That implies not running the tests from a Git Bash environment, where more bin directories are in PATH. So it should be done in PowerShell. This gist shows such a test run.

• It can be contrasted to the far fewer subset of tests that fail in the same environment except with the Git for Windows cmd directory prepended to PATH. That is shown in this second gist.

• Both can be contrasted to a test run on the same system in Git Bash, as shown in this third gist.

[Byron]

Thanks a lot for reporting and the detailed analysis!

I will be looking forward to a follow-up PR that fixes this, everything that was proposed seemed feasible.
Part of me thought that one could even try to keep the implementation of finding git-upload-pack private while there is no motivation to expose a special folder or change the search patterns.
Lastly, I didn't catch why one wouldn't use git upload-pack, as we are already able to find the git binary pretty well.

Thanks again.

[EliahKagan]

I will be looking forward to a follow-up PR that fixes this

Yes, I do plan to do that. It might not be my next PR though.

Part of me thought that one could even try to keep the implementation of finding git-upload-pack private while there is no motivation to expose a special folder or change the search patterns.

If we go the route of finding the actual executable git-upload-pack.exe on Windows, then since the existing logic in gix_path::env::auxiliary is already almost capable of doing that correctly, it would duplicate less logic (and be faster to implement and test) to extend and use it.

Separately from this use, I do want to eventually extend that, though in the absence of this use case I'm uncertain if it should look in the cmd directory:

• We would already benefit from this for bash. Exposing that functionality is all that remains to do for #1886.
• Doing this more broadly would also help in improving shebang handling on Windows. Even if we don't make that open-ended, it seems to me tat we should (eventually) find select executables associated with the Git Bash MSYS2 environment including sh, bash, perl, env, and probably some others. See also the conversation #1815 (comment) on that topic as well as on other environments. However, also from there, I can see your reluctance to go as far as Git finding and running executables residing outside of the caller's path.
• Various parts of the test suite also use *nix commands that Windows may not have. For example, the regression test for #2080 added in #2262 uses cat as a clean and smudge filter. That test fails on Windows systems that don't have cat in PATH. (I'm hoping to include a fix for that by modifyigng the test code, in a forthcoming PR. But if I don't manage to include it then I'll open an issue so it's not forgotten.)
• Also related to filters, the official InnoSetup installer for Git for Windows offers to install Git LFS and, if I recal correctly, does so by default. When it installed Git LFS, git-lfs.exe is placed in the cmd directory. Being able to find and use this seems like it could be helpful.

it seems to me that the right balance may be to offer a facility for finding executables associated with a Git for Windows installation, and use it automatically in select cases where the rationale is especially strong (e.g., being able to run a script that has a shebang for bash), but not rush to use it automatically in other cases (e.g., not automatically resolving cat). With gitoxide being primarily a library, programs could use such a facility or not, according to their design preferences. What do you think of this?

Lastly, I didn't catch why one wouldn't use git upload-pack, as we are already able to find the git binary pretty well.

Whether or not anything is done in gix-path for finding executables, I think it is okay to use git upload-pack for this. That may be better. There is no strong reason not to use it (this is one of the two ways to do this that I advocated as good).

The logic of using it would be a little more complicated because it wouldn't just come down to passing a different path. It has to be done as two separate arguments (since a shell shouldn't be used, for the reasons described in report above). But I don't think this is a problem.

However, there is the question of what should happen if git-upload-pack does exist. Should we look for that, use it if present, and fall back to calling git with upload-pack as its first argument?

There is also the question of whether this should all be specific to Windows or should also be done on Unix-like systems.

I emphasize that none of those are reasons not to use git upload-pack.

[Byron]

If we go the route of finding the actual executable git-upload-pack.exe on Windows, then since the existing logic in gix_path::env::auxiliary is already almost capable of doing that correctly, it would duplicate less logic (and be faster to implement and test) to extend and use it.

I see, and I welcome this. Particularly seeing it in context of the other benefits that doing so would unlock. On Windows this will help, and might even be what's needed to achieve greater/best compatibility.

it seems to me that the right balance may be to offer a facility for finding executables associated with a Git for Windows installation, and use it automatically in select cases where the rationale is especially strong (e.g., being able to run a script that has a shebang for bash), but not rush to use it automatically in other cases (e.g., not automatically resolving cat). With gitoxide being primarily a library, programs could use such a facility or not, according to their design preferences. What do you think of this?

I love it. It's a conservative take on offering a public API for this, but using it internally only when there is a strong rationale to do so.

The logic of using it would be a little more complicated because it wouldn't just come down to passing a different path. It has to be done as two separate arguments (since a shell shouldn't be used, for the reasons described in report above). But I don't think this is a problem.

Ah, I see. I never perceived it as anything worth mentioning because std::process::Command can be used to safely pass any amount of arguments.

However, there is the question of what should happen if git-upload-pack does exist. Should we look for that, use it if present, and fall back to calling git with upload-pack as its first argument?

I wonder what Git does in cases where git-upload-pack isn't in the path. Maybe that can drive the answer to this. My intuition is to use git-upload-pack if possible, and fall back to git upload-pack otherwise. But that's only needed if there is no way for ourselves to find git-upload-pack in auxiliary folders, which would ultimately be my preference. That way, it's very clear what's happening, and a very conscious choice, instead of relying on Git to resolve it.
Of course, this could also be exactly the wrong thing to do, but I'd wait until it becomes an issue.