Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Podman 5.3.0 win-sshproxy.tid: The system cannot find the file specified. (install via scoop) #24557

Closed
johnnykang opened this issue Nov 13, 2024 · 47 comments · Fixed by #25151
Closed
Assignees
Labels
jira kind/bug Categorizes issue or PR as related to a bug. machine windows issue/bug on Windows

Comments

@johnnykang
Copy link

johnnykang commented Nov 13, 2024

Issue Description

after upgrading to the Podman 5.3.0 (via scoop), when starting the podman machine, there is a message

API forwarding for Docker API clients is not available due to the following startup failures.
The system cannot find the path specified.

it turned out that the c:<user_profile>.local\share\containers\podman\machine\wsl\podman-machine-default\win-sshproxy.tid is not created.

Downgrading to Podman 5.2.5 , the message is gone.

Podman machine runs in WSL2.

Steps to reproduce the issue

As above in the description

Describe the results you received

As above in the description

Describe the results you expected

podman machine starts and API forwarding for Docker API clients on Windows machine works as expected without error message.

podman info output

Podman 5.3.0

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

@johnnykang johnnykang added the kind/bug Categorizes issue or PR as related to a bug. label Nov 13, 2024
@glektarssza
Copy link

Calling podman machine stop will also produce the following warning/error in this situation:

Could not stop API forwarding service (win-sshproxy.exe): open C:\Users\<username>\.local\share\containers\podman\machine\wsl\podman-machine-default\win-sshproxy.tid: The system cannot find the file specified.

@baude
Copy link
Member

baude commented Nov 13, 2024

It helps when you provide podman info like the template recommends. In both cases, is it safe to assume WSL is being used?

@baude
Copy link
Member

baude commented Nov 13, 2024

@l0rd thoughts ?

@glektarssza
Copy link

In my case, yes. My podman machine instance is running inside of WSL.

@glektarssza
Copy link

glektarssza commented Nov 13, 2024

podman version output:

Client:       Podman Engine
Version:      5.3.0
API Version:  5.3.0
Go Version:   go1.23.3
Git Commit:   874bf2c301ecf0ba645f1bb45f81966cc755b7da
Built:        Wed Nov 13 06:19:59 2024
OS/Arch:      windows/amd64

Server:       Podman Engine
Version:      5.2.5
API Version:  5.2.5
Go Version:   go1.22.7
Built:        Thu Oct 24 18:00:00 2024
OS/Arch:      linux/amd64

podman info output:

host:
  arch: amd64
  buildahVersion: 1.37.5
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.12-2.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.64
    systemPercent: 0.25
    userPercent: 0.11
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: container
    version: "40"
  eventLogger: journald
  freeLocks: 2048
  hostname: GlekPC
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 5.15.167.4-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 16078487552
  memTotal: 16776019968
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.2-2.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.12.2
    package: netavark-1.12.2-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.12.2
  ociRuntime:
    name: crun
    package: crun-1.17-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.17
      commit: 000fa0d4eeed8938301f3bcf8206405315bc1017
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240906.g6b38f07-1.fc40.x86_64
    version: |
      pasta 0^20240906.g6b38f07-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 0h 5m 17.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 914518016
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 5.2.5
  Built: 1729814400
  BuiltTime: Thu Oct 24 18:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.7
  Os: linux
  OsArch: linux/amd64
  Version: 5.2.5

podman machine info output:

host:
    arch: amd64
    currentmachine: podman-machine-default
    defaultmachine: podman-machine-default
    eventsdir: C:\Users\roblo\.local\share\containers\podman\podman
    machineconfigdir: C:\Users\roblo\.config\containers\podman\machine\wsl
    machineimagedir: C:\Users\roblo\.local\share\containers\podman\machine\wsl
    machinestate: Running
    numberofmachines: 1
    os: windows
    vmtype: wsl
version:
    apiversion: 5.3.0
    version: 5.3.0
    goversion: go1.23.3
    gitcommit: 874bf2c301ecf0ba645f1bb45f81966cc755b7da
    builttime: Wed Nov 13 06:19:59 2024
    built: 1731503999
    osarch: windows/amd64
    os: windows

podman machine inspect output:

[
     {
          "ConfigDir": {
               "Path": "C:\\Users\\roblo\\.config\\containers\\podman\\machine\\wsl"
          },
          "ConnectionInfo": {
               "PodmanSocket": null,
               "PodmanPipe": {
                    "Path": "\\\\.\\pipe\\podman-machine-default"
               }
          },
          "Created": "2024-11-13T14:54:20.3775309-07:00",
          "LastUp": "2024-11-13T14:55:18.3645367-07:00",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 16,
               "DiskSize": 100,
               "Memory": 2048,
               "USBs": []
          },
          "SSHConfig": {
               "IdentityPath": "C:\\Users\\roblo\\.local\\share\\containers\\podman\\machine\\machine",
               "Port": 50275,
               "RemoteUsername": "user"
          },
          "State": "running",
          "UserModeNetworking": false,
          "Rootful": false,
          "Rosetta": false
     }
]

Edit: Apologies, forgot I had downgraded back to 5.2.5 when I dumped all this information. It is now updated with the info dump from 5.3.0 when I was having this issue.

@johnnykang
Copy link
Author

It helps when you provide podman info like the template recommends. In both cases, is it safe to assume WSL is being used?

Yup. Podman machine runs in the WSL2.

@glektarssza
Copy link

Also might be worth mentioning that my podman setup was installed via scoop. Shouldn't impact anything (I think) but you never know.

@johnnykang
Copy link
Author

Also might be worth mentioning that my podman setup was installed via scoop. Shouldn't impact anything (I think) but you never know.

Great point. And that's my installation method (via scoop) as well.

@l0rd
Copy link
Member

l0rd commented Nov 14, 2024

We were able to reproduce this issue with @jeffmaury. The problem was related to the %USERPROFILE%/.ssh/config. As a workaround we renamed the config file.

Can you please provide the output of the command podman info --log-level debug.

@glektarssza
Copy link

@l0rd Here you go:

time="2024-11-13T19:30:32-07:00" level=info msg="C:\\Users\\roblo\\scoop\\apps\\podman\\current\\podman.exe filtering at log level debug"
time="2024-11-13T19:30:32-07:00" level=debug msg="Called info.PersistentPreRunE(C:\\Users\\roblo\\scoop\\apps\\podman\\current\\podman.exe info --log-level debug)"
time="2024-11-13T19:30:32-07:00" level=debug msg="SSH Ident Key \"C:\\\\Users\\\\roblo\\\\.local\\\\share\\\\containers\\\\podman\\\\machine\\\\machine\" SHA256:DW4NTzZN7HgVBWBnF8rA6cmGd9LchILrC9GP0vnMU7Q ssh-ed25519"
time="2024-11-13T19:30:32-07:00" level=debug msg="DoRequest Method: GET URI: http://d/v5.3.0/libpod/_ping"
time="2024-11-13T19:30:32-07:00" level=debug msg="DoRequest Method: GET URI: http://d/v5.3.0/libpod/info"
host:
  arch: amd64
  buildahVersion: 1.37.5
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.12-2.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.47
    systemPercent: 0.35
    userPercent: 0.18
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: container
    version: "40"
  eventLogger: journald
  freeLocks: 2048
  hostname: GlekPC
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 5.15.167.4-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 16092475392
  memTotal: 16776024064
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.2-2.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.12.2
    package: netavark-1.12.2-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.12.2
  ociRuntime:
    name: crun
    package: crun-1.17-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.17
      commit: 000fa0d4eeed8938301f3bcf8206405315bc1017
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240906.g6b38f07-1.fc40.x86_64
    version: |
      pasta 0^20240906.g6b38f07-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 0h 6m 30.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 961994752
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 5.2.5
  Built: 1729814400
  BuiltTime: Thu Oct 24 18:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.7
  Os: linux
  OsArch: linux/amd64
  Version: 5.2.5

time="2024-11-13T19:30:32-07:00" level=debug msg="Called info.PersistentPostRunE(C:\\Users\\roblo\\scoop\\apps\\podman\\current\\podman.exe info --log-level debug)"
time="2024-11-13T19:30:32-07:00" level=debug msg="Shutting down engines"

@l0rd
Copy link
Member

l0rd commented Nov 14, 2024

Thank you. No we were not able to reproduce your error then. The problem was a different one related to the SSH config.

We will try to reproduce it using scoop.

@johnnykang
Copy link
Author

It looks like if Podman is installed via winget, win-sshproxy works perfectly fine.

I suspect this issue is specific to podman installation via scoop.

@Luap99 Luap99 added machine windows issue/bug on Windows labels Nov 14, 2024
@glektarssza
Copy link

I'll try installing via winget after work today to confirm this as well. If I cannot reproduce the issue in that environment I think it's a scoop-specific issue, yeah.

@l0rd
Copy link
Member

l0rd commented Nov 14, 2024

Thank you @johnnykang for checking. And you are right, that should be the problem.

The version of gvproxy has been updated in 5.3 but the scoop installer extracts podman.exe, not gvproxy from the installer.

@baude any idea who maintains the podman scoop bucket? From git history it looks @niheaven is the last one that have updated it.

@l0rd
Copy link
Member

l0rd commented Nov 14, 2024

@glektarssza @johnnykang out of curiosity: why are you installing via scoop rather than winget?

@glektarssza
Copy link

I've been a long time user of scoop so mainly because it's an ecosystem I'm familiar with and, at the time I started using it, had the software I was using.

I'm not opposed to changing but it would be a bit of a pain to migrate everything over at this point depending on what is and is not available in the winget ecosystem.

@baude
Copy link
Member

baude commented Nov 14, 2024

i do not know who maintains scoop.

@glektarssza
Copy link

I believe it's maintained by the community who uses it via a collection of GitHub repositories under the https://github.com/orgs/ScoopInstaller organization. At least that's what their website (https://scoop.sh) seems to indicate down at the bottom.

@glektarssza
Copy link

Actually, I stand corrected. Their website links specifically to https://github.com/orgs/ScoopInstaller/people as the maintainers.

@jeffmaury
Copy link

I can reproduce after having installed Podman through scoop

@johnnykang johnnykang changed the title Podman 5.3.0 win-sshproxy.tid: The system cannot find the file specified. Podman 5.3.0 win-sshproxy.tid: The system cannot find the file specified. (install via scoop) Nov 14, 2024
@johnnykang
Copy link
Author

due to the fact that this issue only occurs when Podman is installed via scoop, i would like to close this issue as i don't believe the team is responsible for fixing it.

closing.

@glektarssza
Copy link

glektarssza commented Nov 14, 2024

I opened ScoopInstaller/Main#6327. Hopefully they can figure this out on their end.

@niheaven
Copy link

Hi all, Scoop uses bundled version of gvproxy.exe and win-sshproxy, which version comes with podman v5.3.0?

And in my podman installation, podman machine start doesn't output any error, but yes, podman machine stop does, even after I replaced existing gvproxy and win-sshproxy in the podman folder with the latest ones (v0.8.0).

@johnnykang
Copy link
Author

And in my podman installation, podman machine start doesn't output any error

Mine output "The system cannot find the path specified." in the startup message. It is not an obvious message, but it warned me that the Docker API stopped working.

@l0rd l0rd self-assigned this Nov 21, 2024
l0rd added a commit to l0rd/podman that referenced this issue Nov 21, 2024
In go 1.23 filepath.EvalSymlink(path) behavior has changed
and it returns an error if path is a Windows junction. The
new behavior breaks some functionalities (see containers#24557).

The GODEBUG option `winsymlink=0` allow to get run `EvalSymlink`
with its legacy behavior.

Signed-off-by: Mario Loriedo <[email protected]>
@niheaven
Copy link

Since I'm unsure whether other applications utilize winsymlink in Go, I implemented a temporary fix for Scoop manifest by adding the versioned directory to the PATH, which seems to be working well.

@Laess3r
Copy link

Laess3r commented Jan 7, 2025

Hi guys!

Are there any updates to this topic? I had to instruct our team to not upgrade / downgrade to 5.2.5

Thanks and best regards.

@l0rd
Copy link
Member

l0rd commented Jan 7, 2025

Hi @Laess3r and happy new year 👋 I should open a PR to fix this problem this week and hopefully the fix will land in v5.4.0.

Out of curiosity: what's the issue you are facing? I am asking because the original problem (after installing podman using scoop, machines fails to start) should be fixed now and I am wondering if there is another junction-related problem we are not aware of.

@disacrol
Copy link

disacrol commented Jan 13, 2025

@l0rd I have the exact same issue as @Laess3r. Had to instruct the team to use a version lower than 5.3.
When I install podman from scoop and start a new podman machine, there is no error.
But if I stop podman machine and re-open the console, the error resurfaces when starting podman machine again.

Maybe there is some environment variable is use as a workaround, that is not persisted?

EDIT: I noticed when I install for example podman 5.3.1, I get that hardcoded on top of the PATH:

PS C:\> Get-ChildItem Env:Path

Name                           Value
----                           -----
Path                           C:\Users\disacrol\scoop\apps\podman\5.3.1;C:\Program Files\PowerShell\7;C:\Windows\system32;C:\Windows; (...)

But this is only working in the scope of the powershell session I installed podman with.

Informing @niheaven

l0rd added a commit to l0rd/podman that referenced this issue Jan 20, 2025
In go 1.23 filepath.EvalSymlink(path) behavior has changed
and it returns an error if path is a Windows junction. The
new behavior breaks some functionalities (see containers#24557).

The GODEBUG option `winsymlink=0` allow to get run `EvalSymlink`
with its legacy behavior.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/podman that referenced this issue Jan 20, 2025
In go 1.23 filepath.EvalSymlink(path) behavior has changed
and it returns an error if path is a Windows junction. The
new behavior breaks some functionalities (see containers#24557).

The GODEBUG option `winsymlink=0` allow to get run `EvalSymlink`
with its legacy behavior.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/podman that referenced this issue Jan 29, 2025
The behavior of function `path/filepath.EvalSymlinks()` has
changed in Go v1.23:

- https://go-review.googlesource.com/c/go/+/565136
- https://go.dev/doc/go1.23#minor_library_changes
- https://tip.golang.org/doc/godebug

As a consequences, starting with Podman 5.3.0, when installing
on Windows (WSL) using scoop, Podman fails to start because it
fails to find helper binaries. Scoop copies Podman binaries in
a folder of type Junction and `EvalSymlinks` returns an error.
The problem is described in containers#24557.

To address this problem we are checking if a path is a `Symlink`
before calling `EvalSymlinks` and, if it's not (hardlinks, mount
points or canonical files), we are calling `path/filepath.Clean`
for consistency. In fact `path/filepath.EvalSymlinks`, after
evaluating a symlink target, calls `Clean` too.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/podman that referenced this issue Jan 29, 2025
The behavior of function `path/filepath.EvalSymlinks()` has
changed in Go v1.23:

- https://go-review.googlesource.com/c/go/+/565136
- https://go.dev/doc/go1.23#minor_library_changes
- https://tip.golang.org/doc/godebug

As a consequences, starting with Podman 5.3.0, when installing
on Windows (WSL) using scoop, Podman fails to start because it
fails to find helper binaries. Scoop copies Podman binaries in
a folder of type Junction and `EvalSymlinks` returns an error.
The problem is described in containers#24557.

To address this problem we are checking if a path is a `Symlink`
before calling `EvalSymlinks` and, if it's not (hardlinks, mount
points or canonical files), we are calling `path/filepath.Clean`
for consistency. In fact `path/filepath.EvalSymlinks`, after
evaluating a symlink target, calls `Clean` too.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/podman that referenced this issue Jan 29, 2025
The behavior of function `path/filepath.EvalSymlinks()` has
changed in Go v1.23:

- https://go-review.googlesource.com/c/go/+/565136
- https://go.dev/doc/go1.23#minor_library_changes
- https://tip.golang.org/doc/godebug

As a consequences, starting with Podman 5.3.0, when installing
on Windows (WSL) using scoop, Podman fails to start because it
fails to find helper binaries. Scoop copies Podman binaries in
a folder of type Junction and `EvalSymlinks` returns an error.
The problem is described in containers#24557.

To address this problem we are checking if a path is a `Symlink`
before calling `EvalSymlinks` and, if it's not (hardlinks, mount
points or canonical files), we are calling `path/filepath.Clean`
for consistency. In fact `path/filepath.EvalSymlinks`, after
evaluating a symlink target, calls `Clean` too.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/podman that referenced this issue Jan 29, 2025
The behavior of function `path/filepath.EvalSymlinks()` has
changed in Go v1.23:

- https://go-review.googlesource.com/c/go/+/565136
- https://go.dev/doc/go1.23#minor_library_changes
- https://tip.golang.org/doc/godebug

As a consequences, starting with Podman 5.3.0, when installing
on Windows (WSL) using scoop, Podman fails to start because it
fails to find helper binaries. Scoop copies Podman binaries in
a folder of type Junction and `EvalSymlinks` returns an error.
The problem is described in containers#24557.

To address this problem we are checking if a path is a `Symlink`
before calling `EvalSymlinks` and, if it's not (hardlinks, mount
points or canonical files), we are calling `path/filepath.Clean`
for consistency. In fact `path/filepath.EvalSymlinks`, after
evaluating a symlink target, calls `Clean` too.

Signed-off-by: Mario Loriedo <[email protected]>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/podman that referenced this issue Jan 29, 2025
The behavior of function `path/filepath.EvalSymlinks()` has
changed in Go v1.23:

- https://go-review.googlesource.com/c/go/+/565136
- https://go.dev/doc/go1.23#minor_library_changes
- https://tip.golang.org/doc/godebug

As a consequences, starting with Podman 5.3.0, when installing
on Windows (WSL) using scoop, Podman fails to start because it
fails to find helper binaries. Scoop copies Podman binaries in
a folder of type Junction and `EvalSymlinks` returns an error.
The problem is described in containers#24557.

To address this problem we are checking if a path is a `Symlink`
before calling `EvalSymlinks` and, if it's not (hardlinks, mount
points or canonical files), we are calling `path/filepath.Clean`
for consistency. In fact `path/filepath.EvalSymlinks`, after
evaluating a symlink target, calls `Clean` too.

Signed-off-by: Mario Loriedo <[email protected]>
@disacrol
Copy link

disacrol commented Feb 4, 2025

@l0rd with podman 5.3.2 installed via scoop it looked to be working at first but no, at least not with user mode networking.

  1. open PowerShell
  2. install podman via scoop
  3. close PowerShell
  4. open PowerShell
  5. create a podman machine with user mode networking (podman machine init --user-mode-networking)
  6. podman machine start
  7. error occurs: Error: could not locate gvproxy.exe, which is necessary for user-mode networking, please reinstall

If PowerShell is not reopened on steps 3 and 4, it works fine.

Could this be strictly an issue with scoop?

@glektarssza
Copy link

@disacrol Can you provide the contents of your $env:PATH after installing via scoop but before reopening PowerShell and then again after reopening PowerShell? That might shed some light on things.

@disacrol
Copy link

disacrol commented Feb 4, 2025

Indeed podman dir is not persisted in the path.
I noticed that scoop persists the path of some apps (e.g. python) but not others (e.g. notepadplusplus). Could this just be a matter of fixing some flag in the podman's scoop manifest or something? Or that ScoopInstaller/Main#6335 was not enough to fix this?

Plain text to allow bold:

PS C:\Users\DOliveira> Write-Host $env:PATH
C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Users\DOliveira\scoop\shims;C:\Users\DOliveira\AppData\Local\Microsoft\WindowsApps;C:\Users\DOliveira\AppData\Local\Microsoft\WinGet\Packages\Microsoft.Sysinternals.TCPView_Microsoft.Winget.Source_8wekyb3d8bbwe;C:\Users\DOliveira\AppData\Local\JetBrains\Toolbox\scripts
PS C:\Users\DOliveira> scoop install podman
Installing 'podman' (5.3.2) [64bit] from 'main' bucket
Loading podman-5.3.2-setup.exe from cache
Checking hash of podman-5.3.2-setup.exe ... ok.
Running installer script...Adding \scoop\apps\podman\5.3.2 to your path.
done.
Installer added '
\scoop\apps\podman\5.3.2' to path. Removing.
Linking ~\scoop\apps\podman\current => ~\scoop\apps\podman\5.3.2
Creating shim for 'podman'.
'podman' (5.3.2) was installed successfully!
PS C:\Users\DOliveira> Write-Host $env:PATH
C:\Users\DOliveira\scoop\apps\podman\5.3.2;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Users\DOliveira\scoop\shims;C:\Users\DOliveira\AppData\Local\Microsoft\WindowsApps;C:\Users\DOliveira\AppData\Local\Microsoft\WinGet\Packages\Microsoft.Sysinternals.TCPView_Microsoft.Winget.Source_8wekyb3d8bbwe;C:\Users\DOliveira\AppData\Local\JetBrains\Toolbox\scripts

(reopen PS)

PS C:\Users\DOliveira> Write-Host $env:PATH
C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Users\DOliveira\scoop\shims;C:\Users\DOliveira\AppData\Local\Microsoft\WindowsApps;C:\Users\DOliveira\AppData\Local\Microsoft\WinGet\Packages\Microsoft.Sysinternals.TCPView_Microsoft.Winget.Source_8wekyb3d8bbwe;C:\Users\DOliveira\AppData\Local\JetBrains\Toolbox\scripts

@glektarssza
Copy link

Yeah, it's strange that it's not being persisted to your path... This is definitely something to do with scoop as I'm seeing the same behaviour on my system too. This is not a podman issue, I think. Best to report it to the scoop team.

@niheaven
Copy link

niheaven commented Feb 5, 2025

Scoop adds something to PATH in two steps: 1. it adds it to the current session; 2. it adds it via Environment Variables. Please check if it works after a reboot (to ensure Environment Variables are applied).

@disacrol
Copy link

Yes it does - and now that I have setup freshly (with newer podman 5.4.0 btw), it also works after re-opening PowerShell. Maybe there was something odd in my system. Thanks @l0rd and @niheaven for the info!

@l0rd
Copy link
Member

l0rd commented Feb 12, 2025

With 5.4.0 the fix provided by @niheaven (the PATH update) should not be necessary anymore. I have created a PR to revert it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jira kind/bug Categorizes issue or PR as related to a bug. machine windows issue/bug on Windows
Projects
None yet
Development

Successfully merging a pull request may close this issue.