fix: permission denied error for /proc/self/mountinfo

[ritesh-harihar]

Problem reported

A workload running under the exec2 driver received Permission denied when reading /proc/self/mountinfo, even though the path was explicitly listed in the task's unveil config. No Landlock violations appeared in the audit log — the access was silently blocked from within the sandboxing layer itself.

Application error observed (Cassandra / DSE on RHEL 9 and Ubuntu):
java.io.FileNotFoundException: /proc/self/mountinfo (Permission denied)

Jobspec that triggered the issue:

config {
  command = "/bin/bash"
  args    = ["-c", "${NOMAD_ALLOC_DIR}/dse/dse-6.9.13/bin/dse cassandra -f"]
  unveil  = [
    "r:/proc/cpuinfo",
    "r:/proc/meminfo",
    "r:/proc/self/mountinfo",   # ← failed with permission denied
    "rwc:/var/lib/cassandra",
  ]
}

Root cause

convert() called os.Stat(filepath) on every unveil entry. For /proc/self/mountinfo, os.Stat follows the /proc/self magic symlink and resolves it to /proc/<shim-pid>/mountinfo —> the shim's own PID entry. Landlock locks that specific inode before unshare --mount-proc runs. After the unshare, the task's private /proc contains completely different inodes; the shim-PID entry no longer exists, so every read returns EPERM.

Paths without symlink indirection — /proc/cpuinfo, /proc/meminfo, r:/proc were unaffected because their inodes are stable across the namespace boundary.

Fix:

For any path matching /proc/self/* or /proc/thread-self/* before calling os.Stat, promote it to Dir("/proc", mode). This is because go-landlock registers rules via O_PATH which pins the inode at registration time. /proc/self resolves to /proc/<shim-pid>, an inode that does not exist in the task's private mount namespace after unshare --mount-proc. Pinning it would always produce EPERM. Promoting to Dir("/proc", mode) uses the stable /proc directory inode instead, which covers all descendants.

The defaults block also adds Dir("/proc", "r") unconditionally when defaults=true, so runtimes (JVM, Go) can read /proc/self/cgroup and /proc/self/mountinfo without needing any explicit unveil entry.

Unveil input	isProcSelfPath?	os.Stat result	Before this PR	After this PR
r:/proc/self/mountinfo	yes	skipped	EPERM — File emitted with shim-PID inode; inode gone after unshare --mount-proc	OK — promoted to Dir("/proc","r"); stable directory inode covers all descendants
r:/proc/self/cgroup	yes	skipped	EPERM — same inode problem	OK — promoted to Dir("/proc","r")
r:/proc/thread-self/mountinfo	yes	skipped	EPERM — same inode problem	OK — promoted to Dir("/proc","r")
r:/proc/cpuinfo	no	succeeds — stable regular file	OK — File emitted, inode stable	OK — unchanged, same path
r:/proc	no	succeeds — directory	OK — Dir emitted	OK — unchanged
r:/sys/fs/cgroup	no	succeeds — directory	OK — Dir emitted	OK — unchanged
r:/etc/passwd	no	succeeds — regular file	OK	OK — unchanged
No explicit /proc unveil, unveil_defaults=true	—	—	EPERM — no /proc rule in defaults	OK — Dir("/proc","r") added unconditionally by defaults block

Testing

Details

1. unveil_defaults=true, unveil_by_task=true

job "proc-specific-files" {
  type = "batch"

  constraint {
    attribute = "${attr.kernel.name}"
    value     = "linux"
  }

  group "group" {
    reschedule {
      attempts  = 0
      unlimited = false
    }

    restart {
      attempts = 0
      mode     = "fail"
    }

    task "read-proc" {
      driver = "exec2"

      config {
        command = "/bin/sh"

        args = [
          "-c",
          "echo '=== mountinfo ===' && cat /proc/self/mountinfo | head -3 && echo '=== cpuinfo ===' && head -1 /proc/cpuinfo && echo '=== meminfo ===' && head -1 /proc/meminfo && echo 'ALL OK'"
        ]

        # Specific sub-paths beneath /proc.
        # The virtualFSRoot logic in convert() should resolve these
        # to the stable /proc mount point directly.
        unveil = [
          "r:/proc/self/mountinfo",
          "r:/proc/cpuinfo",
          "r:/proc/meminfo"
        ]
      }

      resources {
        cpu    = 100
        memory = 32
      }
    }
  }
}

Result Before:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-exec2$ nomad alloc logs --stderr 22ef4a46
cat: /proc/self/mountinfo: Permission denied (os error 13)

Result After:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-exec2$ nomad alloc logs 10b1b25c
=== mountinfo ===
1079 1076 0:38 /scon/containers/01KS4F7XKHAVX614ZKDHGS499Q/rootfs / rw,noatime master:35 - btrfs /dev/vdb1 rw,nodatasum,nodatacow,ssd,discard,space_cache=v2,subvolid=333,subvol=/scon/containers/01KS4F7XKHAVX614ZKDHGS499Q
1080 1079 0:65 / /dev rw,relatime master:36 - tmpfs none rw,size=492k,mode=755
1081 1080 0:7 /fuse /dev/fuse rw,nosuid,noexec,relatime master:39 - devtmpfs devtmpfs rw,size=8206044k,nr_inodes=2051511,mode=755
=== cpuinfo ===
processor       : 0
=== meminfo ===
MemTotal:       16413624 kB
ALL OK

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-exec2$ nomad alloc logs -stderr 10b1b25c

2) Test for panic (unveil_defaults=false + command NOT in any unveil list)
(defaults=false, nothing unveiled → Before: panic(ExitCode 2)→ After: No panic

job "no-defaults-blocked" {
  type = "batch"

  group "group" {
    reschedule {
      attempts  = 0
      unlimited = false
    }
    restart {
      attempts = 0
      mode     = "fail"
    }

    task "task" {
      driver = "exec2"

      config {
        command = "/usr/bin/env" // exists on disk but /usr/bin is NOT unveiled
        // no unveil entries → empty Landlock ruleset
        // NOMAD_TASK_DIR also not unveiled → chdir fails with EACCES
      }

      resources {
        cpu    = 100
        memory = 32
      }
    }
  }
}

Result Before:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-exec2$ nomad alloc logs -stderr 2026dc8b
panic: interface conversion: error is *fs.PathError, not *exec.ExitError

goroutine 1 [running]:
github.com/hashicorp/nomad-driver-exec2/pkg/shim.init.0.func1()
        github.com/hashicorp/nomad-driver-exec2/pkg/shim/z_shim_cmd.go:107 +0xd78
github.com/hashicorp/nomad/helper/subproc.Do({0x12bb9fc, 0xa}, 0x130b9c0)
        github.com/hashicorp/nomad@v1.11.3/helper/subproc/subproc.go:39 +0xa0
github.com/hashicorp/nomad-driver-exec2/pkg/shim.init.0()
        github.com/hashicorp/nomad-driver-exec2/pkg/shim/z_shim_cmd.go:46 +0x38

Result After:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-exec2$ nomad alloc logs --stderr c8578a39
task command failed: open /dev/null: permission denied

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[ritesh-harihar]

Not related to this PR.

If cmd.Dir points to an un-unveiled directory, cmd.Run() returns a *fs.PathError from the kernel-level chdir(2), not an *exec.ExitError. This bare type assertion panicked on the wrong type, leaving the allocation stuck. Replaced with errors.As and returns ExitFailure = 1 .

[tgross]

A couple things seem to be missing here for my understanding:'

• Where does this report come from? Didn't this pop up when you were testing #98 ?
• If unshare is setting up a PID namespace and --mount-proc, won't the inode for /proc in the "container" be different than the one we're reading here regardless?

[ritesh-harihar]

• Where does this report come from? Didn't this pop up when you were testing fix: GOMAXPROCS auto-detection failure #98 ?

In PR #97 , the testing was done mostly when unveil_defaults=true. While testing this PR for unveil_defaults=false received panic(added a jobspec under Testing section)

Before work_dir, cmd.Dir was always NOMAD_TASK_DIR (always unveiled), so cmd.Run() could only ever fail with *exec.ExitError. With work_dir we can now set cmd.Dir to a path that is unveil-blocked or doesn't exist — in that case cmd.Run() returns *fs.PathError from the chdir step, and the bare err.(*exec.ExitError) would panic.

[ritesh-harihar]

If the case is unviel_defaults=true we dont need to do any change as Dir("/proc", "r") is already added in defaults.

The logic added in convert()only makes sense when unviel_defaults=false and unviel=[/proc/self/mountinfo].
Does unviel_defaults=false is used by the users?

• If unshare is setting up a PID namespace and --mount-proc, won't the inode for /proc in the "container" be different than the one we're reading here regardless?

Yes, the device number changes. The specific file inodes like /proc/self/mountinfo also change entirely (330638 → 328614).

But Dir("/proc","r") still works, for two reasons:

1. Inode 1 is always the root of any procfs. Both the host /proc (dev=66) and the task's /proc (dev=158) have inode=1. The root is structurally stable across instances.

2. Landlock PATH_BENEATH is evaluated in the accessing process's mount namespace. When the task reads /proc/self/mountinfo, the kernel asks "is this path beneath /proc?" inside the task's own namespace where /proc is the new dev=158 mount. The containment relationship holds. The registered fd's original device number doesn't matter at check time.

This bug was specifically for a file("/proc/self/mountinfo","r"). A specific file inode has no stable equivalent across a remount, the root always does. So Dir("/proc","r") avoids this entirely by anchoring to the filesystem root rather than a specific file inode.