Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge functionality from github.com/moby/moby/pkg/idtools into user #182

Open
wants to merge 87 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
b8432dc
Add utility/support package for user namespace support
estesp Oct 8, 2015
b39b044
Windows: Daemon broken on master
Oct 12, 2015
9b842bb
Correct build-time directory creation with user namespaced daemon
estesp Oct 14, 2015
2eab947
pkg: idtools: fix subid files parsing
runcom Feb 26, 2016
340c794
Change subordinate range-owning user to be a system user
estesp Mar 16, 2016
f675a09
Lazy init useradd and remove init()
estesp Apr 6, 2016
f67ec21
Don't start daemon in userns mode if graphdir inaccessible
estesp Aug 23, 2016
3afc0fa
Add support for looking up user/groups via `getent`
estesp Oct 20, 2016
896159a
Remove use of pkg/integration in pkg/idtools
vdemeester Nov 8, 2016
f389d73
pkg: return directly without ifs where possible
unclejack Dec 13, 2016
8a9ffb3
Partial refactor of UID/GID usage to use a unified struct.
dnephin May 19, 2017
ef675ed
Remove unused functions from archive.
dnephin May 24, 2017
a048609
Convert tarAppender to the newIDMappings.
dnephin May 24, 2017
da0b07e
Remove ToHost and replace it with IDMappings.ToHost
dnephin May 31, 2017
9124b45
Remove MkdirAllNewAs and update tests.
dnephin May 31, 2017
013fefa
Remove error return from RootPair
dnephin May 31, 2017
8a07e96
LCOW: Create layer folders with correct ACL
Jun 2, 2017
d1b011c
Switch Stat syscalls to x/sys/unix
tklauser Jul 27, 2017
df681c4
Increase Coverage of pkg/idtools
danyalKhaliq10p Sep 26, 2017
96887ee
idtools don't chown if not needed
cpuguy83 Oct 2, 2017
75a95c3
Merge pull request #35059 from cpuguy83/35002_handle_bind_create_errors
yongtang Oct 20, 2017
b7d71bf
Update idtools tests to test non-deprecated functions
thaJeztah Nov 18, 2017
5ac3c73
Minor refactor in idtools
thaJeztah Nov 18, 2017
111ed2c
Remove deprecated MkdirAllAs(), MkdirAs()
thaJeztah Nov 18, 2017
db7d60c
idtools.MkdirAs*: error out if dir exists as file
kolyshkin Nov 27, 2017
fd60e08
Simplify/fix MkdirAll usage
kolyshkin Sep 25, 2017
400e461
Remove redundant build-tags
thaJeztah Dec 18, 2017
a7ddc1e
Merge pull request #35823 from thaJeztah/remove-dead-code
yongtang Dec 19, 2017
20e76a8
Add canonical import comment
dnephin Feb 5, 2018
b286fb1
Automated migration using
dnephin Mar 13, 2018
33772b2
Fix typo in idtools tests
vdemeester Apr 20, 2018
e8f2892
Skip some tests requires root uid when run as user
vdemeester Apr 20, 2018
3aa6a47
Various code-cleanup
thaJeztah May 19, 2018
2c770cc
Update tests to use gotest.tools 👼
vdemeester Jun 11, 2018
95c004f
Merge pull request #37243 from vdemeester/gotestyourself-with-tools
Jun 13, 2018
9494191
Add ADD/COPY --chown flag support to Windows
salah-khan Nov 16, 2017
3d94a79
Stop sorting uid and gid ranges in id maps
dohse May 30, 2019
1790be4
Allow system.MkDirAll() to be used as drop-in for os.MkDirAll()
thaJeztah Aug 8, 2019
50f89a4
unconvert: remove unnescessary conversions
thaJeztah Aug 7, 2019
fbf48b1
Merge pull request #39668 from thaJeztah/replace_gometalinter
yongtang Sep 18, 2019
27cb94c
pkg/idtools: normalize comment formatting
thaJeztah Nov 27, 2019
6c35f19
bump gotest.tools v3.0.1 for compatibility with Go 1.14
thaJeztah Feb 7, 2020
c471a1c
pkg/idtools: fix use of bufio.Scanner.Err
kolyshkin Mar 12, 2020
8f173eb
remove group name from identity mapping
akhilerm May 24, 2020
d237acd
pkg/idtools: refactor to avoid string-splitting
thaJeztah Aug 20, 2020
a72a73d
Merge pull request #41377 from thaJeztah/refactor_idtools
cpuguy83 Aug 20, 2020
8ec694d
Ensure MkdirAllAndChown also sets perms
cpuguy83 Oct 6, 2020
c092f1a
Merge pull request #41964 from thaJeztah/CVE-2021-21284_master
tiborvass Feb 2, 2021
e1535c4
Fix userns-remap option when username & UID match
Rid Feb 9, 2021
6b4f2b8
pkg/system: deprecate some consts and move them to pkg/idtools
thaJeztah Jun 18, 2021
487b6be
Update to Go 1.17.0, and gofmt with Go 1.17
thaJeztah Aug 23, 2021
3c2d209
refactor: move from io/ioutil to io and os package
Juneezee Aug 24, 2021
0ff8db3
Finish refactor of UID/GID usage to a new struct
corhere Mar 14, 2022
08c3d78
pkg/idtools: remove deprecated NewIdentityMapping, UIDS() and GIDS()
thaJeztah Aug 9, 2022
9a4bd8a
pkg/idtools: mkdirAs(): fix infinite loops and repeated "chown"
thaJeztah Sep 27, 2022
b0b487c
pkg/*: fix "empty-lines" (revive)
thaJeztah Sep 23, 2022
754a5e5
Merge pull request #44211 from thaJeztah/more_linters_step1
thaJeztah Sep 28, 2022
966753b
pkg/system: move GetExitCode() to pkg/idtools, and un-export
thaJeztah Oct 5, 2022
bb49a85
pkg/idtools: mkdirAs() be more explicit about ignored args on Windows
thaJeztah Oct 5, 2022
14acd6d
pkg/idtools: mkdirAs(): move var and comment to where it's used
thaJeztah Oct 8, 2022
ac19915
pkg/idtools: remove unused CanAccess() stub for Windows
thaJeztah Oct 5, 2022
b8ca2ac
pkg/idtools: CanAccess(): reorder checks to allow early return
thaJeztah Oct 5, 2022
623c876
pkg/idtools: format code with gofumpt
thaJeztah Oct 8, 2022
4cbf553
pkg/idtools: don't use system.MkdirAll() where not needed
thaJeztah Oct 5, 2022
d8a81d5
pkg/idtools: cleanup errors
thaJeztah Oct 5, 2022
45b8f6c
pkg/idtools: don't use system.Stat() on unix
thaJeztah Oct 8, 2022
54804cf
pkg/idtools: simplify if-statement
thaJeztah Oct 9, 2022
66de297
pkg/idtools: setPermissions() accept Identity as argument
thaJeztah Oct 9, 2022
68d084c
pkg/idtools: remove CanAccess(), and move to daemon
thaJeztah Oct 8, 2022
5fb47b5
pkg/idtools: remove execCmd() utility
thaJeztah Oct 9, 2022
18279d1
remove pre-go1.17 build-tags
thaJeztah May 5, 2023
3aca343
run `getent` with a noop stdin
ndeloof Jun 9, 2023
a8e5f1c
pkg/idtools: use string-literals for easier grep'ing
thaJeztah Jul 5, 2023
8686e26
pkg/idtools: remove sync.Once, and include lookup error
thaJeztah Sep 6, 2023
bc7d145
Merge pull request #46417 from thaJeztah/idtools_preserve_error
thaJeztah Sep 6, 2023
94b31f9
migrate to github.com/moby/sys/user
thaJeztah Oct 24, 2023
5b99a6c
Merge pull request #46711 from thaJeztah/switch_user
AkihiroSuda Oct 25, 2023
be00248
pkg/idtools: fix shadowed variable (govet)
thaJeztah Nov 5, 2024
758bc52
pkg/idtools: remove redundant capturing of loop vars (copyloopvar)
thaJeztah Nov 12, 2024
3d38759
pkg/idtools: remove uses of deprecated system.MkdirAll
thaJeztah Dec 21, 2024
fdf2879
Merge pull request #49162 from thaJeztah/pkg_system_volume_uuid
thaJeztah Dec 23, 2024
5805d70
pkg/idtools: use lazyregexp to compile regexes on first use
thaJeztah Jul 15, 2024
9ecbe7b
pkg/idtools: rewrite to use moby/sys/user
thaJeztah Jan 7, 2025
6134528
Split internal idtools functionality
dmcgowan Dec 13, 2024
7e518ec
Merge pull request #49087 from dmcgowan/split-idtools-internal
thaJeztah Jan 7, 2025
8e08d6b
Merge idtools from github.com/moby/moby/pkg/idtools
dmcgowan Jan 10, 2025
db55716
Update interface to fit into user package
dmcgowan Jan 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions user/idtools.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
package user

import (
"fmt"
"os"
)

// MkdirOpt is a type for options to pass to Mkdir calls
type MkdirOpt func(*mkdirOptions)

type mkdirOptions struct {
onlyNew bool
}

// WithOnlyNew is an option for MkdirAllAndChown that will only change ownership and permissions
// on newly created directories. If the directory already exists, it will not be modified
func WithOnlyNew(o *mkdirOptions) {
o.onlyNew = true
}

// MkdirAllAndChown creates a directory (include any along the path) and then modifies
// ownership to the requested uid/gid. By default, if the directory already exists, this
// function will still change ownership and permissions. If WithOnlyNew is passed as an
// option, then only the newly created directories will have ownership and permissions changed.
func MkdirAllAndChown(path string, mode os.FileMode, uid, gid int, opts ...MkdirOpt) error {
var options mkdirOptions
for _, opt := range opts {
opt(&options)
}

return mkdirAs(path, mode, uid, gid, true, options.onlyNew)
}

// MkdirAndChown creates a directory and then modifies ownership to the requested uid/gid.
// By default, if the directory already exists, this function still changes ownership and permissions.
// If WithOnlyNew is passed as an option, then only the newly created directory will have ownership
// and permissions changed.
// Note that unlike os.Mkdir(), this function does not return IsExist error
// in case path already exists.
func MkdirAndChown(path string, mode os.FileMode, uid, gid int, opts ...MkdirOpt) error {
var options mkdirOptions
for _, opt := range opts {
opt(&options)
}
return mkdirAs(path, mode, uid, gid, false, options.onlyNew)
}

// getRootUIDGID retrieves the remapped root uid/gid pair from the set of maps.
// If the maps are empty, then the root uid/gid will default to "real" 0/0
func getRootUIDGID(uidMap, gidMap []IDMap) (int, int, error) {
uid, err := toHost(0, uidMap)
if err != nil {
return -1, -1, err
}
gid, err := toHost(0, gidMap)
if err != nil {
return -1, -1, err
}
return uid, gid, nil
}

// toContainer takes an id mapping, and uses it to translate a
// host ID to the remapped ID. If no map is provided, then the translation
// assumes a 1-to-1 mapping and returns the passed in id
func toContainer(hostID int, idMap []IDMap) (int, error) {
if idMap == nil {
return hostID, nil
}
for _, m := range idMap {
if (int64(hostID) >= m.ParentID) && (int64(hostID) <= (m.ParentID + m.Count - 1)) {
contID := int(m.ID + (int64(hostID) - m.ParentID))
return contID, nil
}
}
return -1, fmt.Errorf("host ID %d cannot be mapped to a container ID", hostID)
}

// toHost takes an id mapping and a remapped ID, and translates the
// ID to the mapped host ID. If no map is provided, then the translation
// assumes a 1-to-1 mapping and returns the passed in id #
func toHost(contID int, idMap []IDMap) (int, error) {
if idMap == nil {
return contID, nil
}
for _, m := range idMap {
if (int64(contID) >= m.ID) && (int64(contID) <= (m.ID + m.Count - 1)) {
hostID := int(m.ParentID + (int64(contID) - m.ID))
return hostID, nil
}
}
return -1, fmt.Errorf("container ID %d cannot be mapped to a host ID", contID)
}

// IdentityMapping contains a mappings of UIDs and GIDs.
// The zero value represents an empty mapping.
type IdentityMapping struct {
UIDMaps []IDMap `json:"UIDMaps"`
GIDMaps []IDMap `json:"GIDMaps"`
}

// RootPair returns a uid and gid pair for the root user. The error is ignored
// because a root user always exists, and the defaults are correct when the uid
// and gid maps are empty.
func (i IdentityMapping) RootPair() (int, int) {
uid, gid, _ := getRootUIDGID(i.UIDMaps, i.GIDMaps)
return uid, gid
}

// ToHost returns the host UID and GID for the container uid, gid.
// Remapping is only performed if the ids aren't already the remapped root ids
func (i IdentityMapping) ToHost(uid, gid int) (int, int, error) {
var err error
ruid, rgid := i.RootPair()

if uid != ruid {
ruid, err = toHost(uid, i.UIDMaps)
if err != nil {
return ruid, rgid, err
}
}

if gid != rgid {
rgid, err = toHost(gid, i.GIDMaps)
}
return ruid, rgid, err
}

// ToContainer returns the container UID and GID for the host uid and gid
func (i IdentityMapping) ToContainer(uid, gid int) (int, int, error) {
ruid, err := toContainer(uid, i.UIDMaps)
if err != nil {
return -1, -1, err
}
rgid, err := toContainer(gid, i.GIDMaps)
return ruid, rgid, err
}

// Empty returns true if there are no id mappings
func (i IdentityMapping) Empty() bool {
return len(i.UIDMaps) == 0 && len(i.GIDMaps) == 0
}
143 changes: 143 additions & 0 deletions user/idtools_unix.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
//go:build !windows

package user

import (
"fmt"
"os"
"path/filepath"
"strconv"
"syscall"
)

func mkdirAs(path string, mode os.FileMode, uid, gid int, mkAll, onlyNew bool) error {
path, err := filepath.Abs(path)
if err != nil {
return err
}

stat, err := os.Stat(path)
if err == nil {
if !stat.IsDir() {
return &os.PathError{Op: "mkdir", Path: path, Err: syscall.ENOTDIR}
}
if onlyNew {
return nil
}

// short-circuit -- we were called with an existing directory and chown was requested
return setPermissions(path, mode, uid, gid, stat)
}

// make an array containing the original path asked for, plus (for mkAll == true)
// all path components leading up to the complete path that don't exist before we MkdirAll
// so that we can chown all of them properly at the end. If onlyNew is true, we won't
// chown the full directory path if it exists
var paths []string
if os.IsNotExist(err) {
paths = append(paths, path)
}

if mkAll {
// walk back to "/" looking for directories which do not exist
// and add them to the paths array for chown after creation
dirPath := path
for {
dirPath = filepath.Dir(dirPath)
if dirPath == "/" {
break
}
if _, err = os.Stat(dirPath); os.IsNotExist(err) {
paths = append(paths, dirPath)
}
}
if err = os.MkdirAll(path, mode); err != nil {
return err
}
} else if err = os.Mkdir(path, mode); err != nil {
return err
}
// even if it existed, we will chown the requested path + any subpaths that
// didn't exist when we called MkdirAll
for _, pathComponent := range paths {
if err = setPermissions(pathComponent, mode, uid, gid, nil); err != nil {
return err
}
}
return nil
}

// setPermissions performs a chown/chmod only if the uid/gid don't match what's requested
// Normally a Chown is a no-op if uid/gid match, but in some cases this can still cause an error, e.g. if the
// dir is on an NFS share, so don't call chown unless we absolutely must.
// Likewise for setting permissions.
func setPermissions(p string, mode os.FileMode, uid, gid int, stat os.FileInfo) error {
if stat == nil {
var err error
stat, err = os.Stat(p)
if err != nil {
return err
}
}
if stat.Mode().Perm() != mode.Perm() {
if err := os.Chmod(p, mode.Perm()); err != nil {
return err
}
}
ssi := stat.Sys().(*syscall.Stat_t)
if ssi.Uid == uint32(uid) && ssi.Gid == uint32(gid) {
return nil
}
return os.Chown(p, uid, gid)
}

// LoadIdentityMapping takes a requested username and
// using the data from /etc/sub{uid,gid} ranges, creates the
// proper uid and gid remapping ranges for that user/group pair
func LoadIdentityMapping(name string) (IdentityMapping, error) {
// TODO: Consider adding support for calling out to "getent"
usr, err := LookupUser(name)
if err != nil {
return IdentityMapping{}, fmt.Errorf("could not get user for username %s: %w", name, err)
}

subuidRanges, err := lookupSubRangesFile("/etc/subuid", usr)
if err != nil {
return IdentityMapping{}, err
}
subgidRanges, err := lookupSubRangesFile("/etc/subgid", usr)
if err != nil {
return IdentityMapping{}, err
}

return IdentityMapping{
UIDMaps: subuidRanges,
GIDMaps: subgidRanges,
}, nil
}

func lookupSubRangesFile(path string, usr User) ([]IDMap, error) {
uidstr := strconv.Itoa(usr.Uid)
rangeList, err := ParseSubIDFileFilter(path, func(sid SubID) bool {
return sid.Name == usr.Name || sid.Name == uidstr
})
if err != nil {
return nil, err
}
if len(rangeList) == 0 {
return nil, fmt.Errorf("no subuid ranges found for user %q", usr.Name)
}

idMap := []IDMap{}

var containerID int64
for _, idrange := range rangeList {
idMap = append(idMap, IDMap{
ID: containerID,
ParentID: idrange.SubID,
Count: idrange.Count,
})
containerID = containerID + idrange.Count
}
return idMap, nil
}
Loading