Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Native NFSv4-style ZFS ACL support for Linux #13186

Closed
wants to merge 3 commits into from

Conversation

ghost
Copy link

@ghost ghost commented Mar 9, 2022

Motivation and Context

ZFS on Linux has so far not supported the native ZFS ACL type because the Linux kernel lacks support for NFSv4-style ACLs. Instead, ZFS on Linux implemented a new POSIX ACL type. The ACL types are not interchangeable, so existing pools with ACLs on illumos or FreeBSD cannot be used on Linux without loss of those ACLs, and conversely a pool created on Linux with POSIX ACLs cannot use those ACLs on other platforms where only the native ZFS NFSv4-style ACLs are implemented.

See also: #9709

Description

We have implemented NFSv4 ACLs for Linux in ZFS. This is mostly functional with a stock Linux kernel, and we have patches for the kernel to fix the few mishandled edge cases that exist.

Details in the commit messages for now, I'm just getting this PR open early for some initial test coverage across the broad range of Linux distros in the OpenZFS CI.

How Has This Been Tested?

We have a suite of tests in TrueNAS that test this functionality. I intend to port what I can of these tests to ZTS for regular testing outside of our internal infrastructure.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@ghost
Copy link
Author

ghost commented Mar 21, 2022

A few builders seem to be missing the rpcgen tool. I'll see about sorting that out in the zfs-buildbot repo.

@hooklab
Copy link

hooklab commented May 30, 2022

Can I use this feature in production? The Linux kernel 5.8.18 ubuntu zfs

@ghost
Copy link
Author

ghost commented May 30, 2022

Can I use this feature in production? The Linux kernel 5.8.18 ubuntu zfs

This feature is available in TrueNAS SCALE. We are still working on getting it upstreamed here, so it is not in any release of OpenZFS yet.

ghost pushed a commit to zfsonfreebsd/zfs-buildbot that referenced this pull request May 30, 2022
The native NFSv4-style ACL support for Linux PR (openzfs/zfs#13186)
uses rpcgen to generate the XDR codec for encoding/decoding ACLs
transferred between kernel and userspace via an xattr.

CentOS and Fedora builders were missing rpcgen.
ghost pushed a commit to zfsonfreebsd/zfs-buildbot that referenced this pull request May 30, 2022
The native NFSv4-style ACL support for Linux PR (openzfs/zfs#13186)
uses rpcgen to generate the XDR codec for encoding/decoding ACLs
transferred between kernel and userspace via an xattr.

CentOS and Fedora builders were missing rpcgen.
ghost pushed a commit to zfsonfreebsd/zfs-buildbot that referenced this pull request May 30, 2022
The native NFSv4-style ACL support for Linux PR (openzfs/zfs#13186)
uses rpcgen to generate the XDR codec for encoding/decoding ACLs
transferred between kernel and userspace via an xattr.

CentOS 8 and Fedora builders were missing rpcgen.
@hooklab
Copy link

hooklab commented May 31, 2022

Can I use this feature in production? The Linux kernel 5.8.18 ubuntu zfs

This feature is available in TrueNAS SCALE. We are still working on getting it upstreamed here, so it is not in any release of OpenZFS yet.

Are these features tested and stable by you? Can I pull branches and merge your features and apply them?

@ghost
Copy link
Author

ghost commented May 31, 2022

Can I use this feature in production? The Linux kernel 5.8.18 ubuntu zfs

This feature is available in TrueNAS SCALE. We are still working on getting it upstreamed here, so it is not in any release of OpenZFS yet.

Are these features tested and stable by you? Can I pull branches and merge your features and apply them?

We have test coverage for the features in our CI and haven't encountered issues. Feel free to apply the commits here and try things out.

@ghost ghost force-pushed the nfsv4acls branch from 9012635 to b4ada52 Compare May 31, 2022 15:01
@ghost
Copy link
Author

ghost commented May 31, 2022

  • Rebased

@ghost ghost force-pushed the nfsv4acls branch from b4ada52 to 448bc8d Compare May 31, 2022 15:41
@ghost
Copy link
Author

ghost commented May 31, 2022

I tried generating the nfs41acl_xdr.[ch] files into @abs_builddir@ instead of @abs_srcdir@ to keep the source tree clean, but that didn't work so I changed it back.

From the build log:

[...]
rpcgen -h "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/../zfs-2.1.99/module/os/linux/zfs/nfs41acl.x" > "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl.h"
sed -i "/\<rpc\/rpc.h\>/c\\#include \<rpc\/xdr.h\>" "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl.h"
sed -i "9i #include <sys/sysmacros.h>" "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl.h"
rpcgen -c "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/../zfs-2.1.99/module/os/linux/zfs/nfs41acl.x" > "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl_xdr.c"
sed -i "5i #pragma GCC diagnostic push" "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl_xdr.c"
sed -i "6i #pragma GCC diagnostic ignored \"-Wunused-variable\"" "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl_xdr.c"
echo "#pragma GCC diagnostic pop" >> "/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl_xdr.c"
[...]
make[8]: *** No rule to make target '/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/os/linux/zfs/nfs41acl_xdr.o', needed by '/tmp/zfs-build-buildbot-wU9fYGxV/BUILD/zfs-kmod-2.1.99/_kmod_build_4.19.0-20-amd64/module/zfs.o'.  Stop.
[...]

There doesn't seem to be a rule in the Kbuild system to build an object from a source file in the obj dir.

@hooklab
Copy link

hooklab commented Jun 1, 2022

Can I use this feature in production? The Linux kernel 5.8.18 ubuntu zfs

This feature is available in TrueNAS SCALE. We are still working on getting it upstreamed here, so it is not in any release of OpenZFS yet.

Are these features tested and stable by you? Can I pull branches and merge your features and apply them?

We have test coverage for the features in our CI and haven't encountered issues. Feel free to apply the commits here and try things out.

Is Samba ACL mapping supported ?

@ghost
Copy link
Author

ghost commented Jun 1, 2022

Is Samba ACL mapping supported ?

We have patches in our Samba repo for this. @anodos325 can comment on that in more detail.

@anodos325
Copy link
Contributor

anodos325 commented Jun 1, 2022

Yeah, part of the reason for the exact xattr specification is that it allows dropping in on Samba without change. You just need to configure the vfs modules correctly.

	vfs objects = streams_xattr nfs4acl_xattr io_uring
	nfs4:chown = True
	nfs4acl_xattr:encoding = xdr
	nfs4acl_xattr:xattr_name = system.nfs4_acl_xdr
	nfs4acl_xattr:validate_mode = False
	nfs4acl_xattr:nfs4_id_numeric = True

As a stretch goal I'm currently writing a common ACL library for FreeBSD / Linux in our samba repo and on Linux side using it to provide libsunacl so that upstream vfs_zfsacl also works on Linux. Will upstream once done and sufficiently tested (libraries). But kind of need to get this into OpenZFS first. :)

@anodos325
Copy link
Contributor

anodos325 commented Jun 1, 2022

You will also want our NFSv4 ACL management tool: https://github.com/truenas/nfs4xdr-acl-tools
The common ACL library I'm writing has python bindings as well https://github.com/truenas/samba/pull/92/files, but is more experimental. nfs4xdr-acl-tools has JSON input and output support, is stable, and should cover most administrator needs. Formatting for acl_to_text is FreeBSD-style.

behlendorf pushed a commit to openzfs/zfs-buildbot that referenced this pull request Jun 1, 2022
The native NFSv4-style ACL support for Linux PR (openzfs/zfs#13186)
uses rpcgen to generate the XDR codec for encoding/decoding ACLs
transferred between kernel and userspace via an xattr.

CentOS 8 and Fedora builders were missing rpcgen.
@hooklab
Copy link

hooklab commented Jun 7, 2022

You will also want our NFSv4 ACL management tool: https://github.com/truenas/nfs4xdr-acl-tools The common ACL library I'm writing has python bindings as well https://github.com/truenas/samba/pull/92/files, but is more experimental. nfs4xdr-acl-tools has JSON input and output support, is stable, and should cover most administrator needs. Formatting for acl_to_text is FreeBSD-style.

root@ying:/tank# zfs get acltype tank/tech
NAME PROPERTY VALUE SOURCE
tank/tech acltype nfsv4 inherited from tank

root@ying:/tank# mount | grep tank
tank on /tank type zfs (rw,xattr,noacl)
tank/tech on /tank/tech type zfs (rw,xattr,noacl)

root@ying:/tank# nfs4_setfacl -a A::ying@localdomain:rxtncy /tank/tech/my10m.log
Operation to request attribute not supported.
Failed to instantiate ACL.

I pulled your ACL branch and compiled it,But I can't use nfs4_setfacl. Can only use your NFS4xDR-ACL-tools?

@anodos325
Copy link
Contributor

You will also want our NFSv4 ACL management tool: https://github.com/truenas/nfs4xdr-acl-tools The common ACL library I'm writing has python bindings as well https://github.com/truenas/samba/pull/92/files, but is more experimental. nfs4xdr-acl-tools has JSON input and output support, is stable, and should cover most administrator needs. Formatting for acl_to_text is FreeBSD-style.

root@ying:/tank# zfs get acltype tank/tech NAME PROPERTY VALUE SOURCE tank/tech acltype nfsv4 inherited from tank

root@ying:/tank# mount | grep tank tank on /tank type zfs (rw,xattr,noacl) tank/tech on /tank/tech type zfs (rw,xattr,noacl)

root@ying:/tank# nfs4_setfacl -a A::ying@localdomain:rxtncy /tank/tech/my10m.log Operation to request attribute not supported. Failed to instantiate ACL.

I pulled your ACL branch and compiled it,But I can't use nfs4_setfacl. Can only use your NFS4xDR-ACL-tools?

You're still using the wrong tool. nfs4xdr_getfacl nfsxdr_setfacl. I've also written python bindings. nfs4_setfacl expects RFC3530 ACLs, not RFC5661 ACLs. Use the correct tool. If you intend to use kernel NFS server with this ACL type you'll need my patch for that. Will link tomorrow.

@anodos325
Copy link
Contributor

To clarify, xattr path and data structure is different. With non-patched kernel, Samba should work fine with parameters provided above. ACLs can be managed with provided tool.

I wrote a common ACL library (for FreeBSD and Linux) that I currently am building in our samba repo. If desired I can move into this PR and have it get built with ZFS package. In this case on Linux it provides a libsunacl interface as well (provides acl() and facl() for vfs_zfsacl.c in samba. (So that same samba config can be shared between FreeBSD and Linux).

The python bindings use python's C API and seem to work pretty well (cross platform in this case as well).
If desired I can expand scope of this PR to include it.

@ghost
Copy link
Author

ghost commented Jun 7, 2022

Including the tools and libraries here sounds good to me. The Python bindings could help writing cross-platform tests for the test suite.

@hooklab
Copy link

hooklab commented Jun 9, 2022

To clarify, xattr path and data structure is different. With non-patched kernel, Samba should work fine with parameters provided above. ACLs can be managed with provided tool.

I wrote a common ACL library (for FreeBSD and Linux) that I currently am building in our samba repo. If desired I can move into this PR and have it get built with ZFS package. In this case on Linux it provides a libsunacl interface as well (provides acl() and facl() for vfs_zfsacl.c in samba. (So that same samba config can be shared between FreeBSD and Linux).

The python bindings use python's C API and seem to work pretty well (cross platform in this case as well). If desired I can expand scope of this PR to include it.

Great project, thanks for your reply and contribution, but our CEO decided to continue using BTRFS, it's a shame, ZFS should be used in the future.

anodos325 and others added 3 commits September 30, 2022 15:28
This implements NFSv41 (RFC 5661) ACLs in a manner
compatible with vfs_nfs4acl_xattr in Samba and
nfs4xdr-acl-tools.

There are three key areas of change in this commit:
1) NFSv4 ACL management through system.nfs4_acl_xdr xattr.
  Install an xattr handler for "system.nfs4_acl_xdr" that
  presents an xattr containing full NFSv41 ACL structures
  generated through rpcgen using specification from the Samba
  project. This xattr is used by userspace programs to read and
  set permissions.

2) add an i_op->permissions endpoint: zpl_permissions(). This
  is used by the VFS in Linux to determine whether to allow /
  deny an operation. Wherever possible, we try to avoid having
  to call zfs_access(). If kernel has NFSv4 patch for VFS, then
  perform more complete check of avaiable access mask.

3) add capability-based overrides to secpolicy_vnode_access2()
  there are various situations in which ACL may need to be
  overridden based on capabilities. This logic is almost directly
  copied from Linux VFS. For instance, root needs to be able to
  always read / write ACLs (otherwise admin can get locked out
  from files).

This is commit was initially inspired by work from Paul B. Henson
to implement NFSv4.0 (RFC3530) ACLs in ZFS on Linux. Key areas of
divergence are as follows:
- ACL specification, xattr format, xattr name
- Addition of handling for NFSv4 masks from Linux VFS
- Addition of ACL overrides based on capabilities

Signed-off-by: Andrew Walker <[email protected]>
The "permission" inode operation takes a new `struct user_namespace *`
parameter starting in Linux 5.12.

Add a configure check and adapt accordingly.

Signed-off-by: Ryan Moeller <[email protected]>
Properly evaluate edge cases where user credential may grant capability
to override DAC in various situations. Switch to using ns-aware checks
rather than capable().

Expand optimization allow bypass of zfs_zaccess() in case of trivial
ACL if MAY_OPEN is included in requested mask. This will be evaluated
in generic_permission() check, which is RCU walk safe. This means that
in most cases evaluating permissions on boot volume with NFSv4 ACLs
will follow the fast path on checking inode permissions.

Additionally, CAP_SYS_ADMIN is granted to nfsd process, and so override
for this capability in access2 policy check is removed in favor of a
simple check for fsid == 0. Checks for CAP_DAC_OVERRIDE and other
override capabilities are kept as-is.

Signed-off-by: Andrew Walker <[email protected]>
@pbhenson pbhenson mentioned this pull request Apr 5, 2023
7 tasks
@jumbi77
Copy link
Contributor

jumbi77 commented Jul 15, 2023

@anodos325 @ixhamza Can i may ask if iX still plan to upstream this? Would be really great.

Not sure but i guess since the last push here, there were some updates on SCALE regarding this feature...

https://github.com/ truenas/zfs/pull/54
https://github.com/ truenas/zfs/pull/78
https://github.com/ truenas/zfs/pull/121
https://github.com/ truenas/zfs/pull/113
https://github.com/ truenas/zfs/pull/52
https://github.com/ truenas/zfs/pull/17
https://github.com/ truenas/zfs/pull/131
https://github.com/ truenas/zfs/commit/c0d493822bdcfe681bb80c6a16cd6855526c66bd

In any case, much thanks.

@alisaosipova
Copy link

@anodos325
Hello, I would like to refine Samba in my repository to support the vfs_zfsacl module so that I can use NFSv4. Which packages do I need to patch, and is this functionality ready at the moment?

@mvastola
Copy link

mvastola commented Jan 6, 2025

Is there any way to get a status update on this feature? Slash can someone point me to what (if any) work remains?

I would be happy to pitch in if time permits and the task is in my wheelhouse.

@amotin
Copy link
Member

amotin commented Jan 21, 2025

This PR is superseded by #16967.

@amotin amotin closed this Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Work in Progress Not yet ready for general review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants