Getting the ZFS "magic" on GNU Guix System #11453
Replies: 2 comments 4 replies
-
|
It's great to hear about OpenZFS being packaged for another Linux distribution! Let me try and field a few of your questions. You've definitely touched on most (maybe all) of the major integration issues required to get OpenZFS working as intended. What you're planning makes sense and we've tried our best to provide flexible mechanisms to tackles all of these integrations issues. But since each distribution has their own way of doing things you'll need to adapt as appropriate for a GNU Guix System. By all means leverage as much of the provided "magic" integration code as makes sense.
The cache file is optional but histroically preferred. It's primary purpose is to provide a quick way to determine which pools should be imported at boot time and by what paths. You can always run
We provide the
This may not be an issue for you. The sole purpose of this rule is to trigger loading the zfs kmod when a block device containing a pool is detected. If on a GNU Guix System the kmod will always be built in to the kernel then it'll always be available and there's nothing to load.
The only requirement on Linux is that this happens after the kernel modules are loaded. It can be started either before or after pools are imported.
It's recommended, but not because OpenZFS won't be able to import a pool if the paths change. The Hopefully, that helps answer some of your questions! |
Beta Was this translation helpful? Give feedback.
-
Well, Guix would prefer to not have (or at least greatly reduce) mutable files in I notice there's a module parameter that indicates the path to the cache file, does it actually work on Linux? I tried disabling the If for example Guix decides it's better to move the cachefile to Is there a way to refresh the cachefile? Might be something good to do on shutdown...
Yes, I saw that as well. I already added support for this and even created a way to hook it into the existing (not yet very complete) framework Guix has for waiting for devices to come online before mounting filesystems.
I don't intend to do that, licensing concerns mean that Guix can't distribute such a kernel, we can only create instruction for the build system to automate the user's action of compiling, so every user who wants to use ZFS on Guix would be forced to recompile a kernel at each update of the kernel or ZFS. At least as a kernel module only the ZFS would be recompiled. This might not be clear but basically how Guix works is, is that in principle it "should" download sources and compile, for all packages, starting from a tiny self-hosting On the other hand I am putting an
Thanks
Ah, okay, thanks. I guess I'll add that in the manual for Guix. |
Beta Was this translation helpful? Give feedback.
-
Yes, that's right. The cache file will be ignored by default unless You have a couple options for setting the default cache file location. The easiest way you've already discovered and that is to set the default path with the One other thing worth mentioning is that each pool contains a "cachefile" property which can be used to override the default location on a per pool basis. This does include a
The cache file is automatically updated whenever the pool configuration is changed. So you shouldn't need to regenerate it, but it can be done manually by running
Absolutely, this is what's done for several other distributions. The kernel module is distributed as a DKMS package and then binaries are provided for user space from the package manager. A copy of the kmods and utilities is added to the |
Beta Was this translation helpful? Give feedback.
-
|
Thank you!
Guix uses its own homegrown A general policy Guix has is that tools put into What I'm thinking is that, for
If so, it seems to me that Alternately, is there a convenient syscall I can instead use to have ZFS import just |
Beta Was this translation helpful? Give feedback.
-
There may be but this isn't something I've ever really tried. It's easy enough to instruct configure to just build the user components, you can do so like this
That's right.
Yes, it should. The There is an |
Beta Was this translation helpful? Give feedback.
-
|
if it still relevant, i wrote a program that detects all the datasets all mounts all of them, but i wrote in haskell: -- Copyright © 2025 Hashirama Senju
import System.Zfs.Zpool
import System.Zfs.Types
import System.Zfs
import Control.Monad.IO.Class
import System.Process (callCommand)
import Control.Concurrent (threadDelay)
main' :: Zfs ()
main' = do
forZpools_ $ \zpool -> do
nm <- getZpoolName zpool
liftIO $ putStrLn ("Pool: "++nm)
vdevs <- getVdevs zpool
liftIO $ print vdevs
enableDatasets zpool ""
liftIO $ putStrLn "Mounting..."
return ()
main :: IO ()
main = do
callCommand "zpool import -a -N > /dev/null 2>&1" -- detection, a ffi would be more elegant
threadDelay (10 * 1000000)
res <- runZfs main'
case res of
Left e -> print e
Right n -> return ()
it depends only on libZfs , which i did packaged to guix, it's available in that channel: zfs is crucial for me, since i have several TBs of data, and i think haskell is a great choice for dealing with that, because of type-safety, |
Beta Was this translation helpful? Give feedback.
-
I've been doing some work recently to get ZFS into GNU Guix System. Guix System is a GNU/Linux distribution built around the Guix functional package manager. Think "Nix, but using Scheme and only-free-as-in-freedom-software". Guix is willing to package ZFS, on the basis of the fact that users are free to do what they want on hardware they own, so they can use the Guix build system to build ZFS locally and link it to their kernel and run the combined kernel+ZFS, users are just not allowed to redistribute the combined work (and the guix repositories don't contain the binaries, users have to compile it, but it's automated by the Guix package manager). However, support for actually using ZFS is nonexistent (and is what I am working on).
ZFS guides tout many things that ZFS can do automatically:
/dev/zvol/pool/dataset.It turns out that many of the things that ZFS guides take for granted are really magic that is implemented, on Linux, by a variety of programs.
For example, the ZFS kernel module has to be loaded at some point, and that has to be done by the Linux
initprogram. The ZFS release has a bunch of systemd services to do so. Guix does not use systemd, it uses GNU Shepherd. So I have to go understand the load-the-module and other things that the systemd services do and reimplement it in GNU Shepherd.And of course the ZFS automounting behavior turns out to be performed by a systemd service as well.
ZVOLs getting into
/dev/zvol/*are actually implemented by udev. The ZFS release has a bunch of udev rules which implement this. Fortunately Guix uses udev also, but Guix is different enough that I need to add special support for installing the udev rules provided by ZFS into the appropriate place in the Guix system.And then there's ZED, which does a lot of the other magic, like emailing you when a scrub is finished, or replacing a failing device with a hotspare (if you enable the appropriate script).
So I have a bunch of things I need to ask about ZFS and the magic bits that have to be integrated into the distribution in order to make ZFS work the way guides say ZFS is supposed to work.
First, lemme know if the below set of things to make ZFS magic work on Linux is complete. For now I'm targeting
/homeon ZFS, not yet/or/booton ZFS, because just targeting/homeseems to be enough of a challenge. So I'm going to skip overinitrdparts, and start after the bootup has pivoted from theinitrdfilesystem to the "real"/root filesystem.init/etc/modprobe.d/zfs.confis traditional, but Guix developers want to make their own mechanism. Then there's the issue with/on ZFS where you somehow have to pass in the parameters even earlier, possibly before/etc/modprobe.dis accessible, but, sigh. It's why I don't want to think of/on ZFS for Guix yet.zpool import -a -N. However, traditionally it seems to be done byzpool -c /etc/zfs/zpool.cache -a -N. How important is the cache file?? Also do I need to do anything at shutdown?zfs mount -a. On shutdown, dozfs unmount -a./dev/zvol/*, must happen after pool importation. Done byzvol_wait. Also need to have udev rules otherwisezvol_waitwill hang forever, because what the ZFS kernel module actually does is create a/dev/zd*device.zed, but I may need to explicitly pass in locations of PID file, zedlet dir, etc.zfs share -a. Must also happen after samba or NFS services are loaded (?).udevudevrules refer to/sbin/modprobe. Guix does not have a traditional/binor/sbindirectory, instead a tool that usesmodprobeor any other tool should refer to it via an absolute path in/gnu/storewhich contains the specific version that the tool is known to work with. So I have to modify the rules to change this external dependency and point it to the ZFS dependency on thekmodpackage./etc/zfs/zed.d. They are symlinks to zedlets that are built into ZFS. The sysad is supposed to remove or add symlinks, and can add custom zedlets as well./etc, at all. Instead, you have an operating-system file that you process and then it spits out a new configuration that includes stuff in/etcand so on. What I plan to do is that the configuration file can indicate whether a particular ZED trigger should be taken from the ZFS package (cmd/zed/zed.d/*), should be disabled (if it is likescrub_finish-notify.shwhich is enabled by default), or should run custom Scheme code. And for zedlets that come with ZFS, what the various configurations that should go into thezed.rcfile.Does that make sense?
Now an issue I have is the cache file. For now I'm fine with doing an all-devices scan. For one,
/etc/zfsdoes not get created by default and I have to write the code that does that when ZFS gets installed. Do I really need the cache file? The cache file is not created if/etc/zfsdirectory does not exist and currently I don't have this path existing.Also, how important is using
/dev/disk/by-idon Linux nowadays? I tried in a Guix VM with ZFS installed, and created a pool with SLOGs and L2ARCs in various disks, then shut down the VM and restarted it with the devices in jumbled order, and the plain, non-cache usingzpool import -a -Nsemed to be able to import the pool with the jumbled devices correctly. My understanding is that the/dev/disk/by-id/*paths are stored in the cache file since the motherboard could jumble up how devices are sent to the Linux kernel, so it seems to be working with ZFS 2.0.1 at least even without the cache file.Beta Was this translation helpful? Give feedback.
All reactions