zfswin-2.2.6rc1

[lundman]

In terms of development, some changes in this version that could be highlighted.

AddDeviceObject is a single pointer when we are to mount a filesystem, is this going to be a problem if multiple mounts are happening at the same time? Should we have a list and locate the mount it refers to?

I moved all global-variables to the DriverExtension setup, which feels cleaner but is it, it requires the one global still, WIN_DriverObject. So is it better?

Mounting a sub-volume, as in, not on a driveletter, can flash the "Device what to do with ..." as it gets a driveletter initially, then gets mounted on the reparse-point. Dokan disables auto-mount to avoid it (then re-enable right after mount), we can also go back to VIRTUAL_DISK which do not appear to get the popup.

I have occasionally got a "recycle bin is corrupt" message.

DriverUnload appears to no longer work - we do all the unloading work, but the DriverUnload() function does not get called. I have checked DeviceObjects are detached and released, I have checked all HANDLES are closed. I am unsure what might stop it from unloading. For now, people will have to reboot to update OpenZFS versions.

[guenther-alka]

Thanks a lot!

Just updated my Windows 11 23H2 from 2.2.3rc6
Uodate went through without problem

Import pool: ok (small pool 10G) (iSCSI targets)
Create pool: ok (small pool 10G) from .vhdx virtual hd
create filesystem: (encrypted and unencrypted): ok
zfs get all: ok

destroy filesystem: failed, dataset is busy
export pool: failed: pool is busy

[lundman]

Awesome, thanks for the update - that isn't too bad, so I'll check it in the morning :)

[lundman]

Ah ok

$ ./zpool.exe destroy BOOM
cannot destroy 'BOOM': pool is busy
$ ./zfs unmount BOOM/ccm
zunmount(BOOM/ccm,E:/ccm/) running
zunmount(BOOM/ccm,E:/ccm/) returns 0
$ ./zpool.exe destroy BOOM
cannot destroy 'BOOM': pool is busy
$ ./zfs unmount BOOM
zunmount(BOOM,E:/) running
zunmount(BOOM,E:/) returns 0
$ ./zpool.exe destroy BOOM

OK yeah, zpool destroy is missing the logic that things need to be unmounted.

[guenther-alka]

I can't even mount something

zfs mount -a

cannot mount 'tank': Unknown error

[lundman]

Check zpool version

[guenther-alka]

zpool version
zfswin-2.2.6rc1
zfs-kmod-zfswin-2.2.6rc1

[lundman]

OK good, so we are at least on the latest. So it didn't mount at import time?

[lundman]

I am curious if its related to destroy issue, as in, the -a part doesn't work.

[guenther-alka]

zpool import -f tank
.. (ok)

zfs get mounted




NAME      PROPERTY  VALUE    SOURCE
tank      mounted   no       -
tank/enc  mounted   no       -
tank/xx   mounted   no       -
tank/xy   mounted   no       -

The newlines after zfs get are from output

[lundman]

Yeah I have noticed the newlines, not hunted that down yet.

[lundman]

OK so, could you get the 2.2.6rc2 release, and get ready to save cbuf.txt
https://openzfsonosx.org/wiki/Windows_cbuf#Saving_the_debug_print_buffer

Then let's minimize the problem, first import the pool without mounting:

zpool import -N tank

then ask it to just mount the top dataset

zfs mount tank

and when that errors, do the cbuf save, and attach it here.

Hmm i had an unload BSOD, so that is frustrating

[guenther-alka]

Some work for you

what I did

1. update rc1->rc2: ok
   checked zpool version, still rc1

2. retry update (propably rc2->rc2): BSOD
   System Thread Exception not handled, reboot ok

3. Clean install
   rename openzfs.sys, reboot, install rc2: BSOD, reboot ok
   short message like "Driver Cache not less or equal"

Then I tried pool import: ok
pool mount. ok

zfs destroy tank/xx
ok

zpool destroy/export tank: busy

see
https://drive.google.com/file/d/1qHl9yL57EhZcQ2yAxsKXGQLA5T0TN9m1/view?usp=sharing

[lundman]

dont worry about the version number - its not a full release, just added the log thing. but i see now that just adds confusion

[guenther-alka]

The most interesting new feature in 2.2.6rc1 is fast dedup for testings.
I wonder what the default dedup_quota setting "auto" means.

With a dedup vdev the doc is clear, quota is the size of the dedup vdev but without?
Is auto=max/all ram or a certain percentage of RAM what would make more sense
(similar to arc or writecache)

[lundman]

OK, so is anyone having the mounting issues and can save the cbuf? What are the biggest issues?

[guenther-alka]

not working at all

PS C:\Users\me> zfs get mounted






NAME      PROPERTY  VALUE    SOURCE
tank      mounted   no       -
tank/enc  mounted   no       -
tank/xy   mounted   no       -
vhd       mounted   no       -
vhd2      mounted   no       -
vhd2/yy   mounted   no       -
PS C:\Users\me>
PS C:\Users\me> zfs mount vhd2/yy


cannot mount 'vhd2/yy': Unknown error
PS C:\Users\me> zfs mount -a


cannot mount 'tank': Unknown error

cannot mount 'vhd': Unknown error
cannot mount 'vhd2': Unknown error
``
https://drive.google.com/file/d/1iE1_ZLNDG6hVbqF9Qcf-ZVSPPIlF_HD3/view?usp=sharing

[lundman]

Awesome, can you get ready to save cbuf (registry), import -N, mount tank and save cbuf.

[guenther-alka]

rebooted
set cbuf registry to 1

PS C:\Users\me> zpool import -N tank
..
(ok)

PS C:\Users\me> zfs mount tank


PS C:\Users\me> zfs get mounted



NAME      PROPERTY  VALUE    SOURCE
tank      mounted   no       -
tank/enc  mounted   no       -
tank/xy   mounted   no       -
PS C:\Users\me>

https://drive.google.com/file/d/1iE1_ZLNDG6hVbqF9Qcf-ZVSPPIlF_HD3/view?usp=drive_link

[lundman]

Writing "1" saved the cbuf right then, so needs to be done after zfs mount tank, otherwise, perfect.

[guenther-alka]

ok
rebooted
zpool import -N tank
zfs mount tank (not mounted)
set cbuf to 1
uploaded https://drive.google.com/file/d/1iE1_ZLNDG6hVbqF9Qcf-ZVSPPIlF_HD3/view?usp=drive_link

[lundman]

I re-organised the mounting a little, and put in some waits to make sure it is complete, but even if it still fails, it has more prints.
Can you try the OpenZFSOnWindows-debug-.99-500-gcf09d21276-dirty.exe build

[guenther-alka]

There is something critical with this update.
After an update from rc2 i got a bsod (openzfs.sys)

on reboot I run in a loop of bsod with "multiple IRP complete requests",
not even a rename of openzfs.sys helped to boot without openzfs

i need to reinstall Windows from last Aomei backup and will try a clean install then

[lundman]

Nah you can rename openzfs.sys - enter the boot recovery menu.

I am now trying to decide if I should remove the build, in case it is broken

[guenther-alka]

I have renamed openzfs.sys in the boot recovery menu but the bsod (multiple ..) remained

[lundman]

What, how? Windows doesn't have a driver cache afaik, it loads C:/Windows/System32/drivers/openzfs.sys and if it isnt there, it cant load it.

[lundman]

After you find where your C: ends up (D:, E: or whatnot) cd Windows ; dir OpenZFS.sys /s in case you have more than one?

[lundman]

OK installed .exe on VM, created pool, rebooted, imported pool - everything appears normal.

[guenther-alka]

I tried on ESXi 8 with Win11-64
-rename old openzfs.sys, reboot
-install rc2: setup ok, bsod after manual reboot (system thread exception not handled)
reboot ok
terminal: zpool version: bsod (system service exception)

I created a memory dump,
https://drive.google.com/file/d/1triPtbp5qNLdBUCtF96BSIgOUJgmdJXy/view?usp=sharing

[lundman]

OK thanks!

[lundman]

	if ((Status == STATUS_INVALID_DEVICE_REQUEST) && Irp &&
	    zmo != NULL &&
	    zmo->AttachedDevice != NULL &&
	    DriverExtension->ioctlDeviceObject) {
		dprintf("Passing request %s down\n",
		    major2str(IrpSp->MajorFunction, IrpSp->MinorFunction));

		IoSkipCurrentIrpStackLocation(Irp);
		Status = IoCallDriver(zmo->AttachedDevice, Irp);

FFFF8A001143E080: dprintf: zfs_vnops_windows.c:7871:dispatcher(): dispatcher: en
ter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6: fo FFFF8A0FF1FD82D0
FFFF8A001143E080: dprintf: zfs_vnops_windows.c:6625:busDispatcher():   busDispat
cher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL busDeviceObject
FFFF8A001143E080: dprintf: zfs_vnops_windows.c:6737:busDispatcher(): **** unknow
n pnp IRP_MJ_DEVICE_CONTROL: 0x0
FFFF8A001143E080: dprintf: zfs_vnops_windows.c:7975:dispatcher(): Passing reques
t IRP_MJ_DEVICE_CONTROL down bus
-EB-

[lundman]

OK it is curious that it calls AddDevice() 5 times on your machine, so I have changed it to only configure the driver on the first call. I have removed the passing down of AttachedDevice.

Can you check if -501 works better?

[guenther-alka]

Ok, again ESXi8, win64

1. rename openzfs.sys, eeboot

2. install 501 release: bsod loop (System Thread Excepton not handled, driver openzfs.sys)

3. safe mode does not help, tried to rename /windows/system32/drivers/openzfs.sys but the file does not exist

[lundman]

OK so if the file doesn't exist, we are looking in the wrong place. Does your boot disk get mounted on X: during recovery mode? What was weird is that the memory.dmp you sent, it wouldn't match the symbols, as if it was the wrong version.

[lundman]

After install, I have;

$ find /C/Windows/ -name OpenZFS.sys -ls
 -rwxr-xr-x   1 WDKRemoteUser 197121    7102456 Sep 11 12:42 /C/Windows/System32/drivers/OpenZFS.sys
 -rwxr-xr-x   1 WDKRemoteUser 197121    7102456 Sep 11 12:42 /C/Windows/System32/DriverStore/FileRepository/openzfs.inf_amd64_feea5ad729e14052/OpenZFS.sys

[lundman]

Huh chatgpt says:

While Windows doesn't load drivers from DriverStore, it can reinstall drivers from there if the system detects that a driver is missing or corrupt.
During boot, Windows might automatically restore the missing OpenZFS.sys file from DriverStore if it detects that the driver is required for the system's operation.
Windows may also reinstall the driver from the DriverStore if a hardware device associated with that driver is detected, effectively replacing the missing file.

[guenther-alka]

ok, not sure why the command prompt came up with x:
under c: i found the driver, renamed and bootup was ok

upload last memorydump,
https://drive.google.com/file/d/1-PxpjQ3HrdmRpFh9aOjDiW-CD_pSK2P_/view?usp=drive_link

[lundman]

Hurrah! if you can run windbg, you can paste in the stack text instead of uploading the whole file, but carry on if uploading is easier

[lundman]

OK so this is the cause here:

FAULTING_SOURCE_CODE:  
    71: {
    72: 	va_list ap;
    73: 
    74: 	va_start(ap, fmt);
>   75: 	PANIC(fmt, ap);
    76: 	va_end(ap);
    77: }


SYMBOL_NAME:  OpenZFS!spl_panic+4e

kd> k
 # Child-SP          RetAddr               Call Site
00 fffffe88`c6c8ee98 fffff800`1d638195     nt!KeBugCheckEx
01 fffffe88`c6c8eea0 fffff800`1d5d2231     nt!PspSystemThreadStartup$filt$0+0x44
02 fffffe88`c6c8eee0 fffff800`1d62169f     nt!_C_specific_handler+0xa1
03 fffffe88`c6c8ef50 fffff800`1d47fd53     nt!RtlpExecuteHandlerForException+0xf
04 fffffe88`c6c8ef80 fffff800`1d5628ee     nt!RtlDispatchException+0x2f3
05 fffffe88`c6c8f6f0 fffff800`1d62c07c     nt!KiDispatchException+0x1ae
06 fffffe88`c6c8fdd0 fffff800`1d62444e     nt!KiExceptionDispatch+0x13c
07 fffffe88`c6c8ffb0 fffff800`28d7118f     nt!KiBreakpointTrap+0x34e
08 fffffe88`c6c90140 fffff800`2909e82f     OpenZFS!spl_panic+0x4f [C:\src\openzfs\module\os\windows\spl\spl-err.c @ 76] 
09 fffffe88`c6c901a0 fffff800`22163904     OpenZFS!wzvol_HwFreeAdapterResources+0x1ff [C:\src\openzfs\module\os\windows\zfs\zfs_windows_zvol.c @ 1117] 
0a fffffe88`c6c90250 fffff800`221c917e     storport!RaCallMiniportFreeResources+0x40
0b fffffe88`c6c90280 fffff800`2215aa0b     storport!RaDeleteMiniport+0x12
0c fffffe88`c6c902b0 fffff800`221c7062     storport!RaidDeleteAdapter+0xbf
0d fffffe88`c6c902e0 fffff800`2214a7ea     storport!RaidAdapterRemoveDeviceIrp+0x8e
0e fffffe88`c6c90310 fffff800`22127974     storport!RaidAdapterPnpIrp+0x15096
0f fffffe88`c6c903b0 fffff800`290b5ea3     storport!RaDriverPnpIrp+0x94
10 fffffe88`c6c903f0 fffff800`1d512695     OpenZFS!dispatcher+0x4a3 [C:\src\openzfs\module\os\windows\zfs\zfs_vnops_windows.c @ 7919] 
11 fffffe88`c6c90520 fffff800`22eed743     nt!IofCallDriver+0x55
12 fffffe88`c6c90560 fffff800`1d512695     volsnap!VolSnapPnp+0xa3
13 fffffe88`c6c905f0 fffff800`1d89de70     nt!IofCallDriver+0x55
14 fffffe88`c6c90630 fffff800`1da66f40     nt!IopSynchronousCall+0xf8
15 fffffe88`c6c906a0 fffff800`1d5b1f34     nt!IopRemoveDevice+0x108
16 fffffe88`c6c90750 fffff800`1da635f2     nt!PnpRemoveLockedDeviceNode+0x1a8
17 fffffe88`c6c907b0 fffff800`1da63373     nt!PnpDeleteLockedDeviceNode+0x52
18 fffffe88`c6c907f0 fffff800`1da629a3     nt!PnpDeleteLockedDeviceNodes+0xd3
19 fffffe88`c6c90870 fffff800`1d881e3d     nt!PnpProcessQueryRemoveAndEject+0x3b3
1a fffffe88`c6c90950 fffff800`1d88958a     nt!PnpProcessTargetDeviceEvent+0x109
1b fffffe88`c6c90980 fffff800`1d552355     nt!PnpDeviceEventWorker+0x2ca
1c fffffe88`c6c90a00 fffff800`1d554d47     nt!ExpWorkerThread+0x155
1d fffffe88`c6c90bf0 fffff800`1d61b174     nt!PspSystemThreadStartup+0x57
1e fffffe88`c6c90c40 00000000`00000000     nt!KiStartSystemThread+0x34

FFFFA90225920080: dprintf: zfs_windows_zvol.c:1088:wzvol_HwFreeAdapterResources(): MpHwFreeAdapterResources entered, pHBAExt = 0xFFFFA90227541280

So in this case, it is a soft-panic, in that we call PANIC ourselves.
https://github.com/openzfsonwindows/openzfs/blob/zfs-Windows-2.2.6-release/module/os/windows/zfs/zfs_windows_zvol.c#L1115

So we expect the number of MPHBA Objects to be 0 after releasing it.

DrvInfoNbrMPHBAObj : 0x7 [Type: unsigned long]

So uh, Storport is called 8 times on your system?

$ grep "Weird, AddDevice calling StorPort" ../cbuf.txt | wc -l
7

Huh yep. I can fix that, but..

why is your system asking it to remove storport? I should check we can handle that situation or ignore it.

[lundman]

OK there is a -502 build, which will stop it from initialising storport the additional 7 times. Be interesting to see what it does when it is asked to disable.

[guenther-alka]

that helped!

• disable openzfs.sys driver (rename, reboot): ok
• install 502: ok
• manual reboot: ok
• update from 502 to 502:ok
• installer reboots: ok

tried on W10 and W11

[lundman]

Excellent

[guenther-alka]

btw
Just tried a normal update on Win 10 from 2.2.3 to 2.2.6 newest, no problem.
Seems it does not like the boot with driver disabled prior setup.