nvme_driver: don't flr nvme devices

[mattkur]

The default vfio device behavior is to issue a function level reset when attaching or detaching devices. It does so because the device is in an unknown or untrusted state. However, within the context of a trusted virtualization stack, OpenHCL can reasonably trust the state and behavior of the device. So, optimize performance by removing these function level resets for nvme devices. This follows the same model as already exists for MANA devices.

The nvme_driver already shuts down the device (see NvmeDriver::reset()) and waits for the device to become disabled. A well behaved nvme device will not issue DMA after this point. That same device should tolerate a graceful start without an FLR.

Pending work before this PR is ready to commit:

• Initial poc
• Parameterize via command line (provide a way to disable, and also easily run A/B tests)
• Fix nvme_driver: don't flr nvme devices #1714 (comment)
• Check no regressions on CI
• Test servicing locally with real NVMe devices

[Copilot]

Pull Request Overview

This PR modifies the NVMe device initialization logic to disable Function Level Reset (FLR) during device attach/detach operations to improve system startup and shutdown performance. The change implements a fallback mechanism that attempts device initialization without FLR first, and only enables FLR if the initial attempt fails.

• Refactors NVMe device creation into separate functions with retry logic
• Adds reset method configuration to disable FLR by default
• Implements fallback to FLR if device initialization fails without reset

Comments suppressed due to low confidence (1)

openhcl/underhill_core/src/nvme_manager.rs:244

• [nitpick] The closure name 'update_reset' could be more descriptive. Consider renaming to 'set_device_reset_method' to better reflect its purpose.

    let update_reset = |method: PciDeviceResetMethod| {

[github-actions]

At least one Petri test failed.

[mattkur]

Bummer, this doesn't work yet:

paravisor_log: inner_target="underhill_core::nvme_manager" "Failed to update reset_method" fields={"err": ["failed to create file `/sys/bus/pci/devices/182f:00:00.0/reset_method`: Permission denied (os error 13)"], "method": "NoReset

[jstarks]

This is expected with our NVMe emulator, which does not (currently) support any of Linux's reset methods.

[mattkur]

This is expected with our NVMe emulator, which does not (currently) support any of Linux's reset methods.

Got it. I see confirmation that this works when testing with a physical device. In addition, I have a draft PR to add FLR support (so we can test this in CI).