NVME not working

[peacememories]

I've recently bought a bunch of compute blades to create a home cluster. It seems like no matter what I do, the blade does not support NVME SSDs.

Tested boards:

• Compute Blade DEV
• Compute Blade TPM

Tested compute modules:

• CM5 4GB 32GB
• CM4 Lite 8GB

Tested drives:

• Western Digital WD Black SN770
• Western Digital WD Blue SN580

The same compute modules and drives work flawlessly on the Raspberry Pi IO Board.

Excerpt from the kernel log

[    1.865667] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@1000110000 ranges:
[    1.873017] brcm-pcie 1000110000.pcie:   No bus range found for /axi/pcie@1000110000, using [bus 00-ff]
[    1.882458] brcm-pcie 1000110000.pcie:      MEM 0x1b80000000..0x1bffffffff -> 0x0080000000
[    1.890760] brcm-pcie 1000110000.pcie:      MEM 0x1800000000..0x1b7fffffff -> 0x0400000000
[    1.899062] brcm-pcie 1000110000.pcie:   IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[    1.907363] brcm-pcie 1000110000.pcie:   IB MEM 0x1000131000..0x1000131fff -> 0xfffffff000
[    1.916986] brcm-pcie 1000110000.pcie: PCI host bridge to bus 0001:00
[    1.923457] pci_bus 0001:00: root bus resource [bus 00-ff]
[    1.928963] pci_bus 0001:00: root bus resource [mem 0x1b80000000-0x1bffffffff] (bus address [0x80000000-0xffffffff])
[    1.939532] pci_bus 0001:00: root bus resource [mem 0x1800000000-0x1b7fffffff pref] (bus address [0x400000000-0x77fffffff])
[    1.950719] pci 0001:00:00.0: [14e4:2712] type 01 class 0x060400 PCIe Root Port
[    1.958062] pci 0001:00:00.0: PCI bridge to [bus 00]
[    1.963045] pci 0001:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    1.969859] pci 0001:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    1.977648] pci 0001:00:00.0: PME# supported from D0 D3hot
[    1.983737] pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    2.334696] brcm-pcie 1000110000.pcie: link down

The drives are listed as tested by community members on the uptime.industries page. I know that's not the same as certification, but since I do not see any actually certified compatibility lists, I'm assuming that's the best I'll get.

[oculos]

Same problem here. With me, it’s the Patriot P300 (128GB). Works great on the IO dev board (raspberry pi), no go on the CM5.
After like 5 minutes I get a link on the Ethernet, I see the ip address on my switch interface, but can’t really get to it. No ping, no ssh.

[enzian]

Similar situation here: CM5 Lite, with Blade DEV and a WD Blue SN580 2TB running Raspi OS 64bit. All works fine on the CMIO5 Board but does not boot in the Compute Blade.

I notice that the Compute Blade lacks the 32k cristal oscilator that the CMIO5 has which may cause some SSD to not initialise properly.

[oculos]

Similar situation here: CM5 Lite, with Blade DEV and a WD Blue SN580 2TB running Raspi OS 64bit. All works fine on the CMIO5 Board but does not boot in the Compute Blade.

I notice that the Compute Blade lacks the 32k cristal oscilator that the CMIO5 has which may cause some SSD to not initialise properly.

Have you tried to change the boot mode to just boot from NVME? I had one SSD that wasn't working and suddenly it worked by doing this.

[enzian]

It's a CM5 Lite - lacking an SD card it has a fixed boot order. I solved the problem switching to an SSD from another manufacturer all is working now...

Boot order is nowdays in EEPROM.
Which eeprom version are running?
Could you show your boot.conf?

https://www.raspberrypi.com/documentation/computers/compute-module.html#compute-module-eeprom-bootloader

[enzian]

again: CM5 Lites do not have EEPROMs. See my last comment:

It's a CM5 Lite - lacking an SD card it has a fixed boot order. I solved the problem switching to an SSD from another manufacturer all is working now...

I do not have access to a CM5 Lite.
Only a CM4 Lite, still i am able to change the boot order. (rpiboot)

[DracoTomes]

again: CM5 Lites do not have EEPROMs. See my last comment:

that's not true, the CM Lites do contain EEPROMs but not eMMC storage.
You can access the bootloader of the CM Lite by attaching a USB cable and bridging the nRPIBOOT pins on the blade during power up.
In my case, Blade Basic and CM4 Lite, i had to attach a USB-A to USB-C cable to the USB-A port on the board and jump the pins manually, if you have a Dev version, the button for nRPIBOOT should be populated and you should have a USB-C port. I then followed the official documentation to access the bootloader using WSL.

[AMoonRabbit]

Same issue, CM5 with eMMC, OS is on the eMMC, and I am trying to use my SN550 WDS100T2B0C as storage but it does not pick it up at all. lsblk doesn't show it at all.

I note the following in the dmesg output

[    0.000000] Kernel command line: reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe  smsc95xx.macaddr=88:A2:9E:41:DA:C5 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  console=ttyAMA10,115200 multipath=off dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc
[    2.066315] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@110000 ranges:
[    2.073846] brcm-pcie 1000110000.pcie:   No bus range found for /axi/pcie@110000, using [bus 00-ff]
[    2.082983] brcm-pcie 1000110000.pcie:      MEM 0x1b00000000..0x1bfffffffb -> 0x0000000000
[    2.101856] brcm-pcie 1000110000.pcie:      MEM 0x1800000000..0x1affffffff -> 0x0400000000
[    2.117499] brcm-pcie 1000110000.pcie:   IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[    2.136105] brcm-pcie 1000110000.pcie: Forcing gen 2
[    2.161707] brcm-pcie 1000110000.pcie: PCI host bridge to bus 0000:00
[    2.492979] brcm-pcie 1000110000.pcie: link down
[    2.518059] pcieport 0000:00:00.0: PME: Signaling with IRQ 41
[    2.523918] pcieport 0000:00:00.0: AER: enabled with IRQ 41
[    2.541470] brcm-pcie 1000120000.pcie: host bridge /axi/pcie@120000 ranges:
[    2.548465] brcm-pcie 1000120000.pcie:   No bus range found for /axi/pcie@120000, using [bus 00-ff]
[    2.557563] brcm-pcie 1000120000.pcie:      MEM 0x1f00000000..0x1ffffffffb -> 0x0000000000
[    2.565868] brcm-pcie 1000120000.pcie:      MEM 0x1c00000000..0x1effffffff -> 0x0400000000
[    2.574179] brcm-pcie 1000120000.pcie:   IB MEM 0x1f00000000..0x1f003fffff -> 0x0000000000
[    2.582486] brcm-pcie 1000120000.pcie:   IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[    2.591964] brcm-pcie 1000120000.pcie: Forcing gen 2
[    2.597016] brcm-pcie 1000120000.pcie: PCI host bridge to bus 0000:00
[    2.767980] brcm-pcie 1000120000.pcie: clkreq-mode set to default
[    2.774104] brcm-pcie 1000120000.pcie: link up, 5.0 GT/s PCIe x4 (!SSC)
[    2.884199] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[    2.890366] pcieport 0000:00:00.0: PME: Signaling with IRQ 42
[    2.896223] pcieport 0000:00:00.0: AER: enabled with IRQ 42

Edit: I note that my NVMe model is also marked as supported in documentation. https://docs.computeblade.com/blade/compatibility/ssd

Ubuntu 24.04.3 LTS

[DracoTomes]

I'm also having trouble getting CM4 Lites to boot with seemingly compatible SSDs. As my basic blade lacks display output i can't really capture the output during startup but I've tried a "fangxiang" (chinese brand) 2TB drive that didn't work, Crucial P310 500G drives and Kingston NV3 500GB. All drives result in the blue SSD indicator LED blinking a couple of times during power on and then the activity LED blinking three short times which just means "generic failure to boot". Any idea on what I can troubleshoot without ordering more SSDs and waiting for them to arrive?

[urbaniak]

I'm also unable to get ssd (different ssds - samsung and intel) working with compute module, the same ssd works with home assistant yellow, but in compute module I'm getting those logs in the bootroom:

Boot mode: NVME (06) order f25
PCIe timeout: 0x0000008f
Failed to open device: 'nvme'

when installed HA on eMMC - it doesn't recognize the nvme too

[mgcdrd]

Not sure if this will help, but I was able to get Debian to preseed onto my dev blades using CM4 lites, and then boot from them. I ended up using pftf/RPi4 on my SD card to give a UEFI environment. I documented it at: https://github.com/mgcdrd/Rpi-Compute-Blade-EFI-Scripts
I'm currently trying to get Rocky 9 to kickstart on it, using the bland Rocky 9 aarch64 repos.