Accessing PCI devices

[kakaroto]

Hi, I have a question and maybe it's a dumb one but I can't seem to figure it out. When the JTAG is enabled, it's from the BUP process, which is a user space program, at Ring-3. Is there a way to escalate privilege to kernel mode? I can't seem to be able to set a breakpoint in kernel code, modifying the TSS doesn't work and thread.step() doesn't actually seem to work either (does the same as thread.go())
I'm not sure how we're supposed to get the full access to the ME from Ring-3, I'm trying to just read io port 0x80 for example (or any I/O port for that matter) and it all returns 0xFFFFFFF and that's because the TSS for BUP process has IOPB empty.
Any advice?
Thanks!

[h0t]

Hi @kakaroto. You can get it through several ways:

1. setup a breakpoint in kernel space (cs for kernel is 0x8, like "0x8:");
2. manually change cs register;

You can also use the suffix "P" to access physical memory (like "0xf0080004P"). ME doesn't have I/O ports, if I recall correctly.

[kakaroto]

Hi @h0t, thanks for the quick answer.

I guess my issue was the selector, I was setting the breakpoints on the linear address and that doesn't seem to work. Weirdly though even if I break inside the interrupt now (int 0x80) the "flags" register still show ring-3 and yes, it seems you're right, ME has no I/O ports according to datasheet vol.2, I had missed that it seems. I'm still unclear then on how we can access any device, without I/O ports, I can't access PCI devices without using MMIO and I can't figure out what the MMIO addresses for PCI is (or for IOSF-SB or for anything pretty much). in your README.md, you talk about HROMMB at offset 0xe20 of the MISA MMIO, and I can see in datasheet vol3, section 27.13 that HROMMB is at offset 0xe20 of MISA MMIO, but I don't see how you got that MISA MMIO is at 0xf0000000P. Also it doesn't seem to help me figure out how to access the PCI configuration space (or if that's not possible, how to use IOSF-SB to access a PCI device, although vol2 says that CSE has access to all those PCI devices so there must be a way to enumerate them).

FYI: I've been working on reversing the exact details of the exploit used here to write up an analysis/explanation of how it works (since you used syslib_tracer, not sys_shared_mem to achieve it on apollolake, it's technically a different method from what you showed at Blackhat), the only thing missing for me to figure out is what that side_band_mapping in the ROPs does which sets values in segment 0xC7 and why you call it with the values 0x706a8 for DCI and 0x70684 for DfX personality. Any hints would be welcome!

Thanks for the help!

[peterbjornx]

Hi kakaroto,
The base address for MISA MMIO can be glanced from the PCI BARs for it, the ME uses PCIE memory based config space access starting at 0xe000_0000. Device 0:0.0 is the SA. The sideband mapping routine sets up the AddressTranslationTable peripheral to provide a MMIO window into a sideband BAR.

[h0t]

Hi @kakaroto,
In generally ME works with private-PCI devices through MMIO space. Some devices (like SPI) have the
special MMIO space for ME and we didn't see the information about this ranges in documentation. Do you want get access to non-pPCI devices? What kind of pci devices do you need?

[kakaroto]

Thanks for the help.
Two days ago I had dumped all the MMIO ranges from the bup metadata and found 0xe0070000 to be a PCI device and then I used 0xe0000000 as a base address to find the device 0:0.0, 0:1.0, 0:13.2 (SPI) and 0:14.0 (audio). The 13.2 and 14.2 have same device ID as for host, but with 0 instead of 5 in the highest nibble (0x0a96 and 0x0a98 instead of 0x5a96 and 0x5a98).
I've also found that there are other PCI devices at 0xF1000000. Those have IDs like 0xFFF1, 0xFFF2, etc.. so not very useful. I haven't yet looked into them in details to figure out what they are for.
I also found thanks to @peterbjornx's hint about setting up the ATT for the sideband MMIO window that the sideband mapping changes the content of the MMIO at address 0xF6110000 depending on the value in 0xf00a9018. I'm now dumping a bunch of those to see what's in them. I still don't know how the magic number 0x70684 translates to the DFx aggregator at offset 0x8400, especially since the "Inside Intel Management Engine" slides say that the DFx personality register is at offset 0xF5010000 (that MMIO address is not accessible though).

@h0t now that I know how to access the ATT, I'll try and bruteforce my way around it, maybe I'll find something, but with the datasheet saying "The CSE can access IOSF-SB Private CR space from an SB ATT window" and your Visa slides saying "Intel CSME has full access to SB (via its own ATT bridge with a privileged SAI).", I guess I should be able to access the PCI devices with the right DestID sent to the right IOSF CR in the right ATT window... The problem is knowing what values are needed.

I'd like to be able to access PCIe, USB or SATA PCI devices so yes, I want the non-private PCI devices which according to your presentations should be doable from the ME but I can't find information on how to achieve it..
Thanks!

[kakaroto]

By the way, I still don't know how you got the 0x706a8 (for DCI) and 0x70684 for (DfX aggregator) values. I assume those are the Dest ID to the IOSF SB through ATT window, but I don't see where you found them (I did find 0x10070684 in TXE code but not 0x706a8) or how you got that full list of SB ports for LBG in your bonus slides. (I did find the 0x54 IR command for DFX Aggregator from the TDef files, so at least there's that answer).

[peterbjornx]

I'm not entirely sure about the highest nybble of the sideband address, but the breakdown of that bitfield is:

• Bits 7 downto 0 : Sideband port/endpoint ID
• Bits 15 downto 8 : Read opcode
• Bits 23 downto 16 : Write opcode
• Bits 26 downto 24 : BAR number
• Bit 27: Unknown, possibly also part of BAR
• Bit 28: Flag, unknown
• Bit 29: Flag, unknown
• Bit 30: Not used by bup on SPT
• Bit 31: Not used by bup on SPT

Which for the examples yields:

• 0x10070684 : Endpoint 0x84:BAR0, readop=6, writeop=7, flag28 set
• 0x00070684 : Endpoint 0x84:BAR0, readop=6, writeop=7

And to find those numbers, there are tables in bup and busdrv which contain all devices that are available at runtime, which have base addresses, SB endpoints, PCI BDFs and so on.

Source: Own research and https://twitter.com/_markel___/status/1121131631245443074

[kakaroto]

Thank you Peter, that was very helpful. I facepalmed when I realized the sideband port is basically the PCR port id. I just made a loop to dump all of their bars and found quite a few. I'm not sure about those readop/writeop or flag28, it didn't seem to make any difference to me what I set in bits 24 and up.
I found the bridge host PCI device (id 8086:5AF0) at 0x050441, a PCI device with id 8086:5A8E at 0x050488 and id 8086:0008 at 0x050432 (+ the audio PCI device found previously). When I saw the host bridge, I was hoping that the read opcode 4 was for the PCI config space for all the pci devices, but no luck.

I did manage to get the MODPHY (0xa5) and USB Host (0xa2) BARs, just not the PCI config space.
I'm still searching for those, but for now I have enough to attempt my port to skylake!

Unfortunately, I still can't get the main CPU to boot (seems to block in an infinite loop at 0x35644), if I can't figure it out soon, I'll be opening another issue (or re-open #11).

[h0t]

Unfortunately, I still can't get the main CPU to boot (seems to block in an infinite loop at 0x35644), if I can't figure it out soon, I'll be opening another issue (or re-open #11).

Did you try activate HAP-mode? #11 (comment)

[kakaroto]

Did you try activate HAP-mode? #11 (comment)

Yes of course. I've tried the F5 image as is and the Brix boots, I tried with the exploit only, and I get that expected behavior of it shutting down continuously, I enabled HAP bit and I get that infinite loop issue in bup where it doesn't seem to finish that initiatlization (it was at index (ebx) 0xC and byte_97258 has 0x2 and byte_97259 has 0x3). I used JTAG to make it unlock it (changed byte_97258 to 0x3) and BUP did finish but then the kernel went into the hang function. I have the v2 hardware and so far, I haven't been able to debug what's the issue, especially with you saying it works on your side.

[h0t]

Hi @ValZapod. We aren't going to publish your collection of visa.xml files. You can get it from Intel System Studio, installers is available on Intel's website. We don't use USB-C. Unfortunately, I have no idea about DCI throgh USB-C. What is your hardware?

Please create separate issues for different topics!