Add APOB messages to host_sp_comms #2006

mkeeter · 2025-02-07T15:45:23Z

Implements the state machine described in RFD 593

See https://github.com/oxidecomputer/stlouis/issues/707, https://github.com/oxidecomputer/rfd/pull/855 , oxidecomputer/amd-host-image-builder#222

jgallagher · 2025-02-07T16:47:24Z

task/host-sp-comms/src/main.rs

+            hf.bonus_sector_erase(offset)
+                .map_err(|err| APOBError::EraseFailed { offset, err })?;
+        } else {
+            // Read back the page and confirm that it's all empty


Could this fail if there's a blip in the IPCC path and the host resends an APOB request? (I'm not sure what the expectations are for the offsets the host is providing.)

This has changed a bunch since February; messages should now all be idempotent.

hawkw

Some mostly kind of annoying nitpicks.

lib/host-sp-messages/src/lib.rs

task/host-sp-comms/src/main.rs

rmustacc · 2025-08-29T20:10:35Z

task/host-sp-comms/src/main.rs

+    fn apob_write(
+        hf: &HostFlash,
+        mut offset: u64,
+        data: &[u8],
+    ) -> Result<(), ApobError> {


So we need something in the broader API to discover what the erase size granularity is or at least make sure that we're sending stuff that is page size aligned. This gets to what @jgallagher gets at below. But if the host sent things that wasn't page aligned then we'd erase the entire page because our API is not doing a read-modify-write.

This has all changed now – we ensure that the to-be-written page is erased when the state machine starts, and writes are idempotent.

labbott

Will probably do another pass later

drv/cosmo-hf/src/apob.rs

labbott · 2025-10-01T14:25:44Z

drv/cosmo-hf/src/apob.rs

+            checksum: 0, // dummy value
+        };
+        out.checksum = out.expected_checksum();
+        assert!(out.is_valid());


assert becomes panic, is that the behavior we want here?

I think so; the only way this should panic is if someone has broken the code in a dramatic way (e.g. editing the implementation of is_valid so that previously valid data is no longer valid).

But this is also copied from HfRawPersistentData, so I didn't think about it too much!

labbott · 2025-10-01T14:29:42Z

drv/cosmo-hf/src/apob.rs

+    /// Either 0 or 1; directly translatable to [`ApobSlot`]
+    pub slot_select: u32,


There are a couple of places where we have unreachable because of using the u32, maaaybe an enum would be cleaner and reduce a few checks? Or is the issue this needs to match exactly what the host is expecting?

This isn't seen at all by the host, but we're reading / writing this object directly to disk, so we need zerocopy-friendly types.

FWIW, zerocopy::TryFromBytes can be derived for enums (though I'm not sure how annoying that would be to actually use here)

I think it would be usable, but the docs seem to imply that you shouldn't it when round-tripping through bytes (?!).

I've opened google/zerocopy#2722 to ask for clarification

drv/cosmo-hf/src/hf.rs

labbott · 2025-10-01T14:37:22Z

drv/cosmo-hf/src/main.rs

 fn fail(err: drv_hf_api::HfError) {
    let mut buffer = [0; hf::idl::INCOMING_SIZE];
-    let mut server = hf::idl::FailServer::new(err);
+    let mut server = hf::FailServer(err);


Hmmm, do we lose the idl generation of this?

Yeah – I switched to finer-grained error types for a few methods, which means that the IDL generator can't make a FailServer (which assumes a single error type).

labbott · 2025-10-01T14:38:48Z

drv/cosmo-hf/src/apob.rs

+    pub(crate) fn write(
+        &mut self,
+        drv: &mut FlashDriver,
+        offset: u64,


Do we need the u64 everywhere? It seems like everything gets converted/checked against u32 anyway

The initial u64 is sent by the host, but I pushed the u32 conversion upstream into host_sp_comms.

citrus-it · 2025-10-02T15:12:55Z

@mkeeter - here are the IPCC messages which are expected to be seen before the host is finished with APOB. Anything else incoming from the host should trigger the lockdown.

humility: ring buffer task_host_sp_comms::__RINGBUF in host_sp_comms:
   TOTAL VARIANT
      99 Request(ApobRead)
      96 Request(ApobData)
       2 Request(KeyLookup)
       1 Request(GetBootStorageUnit)
       1 Request(GetIdentity)
       1 Request(GetStatus)
       1 Request(AckSpStart)
       1 Request(ApobBegin)
       1 Request(ApobCommit)

mkeeter · 2025-10-02T20:37:20Z

@citrus-it Great, thanks! I've pushed this list to Hubris and to RFD 593.

app/gimlet/base.toml

drv/cosmo-hf/src/apob.rs

drv/cosmo-hf/src/hf.rs

drv/hf-api/src/lib.rs

drv/spartan7-loader/cosmo-seq/README.md

task/host-sp-comms/src/main.rs

drv/cosmo-hf/src/apob.rs

hawkw

I like the latest changes, this looks good to me! I'd be happy to approve this now, but I felt like some of the // XXX: should this lock the state machine? comments could maybe use a second opinion from someone wiser than I am (perhaps @labbott?).

drv/cosmo-hf/src/apob.rs

hawkw · 2025-10-09T17:26:53Z

drv/cosmo-hf/src/apob.rs

+/// See rfd.shared.oxide.computer/rfd/593#_production_strength_implementation
+/// for details on the states and transitions.  Note that the diagram in the RFD
+/// includes fine-grained states (e.g. writing), which the actual implementation
+/// never dwells in; these states are not explicit in `ApobState`.


hawkw · 2025-10-09T17:27:44Z

drv/cosmo-hf/src/apob.rs

+            // This is a little tricky: we allow for bytes to either match our
+            // expected write (for idempotency), _or_ to be `0xFF` (because that
+            // means they're erased).  We have to check every byte to confirm
+            // that they all match, but can bail immediately if we find a
+            // non-matching byte that is *also* not erased.
+            let mut needs_write = false;
+            for (a, b) in buf.scratch[..n].iter().zip(buf.page[..n].iter()) {
                if *a != *b {
-                    all_matches = false;
+                    // You may be tempted to insert a `break` here, but that
+                    // would be incorrect: there could be subsequent bytes which
+                    // do not match *and* are not erased, in which case we must
+                    // return `NotErased`.
+                    needs_write = true;


comments here aer lovely, thank you for adding them --- this is all much clearer now.

drv/cosmo-hf/src/apob.rs

mkeeter requested review from citrus-it, hawkw and jgallagher February 7, 2025 15:45

jgallagher reviewed Feb 7, 2025

View reviewed changes

mkeeter marked this pull request as draft February 7, 2025 17:08

hawkw reviewed Feb 7, 2025

View reviewed changes

lib/host-sp-messages/src/lib.rs Outdated Show resolved Hide resolved

task/host-sp-comms/src/main.rs Outdated Show resolved Hide resolved

task/host-sp-comms/src/main.rs Outdated Show resolved Hide resolved

task/host-sp-comms/src/main.rs Outdated Show resolved Hide resolved

Aaron-Hartwig force-pushed the mkeeter/ipcc-apob branch from 3c34974 to 593980a Compare February 24, 2025 18:19

mkeeter force-pushed the mkeeter/ipcc-apob branch from 26109b2 to 65a65da Compare May 5, 2025 18:55

citrus-it force-pushed the mkeeter/ipcc-apob branch from 65a65da to ebcfe39 Compare July 1, 2025 14:48

citrus-it force-pushed the mkeeter/ipcc-apob branch from ded20cd to 7846e44 Compare August 28, 2025 12:14

rmustacc reviewed Aug 29, 2025

View reviewed changes

mkeeter force-pushed the mkeeter/ipcc-apob branch 3 times, most recently from 14165d3 to b0acdc3 Compare September 26, 2025 15:37

mkeeter force-pushed the mkeeter/ipcc-apob branch from f7e8314 to 4a07f2b Compare October 1, 2025 14:00

mkeeter marked this pull request as ready for review October 1, 2025 14:13

mkeeter force-pushed the mkeeter/ipcc-apob branch from ded1fa1 to ad93477 Compare October 1, 2025 14:27

labbott reviewed Oct 1, 2025

View reviewed changes

mkeeter force-pushed the mkeeter/ipcc-apob branch 2 times, most recently from 49afe3d to 695c7d5 Compare October 2, 2025 14:30

mkeeter force-pushed the mkeeter/ipcc-apob branch from efc7039 to 8d35455 Compare October 2, 2025 20:35

mkeeter force-pushed the mkeeter/ipcc-apob branch from 8d35455 to 58e761d Compare October 2, 2025 20:40

mkeeter added 6 commits October 3, 2025 10:44

Add APOB message to host_sp_comms

350bfd6

s/APOB/Apob

065b31e

Thanks, clippy

698dae6

s/offset/page_offset

6ee0313

Add ApobRead

d1eeebd

Clippy fixes

d4e28d3

mkeeter and others added 17 commits October 3, 2025 10:44

Reinitialize the APOB when muxing to the host

2fe4f2c

Reset the APOB offset under all circumstances

0506062

Fix apob_read call

4cdca32

update grapefruit with the ApobFlashOffset register

d1810bd

Get stuff working with faux-ipcc

3c66b99

Clippy fix

d191501

Changes to match new HSS messages

ec74ca7

Check APOB shape

4a7914c

Erase data if there's a validation error

8a421aa

Small APOB tweaks

d35e4ce

Fix read offset

cac99c2

Fix build

003c154

Push u64 -> u32 conversion upstream

e638119

More memory fixes

a0994ff

clippyyyyyy

2f91e98

Lock APOB state machine on unapproved messages

31e6828

Only erase written region on validation failure

f883776

mkeeter force-pushed the mkeeter/ipcc-apob branch from 58e761d to f883776 Compare October 3, 2025 15:15

labbott approved these changes Oct 8, 2025

View reviewed changes

hawkw reviewed Oct 9, 2025

View reviewed changes

mkeeter and others added 6 commits October 9, 2025 10:22

Feedback from PR review

8d7c269

Add static buffers

ee29443

Add comment explaining tricky loop; switch polarity

4a12383

More tricky commenting

54061dc

Remove dead comment

fd88494

get real FPGA releases

c86b772

hawkw reviewed Oct 9, 2025

View reviewed changes

drv/cosmo-hf/src/apob.rs Outdated Show resolved Hide resolved

drv/cosmo-hf/src/apob.rs Outdated Show resolved Hide resolved

hawkw reviewed Oct 9, 2025

View reviewed changes

More review feedback

7d5c757

hawkw reviewed Oct 9, 2025

View reviewed changes

drv/cosmo-hf/src/apob.rs Show resolved Hide resolved

drv/cosmo-hf/src/apob.rs Show resolved Hide resolved

		/// Either 0 or 1; directly translatable to [`ApobSlot`]
		pub slot_select: u32,

Add APOB messages to host_sp_comms #2006

Are you sure you want to change the base?

Add APOB messages to host_sp_comms #2006

Uh oh!

Conversation

mkeeter commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hawkw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

labbott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

citrus-it commented Oct 2, 2025

Uh oh!

mkeeter commented Oct 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hawkw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

mkeeter commented Feb 7, 2025 •

edited

Loading