WIP: Rough draft for updated generic OCI sealing by cgwalters · Pull Request #226 · composefs/composefs-rs

cgwalters · 2026-02-08T22:44:50Z

This is just some rough draft raw material that builds on:

oci: Add OCI-native manifest/config storage and image management #216 (🆕 now requires manifest/config as external not inline splitstream objeects, also the OCI artifact and referrer support was key)
doc/plans: Update OCI sealing spec (kernel sigs, flattened layers) #224 (Lots of updates there, I redid to go all-in on an external signature+digests by default)

cgwalters · 2026-02-12T16:53:07Z

+                    composefs_oci::signing::FsVeritySigningKey::from_pem(&cert_pem, &key_pem)?;
+
+                // Build subject descriptor from the source image's manifest
+                let manifest_json = img.manifest().to_string()?;


Hmm we actually need to operate on the raw original representation, can't rely on to_string() always giving us the same thing.

cgwalters · 2026-02-12T16:53:56Z

+        /// Image reference (tag name)
+        image: String,
+        /// Path to the OCI layout directory (must already exist)
+        oci_layout_path: PathBuf,


I think we can use clap(value_parser) into an ocidir directly or so

cgwalters · 2026-02-12T16:54:28Z

-/// the container to be mounted with integrity protection.
-///
-/// Returns a tuple of (sha256 content hash, fs-verity hash value) for the updated configuration.
-pub fn seal<ObjectID: FsVerityHashValue>(


Might be cleaner if we do a prep commit that removes the old sealing as we know we're not going to do it anymore.

cgwalters · 2026-02-12T16:54:45Z

+/// # Returns
+///
+/// The number of referrer artifacts exported.
+pub fn export_referrers_to_oci_layout<ObjectID: FsVerityHashValue>(


Something like this could land as a prep commit

cgwalters · 2026-02-12T16:54:50Z

+    use std::fs;
+    use std::io::Write;
+
+    let blobs_dir = oci_layout_path.join("blobs").join("sha256");


cgwalters · 2026-02-12T16:56:27Z

+        format!("{seed:02x}").repeat(32)
+    }
+
+    fn sample_subject() -> Descriptor {


Let's unify this stuff with shared infra to generate an ocidir with known content

cgwalters · 2026-03-11T12:38:12Z

This one will need to logically depend on #225 because that one has a lot of hardening for the EROFS parser

alexlarsson · 2026-06-08T07:56:35Z

I've been doing some experimental work on something similar with the current composefs support in podman at podman-container-tools/podman#28658. That code encodes the expected composefs digests in the manifests and assumes it will be reproducibly generated from the tar layers locally. To get trust in that I then have to validate the manifest signature at runtime. I think long term I prefer the approach proposed here of distributing the erofs images directly, particularly because it allows kernel side signatures that can integrate with IMA, etc.

However... I wonder if not just signing the erofs image is missing something. Don't we actually want to sign the entire container specification, not just the file content? What about other parts of the image, like exposed ports, volumes, cmdline, etc? With the signature on the erofs blob we leave all the stuff in the manifest and config "unprotected".

cgwalters · 2026-06-08T15:58:03Z

Don't we actually want to sign the entire container specification, not just the file content? What about other parts of the image, like exposed ports, volumes, cmdline, etc? With the signature on the erofs blob we leave all the stuff in the manifest and config "unprotected".

In the current design, the manifest and config are always stored as external objects, and we can then apply fsverity signatures to those as well.

It's a bit buried in the spec, but see https://github.com/composefs/composefs-rs/pull/224/changes#diff-def2a4eef2075f93a81da71d729f633c5f748feff036fd3789252ee37cf37dfdR281

cgwalters · 2026-06-08T16:08:33Z

A simple way to say it is, I think with this proposal we don't actually need to sign the manifest via cosign at all - runtime integrity >= transport integrity.

But of course in practice most use cases would do so, because you still want the transport integrity for deployments that aren't using runtime integrity yet.

The debian-bootc base image currently fails to bootstrap test deps: apt hits a circular dependency (systemd -> mount -> libselinux1 -> systemd-tmpfiles, provided only by systemd itself) that only surfaces when installing onto the empty dpkg database debian-bootc ships with by design. This is an upstream Debian sid packaging issue, confirmed reproducible independent of any composefs-rs changes. CI already worked around this by disabling the debian matrix entry and always passing COMPOSEFS_BASE_IMAGE explicitly per job, so this doesn't change CI behavior. It does mean `just test-integration-vm` with no arguments currently fails for local development; switch the default to stream10, one of the two distros CI still exercises, so the plain invocation works out of the box again. Signed-off-by: Colin Walters <walters@verbum.org>

The pull-remote-archive test spawned `python3 -m http.server` and waited a fixed 500ms before connecting, which raced with the server actually binding the port and intermittently failed with "Connection refused". Poll the port with a short retry loop up to a generous timeout so the test is deterministic. Signed-off-by: Colin Walters <walters@verbum.org>

Error markers like ImageNotFound and OciRefNotFound/OciImageNotFound get .context()-wrapped as they propagate through resolve_oci_image and friends, so a top-level downcast_ref() misclassifies them as InternalError instead of the more specific varlink error variant. Add find_in_chain() to walk the whole error chain and use it in the existing run_image_objects/run_inspect paths; upcoming Seal/Sign/Verify varlink methods will reuse the same helper. Signed-off-by: Colin Walters <walters@verbum.org>

Build cfsctl in release mode with rhel9 and oci-client features, then push the binary to GHCR as an OCI artifact tagged by branch name and short SHA. This lets the sealed demo Containerfile fetch a pre-built binary via oras instead of building from source. Restored from an earlier rebase where this workflow was dropped; bumped actions/checkout to v7 to match the version used by the rest of .github/workflows/. Signed-off-by: Colin Walters <walters@verbum.org>

Add the low-level building blocks for OCI image sealing and signing: - fsverity::Algorithm gains block_size()/digest_size() helpers, needed when validating composefs.* artifact annotations. - fsverity::formatted_digest adds a hex-encoded digest type tagged with its algorithm, used to represent composefs digests in a stable text form. - fsverity::ioctl gains fs_ioc_enable_verity_with_sig() and enable_verity_raw_with_sig() to pass a PKCS#7 signature blob when enabling verity, alongside the existing unsigned path. - composefs-ioctls gains a new keyring feature exposing inject_fsverity_cert(), which loads an X.509 certificate into the kernel's .fs-verity keyring so signed images can be verified. The keyring lookup resolves the .fs-verity keyring by scanning /proc/keys for the keyring-type entry named ".fs-verity" (the same approach keyctl(1)'s %:<name> syntax uses), rather than a special key ID: none of the eight KEY_SPEC_* special IDs in the kernel UAPI refer to the fs-verity keyring, so an earlier attempt using KEY_SPEC_FS_FSVERITY_KEYRING actually resolved to the caller's user keyring instead. That bug would have silently enrolled certificates that the kernel's PKCS#7 verification path never looks at whenever CONFIG_FS_VERITY_BUILTIN_SIGNATURES enforces signatures; it went unnoticed in CI because our default kernel doesn't enable that config and accepts any signature bytes unconditionally. Signed-off-by: Colin Walters <walters@verbum.org>

Define how composefs attaches fs-verity digests (and optional kernel PKCS#7 signatures) to OCI manifests, configs, and merged EROFS trees, covering both a separate signature-artifact mode and an inline mode where digests and base64 PKCS#7 signatures live directly in the image manifest annotations so ordinary OCI tooling carries them along. The spec lives as rustdoc in sealing_spec.rs, next to the code implementing it, replacing the older doc/plans/oci-sealing-spec.md. On top of that, add the actual sealing/signing/verification machinery: - image.rs/oci_image.rs gain per-layer and merged digest/EROFS generation (compute_per_layer_digests, compute_merged_digest, generate_per_layer_images, generate_merged_image), always using EROFS format v1 regardless of the repository's default, since the sealing spec requires a stable, deterministic digest. - signing.rs wraps openssl's PKCS#7 support as FsVeritySigningKey and FsVeritySignatureVerifier. Signatures are generated with Pkcs7Flags::NOCERTS: the kernel's builtin fs-verity verification matches a SignerInfo's issuer+serial against the .fs-verity keyring without needing an embedded cert, and FsVeritySignatureVerifier is always given its trusted cert out-of-band, so embedding the cert chain only bloats every signature from a few hundred bytes to over a kilobyte for no benefit. - signature.rs implements building and parsing signature artifacts (in both separate-artifact and inline modes) and seal_image/sign_image entry points, plus verify_image_report as the single source of truth for verification: it regenerates EROFS digests locally, checks them against the recorded values, and verifies signatures with a supplied certificate. It is fail-closed — any digest mismatch or signature failure errors out, the merged-filesystem digest must be signed when a certificate is given (so an attacker cannot strip it and pass on a single layer alone), and missing metadata is reported distinctly (NoSignatureArtifacts) from a verification failure. - referrers.rs fetches and imports OCI Referrers API artifacts after a registry pull, since skopeo doesn't support the Referrers API itself and any composefs signature artifacts attached that way would otherwise remain unknown to the local repo. fetch_referrer_artifacts (network fetch + import) is split from register_referrers (registration only) so a caller can fetch before the local subject digest is finalized and register once it's known — the ordering future kernel signature enrollment during EROFS generation depends on the referrer bytes being available locally before EROFS images are generated. The CLI/varlink surface for seal/sign/verify (cfsctl and the varlink service) lands in the next commit, built on top of this library API. Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters force-pushed the sealing-impl branch 2 times, most recently from 1ce192a to 063ff54 Compare February 12, 2026 16:49

cgwalters commented Feb 12, 2026

View reviewed changes

cgwalters force-pushed the sealing-impl branch 3 times, most recently from 361eeb7 to 2f93e4a Compare March 6, 2026 12:24

cgwalters force-pushed the sealing-impl branch 2 times, most recently from 6b676dd to d226f55 Compare March 14, 2026 21:30

cgwalters mentioned this pull request Mar 25, 2026

WIP: Changes for booting composefs native systems coreos/fedora-coreos-config#3874

Open

cgwalters force-pushed the sealing-impl branch 2 times, most recently from 83ea13e to 13f1957 Compare April 4, 2026 12:53

cgwalters force-pushed the sealing-impl branch from 13f1957 to 40f2426 Compare April 10, 2026 20:26

cgwalters force-pushed the sealing-impl branch 3 times, most recently from 0f06a47 to e0c049a Compare May 23, 2026 12:41

cgwalters force-pushed the sealing-impl branch 2 times, most recently from 5778382 to 8c3ab2d Compare June 7, 2026 12:10

cgwalters force-pushed the sealing-impl branch 2 times, most recently from cdad059 to 8527474 Compare June 29, 2026 18:20

cgwalters mentioned this pull request Jul 9, 2026

doc/plans: Update OCI sealing spec (kernel sigs, flattened layers) #224

Merged

cgwalters force-pushed the sealing-impl branch 2 times, most recently from 237a192 to 64fc110 Compare July 13, 2026 12:17

cgwalters added 4 commits July 13, 2026 09:47

cgwalters added 2 commits July 13, 2026 09:47

cgwalters force-pushed the sealing-impl branch from 64fc110 to d38c814 Compare July 13, 2026 13:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Rough draft for updated generic OCI sealing#226

WIP: Rough draft for updated generic OCI sealing#226
cgwalters wants to merge 6 commits into
composefs:mainfrom
cgwalters:sealing-impl

cgwalters commented Feb 8, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

cgwalters Feb 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters commented Mar 11, 2026

Uh oh!

alexlarsson commented Jun 8, 2026

Uh oh!

cgwalters commented Jun 8, 2026

Uh oh!

cgwalters commented Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

cgwalters commented Feb 8, 2026

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters commented Mar 11, 2026

Uh oh!

alexlarsson commented Jun 8, 2026

Uh oh!

cgwalters commented Jun 8, 2026

Uh oh!

cgwalters commented Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants