0.7.0

A lot going on in the march to 1.0! Among other big things, this release adds support for "v1" composefs EROFS image formats, which we need for compatibility with RHEL9 and for compatibility with the original https://github.com/composefs/composefs project.

What's Changed

• ci: Cover feature matrix, use just more consistently by @cgwalters in #254
• Justfile: Simplify and streamline integration test targets by @cgwalters in #250
• ci: Add workflow to mirror fixture container images to ghcr.io by @cgwalters in #258
• dumpfile: Reject . and // in paths instead of normalizing by @cgwalters in #257
• tests: Add EROFS digest stability tests by @cgwalters in #255
• Sync common files from infra repository by @bootc-bot[bot] in #259
• The story begins by introducing our EROFS parser to cargo fuzz by @cgwalters in #256
• Sync common files from infra repository by @bootc-bot[bot] in #262
• ci: Add merge_group trigger to CI workflows by @cgwalters in #264
• mount: Fix pre-6.15 overlayfs compat for kernels < 6.13 by @cgwalters in #265
• Add renovate.json for centralized dependency management by @cgwalters in #261
• Add fsck support by @cgwalters in #260
• Add repr(c) for SplitStream header structs by @Johan-Liebert1 in #266
• Sync common files from infra repository by @bootc-bot[bot] in #269
• dumpfile: Various canonicalization fixes, limit symlink length by @cgwalters in #270
• dumpfile: Use named escapes and only escape '=' in xattr fields by @cgwalters in #271
• erofs: Validate directory '.'/'..' entries and nlink counts by @cgwalters in #272
• fix(deps): update rust by @bootc-bot[bot] in #274
• chore(deps): update github actions by @bootc-bot[bot] in #273
• Use re-exported oci-spec from containers-image-proxy by @jeckersb in #277
• repository: Add meta.json for repo metadata and cfsctl init by @cgwalters in #268
• composefs-oci: Use OCI Digest type for SHA-256 content digests by @cgwalters in #278
• oci: Generate composefs EROFS at pull time, track via config refs by @cgwalters in #263
• test: Add more tests for SELinux labeling by @cgwalters in #279
• Improve error handling for object write failures and repository checks by @cgwalters in #281
• chore(deps): update extractions/setup-just action to v4 by @bootc-bot[bot] in #282
• repo: Optimize local file reads by @cgwalters in #276
• generic_tree: Replace Rc/RefCell with flat index-based FileSystem by @cgwalters in #284
• Import cstor-rs, use for reflinked imports from containers-storage by @cgwalters in #218
• repo: Support reading and upgrading from older composefs-rs repos by @cgwalters in #285
• Sync common files from infra repository by @bootc-bot[bot] in #286
• ci: A few fixes by @cgwalters in #288
• oci: Add fast path for oci: transport using ocidir crate by @cgwalters in #233
• Prepare for publishing a 0.4 to crates.io by @cgwalters in #289
• tar: Preserve trailing record-padding bytes after end-of-archive by @cgwalters in #292
• setup-root: Fix overlay permissions by @cgwalters in #294
• selabel: Break hardlinks when different SELinux labels are assigned by @cgwalters in #296
• Implement buffered readers for UKI by @Johan-Liebert1 in #298
• progress: Move indicatif to CLI, introduce ProgressReporter API by @cgwalters in #295
• ci: Re-enable bootc revdep CI by @cgwalters in #299
• tests: Add debian-bootc digest checks by @cgwalters in #300
• README.md: Sync with composefs-c by @cgwalters in #301
• cfsctl: Add varlink RPC API and unify structured output by @cgwalters in #304
• Initial reimplementation of composefs-c by @cgwalters in #225
• Add support for mounting upper layers with images by @alexlarsson in #303
• Fix: Avoid a hard panic and use a recoverable error instead by @henrywang in #307
• A bit more v1 EROFS work by @cgwalters in #310
• Sync common files from infra repository by @bootc-bot[bot] in #293
• fuse: Add option for allow_other by @alexlarsson in #313
• test_util: expose and extend OCI image builder API by @cgwalters in #312
• Add support for cfsctl oci apply-delta by @alexlarsson in #305
• Extend v1 EROFS to OCI and booting by @cgwalters in #297
• selabel: Use pcre2 for regex by @orowith2os in #318
• chore(deps): update github actions by @bootc-bot[bot] in #319
• Release 0.7.0 by @cgwalters in #314

New Contributors

• @bootc-bot[bot] made their first contribution in #259
• @henrywang made their first contribution in #307
• @orowith2os made their first contribution in #318

Full Changelog: v0.6.0...v0.7.0

v0.6.0

A lot of cool stuff landing here, and even more in the pipeline! In particular we have a completely revamped interface for oci, our tar parsing is now much more strongly hardened, and much improved EROFS handling.

What's Changed

• gc: Use trace logs by @Johan-Liebert1 in #235
• oci: Revamp cfsctl oci by @cgwalters in #237
• cfsctl: Create lib target by @Johan-Liebert1 in #239
• ioctl: Use raw opcodes to match kernel definitions by @Johan-Liebert1 in #240
• cfsctl: Re-export all composefs-rs library crates by @cgwalters in #243
• oci: Add ObjectStoreMethod, ImportStats, and PullResult by @cgwalters in #241
• ci: Add support for Debian+bcvk by @cgwalters in #245
• tar: Switch to async-only parser by @cgwalters in #248
• ci: Consolidate workflows and add required-checks sentinel job by @cgwalters in #251
• oci: Switch to tar-core crate with sans-IO Parser by @cgwalters in #249
• bootc: Update revdep test for current bootc main by @cgwalters in #242
• erofs: Add erofs_to_filesystem to convert EROFS images back to tree::FileSystem by @cgwalters in #252
• cfsctl: Add cmd to get object ref for a file by @Johan-Liebert1 in #236
• two small prep commits for #218 by @cgwalters in #253

Full Changelog: v0.5.0...v0.6.0

v0.5.0

This project has made huge strides since its creation. We didn't create formal release tags as we were working, so here's the first one! There are two big themes going on:

Format stability

The goal of this effort is to finalize the on-disk format and especially precisely how we generate the EROFS so that checksums are stable.

Replacing composefs-c

The original composefs project was written in C and had a different functionality set (includes a CLI too and C shared library, but does not scope in any specific support for OCI or logic that runs in the initramfs).

Changes in last two months

Featured changes

Splitstream format rework (ostree prep)

The splitstream file format was updated to support future ostree use and to improve GC. The header is no longer compressed. All referenced fs-verity objects (external chunks, mapped splitstreams, and optional references not used in chunks) are stored in the header. The mapping table is separate from the reference table. The format now has a magic value and content-type magic for detection, stores the ObjectID format tag and total stream size in the header, and allows referencing file objects in the repo even when they are not part of the splitstream content. GC can enumerate referenced objects without parsing the entire splitstream.

• Preparatory splitstream format changes for ostree support by @alexlarsson in #185

OCI-native manifest and config storage

OCI images are now stored and identified by explicit oci/ tags that point to a manifest, which points to a config. Manifest and config are still written as splitstreams but the original bytes are stored as external objects, aligning with a future proposal for composefs OCI sealing (fsverity signatures for manifest/config as a detached object). Downstream such as bootc can stop storing the manifest on their own (see bootc#1830). This change also adds general support for OCI artifacts and referrers (e.g. sigstore, future composefs signatures) and an initial sketch for multi-arch image storage.

• oci: Add OCI-native manifest/config storage and image management by @cgwalters in #216

Parallel object storage in tar layers

Tar processing now writes large files to the object store in parallel. Files are streamed to O_TMPFILE via a channel while fs-verity digests are computed in background blocking tasks, avoiding blocking the async runtime. New repository helpers include create_object_tmpfile, finalize_object_tmpfile, and a shared write semaphore for backpressure. For a 2GB tar (10,000 files × 200KB), this yields roughly a 7× speedup (~980ms → ~140ms).

• oci: Parallelize object storage within tar layers by @cgwalters in #208

Container root metadata from /usr

OCI layer tars often omit a root directory entry, and container runtimes ignore it. Root metadata (mode, uid, gid, mtime, xattrs) is now defined from /usr so it is deterministic. New APIs: copy_root_metadata_from_usr() and read_container_root() (which calls read_filesystem() then copies metadata). The previous have_root_stat / ensure_root_stat logic (which only handled mtime) was removed. cfsctl commands use read_container_root() by default; --no-propagate-usr-to-root uses raw read_filesystem() for testing.

• fs: Add read_container_root() and copy metadata from /usr -> / by @cgwalters in #209

Garbage collection fix

The previous GC implementation never added any object IDs to the live set and would mark every object for deletion. It assumed an older repository layout where non-first-level entries under streams/ and images/ could link directly to the object store; in the current layout, refs link to first-level entries which then link to objects. The fix walks all named references into the live set and performs a full recursive walk of the stream tree (config splitstream → layer splitstreams → layer contents) so that only unreferenced objects are marked for deletion. The implementation remains dry-run by default; output can be piped to a shell to perform deletions. Relevant for downstream such as bootc (see bootc#1808).

• cfsctl: Fix garbage collection by @chaserhkj in #200

Containers-storage: use uncompressed layer digest

When pulling from containers-storage, the code was requesting the layer by its compressed digest, which led to skopeo errors (“file does not exist”). The pull path now uses get_blob with the uncompressed digest for the layer. Fixes bootc#1703.

• skopeo: Get uncompressed layer when pulling from containers-storage by @Johan-Liebert1 in #210

Important Bugfixes

• cfsctl: Fix garbage collection by @chaserhkj in #200
• skopeo: Get uncompressed layer when pulling from containers-storage by @Johan-Liebert1 in #210

Bugfixes

• tar: Use next_multiple_of for 512-byte alignment by @cgwalters in #201
• Use u64 for file sizes to handle >4GB on 32-bit by @cgwalters in #204
• composefs-oci: Fix wrong OCI opaque whiteout filename by @chaserhkj in #205
• Fix for ensure_config + re-enable bootc revdep CI by @cgwalters in #207

Other changes

• uki: Get raw section from UKI by @Johan-Liebert1 in #198
• Support for handling sha512-addressed repos in cfsctl and composefs-setup-root by @chaserhkj in #199
• ci: Add bootc/ dir for reverse dependency testing by @cgwalters in #214
• Support SHA-512 digests in composefs-setup-root by @gucci-on-fleek in #221
• composefs-oci: use chunked reading in tar processing by @giuseppe in #222
• tests: Add integration test suite with bcvk VM support by @cgwalters in #227
• Add splitfdstream crate for binary streaming format by @cgwalters in #229
• ioctls: Extract unsafe ioctl code into composefs-ioctls crate by @cgwalters in #230
• splitstream: Add more unit tests by @cgwalters in #206
• examples: Pass --hash sha256 to unified examples by @Johan-Liebert1 in #211
• More repository docs by @cgwalters in #217
• skopeo: Use JoinSet for structured concurrency in layer pulls by @cgwalters in #219
• cfsctl: Help text improvement and some other quality of life improvements by @chaserhkj in #220
• project: Add error contexts to functions by @Johan-Liebert1 in #223