Skip to content

refactor: replace unmaintained bincode with postcard + borsh - #1302

Open
MrCroxx wants to merge 5 commits into
mainfrom
refactor/replace-bincode-with-postcard-and-borsh
Open

refactor: replace unmaintained bincode with postcard + borsh#1302
MrCroxx wants to merge 5 commits into
mainfrom
refactor/replace-bincode-with-postcard-and-borsh

Conversation

@MrCroxx

@MrCroxx MrCroxx commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

Replace the unmaintained bincode crate (RUSTSEC-2025-0141) with two actively maintained serialization backends behind feature gates.

Changes

Dependencies

  • Removed: bincode = "1" (workspace)
  • Added: postcard = "1" (serde-native, community standard replacement)
  • Added: borsh = { version = "1", features = ["derive"] } (NEAR Protocol, high performance)

Feature Gates

Features Active blanket Code impl
(none) Manual impls for numeric types, bool, Vec<u8>, String, Bytes
serde only impl Code for T: Serialize + DeserializeOwned via postcard
borsh only impl Code for T: BorshSerialize + BorshDeserialize via borsh
serde + borsh borsh takes priority (avoids trait coherence conflicts)

Modified Files

  • Cargo.toml — workspace deps
  • foyer-common/Cargo.toml — features and dependencies
  • foyer-common/src/code.rs — three mutually exclusive Code impl paths with #[cfg]
  • foyer-common/src/error.rsbincode_error() -> postcard_error()
  • foyer/Cargo.toml — add borsh pass-through feature
  • examples/Cargo.toml — add borsh pass-through feature

New Benchmark

foyer-common/benches/bench_serde/ rewritten to directly compare postcard, borsh, and manual encoding:

  • postcard.rs — postcard backend (#[cfg(feature = "serde")])
  • borsh.rs — borsh backend (#[cfg(feature = "borsh")])
  • manual.rs — hand-written LE encoding (always available)

Benchmark Results

Test machine: Linux 6.8, 100 samples per benchmark, 3s warmup.
Test data: struct Entry { id: u64, label: String, payload: Vec<u8> }

Encode

Size manual borsh postcard
64 KiB 539 ns 1.04 us (1.9x) 16.8 us (31x)
4 MiB 51.9 us 101 us (1.9x) 1.11 ms (21x)
64 MiB 3.83 ms 19.3 ms (5.0x) 20.6 ms (5.4x)

Decode

Size manual borsh postcard
64 KiB 541 ns 866 ns (1.6x) 27.0 us (50x)
4 MiB 50.7 us 76.9 us (1.5x) 1.74 ms (34x)
64 MiB 12.3 ms 14.6 ms (1.2x) 38.0 ms (3.1x)

Encoded Size (raw = id + label + payload)

Size raw manual borsh postcard
64 KiB 65552 B 65568 B (+16) 65560 B (+8) 65557 B (+5)
4 MiB 4194320 B 4194336 B (+16) 4194328 B (+8) 4194326 B (+6)
64 MiB 67108880 B 67108896 B (+16) 67108888 B (+8) 67108887 B (+7)

Key Findings

  1. borsh stays close to manual — only 1.5-2x slower than raw memcpy for both encode and decode across all sizes
  2. postcard is significantly slower for large payloads: 21x slower encode and 34x slower decode than manual at 4 MiB. The varint encoding overhead dominates for bulk data.
  3. At 64 MiB the gap narrows as memory bandwidth becomes the bottleneck — borsh ~5x, postcard ~5.4x vs manual for encode
  4. All three produce nearly identical encoded sizes — only 5-16 bytes of metadata overhead regardless of payload size
  5. manual encoding is always fastest but requires handwritten encode/decode — the baseline cost is essentially memcpy

Migration Notes

  • Wire format change: postcard and bincode produce different byte representations. Disk caches written with bincode cannot be read with postcard. Users with persistent disk caches must clear or migrate them.
  • Error::bincode_error() has been replaced with Error::postcard_error() (available when the serde feature is active and borsh is not).
  • Users who derived Serialize + DeserializeOwned for Code continue to work with the serde feature, no code changes needed.
  • Users who want maximum performance should consider the borsh feature and derive BorshSerialize + BorshDeserialize.

Generated with Claude Code

@MrCroxx
MrCroxx requested review from Xuanwo and tisonkun as code owners June 16, 2026 15:32
@MrCroxx
MrCroxx force-pushed the refactor/replace-bincode-with-postcard-and-borsh branch 2 times, most recently from f41547b to 31a92de Compare June 16, 2026 15:43
@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 10.00000% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
foyer-common/src/code.rs 12.50% 14 Missing ⚠️
foyer-common/src/error.rs 0.00% 4 Missing ⚠️
Files with missing lines Coverage Δ
foyer-common/src/error.rs 75.26% <0.00%> (+1.16%) ⬆️
foyer-common/src/code.rs 80.82% <12.50%> (-5.31%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

MrCroxx added 4 commits June 17, 2026 00:00
Replace the unmaintained `bincode` (RUSTSEC-2025-0141) dependency with two
actively maintained serialization backends behind feature gates:

- `serde` feature: uses `postcard`, a community-standard serde-native
  binary format. Drop-in replacement for the old bincode-backed blanket
  `Code` impl.

- `borsh` feature (NEW): uses NEAR Protocol `borsh`, a high-performance
  binary serialization with its own trait system. Takes priority over the
  `serde` feature when both are enabled to avoid coherence conflicts.

Feature precedence: `borsh` > `serde` (postcard) > manual impls.

The benchmark has been rewritten to directly compare postcard, borsh,
and manual encoding across three payload sizes (64B, 1KB, 64KB).

Signed-off-by: MrCroxx <mrcroxx.cs@gmail.com>
Signed-off-by: MrCroxx <mrcroxx.cs@gmail.com>
Signed-off-by: MrCroxx <mrcroxx.cs@gmail.com>
Signed-off-by: MrCroxx <mrcroxx.cs@gmail.com>
@MrCroxx
MrCroxx force-pushed the refactor/replace-bincode-with-postcard-and-borsh branch from 57235c7 to f3eca19 Compare June 16, 2026 16:01
Signed-off-by: MrCroxx <mrcroxx.cs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant