feat (storage): spill large clipboard entries to disk-backed pointers

[madsysharma]

feat(storage): spill large clipboard entries to disk-backed pointers

Closes #28

Summary

Large clipboard payloads (images, binary blobs, rich text) used to be held as fully-resident Vec<u8> byte arrays in the active ring or static slots. Copying a 50 MB image cost 50 MB of RAM the moment it was captured and kept costing the same whether or not the entry was ever pasted.

This PR makes those payloads behave like pointers. When a captured entry's payload is at least a configurable threshold (default 1 MiB), its bytes are written to a blob file under ~/.clipwallet/blobs/<id>.blob and the in-RAM ring/slot holds only a lightweight ClipData::Spilled descriptor (a few dozen bytes). The bytes are read back only at paste time and dropped again the instant the system pasteboard has copied them.

This directly addresses the issue's reproduction case: copying a large image and navigating past it without pasting no longer consumes significant RAM.

Design decisions

Resident state never holds large bytes

A new last-position enum variant carries only metadata:

ClipData::Spilled {
    blob_id: EntryId,     // == owning entry id; names the blob file
    kind:    SpilledKind, // RichText | Image { w, h } | Binary
    size:    usize,       // logical byte size (kept accurate for logs/status)
    hash:    u32,         // CRC32 of the bytes, for dedup without loading
}

PlainText (kept resident so previews work) and FilePath (already a pointer) are never spilled.

Load only at paste, release immediately after

• Copy/Cut: the in-hand bytes are synced to the system pasteboard before spilling, then written to a blob; the resident copy is dropped. The ring stores the descriptor only.
• Paste / plain-paste: spill::hydrate reconstructs the original variant from the blob just long enough to place it on the pasteboard, then it drops.
• Navigation: a spilled entry is not loaded when scrolling past it. Instead, it reaches the pasteboard only on an explicit paste. Small/resident entries still sync on navigation exactly as before, so the common case (text snippets) is unchanged.

Blob lifecycle via reference-counted GC

Rather than threading disk deletes through every mutation (evict, delete, overwrite, encrypt-to-vault), blob lifetime is a single invariant: a blob whose id is not referenced by any live entry is reclaimed. GC runs:

• on every flush (a mutation always marks the store dirty, so an orphan is reclaimed within one flush interval),
• on graceful shutdown
• at startup (which also reclaims blobs orphaned by a hard crash / SIGKILL that bypassed the graceful path).

Durability matches the existing store: blobs are written with the same write -> fsync -> rename -> fsync-parent discipline as storage::disk.

Config

New ~/.clipwallet/config.toml field (added with a serde default, so existing configs keep working untouched):

# Entries this size (bytes) or larger are spilled to disk. 0 disables spilling.
spill_threshold_bytes = 1048576   # 1 MiB default

Non-zero values are clamped to [4 KiB, 256 MiB]; 0 disables spilling and keeps every entry resident (prior behavior).

Behavior notes

• For large entries, the native macOS Cmd+V mirrors the cursor only after an explicit ClipWallet paste, since we intentionally don't materialize large payloads onto the system pasteboard just for navigation. This is the trade-off the issue asks for ("loaded into memory only at the moment of a paste").
• Vault: encrypting a spilled slot hydrates the bytes first so the vault stores real content (not a dangling pointer); decrypting re-spills large entries on the way back into RAM.
• clipwallet status now reports on-disk blob count and total size.

Backward compatibility

• ClipData::Spilled is the last enum variant, so MessagePack (which encodes variants by index) reads every pre-existing store/ file unchanged.
• spill_threshold_bytes uses #[serde(default)], so configs written before this change deserialize with the 1 MiB default and no rewrite is required.
• No public CLI or hotkey changes.

Testing

The new logic - the Spilled type semantics, the spill/hydrate round-trip, blob GC, and config parsing/clamping - is covered by unit tests in types.rs, spill.rs, and config.rs (17 tests total). These pass on the project's macOS target via cargo test (tested on macOS Sonoma)

Suggested steps:

1. cargo run -- run
2. Copy a large image (≥ 1 MiB). Confirm ~/.clipwallet/blobs/ gains a .blob file and resident memory does not grow by the image size.
3. Navigate past it in the ring: memory stays flat.
4. Paste it: content is correct; the transient memory spike is released.
5. Delete/evict it and wait one flush: the blob is GC'd.

Checklist

• Branch off main using the feature/ prefix
• Conventional Commits message
• New code documented with inline comments
• Unit tests added for new logic
• Backward compatible (on-disk store + config)
• README + CHANGELOG updated
• cargo fmt, cargo clippy, cargo test run on macOS (reviewer/CI gate)

[madsysharma]

Hi @shaaravraghu , please review this PR. Thank you.

[madsysharma]

Hi @shaaravraghu , please review this PR. Thank you.

[madsysharma]

Hi @shaaravraghu , please review this PR. Thank you.

[madsysharma]

Hi @shaaravraghu , please review this PR. Thank you.