[Feature] Client-side checksum computation

[ben-z]

The feature

Currently, it appears that the SHA1 checksum is calculated and deduplication happens on the server side, after the upload (1, 2). This can take a long time (and takes a lot of bandwidth) if there are multiple devices that share the library (e.g. iCloud Photo Library). Is there a way to implement client-side SHA1 computation, and check for duplicates before the file is uploaded? I feel that this would significantly speed up backing up the same library from multiple devices.

There's some discussion on this in the comments of #1323

Platform

• Server
• Web
• Mobile

[jrasm91]

Sha1 on the web is quite a pain and pretty inefficient and/or complex. There is an option to prehash in the CLI before upload, but the server will still compute it if it ends up being uploaded. And actually I think this only exists in the unreleased CLI in the main repo. In the mobile app, hash to determine if it should be uploaded or not is currently a work in progress.

[ben-z]

Thanks for the info! For the record, I just came across the CLI client side hashing pR: https://github.com/immich-app/CLI/pull/88

[etnoy]

the new v2 cli supports client side hashing

[ben-z]

Awesome! For the record, here are the tickets to track mobile client-side hashing:

• [TECH-DEBT] Mobile upload refactor #2567
• Move to client side hashing #1553