Detect duplicates from messaging apps in albums

[jagjordi]

If i share a image with my partner through WhatsApp and we both sync all our phone library to immich then those shared pictures are duplicated. When you try to create an album you want that the original (not the compressed Whatsapp one) is the one added to the album.

It would be very nice to have a feature that would remove the duplicates using image hash or a similar mechanism.

Example of usage:

1. Both users add pictures to an album including duplicates from Whatsapp or similar.
2. Inside the album there is a button to "remove low quality duplicates" or similar
3. When that option is pressed all the images are checked with their hash for duplicates and if a duplicate is found the lower quality asset is removed from the album

[bo0tzz]

Are you sharing an account?

[jagjordi]

You mean in immich?

[bo0tzz]

Yes

[jagjordi]

We use different accounts in immich Best regards, Jordi Altayó

…

________________________________ From: bo0tzz ***@***.***> Sent: Saturday, March 25, 2023 10:27:45 AM To: immich-app/immich ***@***.***> Cc: Jordi Altayó ***@***.***>; Author ***@***.***> Subject: Re: [immich-app/immich] Detect duplicates from messaging apps in albums (Discussion #2082) Yes — Reply to this email directly, view it on GitHub<#2082 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADLIJKZ6QDT6UVE2XF6YU53W523BDANCNFSM6AAAAAAWHMN2XM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[feerlessleadr]

I'm facing a similar issue (but not exactly the same, so if needed I can open a new thread). My wife and I share an Immich account, and when she takes a picture and sends me that picture in a text message, both the picture she took and the picture in the text message I receive on my phone are uploaded to immich, causing a duplicate.

I'm assuming this is because the file names are different, hence they are not being detected as duplicate, but I'm not sure how the duplicate process works to be sure.

[feerlessleadr]

Thanks - I think you're right (although I'm on Android, but I believe the same scenario applies). The first picture is the sms version (that I recieved), the second is the original that my wife took and was backed up to immich directly.


@alextran1502

[alextran1502]

@feerlessleadr from the width and height of the images you can see the sms version is compressed. So the hash is different. Is there any folder on Android that store only assets received from sms? You can exclude that album not be uploaded

[feerlessleadr]

Thanks - this is definitely an issue on my end then. I had explicitly added this in to immich to be backed up since I'd like to backup the pictures I receive (at least from people other than my wife at this point).

I'll continue to play with it - thanks!

[x42x64]

I'm running into the same thing with WhatsApp (it's compressed through it as well). Even worse - as the images in WhatsApp have different Metadata, the same image appears even at different points in time. I want to backup/store those photos as well. I'm not sure how to deal with it, maybe someone else has an idea...

[konradr25]

It would be nice to have feature to remove duplicates.

[TheOriginal92]

Second this. Duplicate detection would be very very useful!