[Feature] Pet detection

[alternativesurfer]

The feature

I notice that dogs get picked up with the "people" facial recognition but cats don't.
Any thoughts on how we could add that?

Platform

• Server
• Web
• Mobile

[arhills]

One of my cats got detected as a person. The other 3 did not. None of my dogs got detected. This might not help at all :)

[aviv926]

There are few models trained for such an operation, do you know a model good enough for such a purpose?
The model we currently use for facial recognition does not support animal recognition.

As a temporary alternative: you can describe your pet in the search, I believe that the CLIP model on which the search is based will be able to find your pet with a fairly high accuracy.

[alternativesurfer]

True, CLIP does appear to search properly.

[rodrigopolo]

Here is one:
https://github.com/MarQuisCheshire/Pets-Face-Recognition

[Chuckame]

How to use it in immich? Wouldn't it replace the current face detection so we lose human faces?

[Roachforlife]

Would be interested as well. Is it possible to add 2 rounds of detection, one for humans and one for pets?

[tekjoey]

I just realized Apple Photos does pet detection, so this is a feature I would like for sure. Immich picked up some of my dogs, but not all of them, and not all photos of dogs. I certainly would like to be able to link photos with a pet. My current implementation is manually adding to albums, which works but isn't great.

(Some musings for the devs) If this feature ever does get implemented, I think there should be a per-user setting to display the pet faces on the same page as people, on a separate (pet only) page, or not at all (currently Apple Photos displays all faces together). Perhaps the database could be updated with a new value of "face_type" with options of "person" or "pet" (maybe specify the type of pet? Maybe the type of pet can be configurable by the user?) Also, when a model does get chosen/trained, please remember that cats and dogs are not the only pets that people have! My wife has horses, so that is a must for us, but lots of others exist as well. (again, these are just some ideas. Take all, some, or none as you see fit)

[gilgwath]

Being able to recognise our cats would probably quadruple the spouse approval factor. I think about a third of our photos are of our cats :-)

[mrshmllow]

Unironically this is blocking me from switching from google photos

[datasian]

Same. I do have all of my photos cataloged onto immich finally but Google Photos helps me find specific pets to a degree. Google Photos does recognition and lets me search by their proper name. That's a killer feature if I'm looking for two specific cats in the same photo like the twins doing cute twin things. I do have unique cats that a general recognition would find. One Siamese, one void, one nebelung and one tuxedo. Distinct colorings helps it. Google Photos will mix up my late Siamese and my current one, and 4 different tuxedo cats that we either babysat or rescued but that's something acceptable for a pet recognition feature.

[VACIndustries]

It's worth noting that it may drive user adoption adding this properly. Google Photos does this at a very basic level. If your pet passes and you get another pet (similar breed), you cannot create a new pet and have it recognize it as a new pet. It continues to tag it as the old pet. No way to delete said old pet either.

This is what drove me to at least find Immich (at which point I realized there was zero pet recognition).

[TAT-Hins]

Hi everyone. This is my first time in the discussion, so sorry if I'm not expressing in the right way.

It's worth noting that it may drive user adoption adding this properly. Google Photos does this at a very basic level. If your pet passes and you get another pet (similar breed), you cannot create a new pet and have it recognize it as a new pet. It continues to tag it as the old pet. No way to delete said old pet either.

This is what drove me to at least find Immich (at which point I realized there was zero pet recognition).

I have a cat and two dogs. It's very easy to use smart search by ML clip models to locate the photos of them, so I suppose it should also work with facial recognition, so that I can tag and name them just like people to make it more easily, which seems to be a possible solution for the doubt.

I'm pretty sure that the feature is technically feasible, and I notice that Google Photos has this feature for a long time, but I have no idea how much work it should take to achieve this. It would be awesome if immich can make this happen, and I hope to be a part of the contribution, no matter as a code debugger or just as a user.

[WizenThorne]

I'd love to have this feature as well.

[Chuckame]

Just putting some random links about Dave detection for pets. It seems to be really hard to find on the internet...!

The most advanced source I've found: https://github.com/GuillaumeMougeot/DogFaceNet

ByteDance seems to provide api for cat and dog recognition (at which price?) : https://docs.byteplus.com/en/docs/effects/docs-pet-face-recognition

As generally people have only a few pets, maybe we could more have a clustering of similar dog than pure face recognition 🤔

[Chuckame]

For cats : https://levelup.gitconnected.com/detect-your-cats-face-with-opencv-21893210737e

[nagyv]

Related model definitions(?) are available at https://www.kaggle.com/datasets/wutheringwang/dog-face-recognition/data

[KuramaSyu]

Where in the code is the current face detection done? If it's not too hard, I would consider making a PR. I don't think that it would be so hard to use 2 AI models for scanning images.

[aviv926]

I'm pretty sure it's done here
https://github.com/immich-app/immich/tree/main/machine-learning/app/models/facial_recognition

You might want to consult with @mertalev before starting work on such a PR, to check if it is something that will be accepted for the project.

[mertalev]

Adding it to immich is of course fine, but getting a pet recognition (not detection) model in the first place is more involved. There's some discussion on training a model for this on Discord.

[nagyv]

@mertalev As frigate already ships with models to detect people, cars, cats, dogs out of the box and is MIT licensed, immich might be able to use the same models.

I think they load the models into their docker images around here: https://github.com/blakeblackshear/frigate/blob/dev/docker/main/Dockerfile#L115

[bo0tzz]

AFAIK, Frigate's models are detection models ("Is this a cat"), not recognition models ("Is this cat the same as that other one").

[pilot51]

I just discovered Immich today (thanks Google feed) and I'm interested in switching from Google Photos to a self-hosted solution. I have it running and I really like what I see, but it'll be hard for me to commit to a full switch if cat detection isn't working.

My cats are like 90% of all the faces recognized in Google Photos, which automatically adds the photos to an album for both cats shared with my family.

[aviv926]

I just discovered Immich today (thanks Google feed) and I'm interested in switching from Google Photos to a self-hosted solution. I have it running and I really like what I see, but it'll be hard for me to commit to a full switch if cat detection isn't working.

My cats are like 90% of all the faces recognized in Google Photos, which automatically adds the photos to an album for both cats shared with my family.

You can always use the smart search for now (hint, it really is smart) Check it out in its demo (this is the basic model) Read more here:
https://immich.app/docs/features/smart-search

Also:

https://demo.immich.app

[pilot51]

You can always use the smart search for now (hint, it really is smart) Check it out in its demo (this is the basic model) Read more here: https://immich.app/docs/features/smart-search

That works well for generic searches like "cat". However, what if I only want to find photos of a pet by name?

I would also like to ensure accurate results that show all of the desired photos and no false positives. It's practically impossible to be sure with on-demand results like that.
For example, I just searched for "cat" and it's very accurate at the top of the results but quickly becomes interspersed with other animals and eventually random photos with 0% cat-like objects as you scroll down (it's even worse in the demo with only one cat photo), which tells me there's no certainty cut-off.
That's why I like to use albums that I can curate myself, in this case assisted by ML to add detected faces to the album, which is essential for such high volumes of cat photos.

One thing Google Photos really struggles with is identifying which cat is which (they're both the same color but with distinct features), so I regularly have to correct it by removing the wrong detected cat and selecting the correct one. I don't expect any solution here to be much better in the short term, but if anything comes close to that and lets me easily correct detection errors, I'll be happy. Google hasn't improved at all in the 5 years I've had these cats, so I can see the open source community eventually doing it better.

[Aevaris]

I 100% would love a way to automatically identify objects and pets, so take my upvote. That said...

However, what if I only want to find photos of a pet by name?

I would also like to ensure accurate results that show all of the desired photos and no false positives.

You can't do it by name, but, if you use context searching, it does a better job than Google Photos does in my experience.

For example, I have 2 cats, both ragdolls, one is the Sealpoint color and one is a Bluepoint. They are fairly different in color but Google Photos consistently mixes them up. However, in Immich, if I enter a search of 'White cat with a dark grey face', it will find my Bluepoint almost perfectly (certainly better than Google does).

It's not automatic, but you could from there either:

1. Add all results to an album so then there is no risk / effort of re-running the search
2. Add a tag to your photos, and then there is no risk / effort of re-running the search.

Edit: I forgot that you can't context search at all in Google. As a result, I forgot that I gave up on the pet feature because it identified my Bluepoint as my only pet and mixes up between my 2 cats (but also fails to identify my other cat as a pet about 50% of the time entirely). Since I am at the mercy of GP's automation, I can't really use it to find my pets. In this way, I've found Immich to perform better.

[pilot51]

You can't do it by name, but, if you use context searching, it does a better job than Google Photos does in my experience.

That wouldn't work for me. Both of my cats are long haired cream colored flame points and the only way I know to describe the difference is in relative terms. One has longer hair, bigger eyes, and slightly darker nose and ears than the other.

GP is pretty bad at telling the difference, but at least it reliably detects that they're one of my cats and automatically adds them to the shared album I have for both cats. Occasionally it fails to detect and I need to manually add the photo, though I can't really blame it when their face isn't showing and they're basically camouflaged in their cream colored bed.

1. Add all results to an album so then there is no risk / effort of re-running the search

That wouldn't work either because, as I previously mentioned, there is no certainty cut-off in the results. As I scroll down, it transitions to cats that don't match the description, then other animals, then endless non-animals. As far as I can tell, adding all context-only search results to an album would effectively add my entire library to the album regardless of the search.

2. Add a tag to your photos, and then there is no risk / effort of re-running the search.

How is that any better than manually adding every photo of my cats to the shared album, as is the desired end result?

I forgot that you can't context search at all in Google.

Yes you can. I just tested and photos that matched the context appeared at the top. I compared against Immich with the exact same search terms, like "mouse" or "eagle" (which I knew I had only a handful of photos) and it did far better. For "eagle", GP found 4 of my 5 photos and Immich missed entirely and gave me a bunch of military museum photos at the top. I scrolled quite a ways and didn't see any eagle photos and very few of my many bird photos. GP still has an issue with many results having nothing to do with the search term, but at least it does have a somewhat reasonable certainty cut-off, for example only giving 9 results for "eagle" and 41 (of 16) for "mouse". Though it does appear to use albums for an assist since I do have an album for pests and the results included some other critters in the album. I haven't bothered recreating my albums in Immich yet because it's such a huge chore and I'm procrastinating.

In my experience, GP wins hands down for object detection and automation.

[mertalev]

If you're using the default search model, you can try using a different one with this guide. There are much better models.

[david-askari]

For the record: The Immich team talked about pet recognizion in a recent livestream, starting around 17:24.

TL;DR

• No good enough and free model for this task is available
• No training data available to train a model themselves
• Idea to crowdsource the training data from user uploads

[Aevaris]

Idea to crowdsource the training data from user uploads

I'd help.

[Roachforlife]

I have loads of pictures of about 20 pets in my library and would be happy to contribute to any training

[nagyv]

I just learned about a large cat face database used for cat-ID training: https://arxiv.org/html/2501.02112v1 and https://www.kaggle.com/datasets/tobiastrein/heellostreetcat-individuals

Similarly, there are the MegaDescriptor models (HuggingFace) that were created primarily for ecologists for species-independent animal identification in the wild.

Could DeepSeek-VL2 be used as the basis for pet-identification training?

On a related note: how can I help? (I used to be a Python programmer before moving to product management. I can still write hopefully good enough quality code if needed. I don't have any serious ML training.)

[daltimus11]

I just made the switch to Immich and I am extremely impressed. However, pet detection is something I am missing after leaving Google Photos. A lot of people are passionate about their pets, seeing them as part of the family, I imagine this could be important to many.

Having the ability to manually add "people" could be another solution. #7301

[Roachforlife]

I just made the switch to Immich and I am extremely impressed. However, pet detection is something I am missing after leaving Google Photos. A lot of people are passionate about their pets, seeing them as part of the family, I imagine this could be important to many.

Having the ability to manually add "people" could be another solution. #7301

I agree. If the ability to manually add a person (or animal in this case) and then manually tag the photo to that person, would be great. At least as an alternative until there is a model that does it all automatically

[eleaner]

don’t forget about cats!

…

On 18 Mar 2025, at 16:56, Viktor Nagy 'notifications at github.com' ***@***.***> wrote: Related model definitions(?) are available at https://www.kaggle.com/datasets/wutheringwang/dog-face-recognition/data — Reply to this email directly, view it on GitHub <#7151 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHOAM7LO6BAZVB7UGL4OJT2VBF2HAVCNFSM6AAAAABDMOT7T2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJUGE2DQMQ>. You are receiving this because you are subscribed to this thread.

[OfficiallyCrazy]

And parrots, rats, monkeys, peguins, and tiny little lady bugs

[allmazz]

I occasionally found my pets in People after switching to antelopev2 model

Minimum detection score - 0.6
Maximum recognition distance - 0.5
Minimum recognized faces - 1

Of course with mistakes, but I will try to tweak the settings

[OfficiallyCrazy]

Can you guys please add "Pet face Recognition" to the official roadmap?

[Lesani]

I 13.798th this, I have switched to immich despite it not being implemented, but as apple photos does it so well, I would really like to have this. Apples implementation even separates nicely two very similar (tuxedo) cats completely accurately. I have not seen a single mixed up photo there.

it could be an additional detection step, maybe even where a user can define it as an optional feature to be enabled, so not every user runs it, only those with and interested in detecting their pets

[akshat-khare]

This feature is very much needed. Pet are family and one of the primary usecase of google photos because its so good at categorising pets too. Lets add that!

[akshat-khare]

Does all the target user will have infrastructure lying around to power deepseek. I'm afraid not. It's better to fine tune the already available and used models and maybe see with them SAM from meta can be a good model here imho

…

On Fri, Apr 25, 2025, 13:11 Viktor Nagy ***@***.***> wrote: I just learned about a large cat face database used for cat-ID training: https://arxiv.org/html/2501.02112v1 and https://www.kaggle.com/datasets/tobiastrein/heellostreetcat-individuals Similarly, there are the MegaDescriptor models <https://arxiv.org/html/2311.09118v2> (HuggingFace <https://huggingface.co/BVRA>) that were created primarily for ecologists for species-independent animal identification in the wild. Could DeepSeek-VL2 <https://github.com/deepseek-ai/DeepSeek-VL2> be used as the basis for pet-identification training? On a related note: how can I help? (I used to be a Python programmer before moving to product management. I can still write hopefully good enough quality code if needed. I don't have any serious ML training.) — Reply to this email directly, view it on GitHub <#7151 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGF2JQRXRANCLCGUAOPI3XD23HRLHAVCNFSM6AAAAABDMOT7T2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEOJUGM4DQOI> . You are receiving this because you commented.Message ID: ***@***.***>

[Knuppel1983]

How far are we on this? 75% of my database consists of dog photos. We do all our (hiking) holidays with our dogs, they're like our children. We need this feature, right now it's stopping me from ditching Google Photos.

[mertalev]

We're lucky to be working on a project with so many exciting ideas and possibilities! Unfortunately, we can only work on a tiny fraction of those ideas at any given time.

We look forward to implementing pet recognition when we can, unless a contributor beats us to it 😉 (if this is you, join our Discord!)

[LukeCarrier]

Hold your horses… We all appreciate the work being done and would love to help (at least i would). We’re just not all developers.

I wasn't born a developer, and neither were the maintainers 😉

If you don't care enough to learn, the best thing you can do to support the maintainers is to avoid cluttering issues with +1 comments that add no value to the discussion and actively put off contributors.

[Knuppel1983]

I know how it works, i work in IT. The ones that moan the most get what they want. ;-)

[bo0tzz]

The ones that moan the most get their comments minimized as duplicate by annoyed maintainers ;)

[pilot51]

I've done IT and currently software development. They're very different beasts when it comes to solving problems.

Typically, IT issues can be solved within a few hours and it's fairly easy to keep up and get some downtime.

In software dev, on the other hand, issues typically take days or weeks but have a very wide range between minutes and years. That means for many if not most active public projects, new issues are created at a higher rate than there is capacity to work through them, so they need to be triaged and many are ignored if they aren't a major bug.

Moaning rarely helps prioritize an issue. It's about severity (including votes), how easy it is to resolve, and especially for volunteer projects, how much the dev is personally interested in working on the issue.

Also, just because I'm a developer with some applicable experience and I want this feature doesn't mean I have the time or passion to contribute. I have a hard enough time getting motivated to work on my own FOSS projects when I have a full-time dev job.

However, I do believe anyone who impatiently moans, especially without adding value to the conversation, would be much better off spending their time learning how to do it themselves. Who knows, maybe they'd kick off a fulfilling career kind of like I did.

[PseudoResonance]

For the sake of everyone's notifications, please do not make comments that don't add anything new... Just upvote the discussion.

I want the feature as much as anyone, but I also don't need daily emails of "+1". We know.

[nagyv]

Could AutoGlon's Image-to-Image Semantic Matching be used to train a pet recognising model? https://auto.gluon.ai/stable/tutorials/multimodal/semantic_matching/image2image_matching.html

I'm not an ML researcher. As a result, I only have questions.

Theoretically:

• There were many people here who offered to share their pet images.
• We could collect pet images (let's say 5-20 images / pet)
• Combine every image with every other image to get a big variation of matches and no-matches
• Train and the model based on these combinations

Questions for data collection

• How many (individual) pets and how many images / pet would be needed for a meaningful dataset?
• How clean the images should be? (What if one image shows two dogs? Each dog has multiple images, but some of those are "shared".)