[Feature] On demand thumbnail and transcoding

[grmrgecko]

The feature

Hello,

I'd like to see an option to disable thumbnail and transcode creation and to see support for on demand thumbnail creation. I run immich on a 32 core threadripper with an RTX 3080 GPU, so transcoding in real time isn't difficult for my system.

I have searched around, and the closest discussion similar to this is #1334 however my request is that we save disk space by not transcoding and creating thumbnails, but doing so in real time. Maybe the thumbs and encode-video folder can be periodically flushed of files older than a day, that way we can use those folders to temporarily store the needed thumbnail and transcode in a way that's similar to cache in that it gets cleared over time.

The reason I'm wondering about this is currently both of these folders add up to about 400 GB, which is not insignificant even though I have 25 TB free disk space.

Platform

• Server
• Web
• Mobile

[mmomjian]

Are you suggesting that the small WebP (scroll view) thumbnails would also be generated on demand? Have you considered the I/O requirements to read in that many images in full resolution when scrolling through the timeline? Or is this only for the "mid-size" JPEGs? I guess my point is that there are more things to think about than just raw CPU and GPU power, especially for pictures - video transcoding is a much more doable because videos are viewed at a fixed speed.

[grmrgecko]

That is definitely something that can be considered in a feature like this, maybe allow the option of small thumbnails to be kept around but the larger previews to on demand. I am not too concerned about it with my hardware, but I can see how it may be a concern for some people.

[FoxxMD]

There's also this issue asking for hardware-accelerated decoding #4321

[AngelaDMerkel]

Having the option to transcode video on the fly is probably where this feature would be most useful. The (tiny) still image and video thumbnails are hardly a storage concern, but even 720p video adds up. The problem is that this can only be realistically accomplished with hardware decoding and NVENC, QuickSync, etc are much much less robust and flexible than ffmpeg using x265 for example. I sometimes add video in strange resolutions or uncommon bit depths (12bit 4:4:4) which break hardware encoders. It seems like it would be very very tough to account for all the weirdness without having a whitelist for video which should be converted on the fly and which should be pretranscoded in software.

[ph0b0z]

I just wanted to add: I recently discovered Immich and set it up on my NAS and love it so far. Since one of my main reasons to search for a Synology Photos alternative was the removal of its transcoding ability i'd really like to have the option to use on-the-fly transcoding for videos, maybe even when i use the Immich app on a mobile network.

[dronnikovigor]

Well, now there are a lot of options to specify what to trancode and what not. Also there is an option to enable/disable hardware transcoding. I don't see a problem to add option to use on-fly transcoding or not.

[bo0tzz]

That's a gross misunderstanding, on-the-fly transcoding isn't just a line we add to the settings to have everything magically work, it's a complicated feature that would take a lot of effort to implement.

[dronnikovigor]

Sure it is. I just wanted to point, that feature may be optional as now it's for hardware transcoding (e.g. I keep it disabled as it's stuck at HDR videos) for those, who have problems with specific videos.

[AlmightyFrog]

I see no real point in having thumbnails generated on the fly, it would be instead worth to have it kind of be able to detect moved/duplicate files and not generate a new thumbnail again cluttering up more disk space than necessary.

When it comes to the video I instead see a real use for on the fly transcoding: having all those transcoded video files adding up does not make much sense for a mostly single user system where even my simple i5 with quicksync could enode on the fly...

[AngelaDMerkel]

I agree that this is the scenario where this would make sense, but getting around all the systems which don't have capable hardware transcoding would be a nuisance

[Nodiink]

I've been experimenting with Immich for a while now and I have to say that I'm quite happy with it.
When I found the settings for HW encoding I honestly assumed that it was on demand, also because encoding via GPU produces larger files, although it takes much less time. The doubt came to me at the two-pass parameter, which didn't make sense with my assumption.
In my opinion, on demand encoding should be a must-have, and for those who can only offer the CPU the existing approach is more than valid, but for those who can offer the GPU, on demand transcoding has no cons and several pros, including the ability to change only the container while keeping the original tracks, or convert only one of them, and to convert based on the client and the network speed.

Clearly, implementing this feature, which impacts both frontend and backend, is far from simple: two of the four famous media servers currently available have it as a paid key feature, and for a reason.

While browsing, I noticed that Immich seems to use the ffmpeg library compiled by Jellyfin, and I thought of a possible solution.
I'll be very abstract because at the moment I have no idea what the limitations are and therefore if it is really possible to do it or not, but ... have you considered the possibility of using Jellyfin itself, via a plugin, for transcoding?
In my head I see two options:

• Immich offers to Jellyfin via its plugin the original file, and the plugin returns a transcode that Immich then offers to the client (the video would take a long path, but maybe it is easier to implement?);
• or that jellyfin itself is started, like a third-party app to manage the video file (such as when you are on a cloud app, and a third-party app opens the actual file) so if you are on a browser it opens the jellyfin url, if you use the app it opens the jellyfin app (or asks you to install it);

Again, I have no idea if it is possible and I think the feasibility is mostly linked to the limitations of the jellyfin plugins, I hope someone can be more helpful in this regard.

[bo0tzz]

AFAIK Jellyfin doesn't expose anything that would allow it to be used in this way. A similar project, Kyoo, does and it has crossed my mind to use its transcoding backend (https://github.com/zoriya/Kyoo/tree/master/transcoder) like this before.

[Swell61]

On-the-fly video transcoding would be a killer feature for me too. I've been using the Nextcloud app Memories, which I'm trying to replace with Immich, and it has on-the-fly transcoding. Perhaps their implementation may offer some ideas?

[dionorgua]

On fly transcoding for 'playback' is surely the best thing to have.

But as temporary it would be cool to have something like lazy on-demand transcoding. So that system will generate these video files only when actually needed and cache results just for some time (days/months/years depending on settings). User will need to wait until it's done to start playback.

This will add way to not store tons of transcoded videos for old content that may be not played at all..

[AngelaDMerkel]

This is a great compromise and would also avoid reencoding a newer video if it's being routinely viewed. I'd like to see a cache duration though, so that older videos can be cleared from the cache. Assuming I used Immich for 5 or 10 years, it adds up.