[Feature] Do not transcode videos with certain audio codecs

[wlamers]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

Loving Immich thus far! There is however one issue that I cannot fix/solve, and that is that Immich wants to transcode almost all of my video's in my main external library (1.4TB large...). Since I do not have a spare 1.4TB SSD to store these I need another solution. I did some digging and this is what I found:

Getting the asset ID info:

curl -L 'https://IMMICHSERVERURL.COM/api/assets/00628edf-8532-4af5-8266-1b522f0fb699' -H 'Accept: application/json' -H 'x-api-key: APIKEY'

{"id":"00628edf-8532-4af5-8266-1b522f0fb699","deviceAssetId":"MOVIE.MOV","ownerId":"9a51fae9-e388-41bb-98cf-54ed7309688e","owner":{"id":"9a51fae9-e388-41bb-98cf-54ed7309688e","email":"[email protected]","name":"NAME","profileImagePath":"","avatarColor":"red"},"deviceId":"Library Import","libraryId":"02bf9909-d76e-4480-8b58-d446f3b1887f","type":"VIDEO","originalPath":"/mnt/PhotosVideos/MOVIE.MOV","originalFileName":"MOVIE.MOV","originalMimeType":"video/quicktime","resized":true,"thumbhash":"mQgGDIJoZnd7d3efh6p/cU32Jg==","fileCreatedAt":"2018-04-23T10:41:50.000Z","fileModifiedAt":"2018-04-23T08:42:00.000Z","localDateTime":"2018-04-23T10:41:50.000Z","updatedAt":"2024-07-08T09:17:56.458Z","isFavorite":false,"isArchived":false,"isTrashed":false,"duration":"00:00:08.880","exifInfo":{"make":null,"model":"Canon EOS 550D","exifImageWidth":1920,"exifImageHeight":1080,"fileSizeInByte":51722217,"orientation":"1","dateTimeOriginal":"2018-04-23T10:41:50.000Z","modifyDate":"2018-04-23T10:41:50.000Z","timeZone":"UTC","lensModel":null,"fNumber":null,"focalLength":null,"iso":null,"exposureTime":null,"latitude":null,"longitude":null,"city":null,"state":null,"country":null,"description":"","projectionType":null},"livePhotoVideoId":null,"tags":[],"people":[],"unassignedFaces":[],"checksum":"LhfaQ/Cq+aXwHeLzs2ODYhaTj+c=","stackCount":null,"isOffline":false,"hasMetadata":true,"duplicateId":null}

Transcode log:

Started encoding video 00628edf-8532-4af5-8266-1b522f0fb699 {"inputOptions":["-hwaccel cuda","-hwaccel_output_format cuda","-noautorotate","-threads 1"],"outputOptions":["-tune hq","-qmin 0","-rc-lookahead 20","-i_qfactor 0.75","-c:v copy","-c:a aac","-movflags faststart","-fps_mode passthrough","-map 0:0","-map 0:1","-g 256","-v verbose","-preset p4","-cq:v 30"],"twoPass":false}

Seems that Immich copies over the video stream (since that is in H264, and therefore fine), but transcodes the audio stream. Please note that I set AAC, MP3 and Opus as 'accepted audio codecs' in the settings (to prevent transcoding of those).

If I then look at the ffprobe output of the original and transcodes file I get:

ffprobe MOVIE.MOV
ffprobe version 4.4.4 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 13.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --enable-gpl --enable-gnutls --disable-stripping --enable-libcdio --enable-version3 --enable-runtime-cpudetect --enable-libmp3lame --enable-libvorbis --enable-libxvid --enable-libx264 --enable-libvpx --enable-libtheora --enable-shared --enable-static --enable-libxcb --enable-libpulse --enable-libfreetype --enable-libmodplug --enable-libspeex --enable-libcelt --enable-libass --enable-libopus --enable-librtmp --enable-libjack --disable-libopencore_amrnb --disable-libopencore_amrwb --disable-libopenjpeg --enable-libbluray --enable-postproc --enable-opencl --enable-libvmaf --enable-libx265 --enable-libv4l2 --enable-libaom --enable-vaapi --enable-vdpau --enable-libbs2b --enable-avresample --enable-libvidstab --enable-libdav1d --disable-libzimg --enable-libwebp --disable-libmysofa --enable-vulkan --enable-libdrm --enable-libsvtav1 --enable-libsrt --enable-librist --enable-nvenc --enable-nvdec
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'MOVIE.MOV':
  Metadata:
    major_brand     : qt
    minor_version   : 537331968
    compatible_brands: qt  CAEP
    creation_time   : 2018-04-23T10:41:50.000000Z
  Duration: 00:00:08.88, start: 0.000000, bitrate: 46596 kb/s
  Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/bt709/bt709), 1920x1080, 45056 kb/s, 25 fps, 25 tbr, 25k tbn, 50k tbc (default)
    Metadata:
      creation_time   : 2018-04-23T10:41:50.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2018-04-23T10:41:50.000000Z
      vendor_id       : [0][0][0][0]

And the transcoded one:

ffprobe 00628edf-8532-4af5-8266-1b522f0fb699.mp4
ffprobe version 4.4.4 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 13.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --enable-gpl --enable-gnutls --disable-stripping --enable-libcdio --enable-version3 --enable-runtime-cpudetect --enable-libmp3lame --enable-libvorbis --enable-libxvid --enable-libx264 --enable-libvpx --enable-libtheora --enable-shared --enable-static --enable-libxcb --enable-libpulse --enable-libfreetype --enable-libmodplug --enable-libspeex --enable-libcelt --enable-libass --enable-libopus --enable-librtmp --enable-libjack --disable-libopencore_amrnb --disable-libopencore_amrwb --disable-libopenjpeg --enable-libbluray --enable-postproc --enable-opencl --enable-libvmaf --enable-libx265 --enable-libv4l2 --enable-libaom --enable-vaapi --enable-vdpau --enable-libbs2b --enable-avresample --enable-libvidstab --enable-libdav1d --disable-libzimg --enable-libwebp --disable-libmysofa --enable-vulkan --enable-libdrm --enable-libsvtav1 --enable-libsrt --enable-librist --enable-nvenc --enable-nvdec
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '00628edf-8532-4af5-8266-1b522f0fb699.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Duration: 00:00:08.88, start: 0.000000, bitrate: 45191 kb/s
  Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/bt709/bt709), 1920x1080, 45056 kb/s, 25 fps, 25 tbr, 25k tbn, 50k tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

To me it seem like Immich wants to transcode this file because of the audio codec. From:

  Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)

to:

  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)

These videos are shot on a Canon EOS 550d. And I have many, many videos from that camera.

Would it be possible to, either:

1. Add an option under:

'Accepted audio codecs' Select which audio codecs do not need to be transcoded. Only used for certain transcode policies.

Such that the videos with the above audio codec is not transcoded?

2. Add an option that prevents transcoding of videos with a mis-matching audio codec all together

3. Add an option (ultimate solution) that real-time transcodes videos. Like Plex is doing. This might increase latency a bit, but with the benefit of not needing to transcode large amount of videos.

Platform

• Server
• Web
• Mobile

[mertalev]

I think someone else mentioned pcm as well. I'm fine with adding that as an accepted audio codec. In the meantime, if you know you don't need transcoding, you can disable it in the transcode policy.

We do plan to add realtime transcoding at some point, but there's a lot that needs to happen for that.

[wlamers]

That would be really great if you could add it!

Yes, I am aware of disabling transcoding altogether. The thing is that I also have several old mpeg's that need to be transcoded, otherwise that will not be played on mobile.

Real-time transcoding would be awesome! I will be watching the roadmap ;)

Thanks for your great work.

[mertalev]

Based on this post, it seems PCM support for MP4 is fairly new. I'm not sure if FFmpeg implemented this yet. I can't promise that we can implement this given that, but I'll give it a test.

[wlamers]

Got it thanks. Great! The original is indeed a mov container. The (current) implementation transcodes to an MP4 container. I'm totally fine with transcoding the audio stream (stored into a MP4 container) in case the video stream requires transcoding, and not transcoding if the audio stream is PCM and the video stream does not require transcoding.

Would you be able to add this to the 110 release you think?

[mertalev]

We don't really do ETAs, but I can take a look at it this weekend.

[wlamers]

Hi @mertalev, just out of curiosity, can you say something about the status of the above? Do you have it on the roadmap to add this? Currently many video's cannot be played because of this. I expect more people have this issue, since those video's are recorded with a Canon EOS camera. Looking forward hearing back from you!

[mertalev]

#13418 just got merged so it'll be in the next release

[wlamers]

Great, tnx!