iOS 9 and iOS 10 rendering issue

[blitzcrank]

It seems that on iOS 9 & 10, the pixels from the previous layer are not cleared and the next frame is just rendered on top. The issue is not with the createVideoComposition method. This returns a correct outputImage which is masked perfectly for each frame. It seems the issue is that all the video's frames are drawn on AVPlayerLayer and never cleared. The latest frame that needs to be displayed is simply drawn on top of the rest.

Note: I tested this on a real device (not simulator): iPod touch & iPad mini
Any idea what might be causing this issue or how I could fix this?

[quentinfasquel]

Currently traveling, will try and come back to you next week.

[quentinfasquel]

I am currently trying to get back to iOS 10 on one of my devices. I was definitely not aware of this issue but noticed someone else opened a similar issue back in January.

The first thing that comes to mind would be contentMode = .redraw but I am not sure that exists on this type of view/layer.

[quentinfasquel]

@blitzcrank I was able to reproduce the issue with an old device on iOS 10.x

This is unfortunate but it seems iOS wasn't able to properly apply the AVPlayerLayer's pixelBufferAttributes. If your need is really to have support of transparent video rendering on iOS 9 and iOS 10, you will have to go with another solution, not directly with UIKit.

As another quick solution, I managed to have transparent video over UIKit component using a SpriteKit scene like I implemented in an Xcode Playground here.
The trick is to set allowsTransparency to true on your SKView and a clear backgroundColor on your SKScene.

I will try to push a new branch to show the use with SpriteKit.

Regardless of your issue, I do have an update of the current CIColorKernel for a Metal kernel. It's important to note that Metal is now supported on simulator since macOS Catalina.

[blitzcrank]

@quentinfasquel Thanks for taking the time to investigate this issue.

I have tried your alternative solution with SpriteKit but it seems AVPlayerLooper is only available since iOS 10. Unfortunately I do still need to support iOS 9.

My current solution for iOS 9 & 10 is using Mo Dejong's AVAnimator Library to render transparent videos. The only downside I have noticed is that it requires a lot of diskspace (in tmp folder) and time to produce a mvid container with apng files. Just to give you an example of my findings:
A 416x640 12fps 782KB RGB video + 416x640 12fps 541KB alpha video of roughly 4 seconds creates a 65MB mvid container. Using higher quality videos quickly goest into the 250-300MB range. Other than that it works quite well!

If you have any other ideas, I would be very interested to hear them! thanks again

[quentinfasquel]

What's the timeline on your need? I am glad you can use Mo Dejong's library AVAnimator. I actually met with him this year to collaborate on his new project for transparent videos, which basically is the continuity of *AVAnimator* and provides a dedicated encoding/decoding to avoid color loss. I began to implement it in Swift with a different architecture. With the lack of time I haven't been able to continue the project and I don't remember if looping was fully supported on my implementation. I am giving you access to it and we'll see if you are be able to use it. I'm guessing if someone gets to use it I may be more subject to maintaining it. Le mer. 4 déc. 2019 à 09:52, Jovan <notifications@github.com> a écrit :

…

@quentinfasquel <https://github.com/quentinfasquel> Thanks for taking the time to investigate this issue. I have tried your alternative solution with SpriteKit but it seems AVPlayerLooper is only available since iOS 10. Unfortunately I do still need to support iOS 9. My current solution for iOS 9 & 10 is using Mo Dejong's AVAnimator Library <https://github.com/mdejong/AVAnimator> to render transparent videos. The only downside I have noticed is that it requires a lot of diskspace (in tmp folder) and time to produce a mvid container with apng files. Just to give you an example of my findings: A 416x640 12fps 782KB RGB video + 416x640 12fps 541KB alpha video of roughly 4 seconds creates a 65MB mvid container. Using higher quality videos quickly goest into the 250-300MB range. Other than that it works quite well! If you have any other ideas, I would be very interested to hear them! thanks again — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3?email_source=notifications&email_token=AAEFE6I37X3GEX4DFKWIXO3QW5VUVA5CNFSM4JSUZN62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF4GV2A#issuecomment-561539816>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEFE6KMZNT5BTLXBMTXLOTQW5VUVANCNFSM4JSUZN6Q> .

[quentinfasquel]

Here : https://github.com/quentinfasquel/TransparentVideoPlayer Le mer. 4 déc. 2019 à 10:31, Quentin Fasquel <quentin.fasquel@gmail.com> a écrit :

…

What's the timeline on your need? I am glad you can use Mo Dejong's library AVAnimator. I actually met with him this year to collaborate on his new project for transparent videos, which basically is the continuity of *AVAnimator* and provides a dedicated encoding/decoding to avoid color loss. I began to implement it in Swift with a different architecture. With the lack of time I haven't been able to continue the project and I don't remember if looping was fully supported on my implementation. I am giving you access to it and we'll see if you are be able to use it. I'm guessing if someone gets to use it I may be more subject to maintaining it. Le mer. 4 déc. 2019 à 09:52, Jovan ***@***.***> a écrit : > @quentinfasquel <https://github.com/quentinfasquel> Thanks for taking > the time to investigate this issue. > > I have tried your alternative solution with SpriteKit but it seems > AVPlayerLooper is only available since iOS 10. Unfortunately I do still > need to support iOS 9. > > My current solution for iOS 9 & 10 is using Mo Dejong's AVAnimator > Library <https://github.com/mdejong/AVAnimator> to render transparent > videos. The only downside I have noticed is that it requires a lot of > diskspace (in tmp folder) and time to produce a mvid container with apng > files. Just to give you an example of my findings: > A 416x640 12fps 782KB RGB video + 416x640 12fps 541KB alpha video of > roughly 4 seconds creates a 65MB mvid container. Using higher quality > videos quickly goest into the 250-300MB range. Other than that it works > quite well! > > If you have any other ideas, I would be very interested to hear them! > thanks again > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3?email_source=notifications&email_token=AAEFE6I37X3GEX4DFKWIXO3QW5VUVA5CNFSM4JSUZN62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF4GV2A#issuecomment-561539816>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAEFE6KMZNT5BTLXBMTXLOTQW5VUVANCNFSM4JSUZN6Q> > . >

[blitzcrank]

Thanks for providing the link. The timeline is very short but I will try to look into it.
On a sidenote, do you happen to have an objective-c version of MyTransparentVideoExample that you would be willing to share?

[quentinfasquel]

No I don't. I am aware Objective-C still exists but from now on I mostly use Swift, specifically for small demos, since it provides a more lightweight syntax that's easier to read. It should however be easy to write. But Mo's last project is available and in Objective-C, see here : https://github.com/mdejong/AlphaOverVideo Le mer. 4 déc. 2019 à 13:45, Jovan <notifications@github.com> a écrit :

…

Thanks for providing the link. The timeline is very short but I will try to look into it. On a sidenote, do you happen to have an objective-c version of MyTransparentVideoExample that you would be willing to share? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3?email_source=notifications&email_token=AAEFE6O3MBAYDISNSAAETOLQW6Q7XA5CNFSM4JSUZN62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF44GYA#issuecomment-561628000>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEFE6MVWHLVBJK2KFK3XC3QW6Q7XANCNFSM4JSUZN6Q> .

[blitzcrank]

Thank you @quentinfasquel. You have been a big help!