I have a research project where I need to apply very, very computationally heavy image filters on a h264 video. Due some restrictions outside my control I have to use proprietary filter that works only on images and not video. Filter has two modes, A mode is very computationally heavy but produces production ready quality and B mode is fast but produces less than stellar quality.
My theory is that given how h264 uses I-frames to get the reference and rest of the frames are "non-essential", if I can find apply the high quality A mode on I-frames and B mode on other frames, I should be able to leverage h264's own compression to get the result without having to use filter on A mode to every frame. Another avenue would be to try with h265 because it's more robust frame prediction capabilities.
What I've figured out so far is that I need to find a way to extract all frames from a video while knowing which ones are I-frames, then do a filter on all the frames, and then re-encode the video back together with minimum quality loss.
So first question is that how can I extract frames with ffmpeg so that it somehow identifies each frame type on the filename?
ffmpeg -i C:\test.mp4 -vf select='eq(pict_type\,I)',setpts='N/(25*TB)' C:\testTemp\%09d.jpg
Above gives you i-frames but I haven't figured out how I can get this to export all the frames and to put the type on the filename?
Second question is that how can I re-encode the video so that it keeps the I-frame positions?
ffmpeg -framerate 24 -i frame-%03d.png output.mp4
Above would make the video out of frames but the I-frame positioning eludes me.
I know this is a complex problem so all help is truly appreciated!