How to optimize already-encoded videos of still images (e.g. presentation slides)?

Question

I have a H264 video that is, for the most part, just a video of presentation slides.
It doesn't seem to me that it's as well-compressed as it could be.

Given that it's a video of presentation slides, the vast, vast majority of the frames should be exactly the same as each other. To the extent that they are not identical, it would be due to encoding noise.

My question is, is there any way (e.g. in FFMPEG, or via other tools) I could automatically "detect and repeat" such similar frames in these videos, so that any consecutive frames that are identical to within some small epsilon tolerance are automatically deduplicated and merely repeated from prior frames, so that the frames are encoded once?

Note that there are random bits of normal film video in between, so I can't assume the entire video is like this. But maybe 90% of it is like this.

(Ideally unaffected parts of the stream would be copied directly without being re-encoded, but if this isn't possible then I can live with a bit of quality loss.)

Answer 1

(Ideally unaffected parts of the stream would be copied directly without being re-encoded, but if this isn't possible then I can live with a bit of quality loss.)

First of all, I doubt there's a way to do this losslessly. Very few things that touch the actual, encoded video content in any way can be done without re-encoding.

That said, I tried out a few things just now and found that a very effective way of doing this is making the video frame rate variable and dropping duplicate frames using FFmpeg's mpdecimate filter:

ffmpeg -i input.mp4 -vsync vfr -vf mpdecimate output.mp4

The mpdecimate filter has a bunch of additional options that can be set but I found the default worked just fine. I downloaded this video with youtube-dl, encoded the first 5 minutes of it with standard options -c:v libx264 -an -t 300 -crf 22 -preset medium. I then encoded the same part of the video with the additional options -vsync vfr -vf mpdecimate and the resulting file was 42% smaller than the one encoded with a constant framerate. 30.8 MB vs 18.0 MB. The encoding process was also significantly faster since there were much fewer frames to encode.

However, I've heard that some players may have trouble playing variable frame rate video and that it can also cause audio sync issues. So I also encoded a version with the same settings but included audio as 64k Opus. I played the video in mpv and Firefox and didn't observe any audio synchronisation issues.

What I did notice was that this does affect seeking, sometimes quite dramatically. Firefox was able to seek fine for the most part, but when seeking to some parts of the video FF would display a frame that was long past (like a picture of the speaker at 2:03) and only start playing from a more recent frame about two seconds later but sometimes this would continue for a long time, like 40 seconds. However, when playing the video without seeking playback was perfect.

mpv played the file perfectly too and seemed more adept at seeking. However, some parts of the video couldn't be seeked to (presumably due to the same issue/property of the file which caused FF to have trouble with seeking) and would jump ahead or behind the spot I attempted to seek to. Both of these player issues probably have something to do with the general lack of frames inside the video. AFAIK most players use keyframes for seeking and services like Youtube use a keyframe interval of about two seconds to make videos easier to seek. So this issue probably can't be fixed without sacrificing the file size decrease gained from throwing away duplicate frames.

Due to these issues I wouldn't really recommend this method for video that's intended for wide distribution. However, it's probably fine for video that's inteded for personal use or archival. With the latter purpose in mind I attempted to re-encode the variable framerate video back into a constant framerate one using the -r 30000/1001 switch, just to check that it works, and ran into this issue with the audio. I used the -max_muxing_queue_size 9999 switch and the encode worked just fine and the resulting file played without issue. So as long as your tools are up to the task this should be a viable method of archiving video material.

Since this method does involve re-encoding you're going to lose some quality. However, since getting rid of duplicate frames significantly reduces the file size with this type of content, you can probably lower the CRF value to a point where the visible quality loss is almost insignificant and still end up saving a lot of space.

Answer 2

Video codecs already do this. They will insert key frames regularly however to make sure you can still seek to random times within the file without having to decode the entire thing. Most encoding tools let you change that interval.