1

I have a H264 video that is, for the most part, just a video of presentation slides.
It doesn't seem to me that it's as well-compressed as it could be.

Given that it's a video of presentation slides, the vast, vast majority of the frames should be exactly the same as each other. To the extent that they are not identical, it would be due to encoding noise.

My question is, is there any way (e.g. in FFMPEG, or via other tools) I could automatically "detect and repeat" such similar frames in these videos, so that any consecutive frames that are identical to within some small epsilon tolerance are automatically deduplicated and merely repeated from prior frames, so that the frames are encoded once?

Note that there are random bits of normal film video in between, so I can't assume the entire video is like this. But maybe 90% of it is like this.

(Ideally unaffected parts of the stream would be copied directly without being re-encoded, but if this isn't possible then I can live with a bit of quality loss.)

  • 1
    you can improve the x264 encoding performance for still images by using -tune stillimage in your ffmpeg command. – stib Sep 27 '18 at 10:01
  • @stib: Yeah, I've already tried that, it increases the file size instead of decreasing it. – Mehrdad Sep 30 '18 at 12:04
2

Video codecs already do this. They will insert key frames regularly however to make sure you can still seek to random times within the file without having to decode the entire thing. Most encoding tools let you change that interval.

  • This is frustrating. Your idea that the codec is somehow already achieving this is simply wrong. I have a 331 KB, 1-minute MPEG-4 AVC (part 10) video that is entirely a video of a single presentation slide. Its key frames come every 5 seconds. Yet no matter where I cut it by 1 second, its space usage decreases approximately proportionally, with only a minor difference if I hit a key frame. It is very obvious to me that there is a lot of room for improvement in whatever the codec has encoded in between the key frames. The problem is very clearly not the key frame spacing. – Mehrdad Sep 30 '18 at 12:02
  • @Mehrdad Can you share this file? – Gyan Oct 15 '18 at 15:40
2

(Ideally unaffected parts of the stream would be copied directly without being re-encoded, but if this isn't possible then I can live with a bit of quality loss.)

First of all, I doubt there's a way to do this losslessly. Very few things that touch the actual, encoded video content in any way can be done without re-encoding.

That said, I tried out a few things just now and found that a very effective way of doing this is making the video frame rate variable and dropping duplicate frames using FFmpeg's mpdecimate filter:

ffmpeg -i input.mp4 -vsync vfr -vf mpdecimate output.mp4

The mpdecimate filter has a bunch of additional options that can be set but I found the default worked just fine. I downloaded this video with youtube-dl, encoded the first 5 minutes of it with standard options -c:v libx264 -an -t 300 -crf 22 -preset medium. I then encoded the same part of the video with the additional options -vsync vfr -vf mpdecimate and the resulting file was 42% smaller than the one encoded with a constant framerate. 30.8 MB vs 18.0 MB. The encoding process was also significantly faster since there were much fewer frames to encode.

However, I've heard that some players may have trouble playing variable frame rate video and that it can also cause audio sync issues. So I also encoded a version with the same settings but included audio as 64k Opus. I played the video in mpv and Firefox and didn't observe any audio synchronisation issues.

What I did notice was that this does affect seeking, sometimes quite dramatically. Firefox was able to seek fine for the most part, but when seeking to some parts of the video FF would display a frame that was long past (like a picture of the speaker at 2:03) and only start playing from a more recent frame about two seconds later but sometimes this would continue for a long time, like 40 seconds. However, when playing the video without seeking playback was perfect.

mpv played the file perfectly too and seemed more adept at seeking. However, some parts of the video couldn't be seeked to (presumably due to the same issue/property of the file which caused FF to have trouble with seeking) and would jump ahead or behind the spot I attempted to seek to. Both of these player issues probably have something to do with the general lack of frames inside the video. AFAIK most players use keyframes for seeking and services like Youtube use a keyframe interval of about two seconds to make videos easier to seek. So this issue probably can't be fixed without sacrificing the file size decrease gained from throwing away duplicate frames.

Due to these issues I wouldn't really recommend this method for video that's intended for wide distribution. However, it's probably fine for video that's inteded for personal use or archival. With the latter purpose in mind I attempted to re-encode the variable framerate video back into a constant framerate one using the -r 30000/1001 switch, just to check that it works, and ran into this issue with the audio. I used the -max_muxing_queue_size 9999 switch and the encode worked just fine and the resulting file played without issue. So as long as your tools are up to the task this should be a viable method of archiving video material.

Since this method does involve re-encoding you're going to lose some quality. However, since getting rid of duplicate frames significantly reduces the file size with this type of content, you can probably lower the CRF value to a point where the visible quality loss is almost insignificant and still end up saving a lot of space.

  • +1 thank you! I will check this out when I get the chance. – Mehrdad Oct 15 '18 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.