7

When making a video from single image files whereas each image file should be visible for about one second, it does make sense to encode a video with an extremely low frame rate such as 1 frame per second. For this type of application, every frame rate greater than this would be a waste of resources.

I am wondering if the H264 codec (or any specific implementation, such as x264) itself has any lower limit for the frame rate below which it comes to technical problems or some kind of instabilities. In case there is no problem with encoding, can we expect video players to properly deal with such an unusual low frame rate?

Thanks for sharing your experience!

|improve this question
2

I'm with AJ. Unless you know the characteristics of every player that might view this, it would be unwise to rely on a small sample of test results. Using a standard frame rate like 24 fps with a keyframe interval of 24 frames will give you essentially the same thing with no compromise in compatibility. The intermediate frames will be minimally small because there will be no detectable changes to encode.

|improve this answer
  • 1
    yup, a bit-identical frame only takes about 15 bytes. All macroblocks = skip, and CABAC compresses the repeated bit pattern for that very well. – Peter Cordes Mar 1 '15 at 2:06
  • 2
    I'd only worry about hardware players that are assuming they'll output to a 60 or 50Hz TV signal, though. h.264 doesn't care about timing, it's just frames, even in a VFR video. Frame timestamps is a container issue. Container formats are very flexible. It's easily possible to have a single frame shown for 1 minute, then 150fps for several frames, then another frame shown for a while, or anything you like. Storing VFR video in mkv, mp4, and some other modern containers is a solved problem. – Peter Cordes Mar 1 '15 at 2:08
5

I'm not sure how it will behave at very low frame rates, but it is worth pointing out this would also limit your options on how and when you could change frames since they would have to follow on the clock cycles. What is more likely to work in this case is a long keyframe interval. The majority of frames in a compression like H.264 only store the changes from the previous frame. In the case of a still image, the compression ratios will be huge because very little (no) change occurs between frames. I'm not sure that you would really pick up enough of a savings from dropping the frame rate to be worth the loss of control over when you can make a change to the frame.

The best bet would be to try it with your media and see the results. Compression is a highly content dependent thing and the best quality and compression for any particular clip is going to depend a lot on the nature of that clip, so trial is still the best way to test it out.

|improve this answer
  • There is a compression downside beyond what my earlier comment on another answer said: If there is much redundancy between the different images (i.e. it's still a video, not a slideshow), padding with identical images will make it harder for the encoder to find and exploit that. Depending on encoding settings, the encoder will only keep some number of old frames as possible references for new frames, and can only search within a GOP (e.g. default 250 frames for x264). If all those candidates are the same image, that doesn't give it multiple options to find a better reference for each block... – Peter Cordes Apr 18 at 8:40
  • ... e.g. after a foreground object moves in front of some background detail, the encoder can save bits by referencing what it looked like in an older frame before it was obscured. h.264 can choose reference frames on a per-block basis. This is a relatively small effect; good h.264 encoders do ok with only 1 reference frame, but it's still somewhat harmful to compression efficiency – Peter Cordes Apr 18 at 8:41
  • Sure, you still need proper encoding settings, but you can grow your GOP size rather than reduce your frame rate if things are all that static. If they aren't, then dropping frame rate isn't a great option to begin with. I wonder if there has been any work on a variable GOP format. – AJ Henderson♦ Apr 18 at 11:19
  • I think repeated images are still going to reduce the opportunity for useful B-pyramid and multiple reference P-frames options. But I guess an encoder can keep an old P frame from anywhere within the GOP so losing out on reference B-frames is probably all in theory, but IDK about in practice. – Peter Cordes Apr 18 at 11:41
  • 1
    Good MPEG-2 encoders can make keyframe decisions based on scenecuts, and P vs B frame decisions based on content. :P ffmpeg's mpeg2video encoder lists a -sc_threshold option, and a -b_strategy option to control I/P/B selection strategy. But anyway, h.265 is neat, with up to 32x32 DCT blocks, and very large 64x64 prediction units which can break down into smaller blocks if needed. sonnati.wordpress.com/2014/06/20/h265-part-i-technical-overview. vs. h.264 16x16 macroblocks with only 4x4 or 8x8 (high profile only) DCT blocks. Also forum.doom9.org/showthread.php?t=167081 – Peter Cordes Apr 18 at 12:27
2

I've played around with turning a bunch of still photos into a h.264 slideshow, mostly to compare the compression efficiency of JPG vs. h.264. I got some useful replies about the technical implications of this from x264 devs on doom9. e.g. force x264 to not use B frames for this, because not-very-related images will need a lot of I macroblocks, and coding them in B frames is more expensive.

Software player behaviour with low-fps video isn't ideal, in the past. I think one older player only checked for keyboard input when it displayed a frame. So there was lag between user input and player response. mplayer2 and mpv don't have this problem. Also, players that can only seek to keyframes will seek in really large chunks (2 minutes or so!) if you don't reduce the keyframe interval. x264 won't insert IDR (GOP boundaries) all over the place if the images are somewhat related to each other.

Use x264 -tune stillimage. It cranks up the psy optimizations, because temporal stability isn't an issue for this use-case. Further search results: from google.

I'd agree with other suggestions to have some duplicated frames, to bring the FPS up to at least 5 or something, just in case of bad players. However, smartphones / tablets should have no problem playing variable-FPS video, since they usually record that way when light levels drop. Since variable-FPS videos from phones are now out there, hardware player support for them should be expected. I wouldn't expect problems, but I also wouldn't be surprised if there at least some old hardware players that don't handle it well.

A frame of all "skip" macroblocks only takes about 20bytes at 1080p, IIRC. One reason I don't like duplicated frames, though, is that it interferes with single-stepping to go through the images manually.

There is a compression downside to duplicating frames, though: If there is much redundancy between the different images (i.e. it's still a video, not a slideshow), padding with identical images will make it harder for the encoder to find and exploit that.

Depending on encoding settings, the encoder will only keep some number of old frames as possible references for new frames, and can only search within a GOP (e.g. default 250 frames for x264). If all those candidates are the same image, that doesn't give it multiple options to find a better reference for each block.

e.g. After a foreground object moves in front of some background detail, the encoder can save bits by referencing what it looked like in an older frame before it was obscured. h.264 can choose reference frames on a per-block basis. This is a relatively small effect; good h.264 encoders do ok with only 1 reference frame, but it's still somewhat harmful to compression efficiency, and a waste of power / battery life / CPU time on the decompression side to copy memory around decoding and displaying extra frames.

Recovering VFR after an NLE forces all your clips to some high frame rate:

FFmpeg has an mpdecimate filter that drops similar frames. You can set limits on how many frames in a row it can drop. With a tight similarity threshold, you should get it to only drop actual duplicates.

e.g. ffmpeg -i input.mp4 -vf mpdecimate=max=9:hi=400 -c:a copy -c:v libx264 -preset veryslow -tune film output_vfr.mkv drops up to 9 frames in a row, and only if the most-different block was different under "400", and (defaults): no more than 33% of the blocks were different by "320" units. IIRC, it's basically an 8x8 SAD on pixel components.

(FFmpeg defaults to CFR for .mp4 outputs, though, so use -vsync 2 for variable-frame-rate .mp4 output. I think that's safe: Problems with frame rate on video conversion using ffmpeg with libx264)

|improve this answer
1

Most NLE's will allow you to import a still image in the form of how long you want it to appear in the timeline assuming you have set the project properties to some standard frame rate such as 30 fps or 24 fps etc.

In Vegas Pro I can set the time a still picture should appear on the timeline, from a fraction of second to several seconds. If I set this to 1 second then when I drag and drop a still image in the timeline, Vegas will generate enough frames to meet my request. I usually edit with 30 fps videos, and when I add a still picture I am mixing a timeline with 30 fps video that is already there (AVCHD 1080p).

To give you a specific answer, I would need to know what NLE you are using.

|improve this answer
  • I just apply a raw encoding software such as ffmpeg or avconv, so no need to talk about any NLE. I think the question is pretty much answered with "Just go by a standard frame rate which all players can deal with properly. There is no real 'waste of resources', because the encoding scheme is good enough to efficiently deal with still images". – Jan-Philip Gehrcke Aug 17 '13 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.