I've played around with turning a bunch of still photos into a h.264 slideshow, mostly to compare the compression efficiency of JPG vs. h.264. I got some useful replies about the technical implications of this from x264 devs on doom9. e.g. force x264 to not use B frames for this, because not-very-related images will need a lot of I macroblocks, and coding them in B frames is more expensive.
Software player behaviour with low-fps video isn't ideal, in the past. I think one older player only checked for keyboard input when it displayed a frame. So there was lag between user input and player response. mplayer2 and mpv don't have this problem. Also, players that can only seek to keyframes will seek in really large chunks (2 minutes or so!) if you don't reduce the keyframe interval. x264 won't insert IDR (GOP boundaries) all over the place if the images are somewhat related to each other.
Use x264 -tune stillimage. It cranks up the psy optimizations, because temporal stability isn't an issue for this use-case. Further search results: from google.
I'd agree with other suggestions to have some duplicated frames, to bring the FPS up to at least 5 or something, just in case of bad players. However, smartphones / tablets should have no problem playing variable-FPS video, since they usually record that way when light levels drop. Since variable-FPS videos from phones are now out there, hardware player support for them should be expected. I wouldn't expect problems, but I also wouldn't be surprised if there at least some old hardware players that don't handle it well.
A frame of all "skip" macroblocks only takes about 20bytes at 1080p, IIRC. One reason I don't like duplicated frames, though, is that it interferes with single-stepping to go through the images manually.
There is a compression downside to duplicating frames, though: If there is much redundancy between the different images (i.e. it's still a video, not a slideshow), padding with identical images will make it harder for the encoder to find and exploit that.
Depending on encoding settings, the encoder will only keep some number of old frames as possible references for new frames, and can only search within a GOP (e.g. default 250 frames for x264). If all those candidates are the same image, that doesn't give it multiple options to find a better reference for each block.
e.g. After a foreground object moves in front of some background detail, the encoder can save bits by referencing what it looked like in an older frame before it was obscured. h.264 can choose reference frames on a per-block basis. This is a relatively small effect; good h.264 encoders do ok with only 1 reference frame, but it's still somewhat harmful to compression efficiency, and a waste of power / battery life / CPU time on the decompression side to copy memory around decoding and displaying extra frames.
Recovering VFR after an NLE forces all your clips to some high frame rate:
FFmpeg has an mpdecimate filter that drops similar frames. You can set limits on how many frames in a row it can drop. With a tight similarity threshold, you should get it to only drop actual duplicates.
e.g. ffmpeg -i input.mp4 -vf mpdecimate=max=9:hi=400 -c:a copy -c:v libx264 -preset veryslow -tune film output_vfr.mkv drops up to 9 frames in a row, and only if the most-different block was different under "400", and (defaults): no more than 33% of the blocks were different by "320" units. IIRC, it's basically an 8x8 SAD on pixel components.
(FFmpeg defaults to CFR for .mp4 outputs, though, so use -vsync 2 for variable-frame-rate .mp4 output. I think that's safe: Problems with frame rate on video conversion using ffmpeg with libx264)
