1

Due to a bug with Nvidia Shadowplay, I have a bunch of videos that look like this. A bunch of random frames have been replaced with pure black frames.

I'd like to fix one of these videos to remove the flickering. I saw an answer here which suggests the following ffmpeg command:

ffmpeg -i in.mp4 -vf blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=50:function=less -vsync cfr -c:a copy out.mp4

However, this replaces all the black frames with the most recent non-black frame, which makes the video look like a slideshow.

Is there any way to use a similar ffmpeg command (or any other tool) to use motion interpolation to fix the missing frames?

Improve this question

3 Answers 3

Reset to default
2

After poring over the documentation all weekend, I figured out how to use the minterpolate filter to do motion interpolation. The full command is

ffmpeg -i in.mp4 -vf blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=50:function=less,minterpolate=mi_mode=mci:mc_mode=aobmc:me_mode=bidir:vsbmc=1:me=fss -c:a copy out.mp4

These were the settings that gave me the best-looking results. If these are too slow for you, you can try something like

ffmpeg -i in.mp4 -vf blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=50:function=less,minterpolate=mi_mode=mci:mc_mode=obmc:me_mode=bilat:vsbmc=0:me=tdls -c:a copy out.mp4

which went about 3x as fast in my testing. You can find the documentation on the different minterpolate options here.

Improve this answer
1
+100

After removing the black frames, you can insert the framerate filter to fill the gaps with blended frames.

ffmpeg -i in.mp4 -vf blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=50:function=less,framerate=fps=60000/1001 -c:a copy out.mp4

See docs for framerate filter. The filter will produce a CFR stream so vsync is not required.

Improve this answer
3
  • This appears to fade out the previous frame and fade in the next one (either that or the motion interpolation algorithm is extremely poor). While it's definitely better than just holding a still frame, the results are still not great. I'll accept this if no one finds anything better...
    – BlueRaja - Danny Pflughoeft
    Sep 6, 2019 at 19:18
  • The filter performs blending, so no motion interpolation. You can try the minterpolate filter instead of framerate but it will be slow.
    – Gyan
    Sep 6, 2019 at 21:08
  • Thanks for pointing me to minterpolate! It works better than the 'framerate' filter, except in areas with fast movement or lots of consecutive missing frames.
    – BlueRaja - Danny Pflughoeft
    Sep 8, 2019 at 22:47
0

Not sure about FFMPEG, but it wouldn't hurt to try the free version of DaVinci Resolve. From page 1351 of the user manual (yes, quite TLDR):

RepairFrame, Tween

The Tween and Repair Frame nodes are different from standard optical flow nodes because they have the OpticalFlow analysis and motion vector generation built in. Tween will compare two frames and create an in-between frame, which is good for recreating a missing or flawed frame. Repair Frame will look at frames on either side of the current frame and repair scratches, dust marks, and so on. Because these nodes work with flow values between non-sequential frames, they cannot use the optical flow stored in the input image’s Vector/BackVector channels, but rather must regenerate the flow of each frame, do their processing, and then destroy the flow channels. This being the case, these nodes are computationally expensive. For more information on Tween or Repair Frame see the Nodes Reference guide.

Improve this answer
2
  • It seems "RepairFrame" has been renamed to "Remove dust and scratches" in the latest version? I tried it but it didn't work at all.
    – BlueRaja - Danny Pflughoeft
    Sep 6, 2019 at 19:15
  • Works fine for me. It's not called dust and scratches. It's in the fusion tab. Type shift+spacebar with the node editor active, and start typing "repairframe" or check under the "optical flow" subcategory of "tools" in the Effects Library. If you don't see it there, it's a paid version only feature, in which case, I apologize for sending you on a wild goose chase. It's hard to remember which things are and which aren't. There isn't much that you can't do with the free version, but every once in a while, you run into something.
    – Jason Conrad
    Sep 8, 2019 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.