My video file has some blackish frames (it's almost black, but not exactly black, i.e. #000000), and I want to filter out them, replacing them with neighboring frames.

However, I can't find a simple way to do that.

Of course, I can extract all the frame images from the video file with FFmpeg:

ffmpeg -i foo.mp4 -an -framerate 30 -s 1280x720 -f image2 foo-%05d.jpeg

and then list all the blackish images with some image tools, replace them with neighboring-numbered files, and merge them:

ffmpeg -i foo.mp4 -vn -acodec copy foo.m4a
ffmpeg -i foo.m4a -f image2 -framerate 30 -i foo-%05d.jpeg -r 30 -s 1280x720 output.mp4

but this will create over 10,000 intermediate image files and it is also time-consuming.

Is there any software or programming language that helps this work?

I am using Windows x64.

  • 1
    Remember that to do the image processing, whatever software you use is going to have to decode each frame so it's always going to be time consuming. Writing them to disk as separate image files will take more time I suppose, but you could perhaps use something like a named pipe to keep it in memory, depending on what image tools you use.
    – stib
    Commented Jan 1, 2015 at 12:13
  • 1
    As a side note, you really want to interpolate a frame rather than just using the frame to one side. 2 frames in sequence being the same will be a noticeable stutter.
    – AJ Henderson
    Commented Jan 2, 2015 at 17:01
  • Using something like Twixtor would attempt to interpolate movement from the two frames either side - might be a better option than just repeating or interpolating a crossfade of two adjacent frames? revisionfx.com/products/twixtor
    – tomh
    Commented Jan 2, 2015 at 17:22

1 Answer 1


You could do this in After Effects. There are lots of examples online where people use Expressions to examine the average brightness of each frame, then act upon it. For example, you could add a marker to each frame that fell below a certain threshold level. Or make the frame's opacity go to 0%, to allow another track to show through from behind.

Tutorial examples here: http://markos.co.nz/2012/05/after-effects-expression-use-the-luminance-of-one-layer-to-drive-another-layer/


Here's an example expression I wrote based on the blog posts above -

threshold = 0.01;  // this is the threshold below which the track becomes transparent - black = 0.
driverLayer = thisComp.layer("Pre-comp 1");
samplePoint = [0,0]; // examine from the top left pixel
sampleSize = [640,360]; // examine the full comp size (in this case 640x360)
lightnessSample = driverLayer.sampleImage(samplePoint,sampleSize);  // this samples the image as rgb
driverLightness = rgbToHsl(lightnessSample)[2]; // converts to hue, saturation and lightness, then only takes lightness
if (driverLightness >=threshold) transform.opacity = 100 else 0; // tell opacity to be either 100 or zero depending upon the threshold point chosen in line 1

I applied this to the opacity layer. You would need to adjust the threshold value, and the size of your comp ( in my case 640x360). (the red solid below was to see whether the opacity change was working. I guess you would put a copy of your video showing the previous frame here? Will look as though the video got stuck though...

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.