2

I have a series of high framerate (slow motion) videos and need to extract all frames with content to series of images.

Most of the videos are of electrical events that took place in the dark and just the frames with a certain degree of lighting / actual visual output need to be extracted. Thus the dark / black frames will not be extracted, but the other frames will be.

Any ideas on how to accomplish this?

  • Sounds like you need a custom tool to be coded. Do you know AS3, Java, C# or Python? I know those can plug into FFmpeg executable, so you could decode per frame and search for any first pixel of above certain "brightness" then extract range of until black/darker frames. – VC.One Aug 21 '17 at 12:26
  • 1
    Don't have time now, but I did a similar task a while back. Will write it up within a day or two. I used ffmpeg. – Gyan Aug 21 '17 at 16:52
  • Wow, that will be quite interesting to see @Mulvya. – ylluminate Aug 21 '17 at 18:48
2

Here's the gist of the ffmpeg method: use the blackframe filter to identify frames with a certain amount of non-black content. Use the metadata filter to pass through those frames and discard the rest.

Inspection:

ffmpeg -i in.mp4 -vf blackframe=amount=0:threshold=92 -f null -

The amount (0) in the blackframe parameters refers to the percentage of pixels that have to be below the threshold (92) for the filter to tag it as a black frame. The way we'll using it, it won't matter what the amount is, so keep it at 0.

You will get a readout like this:

[Parsed_blackframe_0 @ 0000000002be5aa0] frame:1 pblack:35 pts:1 t:0.040000 type:I last_keyframe:1
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:2 pblack:35 pts:2 t:0.080000 type:I last_keyframe:2
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:3 pblack:35 pts:3 t:0.120000 type:I last_keyframe:3
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:4 pblack:35 pts:4 t:0.160000 type:I last_keyframe:4
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:5 pblack:34 pts:5 t:0.200000 type:I last_keyframe:5
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:6 pblack:34 pts:6 t:0.240000 type:I last_keyframe:6
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:7 pblack:34 pts:7 t:0.280000 type:I last_keyframe:7
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:8 pblack:34 pts:8 t:0.320000 type:I last_keyframe:8
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:9 pblack:34 pts:9 t:0.360000 type:I last_keyframe:9
[Parsed_blackframe_0 @ 0000000002be5aa0] frame:10 pblack:34 pts:10 t:0.400000 type:I last_keyframe:10

The pblack value indicates that 35% of the pixels in frame 1 had a luminance value below 92 (out of 255, since I ran it on a video with 8-bit luma).

With some trial and error, figure out what threshold and pblack values will identify the frames you want to discard/keep.

Extraction:

With those values, use

ffmpeg -i in.mp4
       -vf "blackframe=amount=0:threshold=X,
            metadata=select:key='lavfi.blackframe.pblack':value=Y:function=less"
       -vsync 0 img%d.jpg

X is the threshold value you calibrated to identify black pixels, and determines the pblack value (out of 100) reported by the blackframe filter. The metadata filter then keeps those frames which have at most Y% of pixels that qualify as black. These frames are then saved as images.

| improve this answer | |
1

AVISynth is more powerful than using FFMPEG (as suggested by @Mulvya) but I do like the simplicity.

This Webpage from AVISynth's Help demonstrates: http://avisynth.nl/index.php/ConditionalFilter .

Not only will AVISynth allow you to operate on individual frames automatically and manually with various Functions but it can also save to numbered Images if desired.

A fair learning curve is involved but 10's of thousands of examples are available. There are also Plugins available which can skip Frames that are unchanged.

Loading the AVISynth Script into VirtualDub makes for easy viewing and editing.

Once your Script is tweaked perfectly it can be fed to FFMPEG and most other Editors or Compressors to produce a final version.

| improve this answer | |
0

Regardless, the footage would have to be analyzed/scanned with a first-pass to identify the frames (provided you coded / scripted a tool that would create a N value based on luminance characteristics of each frame- which would then need to be logged to a file).

I would recommend just exporting the footage as a JPG sequence, which is very fast to do.

Then using a program like ACDSEE, which works a lot like Bridge; but has more features and tools; you can sort the images based on luminance; discarding those with levels below an N value.

Also; ACDSEE has a tool to "Find Duplicate Images".

It would thus find all black frames as duplicate images (provided there wasn't any motion); and you could delete those.

Re-import the JPEG sequence as footage; and ignore the errors for missing frames. Then ripple delete the spaces in one swoop.

Lastly; you could export TIF uncompressed as well. Not sure if you're dealing with just 120fps GoPro footage or Uncompressed 4:4:4 RED footage... which you'd want to retain the color depth for down the pipeline corrections.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.