3

I am doing an animation, and if I do things the easy way, fast moving objects end up jerking across the screen, rather than appearing to move smoothly – in other words, I have a temporal aliasing problem. One possible solution is to oversample the animation and blend the frames back together at the output frame rate.

To illustrate, at 24 frames per second:

frame
  0    **
  1        **
  2            **
  3                **
 ...                   **

Oversampled at double speed (though realistically I'd probably oversample quite a bit more than double):

frame
  0    **
 0.6     **
  1        **
 1.5         **
  2            **
 2.5             **
  3                **
 3.5                 **
 ...                   **

Blended back to 24 frames per second (all mixed with the background):

frame
  0    ****
  1        ****
  2            ****
  3                ****
 ...                   ****

The problem with oversampling is that if I just use Premiere Elements 14, if I take the oversampled clip and slow it back down to regular speed, it appears to just skip frames rather than blending them.

This seems like something that ffmpeg probably does, but I've barely started trying to understand it, and my reading of the documentation, tutorials, and the forum messages that have turned up with web searches hasn't been enough to figure out what switches would do the job.

So, questions:

  1. Is there an option in Premiere Elements 14 to blend frames from an oversampled clip, rather than just skipping frames?
  2. If ffmpeg is the smart way to do this, what kind of switches do I need to use to get the job done?
  3. Is there some other reasonably easy-to-use software that could do the frame blending instead of skipping them? (And if so, how do I use that?)

Even if there's a way to do it directly in Premiere Elements, answers to #2 and #3 are likely to be helpful to people who use other editors.

  • 1
    I just wanted to ask the same! Important question. – Crouching Kitten May 16 '18 at 22:15
  • Please share how did you eventually solve the problem? I'm considering the tmix filter to evenly blend, say, 8 frames at 8x framerate. Then drop 7 out of 8 frames going to original framerate. – Serge Mar 17 at 11:25
  • 1
    @Serge , I did the blur entirely within Premiere. It was pretty tedious, however. I split the frames into groups (for example, 0, 1, 2, 3, 4), and overlay them at alpha 20%, 25%, 33%, 50%, and 100%, respectively. With some mathematical reasoning, or with experimentation, you can find that to equally blend an n frame oversample, the top later should have alpha 1 ÷ n, the next 1 ÷ (n - 1), . . ., and the bottom layer 100%. – Steve Mar 29 at 17:48
  • Although tmix may be a good general solution, I struggled with ffmpeg enough that I went ahead and did it entirely within Premiere. Maybe I should write up a full explanation of what I did, since this is now a "notable question", and a detailed explanation might be useful across all video editors if I explain it well. Doing it entirely within an editor may be preferable in some cases because it doesn't add a generation of codec artifacts and distortion to the video. – Steve Mar 29 at 17:59
2

I can't test the ffmpeg method below now but it notionally does what you ask:

Let's say your source is 25 fps.

ffmpeg -i in.mp4 -vf minterpolate=50,tblend=all_mode=average,framestep=2 out.mp4
  • I'm looking at the manual, and I can see what some of this means: -i for the input file name, -vf to define a filtergraph, and no prefix for the output file name. But I'm not clear which of minterpolate, tblend, all_mode, average, and framestep are keywords. None of them show up on ffmpeg.org/ffmpeg.html . I'll test and tinker, but I need to generate an oversample file for the testing first. – Steve Apr 12 '17 at 6:36
  • 1
    minterpolate, tblend and framestep are filters. The first does the oversampling. The 2nd, the blended frames, and 3rd reduces the stream back to 25 fps. – Gyan Apr 12 '17 at 6:54
  • That's a big help. That list of filters is not linked in context on ffmpeg.org/ffmpeg.html – only in the "see also" section at the end, where I didn't know to look for it. – Steve Apr 12 '17 at 20:46
  • I tried a test case, and got the error message: "[Parsed_minterpolate_0 @ 0000000001fa9000] Failed to configure input pad on Parsed_minterpolate_0 Error opening filters!" (The long hex number changes each time I run it.) Web searches for "Failed to configure input pad" have been unhelpful so far. – Steve Apr 15 '17 at 15:36
  • Need to see full console output. The hex numbers is the address of the filter in memory. Ignore that. Add -report and run. Share the report. – Gyan Apr 15 '17 at 15:58
2

Newer ffmpeg since version 4.1 includes new tmix video filter that mixes N adjacent frames together using weights.

Now it is possible to do exactly what you have suggested: take an animation rendered with a N times higher framerate, merge every N frames together and lower framerate N times down to normal one, dropping out extra frames.

Here's a script I created to test this flow:

#!/bin/bash

let WIDTH=640
let HEIGHT=480

let MID=$WIDTH/2

let BUFFER=16
let FRAMERATE=30

let XRATE=$BUFFER\*$FRAMERATE

let DURATION=8
let FPERIOD=$XRATE\*$DURATION


ffmpeg \
  -filter_complex \
  " \
    color=white:${WIDTH}x${HEIGHT}:d=$DURATION:r=$XRATE,format=rgb24[bg];  \
    color=black:8x${HEIGHT}:d=$DURATION:r=$XRATE,format=rgb24[bar];  \
    [bg][bar]overlay=x='$MID + $MID * sin(8 * PI * n / $FPERIOD)':y=0:format=yuv444,format=yuv420p[motion]; \
    [motion]tmix=frames=$BUFFER \
  " \
  -c:v libx264  \
  -r $FRAMERATE  \
  -an  \
  -hide_banner \
  -y  \
  output.mp4

It creates an 8-seconds long animation of a black bar running left-right in a sine speed motion. See it on YouTube.

Thanks Gyan for the tip on achieving smooth animation by forcing full chroma sampling.

  • 1
    Excellent! I found a work-around within Premiere, but it was pretty tedious: split the frames into groups (for example, 0, 1, 2, 3, 4), and overlay them at alpha 20%, 25%, 33%, 50%, and 100%. Yours looks much better if I need to do this again. (And isn't there a badge for a good answer to an old question? ) – Steve Mar 18 at 7:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.