0

I have a python script, that generates a ffmpeg command for creating a slideshow with the "Ken Burns" filter.

For example, I have 10 images and want to create an 1-minute video at 30 FPS. It generates the following command:

/usr/local/bin/ffmpeg -i /Users/user/src/python/VideoMaker2/input/images/test/1.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/2.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/3.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/4.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/5.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/6.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/7.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/8.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/9.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/10.jpg -r 30 -t 60 -vcodec libx264 -preset ultrafast -tune zerolatency -pix_fmt yuv420p -filter_complex [0:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=0.0:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+0.0/TB[0v];[1:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+5.28/TB[1v];[2:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+10.56/TB[2v];[3:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+15.84/TB[3v];[4:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+21.12/TB[4v];[5:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+26.400000000000002/TB[5v];[6:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+31.68/TB[6v];[7:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+36.96/TB[7v];[8:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+42.24/TB[8v];[9:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=10.0:alpha=0,setpts=PTS-STARTPTS+47.52/TB[9v];[0v][1v]overlay[1v];[1v][2v]overlay[1v];[1v][3v]overlay[1v];[1v][4v]overlay[1v];[1v][5v]overlay[1v];[1v][6v]overlay[1v];[1v][7v]overlay[1v];[1v][8v]overlay[1v];[1v][9v]overlay -y /Users/user/src/python/VideoMaker2/output/videos/1547957911138.mp4

And the resulting video's duration ends up as 57 seconds (3 seconds difference) for some reason. If I create a longer video, the difference will grow proportionally.

But if I create a video with same parameters but at 60 FPS, it's OK:

/usr/local/bin/ffmpeg -i /Users/user/src/python/VideoMaker2/input/images/test/1.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/2.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/3.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/4.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/5.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/6.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/7.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/8.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/9.jpg -i /Users/user/src/python/VideoMaker2/input/images/test/10.jpg -r 60 -t 60 -vcodec libx264 -preset ultrafast -tune zerolatency -pix_fmt yuv420p -filter_complex [0:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=0.0:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+0.0/TB[0v];[1:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+5.28/TB[1v];[2:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+10.56/TB[2v];[3:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+15.84/TB[3v];[4:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+21.12/TB[4v];[5:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+26.400000000000002/TB[5v];[6:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+31.68/TB[6v];[7:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+36.96/TB[7v];[8:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=0.6600000000000001:alpha=0,setpts=PTS-STARTPTS+42.24/TB[8v];[9:v]format=yuv420p,format=yuv420p,scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=475.2,scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,fade=t=in:st=0:d=1.3200000000000003:alpha=1,fade=t=out:st=7.920000000000001:d=10.0:alpha=0,setpts=PTS-STARTPTS+47.52/TB[9v];[0v][1v]overlay[1v];[1v][2v]overlay[1v];[1v][3v]overlay[1v];[1v][4v]overlay[1v];[1v][5v]overlay[1v];[1v][6v]overlay[1v];[1v][7v]overlay[1v];[1v][8v]overlay[1v];[1v][9v]overlay -y /Users/user/src/python/VideoMaker2/output/videos/1547958267849.mp4

Can someone please say where the mistake is in this command?

Upd. Class that is responsible for filter creation:

class KenBurnsCreator(object):
    HALF_TRANSITION_RATIO = 0.1

    __width: float
    __height: float
    __images: List[str]
    __duration: float
    __frame_rate: int

    __half_transition_duration: float
    __animation_duration: float

    __logger = get_logger(__qualname__)

    def __init__(self, width: float, height: float, images: List[str], duration: float, frame_rate: int) -> None:
        super().__init__()

        self.__width = width
        self.__height = height
        self.__images = images
        self.__duration = duration
        self.__frame_rate = frame_rate

        self.__animation_duration = duration / float(len(self.__images)) * (1 + self.HALF_TRANSITION_RATIO)
        self.__half_transition_duration = self.__animation_duration * self.HALF_TRANSITION_RATIO

        self.__logger.debug(f'Animation duration: {self.__animation_duration}, half transition duration: {self.__half_transition_duration}')

    def __create_ken_burns_translation_filter(self) -> str:
        upscale_ratio = max(3, int(3000 / self.__height))

        w = int(self.__ceil(self.__width * upscale_ratio))
        h = int(self.__ceil(self.__height * upscale_ratio))

        translation_duration = self.__frame_rate * (self.__animation_duration + self.__half_transition_duration * 2)

        filter_str = 'format=yuv420p'
        filter_str += ','
        filter_str += f'scale=trunc(min(iw*{h}/ih\\,{w})/2)*2:-2'
        filter_str += ','
        filter_str += f'pad={w}:{h}:'
        filter_str += f'({w}-iw)/2:'
        filter_str += f'({h}-ih)/2:'
        filter_str += f'color=black'
        filter_str += ','
        filter_str += 'zoompan='
        filter_str += 'z=\'zoom+0.002\':'
        filter_str += 'x=\'iw/2-(iw/zoom/2)\':'
        filter_str += 'y=\'ih/2-(ih/zoom/2)\':'
        filter_str += f's={w}x{h}:'
        filter_str += f'd={translation_duration:.1f}'

        return filter_str

    def __stitch(self, count: int) -> str:
        filter_str = '[0v][1v]'
        filter_str += 'overlay'

        for i in range(2, count):
            filter_str += '[1v];'
            filter_str += '[1v]'
            filter_str += f'[{i}v]'
            filter_str += 'overlay'

        return filter_str

    def __create_filter_content_for_ken_burns(self, count: int, index: int, animation: str) -> str:
        fade_duration_1 = 0.0 if index == 0 else self.__half_transition_duration * 2.0
        fade_start = self.__animation_duration + self.__half_transition_duration * 2.0
        fade_duration_2 = 10.0 if index == count - 1 else self.__half_transition_duration
        pts = (self.__animation_duration - self.__half_transition_duration * 2.0) * float(index)

        filter_str = f'[{index}:v]'
        filter_str += 'format=yuv420p'
        filter_str += ','
        filter_str += animation
        filter_str += ','
        filter_str += 'scale='
        filter_str += f'trunc(min(iw*{int(self.__height)}/ih\\,{int(self.__width)})/2)*2:-2'
        filter_str += ','
        filter_str += 'fade=t=in:st=0:'
        filter_str += f'd={fade_duration_1}:'
        filter_str += 'alpha=1'
        filter_str += ','
        filter_str += 'fade=t=out:'
        filter_str += f'st={fade_start}:'
        filter_str += f'd={fade_duration_2}:'
        filter_str += 'alpha=0'
        filter_str += ','
        filter_str += f'setpts=PTS-STARTPTS+{pts}/TB'
        filter_str += f'[{index}v];'

        return filter_str

    # ceil and round to 2-divisible
    def __ceil(self, number: float):
        new = math.ceil(number)
        if new % 2 != 0:
            new += 1
        return new

    def create_animation_filter(self) -> str:
        filter_str = ''

        images_count = len(self.__images)

        for index in range(images_count):
            ken_burns_translation_filter = self.__create_ken_burns_translation_filter()
            content = self.__create_filter_content_for_ken_burns(images_count, index, ken_burns_translation_filter)

            filter_str += content

        filter_str += self.__stitch(images_count)

        return filter_str
1

Within each image filtering block,

[9:v]format=yuv420p,
format=yuv420p,
scale=trunc(min(iw*2880/ih\,5120)/2)*2:-2,
pad=5120:2880:(5120-iw)/2:(2880-ih)/2:color=black,
zoompan=z='zoom+0.002':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=5120x2880:d=237.6,
scale=trunc(min(iw*720/ih\,1280)/2)*2:-2,
fade=t=in:st=0:d=1.3200000000000003:alpha=1,
fade=t=out:st=7.920000000000001:d=10.0:alpha=0,
setpts=PTS-STARTPTS+47.52/TB[9v];

zoompan is generating output at 25 fps. so you'll want to add fps=30 to see smoother zoom in the output.

The main issue is that the start time of the last image. It's 47.52 s and at present, 237 frames at 25 fps generates 10.48 s, leading to a 58 s output. After changing to 30 fps, each zoompan filter will generate 7.9 s stream. If that is desired, then for the last image, starting PTS should be 53.1 and all other starting timestamps should be adjusted as well.

The fade filter can only operate effectively if the stream is long enough. Take fade=t=out:st=7.920000000000001:d=10.0. Since, at present, each image is 237/25 = 10.48 s long, fade out will start at 7.92s and continue upto 10.48s and then end with a partial fade-out. Don't think this is what you want. st+d should be equal to or less than stream duration.

Finally, you don't need to set format twice, and since overlay defaults to yuv420p output, it's not needed at all.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.