Merging two videos with hstack and play them back sequentially

Question

I am merging two mp4 files using hstack. I am using amix to merge the audio. Everything works perfectly.

Now I want that the hstacked videos should play sequentially, that is one after the other. Here is what I have done

ffmpeg -i left.mp4 -i right.mp4 -filter_complex [0:v]setpts=PTS-STARTPTS[l]; [1:v]setpts=PTS-STARTPTS+14/TB[r]; [l][r]hstack=inputs=2[stacked]; [0:a][1:a]amix=inputs=2[a] -map [stacked] -map [a] -c:a aac -preset superfast result.mp4

In the above example, I tried to delay the play back of the video on the right by 14 seconds (which happens to be the length of the left video). In essence I want the left video to play first and after it finishes the left video should play.

What I am getting instead from the command is, the left video remains still for 14 seconds with only displaying the first frame (the audio plays) and after 14 seconds the right video plays.

I am at my wits end trying to figure it out. What am I doing wrong ?

Answer 1

The h/vstack filters are designed to pair frames so they will hold the first frame of an input if the other input isn't active. Use the tpad filter to add blank frames for the required duration.

ffmpeg -i left.mp4 -i right.mp4 -filter_complex [0:v]setpts=PTS-STARTPTS[l]; [1:v]setpts=PTS-STARTPTS,tpad=start_duration=14:start_mode=add:color=black[r]; [l][r]hstack=inputs=2[stacked]; [0:a][1:a]amix=inputs=2[a] -map [stacked] -map [a] -c:a aac -preset superfast result.mp4

If you don't have the tpad filter available, a slightly longer approach is to use the drawbox filter with ffmpeg 4.0+.

ffmpeg -i left.mp4 -i right.mp4 -filter_complex [0:v]setpts=PTS-STARTPTS[l]; [1:v]setpts=PTS-STARTPTS,split[rb][rv];[rb]trim=0:14,drawbox=t=fill:c=black[rb];[rb][rv]concat=n=2:v=1:a0[r]; [l][r]hstack=inputs=2[stacked]; [0:a][1:a]amix=inputs=2[a] -map [stacked] -map [a] -c:a aac -preset superfast result.mp4