0

I have two .webm videos which I'm trying to process into a horizontally stacked single video with one video on the left and one video on the right.

The individual video files are called left.webm and right.webm. They both have perfect audio/video sync when I play them. I'm using the following command to do the hstack:

/usr/bin/ffmpeg -y -i left.webm -i right.webm -filter_complex "[0:v]setpts=PTS-STARTPTS[left];[1:v]setpts=PTS-STARTPTS[right];[left][right]hstack=inputs=2[v];[a:0][1:a]amerge[a]" -map "[v]" -map "[a]" -ac 2 merged.webm -loglevel error

The command works fine, but I have a synchronisation issue between the audio and the video on the merged result.

Reading through some SE posts (audio sync appears to be quite a common issue) lead me to using ffprobe to try and understand the parameters of each of the left and right videos and to look for anomalies like frame rate differences.

ffprobe produces this for left.webm:

Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv), 320x240, SAR 1:1 DAR 4:3, 1k fps, 1k tbr, 1k tbn, 1k tbc (default)
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp

And right.webm:

Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv), 320x240, SAR 1:1 DAR 4:3, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)

I noticed that left.webm has an fps of 1k, so I tried transcoding that down to 30fps to match right.webm with:

ffmpeg -i left.webm -r 30 -vcodec copy left2.webm

But I still have the audio synchronisation (it's about 200-300ms off, at my best guess) in the resulting merged video when I do the hstack filter.

I'm not sure what to check next given both videos appear to have identical parameters and play perfectly individually.

Any thoughts on what to look at or try next would be gratefully received!

Thanks.

ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 left.webm
0.438000
0.439000
0.439000
0.459000
0.487000
0.516000
0.547000
0.579000
0.610000
0.642000
0.675000
0.708000
0.742000
0.774000
0.807000
0.840000
0.873000
0.906000
0.907000
0.940000
0.973000
1.006000
1.039000
1.072000
1.105000
1.139000
1.205000
1.272000
1.338000
1.339000
1.405000
1.471000
1.538000
1.604000
1.671000
1.738000
1.804000
1.871000
1.872000
1.904000
1.937000
1.971000
2.004000
2.037000
2.071000
2.104000
2.137000
2.138000
2.170000
2.204000
2.237000
2.270000
2.304000
2.337000
2.370000
2.404000
2.437000
2.470000
2.471000
2.504000
2.537000
2.570000
2.604000
2.637000
2.670000
2.704000
2.737000
2.770000
2.803000
2.804000
2.837000
2.870000
2.903000
2.937000
2.970000
3.003000
3.037000
3.070000
3.103000
3.137000
3.138000
3.170000
3.203000
3.237000
3.270000
3.303000
3.337000
3.370000
3.403000
3.437000
3.470000
3.471000
3.503000
3.537000
3.570000
3.603000
3.636000
3.670000
3.703000
3.736000
3.770000
3.803000
3.804000
3.836000
3.870000
3.903000
3.936000
3.970000
4.003000
4.036000
4.070000
4.103000
4.136000
4.137000
4.170000
4.203000
4.236000
4.270000
4.303000
4.336000
4.369000
4.403000
4.436000
4.469000
4.470000
4.503000
4.536000
4.569000
4.603000
4.636000
4.669000
4.703000
4.736000
4.769000
4.803000
4.804000
4.836000
4.869000
4.902000
4.936000
4.969000

ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 right.webm
0.007000
0.040000
0.040000
0.074000
0.107000
0.140000
0.174000
0.207000
0.240000
0.274000
0.307000
0.340000
0.373000
0.374000
0.407000
0.440000
0.474000
0.507000
0.540000
0.574000
0.607000
0.640000
0.674000
0.707000
0.707000
0.740000
0.774000
0.807000
0.840000
0.874000
0.907000
0.940000
0.974000
1.007000
1.040000
1.040000
1.074000
1.107000
1.140000
1.174000
1.207000
1.240000
1.274000
1.307000
1.340000
1.373000
1.374000
1.407000
1.440000
1.474000
1.507000
1.540000
1.574000
1.607000
1.640000
1.674000
1.707000
1.707000
1.740000
1.774000
1.807000
1.840000
1.874000
1.907000
1.940000
1.974000
2.007000
2.040000
2.040000
2.074000
2.107000
2.140000
2.174000
2.207000
2.240000
2.274000
2.307000
2.340000
2.373000
2.374000
2.407000
2.440000
2.474000
2.507000
2.540000
2.574000
2.607000
2.640000
2.674000
2.707000
2.707000
2.740000
2.774000
2.807000
2.840000
2.874000
2.907000
2.940000
2.974000
3.007000
3.040000
3.040000
3.074000
3.107000
3.140000
3.174000
3.207000
3.240000
3.274000
3.307000
3.340000
3.373000
3.374000
3.407000
3.440000
3.474000
3.507000
3.540000
3.574000
3.607000
3.640000
3.674000
3.707000
3.707000
3.740000
3.774000
3.807000
3.840000
3.874000
3.907000
3.940000
3.974000
4.007000
4.040000
4.040000
4.074000
4.107000
4.140000
4.174000
4.207000
4.240000
4.274000
4.307000
4.340000
4.373000
4.374000
4.407000
4.440000
4.474000
4.507000
4.540000
4.574000
4.607000
4.640000
4.674000
4.707000
4.707000
4.740000
4.774000
4.807000
4.840000
4.874000
4.907000
4.940000
4.974000

ffprobe -show_entries packet=pts_time -select_streams a -read_intervals %5 -of compact=p=0:nk=1 -v 0 left.webm
-0.007000
0.014000
0.034000
0.054000
0.074000
0.094000
0.114000
0.134000
0.154000
0.174000
0.194000
0.214000
0.234000
0.254000
0.274000
0.294000
0.314000
0.334000
0.354000
0.374000
0.394000
0.414000
0.434000
0.454000
0.474000
0.494000
0.514000
0.534000
0.554000
0.574000
0.594000
0.614000
0.634000
0.654000
0.674000
0.694000
0.714000
0.734000
0.754000
0.774000
0.794000
0.814000
0.834000
0.854000
0.874000
0.894000
0.914000
0.934000
0.954000
0.974000
0.994000
1.014000
1.034000
1.054000
1.074000
1.094000
1.114000
1.134000
1.154000
1.174000
1.194000
1.214000
1.234000
1.254000
1.274000
1.294000
1.314000
1.334000
1.354000
1.374000
1.394000
1.414000
1.434000
1.454000
1.474000
1.494000
1.514000
1.534000
1.554000
1.574000
1.594000
1.614000
1.634000
1.654000
1.674000
1.694000
1.714000
1.734000
1.754000
1.774000
1.794000
1.814000
1.834000
1.854000
1.874000
1.894000
1.914000
1.934000
1.954000
1.974000
1.994000
2.014000
2.034000
2.054000
2.074000
2.094000
2.114000
2.134000
2.154000
2.174000
2.194000
2.214000
2.234000
2.254000
2.274000
2.294000
2.314000
2.334000
2.354000
2.374000
2.394000
2.414000
2.434000
2.454000
2.474000
2.494000
2.514000
2.534000
2.554000
2.574000
2.594000
2.614000
2.634000
2.654000
2.674000
2.694000
2.714000
2.734000
2.754000
2.774000
2.794000
2.814000
2.834000
2.854000
2.874000
2.894000
2.914000
2.934000
2.954000
2.974000
2.994000
3.014000
3.034000
3.054000
3.074000
3.094000
3.114000
3.134000
3.154000
3.174000
3.194000
3.214000
3.234000
3.254000
3.274000
3.294000
3.314000
3.334000
3.354000
3.374000
3.394000
3.414000
3.434000
3.454000
3.474000
3.494000
3.514000
3.534000
3.554000
3.574000
3.594000
3.614000
3.634000
3.654000
3.674000
3.694000
3.714000
3.734000
3.754000
3.774000
3.794000
3.814000
3.834000
3.854000
3.874000
3.894000
3.914000
3.934000
3.954000
3.974000
3.994000
4.014000
4.034000
4.054000
4.074000
4.094000
4.114000
4.134000
4.154000
4.174000
4.194000
4.214000
4.234000
4.254000
4.274000
4.294000
4.314000
4.334000
4.354000
4.374000
4.394000
4.414000
4.434000
4.454000
4.474000
4.494000
4.514000
4.534000
4.554000
4.574000
4.594000
4.614000
4.634000
4.654000
4.674000
4.694000
4.714000
4.734000
4.754000
4.774000
4.794000
4.814000
4.834000
4.854000
4.874000
4.894000
4.914000
4.934000
4.954000
4.974000
4.994000

ffprobe -show_entries packet=pts_time -select_streams a -read_intervals %5 -of compact=p=0:nk=1 -v 0 right.webm
-0.007000
0.014000
0.034000
0.054000
0.074000
0.094000
0.114000
0.134000
0.154000
0.174000
0.194000
0.214000
0.234000
0.254000
0.274000
0.294000
0.314000
0.334000
0.354000
0.374000
0.394000
0.414000
0.434000
0.454000
0.474000
0.494000
0.514000
0.534000
0.554000
0.574000
0.594000
0.614000
0.634000
0.654000
0.674000
0.694000
0.714000
0.734000
0.754000
0.774000
0.794000
0.814000
0.834000
0.854000
0.874000
0.894000
0.914000
0.934000
0.954000
0.974000
0.994000
1.014000
1.034000
1.054000
1.074000
1.094000
1.114000
1.134000
1.154000
1.174000
1.194000
1.214000
1.234000
1.254000
1.274000
1.294000
1.314000
1.334000
1.354000
1.374000
1.394000
1.414000
1.434000
1.454000
1.474000
1.494000
1.514000
1.534000
1.554000
1.574000
1.594000
1.614000
1.634000
1.654000
1.674000
1.694000
1.714000
1.734000
1.754000
1.774000
1.794000
1.814000
1.834000
1.854000
1.874000
1.894000
1.914000
1.934000
1.954000
1.974000
1.994000
2.014000
2.034000
2.054000
2.074000
2.094000
2.114000
2.134000
2.154000
2.174000
2.194000
2.214000
2.234000
2.254000
2.274000
2.294000
2.314000
2.334000
2.354000
2.374000
2.394000
2.414000
2.434000
2.454000
2.474000
2.494000
2.514000
2.534000
2.554000
2.574000
2.594000
2.614000
2.634000
2.654000
2.674000
2.694000
2.714000
2.734000
2.754000
2.774000
2.794000
2.814000
2.834000
2.854000
2.874000
2.894000
2.914000
2.934000
2.954000
2.974000
2.994000
3.014000
3.034000
3.054000
3.074000
3.094000
3.114000
3.134000
3.154000
3.174000
3.194000
3.214000
3.234000
3.254000
3.274000
3.294000
3.314000
3.334000
3.354000
3.374000
3.394000
3.414000
3.434000
3.454000
3.474000
3.494000
3.514000
3.534000
3.554000
3.574000
3.594000
3.614000
3.634000
3.654000
3.674000
3.694000
3.714000
3.734000
3.754000
3.774000
3.794000
3.814000
3.834000
3.854000
3.874000
3.894000
3.914000
3.934000
3.954000
3.974000
3.994000
4.014000
4.034000
4.054000
4.074000
4.094000
4.114000
4.134000
4.154000
4.174000
4.194000
4.214000
4.234000
4.254000
4.274000
4.294000
4.314000
4.334000
4.354000
4.374000
4.394000
4.414000
4.434000
4.454000
4.474000
4.494000
4.514000
4.534000
4.554000
4.574000
4.594000
4.614000
4.634000
4.654000
4.674000
4.694000
4.714000
4.734000
4.754000
4.774000
4.794000
4.814000
4.834000
4.854000
4.874000
4.894000
4.914000
4.934000
4.954000
4.974000
4.994000
Improve this question
11
  • Can you share the files?
    – Gyan
    Commented Jul 20, 2017 at 9:55
  • Hi Mulvya - that would be tricky as it's two people talking in a video conference (confidentiality etc). I can see if I can produce a couple of test videos to use. Is there something in particular you would look at in the two videos that I could look at and report on?
    – trillianjedi
    Commented Jul 20, 2017 at 10:19
  • The timestamps for the audio and video in both the files. Run and capture output of ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 file.webm. Run same command with v changed to a. Do this for both files.
    – Gyan
    Commented Jul 20, 2017 at 10:37
  • OK, thanks for the help. Output is <too long for SE - will past up somewhere else and link>
    – trillianjedi
    Commented Jul 20, 2017 at 12:24
  • I have edited the main post and added the output there. Thanks Mulvya.
    – trillianjedi
    Commented Jul 20, 2017 at 12:32

1 Answer 1

Reset to default
0

For files with differing start times of video and audio streams, the PTS values should not be changed else sync will be lost.

So,

ffmpeg -y -i left.webm -i right.webm -filter_complex "[0:v][1:v]hstack=inputs=2[v];[a:0][1:a]amerge[a]" -map "[v]" -map "[a]" -ac 2 merged.webm
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.