0

I have two .webm videos which I'm trying to process into a horizontally stacked single video with one video on the left and one video on the right.

The individual video files are called left.webm and right.webm. They both have perfect audio/video sync when I play them. I'm using the following command to do the hstack:

/usr/bin/ffmpeg -y -i left.webm -i right.webm -filter_complex "[0:v]setpts=PTS-STARTPTS[left];[1:v]setpts=PTS-STARTPTS[right];[left][right]hstack=inputs=2[v];[a:0][1:a]amerge[a]" -map "[v]" -map "[a]" -ac 2 merged.webm -loglevel error

The command works fine, but I have a synchronisation issue between the audio and the video on the merged result.

Reading through some SE posts (audio sync appears to be quite a common issue) lead me to using ffprobe to try and understand the parameters of each of the left and right videos and to look for anomalies like frame rate differences.

ffprobe produces this for left.webm:

Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv), 320x240, SAR 1:1 DAR 4:3, 1k fps, 1k tbr, 1k tbn, 1k tbc (default)
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp

And right.webm:

Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv), 320x240, SAR 1:1 DAR 4:3, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)

I noticed that left.webm has an fps of 1k, so I tried transcoding that down to 30fps to match right.webm with:

ffmpeg -i left.webm -r 30 -vcodec copy left2.webm

But I still have the audio synchronisation (it's about 200-300ms off, at my best guess) in the resulting merged video when I do the hstack filter.

I'm not sure what to check next given both videos appear to have identical parameters and play perfectly individually.

Any thoughts on what to look at or try next would be gratefully received!

Thanks.

ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 left.webm
0.438000
0.439000
0.439000
0.459000
0.487000
0.516000
0.547000
0.579000
0.610000
0.642000
0.675000
0.708000
0.742000
0.774000
0.807000
0.840000
0.873000
0.906000
0.907000
0.940000
0.973000
1.006000
1.039000
1.072000
1.105000
1.139000
1.205000
1.272000
1.338000
1.339000
1.405000
1.471000
1.538000
1.604000
1.671000
1.738000
1.804000
1.871000
1.872000
1.904000
1.937000
1.971000
2.004000
2.037000
2.071000
2.104000
2.137000
2.138000
2.170000
2.204000
2.237000
2.270000
2.304000
2.337000
2.370000
2.404000
2.437000
2.470000
2.471000
2.504000
2.537000
2.570000
2.604000
2.637000
2.670000
2.704000
2.737000
2.770000
2.803000
2.804000
2.837000
2.870000
2.903000
2.937000
2.970000
3.003000
3.037000
3.070000
3.103000
3.137000
3.138000
3.170000
3.203000
3.237000
3.270000
3.303000
3.337000
3.370000
3.403000
3.437000
3.470000
3.471000
3.503000
3.537000
3.570000
3.603000
3.636000
3.670000
3.703000
3.736000
3.770000
3.803000
3.804000
3.836000
3.870000
3.903000
3.936000
3.970000
4.003000
4.036000
4.070000
4.103000
4.136000
4.137000
4.170000
4.203000
4.236000
4.270000
4.303000
4.336000
4.369000
4.403000
4.436000
4.469000
4.470000
4.503000
4.536000
4.569000
4.603000
4.636000
4.669000
4.703000
4.736000
4.769000
4.803000
4.804000
4.836000
4.869000
4.902000
4.936000
4.969000

ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 right.webm
0.007000
0.040000
0.040000
0.074000
0.107000
0.140000
0.174000
0.207000
0.240000
0.274000
0.307000
0.340000
0.373000
0.374000
0.407000
0.440000
0.474000
0.507000
0.540000
0.574000
0.607000
0.640000
0.674000
0.707000
0.707000
0.740000
0.774000
0.807000
0.840000
0.874000
0.907000
0.940000
0.974000
1.007000
1.040000
1.040000
1.074000
1.107000
1.140000
1.174000
1.207000
1.240000
1.274000
1.307000
1.340000
1.373000
1.374000
1.407000
1.440000
1.474000
1.507000
1.540000
1.574000
1.607000
1.640000
1.674000
1.707000
1.707000
1.740000
1.774000
1.807000
1.840000
1.874000
1.907000
1.940000
1.974000
2.007000
2.040000
2.040000
2.074000
2.107000
2.140000
2.174000
2.207000
2.240000
2.274000
2.307000
2.340000
2.373000
2.374000
2.407000
2.440000
2.474000
2.507000
2.540000
2.574000
2.607000
2.640000
2.674000
2.707000
2.707000
2.740000
2.774000
2.807000
2.840000
2.874000
2.907000
2.940000
2.974000
3.007000
3.040000
3.040000
3.074000
3.107000
3.140000
3.174000
3.207000
3.240000
3.274000
3.307000
3.340000
3.373000
3.374000
3.407000
3.440000
3.474000
3.507000
3.540000
3.574000
3.607000
3.640000
3.674000
3.707000
3.707000
3.740000
3.774000
3.807000
3.840000
3.874000
3.907000
3.940000
3.974000
4.007000
4.040000
4.040000
4.074000
4.107000
4.140000
4.174000
4.207000
4.240000
4.274000
4.307000
4.340000
4.373000
4.374000
4.407000
4.440000
4.474000
4.507000
4.540000
4.574000
4.607000
4.640000
4.674000
4.707000
4.707000
4.740000
4.774000
4.807000
4.840000
4.874000
4.907000
4.940000
4.974000

ffprobe -show_entries packet=pts_time -select_streams a -read_intervals %5 -of compact=p=0:nk=1 -v 0 left.webm
-0.007000
0.014000
0.034000
0.054000
0.074000
0.094000
0.114000
0.134000
0.154000
0.174000
0.194000
0.214000
0.234000
0.254000
0.274000
0.294000
0.314000
0.334000
0.354000
0.374000
0.394000
0.414000
0.434000
0.454000
0.474000
0.494000
0.514000
0.534000
0.554000
0.574000
0.594000
0.614000
0.634000
0.654000
0.674000
0.694000
0.714000
0.734000
0.754000
0.774000
0.794000
0.814000
0.834000
0.854000
0.874000
0.894000
0.914000
0.934000
0.954000
0.974000
0.994000
1.014000
1.034000
1.054000
1.074000
1.094000
1.114000
1.134000
1.154000
1.174000
1.194000
1.214000
1.234000
1.254000
1.274000
1.294000
1.314000
1.334000
1.354000
1.374000
1.394000
1.414000
1.434000
1.454000
1.474000
1.494000
1.514000
1.534000
1.554000
1.574000
1.594000
1.614000
1.634000
1.654000
1.674000
1.694000
1.714000
1.734000
1.754000
1.774000
1.794000
1.814000
1.834000
1.854000
1.874000
1.894000
1.914000
1.934000
1.954000
1.974000
1.994000
2.014000
2.034000
2.054000
2.074000
2.094000
2.114000
2.134000
2.154000
2.174000
2.194000
2.214000
2.234000
2.254000
2.274000
2.294000
2.314000
2.334000
2.354000
2.374000
2.394000
2.414000
2.434000
2.454000
2.474000
2.494000
2.514000
2.534000
2.554000
2.574000
2.594000
2.614000
2.634000
2.654000
2.674000
2.694000
2.714000
2.734000
2.754000
2.774000
2.794000
2.814000
2.834000
2.854000
2.874000
2.894000
2.914000
2.934000
2.954000
2.974000
2.994000
3.014000
3.034000
3.054000
3.074000
3.094000
3.114000
3.134000
3.154000
3.174000
3.194000
3.214000
3.234000
3.254000
3.274000
3.294000
3.314000
3.334000
3.354000
3.374000
3.394000
3.414000
3.434000
3.454000
3.474000
3.494000
3.514000
3.534000
3.554000
3.574000
3.594000
3.614000
3.634000
3.654000
3.674000
3.694000
3.714000
3.734000
3.754000
3.774000
3.794000
3.814000
3.834000
3.854000
3.874000
3.894000
3.914000
3.934000
3.954000
3.974000
3.994000
4.014000
4.034000
4.054000
4.074000
4.094000
4.114000
4.134000
4.154000
4.174000
4.194000
4.214000
4.234000
4.254000
4.274000
4.294000
4.314000
4.334000
4.354000
4.374000
4.394000
4.414000
4.434000
4.454000
4.474000
4.494000
4.514000
4.534000
4.554000
4.574000
4.594000
4.614000
4.634000
4.654000
4.674000
4.694000
4.714000
4.734000
4.754000
4.774000
4.794000
4.814000
4.834000
4.854000
4.874000
4.894000
4.914000
4.934000
4.954000
4.974000
4.994000

ffprobe -show_entries packet=pts_time -select_streams a -read_intervals %5 -of compact=p=0:nk=1 -v 0 right.webm
-0.007000
0.014000
0.034000
0.054000
0.074000
0.094000
0.114000
0.134000
0.154000
0.174000
0.194000
0.214000
0.234000
0.254000
0.274000
0.294000
0.314000
0.334000
0.354000
0.374000
0.394000
0.414000
0.434000
0.454000
0.474000
0.494000
0.514000
0.534000
0.554000
0.574000
0.594000
0.614000
0.634000
0.654000
0.674000
0.694000
0.714000
0.734000
0.754000
0.774000
0.794000
0.814000
0.834000
0.854000
0.874000
0.894000
0.914000
0.934000
0.954000
0.974000
0.994000
1.014000
1.034000
1.054000
1.074000
1.094000
1.114000
1.134000
1.154000
1.174000
1.194000
1.214000
1.234000
1.254000
1.274000
1.294000
1.314000
1.334000
1.354000
1.374000
1.394000
1.414000
1.434000
1.454000
1.474000
1.494000
1.514000
1.534000
1.554000
1.574000
1.594000
1.614000
1.634000
1.654000
1.674000
1.694000
1.714000
1.734000
1.754000
1.774000
1.794000
1.814000
1.834000
1.854000
1.874000
1.894000
1.914000
1.934000
1.954000
1.974000
1.994000
2.014000
2.034000
2.054000
2.074000
2.094000
2.114000
2.134000
2.154000
2.174000
2.194000
2.214000
2.234000
2.254000
2.274000
2.294000
2.314000
2.334000
2.354000
2.374000
2.394000
2.414000
2.434000
2.454000
2.474000
2.494000
2.514000
2.534000
2.554000
2.574000
2.594000
2.614000
2.634000
2.654000
2.674000
2.694000
2.714000
2.734000
2.754000
2.774000
2.794000
2.814000
2.834000
2.854000
2.874000
2.894000
2.914000
2.934000
2.954000
2.974000
2.994000
3.014000
3.034000
3.054000
3.074000
3.094000
3.114000
3.134000
3.154000
3.174000
3.194000
3.214000
3.234000
3.254000
3.274000
3.294000
3.314000
3.334000
3.354000
3.374000
3.394000
3.414000
3.434000
3.454000
3.474000
3.494000
3.514000
3.534000
3.554000
3.574000
3.594000
3.614000
3.634000
3.654000
3.674000
3.694000
3.714000
3.734000
3.754000
3.774000
3.794000
3.814000
3.834000
3.854000
3.874000
3.894000
3.914000
3.934000
3.954000
3.974000
3.994000
4.014000
4.034000
4.054000
4.074000
4.094000
4.114000
4.134000
4.154000
4.174000
4.194000
4.214000
4.234000
4.254000
4.274000
4.294000
4.314000
4.334000
4.354000
4.374000
4.394000
4.414000
4.434000
4.454000
4.474000
4.494000
4.514000
4.534000
4.554000
4.574000
4.594000
4.614000
4.634000
4.654000
4.674000
4.694000
4.714000
4.734000
4.754000
4.774000
4.794000
4.814000
4.834000
4.854000
4.874000
4.894000
4.914000
4.934000
4.954000
4.974000
4.994000
  • Can you share the files? – Gyan Jul 20 '17 at 9:55
  • Hi Mulvya - that would be tricky as it's two people talking in a video conference (confidentiality etc). I can see if I can produce a couple of test videos to use. Is there something in particular you would look at in the two videos that I could look at and report on? – trillianjedi Jul 20 '17 at 10:19
  • The timestamps for the audio and video in both the files. Run and capture output of ffprobe -show_entries packet=pts_time -select_streams v -read_intervals %5 -of compact=p=0:nk=1 -v 0 file.webm. Run same command with v changed to a. Do this for both files. – Gyan Jul 20 '17 at 10:37
  • OK, thanks for the help. Output is <too long for SE - will past up somewhere else and link> – trillianjedi Jul 20 '17 at 12:24
  • I have edited the main post and added the output there. Thanks Mulvya. – trillianjedi Jul 20 '17 at 12:32
0

For files with differing start times of video and audio streams, the PTS values should not be changed else sync will be lost.

So,

ffmpeg -y -i left.webm -i right.webm -filter_complex "[0:v][1:v]hstack=inputs=2[v];[a:0][1:a]amerge[a]" -map "[v]" -map "[a]" -ac 2 merged.webm
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.