1

I'm trying to combine (side by side) two live video streams coming over RTMP, using the following ffmpeg command:

ffmpeg -i "rtmp://first" -i "rtmp://first" -filter_complex "[0v][1v]xstack=inputs=2:layout=0_0|1920_0[stacked]" -map "[stacked]" -preset ultrafast -vcodec libx264 -tune zerolatency -an -f flv output.flv

In this example I actually use the same input stream two times, because the issue is more visible this way. And the issue is in the output two streams are out of sync by about 2-3 seconds. That is - I expect (since I have two identical inputs) to have exactly the same left and right sides in the output. Instead - left side is behind right side by 2-3 seconds.

What I believe is happening is ffmpeg connects to inputs in order (I see this in output log) and connection to each one takes 2-3 seconds (maybe it waits for I-frame, those streams have I-frame interval of 3 seconds). Then probably, it buffers frames received from first (already connected) input, while connecting to the second one. When second one is connected and frames from both inputs are ready to be put through the filter - first input buffer already contains 2-3 seconds of video - and result is out of sync.

Again, that's just my assumptions. So, how can achieve my goal? What I basically want is for ffmpeg to discard all "old" frames received before BOTH inputs are connected OR somehow put "empty" (black?) frames for second input, while waiting for that second input to become available. I tried play with various flags, with PTS (setpts filter), but to no avail.

  • Can you expose or share one of these URLs? – Gyan Mar 6 at 6:49
  • @Gyan I cannot share such url, but it's produced by streaming my desktop via OBS Studio to rtmp endpoint on server (which is private, so I cannot share it) and consuming that endpoint. Server doesn't do any processing of that RTMP stream, so I think it can be reproduced. Same happens when I stream my camera (or any other "live" source). I consume the same stream 2 times in my example in question, so that synchronization issue is very visible. – Evk Mar 9 at 7:55
  • I'm trying to do something similar and have tried various combinations of -vsync, -max_delay, -copyts and -start_at_zero but not managed to synchronise my two live streams yet. – IpsRich Apr 30 at 11:37
  • @RichardWiseman yes, I've given up on this and had to manually mix frames using C++ (still with the help of ffmpeg, but not with command-line utility). – Evk Apr 30 at 12:14
1

I got it working by adding -fflags nobuffer -flags low_delay -strict experimental before all the inputs. Got helped reading this thread

| improve this answer | |
0

-vsync parameter

Video sync method.

  • 0: Each frame is passed with its timestamp from the demuxer to the muxer
  • 1: Frames will be duplicated and dropped to achieve exactly the requested constant framerate.
  • 2: Frames are passed through with their timestamp or dropped to prevent 2 frames from having the same timestamp
  • -1: Chooses between 1 and 2 depending on muxer capabilities. This is the default method.

With -map you can select from which stream the timestamps should be taken. You can leave either video or audio unchanged and sync the remaining stream(s) to the unchanged one.

-async samples_per_second

Audio sync method. "Stretches/squeezes" the audio stream to match the timestamps, the parameter is the maximum samples per second by which the audio is changed. -async 1 is a special case where only the start of the audio stream is corrected without any later correction.

-copyts

Copy timestamps from input to output.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.