I'm trying to combine (side by side) two live video streams coming over RTMP, using the following ffmpeg command:
ffmpeg -i "rtmp://first" -i "rtmp://first" -filter_complex "[0v][1v]xstack=inputs=2:layout=0_0|1920_0[stacked]" -map "[stacked]" -preset ultrafast -vcodec libx264 -tune zerolatency -an -f flv output.flv
In this example I actually use the same input stream two times, because the issue is more visible this way. And the issue is in the output two streams are out of sync by about 2-3 seconds. That is - I expect (since I have two identical inputs) to have exactly the same left and right sides in the output. Instead - left side is behind right side by 2-3 seconds.
What I believe is happening is ffmpeg connects to inputs in order (I see this in output log) and connection to each one takes 2-3 seconds (maybe it waits for I-frame, those streams have I-frame interval of 3 seconds). Then probably, it buffers frames received from first (already connected) input, while connecting to the second one. When second one is connected and frames from both inputs are ready to be put through the filter - first input buffer already contains 2-3 seconds of video - and result is out of sync.
Again, that's just my assumptions. So, how can achieve my goal? What I basically want is for ffmpeg to discard all "old" frames received before BOTH inputs are connected OR somehow put "empty" (black?) frames for second input, while waiting for that second input to become available. I tried play with various flags, with PTS (setpts filter), but to no avail.