1

Why video call has no (even if there is, it is lower than human perception can discern) latency, but no matter what options of ffmpeg I tried to transmit video to a remote server, the lowest I can get is ~0.2s latency (estimated, not sure how to profile)?

What is the technology behind video call that allows it to be much faster than ffmpeg?

I tried sending to the server and make the server send back the stream using the following commands:

CLIENT:

termimal 1-

ffmpeg -pixel_format mjpeg -s hd720 -r 30 -i /dev/video1 -c libx264 -preset ultrafast -tune zerolatency -crf 18 -b 5M -f matroska -vf hflip tcp://X:5000

termimal 2-

mplayer -benchmark ffmpeg://tcp://Y:5000?listen

SERVER:

ffmpeg -i tcp://X:5000 -c copy -f matroska tcp://Y:5000

However, the video I got back has a slight delay of about 0.5s (estimated, not sure how to profile). Is sending to and fro different that causes it to have longer delay?

8
  • Try skipping format probe in server cmd: ffmpeg -f matroska -i tcp://X:5000 -c copy -f matroska tcp://Y:5000
    – Gyan
    Apr 17, 2018 at 12:37
  • What software is your video call made with, and who are you calling? Recipient on server X, I assume?
    – Tony Sepia
    Apr 17, 2018 at 12:56
  • @Gyan same latency Apr 17, 2018 at 16:01
  • @TonySepia facetime between mac Apr 17, 2018 at 16:02
  • Have you looked at using RTP with ffmpeg? FaceTIme appears to use it. Check here: en.wikipedia.org/wiki/FaceTime
    – Tony Sepia
    Apr 17, 2018 at 16:04

3 Answers 3

1
+50

Try using UDP. It should be faster than TCP but with possibility of losing paackets.

0

Your ffmpeg command is very strange at best:

  1. Using hflip filter: you need fairly recent ffmpeg, otherwise it will add additional processing overhead.
  2. Conflicting encoding options, you should use only preset for h264.

Some other remarks: have you benchmarked your encoding command without network transfer, how fast it is, 1x or 10x realtime?

1
  • Which is conflicting with which for 2? How should I benchmark? Apr 18, 2018 at 14:32
0

Sources of any delays:

  1. TCP protocol. Long handshake, possible error with time loss in future
  2. Long encoder initialization (video call already prepared it)
  3. Long decoder initialization (video call already prepared it), buffering for analyze data
  4. Unoptimized protocol / format for low-time communication, like MKV
  5. Prebuffering in encoder for B-frames and other optimizations

Video conferencing software also have many tricks to detect delays and resync streams. It why in video conferencing has frame drops and it not suitable for transmitting high quality video.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.