Why video call has no (even if there is, it is lower than human perception can discern) latency, but no matter what options of ffmpeg I tried to transmit video to a remote server, the lowest I can get is ~0.2s latency (estimated, not sure how to profile)?

What is the technology behind video call that allows it to be much faster than ffmpeg?

I tried sending to the server and make the server send back the stream using the following commands:


termimal 1-

ffmpeg -pixel_format mjpeg -s hd720 -r 30 -i /dev/video1 -c libx264 -preset ultrafast -tune zerolatency -crf 18 -b 5M -f matroska -vf hflip tcp://X:5000

termimal 2-

mplayer -benchmark ffmpeg://tcp://Y:5000?listen


ffmpeg -i tcp://X:5000 -c copy -f matroska tcp://Y:5000

However, the video I got back has a slight delay of about 0.5s (estimated, not sure how to profile). Is sending to and fro different that causes it to have longer delay?

  • Try skipping format probe in server cmd: ffmpeg -f matroska -i tcp://X:5000 -c copy -f matroska tcp://Y:5000 – Gyan Apr 17 '18 at 12:37
  • What software is your video call made with, and who are you calling? Recipient on server X, I assume? – Tony Sepia Apr 17 '18 at 12:56
  • @Gyan same latency – user10024395 Apr 17 '18 at 16:01
  • @TonySepia facetime between mac – user10024395 Apr 17 '18 at 16:02
  • Have you looked at using RTP with ffmpeg? FaceTIme appears to use it. Check here: en.wikipedia.org/wiki/FaceTime – Tony Sepia Apr 17 '18 at 16:04

Try using UDP. It should be faster than TCP but with possibility of losing paackets.


Your ffmpeg command is very strange at best:

  1. Using hflip filter: you need fairly recent ffmpeg, otherwise it will add additional processing overhead.
  2. Conflicting encoding options, you should use only preset for h264.

Some other remarks: have you benchmarked your encoding command without network transfer, how fast it is, 1x or 10x realtime?

  • Which is conflicting with which for 2? How should I benchmark? – user10024395 Apr 18 '18 at 14:32

Sources of any delays:

  1. TCP protocol. Long handshake, possible error with time loss in future
  2. Long encoder initialization (video call already prepared it)
  3. Long decoder initialization (video call already prepared it), buffering for analyze data
  4. Unoptimized protocol / format for low-time communication, like MKV
  5. Prebuffering in encoder for B-frames and other optimizations

Video conferencing software also have many tricks to detect delays and resync streams. It why in video conferencing has frame drops and it not suitable for transmitting high quality video.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.