Currently, I'm trying to create a live streaming service for internal purposes of my company; The scenario of work is pretty simple:

  1. We are receiving the RTMP stream from the user(over the internet) to Nginx with the nginx-rtmp module.
  2. Internal service(on a different host) starts the ffmpeg process, which subscribes to nginx-rtmp and transcodes the stream on the fly.

Most of the time everything works ok, but sometimes we are receiving problems with unsync audio/video.

But sometimes during the stream, the problem of video/audio sync occurs: the moves of the speaker's lips and gestures do not match with the audio of the speech.

I've tried to record stream via rtmpdump: the dump is ok; the gap between audio and video seems like worsening during the transcoding. My suspicion is on ffmpeg parameters

ffmpeg logs do not show anything unusual: only start/progress logs;

FFmpeg version is 4.2.4

My current ffmpeg setup:

ffmpeg \

-xerror -rtbufsize 50M -analyzeduration 5000000 -hide_banner -noautorotate -progress - -f live_flv -threads 1 -hwaccel cuda -hwaccel_output_format cuda -hwaccel_device 1 \

-i rtmp://nginx-rtmp.internal/source \

-map 0:v:0 -map 0:a:0? -profile:v main -vf scale_npp=256:144:interp_algo=lanczos -c:v h264_nvenc -preset hp -async 1 -vsync 1 -r 25 -g 100 -minrate 20k -maxrate 200k -b:v 160k -bufsize 100k -c:a libfdk_aac -b:a 128k -f hls -hls_segment_filename 144p/segment_%Y%m%d_%H%M%S-%%04d.ts -strftime_mkdir 1 -hls_time 4 -hls_list_size 900 -hls_flags second_level_segment_index+split_by_time   -strftime 1 144p_stream.m3u8 \

-map 0:v:0 -map 0:a:0? -c:v h264_nvenc -preset hp -profile:v main -vf scale_npp=416:240:interp_algo=lanczos -async 1 -vsync 1 -r 25 -g 100 -minrate 40k -maxrate 500k -b:v 400k -bufsize 250k -c:a libfdk_aac -b:a 128k -f hls -hls_segment_filename 240p/segment_%Y%m%d_%H%M%S-%%04d.ts -hls_time 4 -hls_list_size 900 -hls_flags second_level_segment_index+split_by_time -strftime 1 -strftime_mkdir 1 240p_stream.m3u8 \

-map 0:v:0 -map 0:a:0? -c:v h264_nvenc -preset hp -profile:v main -vf scale_npp=640:360:interp_algo=lanczos -async 1 -vsync 1 -r 25 -g 100 -minrate 100k -maxrate 1200k -b:v 1000k -bufsize 500k -c:a libfdk_aac -b:a 128k -f hls -hls_segment_filename 360p/segment_%Y%m%d_%H%M%S-%%04d.ts -strftime 1 -strftime_mkdir 1  -hls_time 4 -hls_list_size 900 -hls_flags second_level_segment_index+split_by_time 360p_stream.m3u8 \

-map 0:v:0 -map 0:a:0? -c:v h264_nvenc -preset hq -profile:v high -vf scale_npp=1280:720:interp_algo=lanczos -async 1 -vsync 1 -r 25 -g 100 -minrate 1000k -maxrate 4500k -b:v 4000k -bufsize 2000k -c:a libfdk_aac -b:a 128k -f hls -hls_segment_filename 720p/segment_%Y%m%d_%H%M%S-%%04d.ts -strftime 1 -strftime_mkdir 1 -hls_time 4 -hls_list_size 900 -hls_flags second_level_segment_index+split_by_time 720p_stream.m3u8 \

-map 0:v:0 -map 0:a:0? -preset hq -profile:v high -c:v h264_nvenc -async 1 -vsync 1 -r 25 -g 100 -minrate 1500k -maxrate 7500k -b:v 6800k -bufsize 2500k -c:a libfdk_aac -b:a 128k -f hls -hls_segment_filename 1080p/segment_%Y%m%d_%H%M%S-%%04d.ts -strftime 1 -strftime_mkdir 1 -hls_time 4 -hls_list_size 900 -hls_flags second_level_segment_index+source_namesplit_by_time 1080p_stream.m3u8

Yes, there is a lot of resolutions.

The problem is unstable(most of the time everything is ok) but very serious and hard to debug.

I'm not a professional in video transcoding, so my question is:

What can cause the problem of video/audio unsync, what would you suggest to check(i really have no more ideas of how to reproduce this bug more or less stable);

Do you have any advice on ffmpeg parameters?

Thank you

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.