FFmpeg hardware cuda decoding and overlay picture

Question

I can overlay a picture to the video stream with the following command:

ffmpeg -thread_queue_size 1024 -i udp_source -i watermark.png -filter_complex "[0:v]scale=1920:1080,yadif=0 [base]; [base][1:v] overlay=main_w-overlay_w-10:10 [marked]" -map "[marked]" -vcodec h264_nvenc -acodec libfdk_aac -ab 192k -ar 48000 -f flv rtmp_output

But if I made a hardware decoding with deinterlacing and resizing at the GPU with the following command:

ffmpeg -thread_queue_size 1024 -hwaccel cuvid -c:v mpeg2_cuvid -deint 2 -drop_second_field 1 -vsync 0 -i udp_source -i watermark.png -filter_complex "[0:v]scale_npp=1920:1080 [base]; [base][1:v] overlay=main_w-overlay_w-10:10 [marked]" -map "[marked]" -vcodec h264_nvenc -acodec libfdk_aac -ab 192k -ar 48000 -f flv rtmp_output

I got the following error:

Error while decoding stream #0:1: Invalid data found when processing input
Impossible to convert between the formats supported by the filter 'Parsed_scale_npp_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #1:0

As I understand we can't overlay the picture to the video via NVIDIA. I added the "hwdownload" filter in the command to copy data to CPU:

ffmpeg -thread_queue_size 1024 -hwaccel cuvid -c:v mpeg2_cuvid -deint 2 -drop_second_field 1 -vsync 0 -i "udp://226.58.24.4:2000?overrun_nonfatal=1" -i /home/anis/logo/2mbps.png -filter_complex "[0:v]scale_npp=1920:1080,hwdownload [base]; [base][1:v] overlay=main_w-overlay_w-10:10 [marked]" -map "[marked]" -vcodec h264_nvenc -aspect 16:9 -b:v 1808k -minrate 1808k -maxrate 1808k -bufsize 904k -g 100 -no-scenecut 1 -forced-idr 0 -preset:v medium -vprofile main -level 4.1 -map 0:#0x091d -acodec libfdk_aac -ab 192k -ar 48000 -f flv rtmp://10.150.12.104/virginandtv/abr2k_test

After this I got the following error:

[udp @ 0x39abda0] Circular buffer overrun. Surviving due to overrun_nonfatal option

I tried to increase a fifo size up to 1M, but this issue still exists.
Is it possible to overlay a picture directly on NVIDIA card?

Answer 1

For 1st question - Most likely you're looking at not applicable error (the default yuv420p is not usable here - you may also need to get newer ffmpeg) - while using the hwdownload you need to tell explicitly to the NPP scaler the output format of the video frames. For your case the below command should make the trick:

ffmpeg -thread_queue_size 1024 \
    -hwaccel cuvid -c:v mpeg2_cuvid -deint 2 \
    -drop_second_field 1 -vsync 0 \
    -i "udp://226.58.24.4:2000?overrun_nonfatal=1" \
    -i /home/anis/logo/2mbps.png \
    -filter_complex "[0:v]scale_npp=1920:1080,hwdownload,format=nv12 [base]; [base][1:v] overlay=main_w-overlay_w-10:10 [marked]" \
    -map "[marked]" -vcodec h264_nvenc -aspect 16:9 \
    -b:v 1808k -minrate 1808k -maxrate 1808k -bufsize 904k \
    -g 100 -no-scenecut 1 -forced-idr 0 -preset:v medium -vprofile main -level 4.1 \
    -map 0:#0x091d -acodec libfdk_aac -ab 192k -ar 48000 \
    -f flv rtmp://10.150.12.104/virginandtv/abr2k_test

If you could please always include full ffmpeg output as this makes debugging always easier.

For 2nd question - I wasn't able to find any way how to mix frames on the GPU using the current NPP library w/ ffmpeg. It certainly should be possible so whenever you find a solution let others know. I personally don't have enough time to read through all the NVIDIA docs and work on implementation but somebody else may get more time :-)