Using ffmpeg nvidia/cuda overlay processing results in green output - how to get the correct colours?

Question

Background: I'm trying to overlay a full frame image on top of an MP4, in ffmpeg, using as much hardware-assistance as possible. The inputs to ffmpeg are:

• a stream from an MP4 file - with 1 audio and 1 video stream
• frames from some software, which are supplied as raw RGBA frames, at the same frame size as the movie.

I struggled for some time to get something working and through various iterations, ended up with this:

ffmpeg -y -report -hide_banner -loglevel verbose 
-hwaccel cuda -hwaccel_output_format yuv420p 
-i movie.mp4 
-f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
-filter_complex [0:v]hwupload[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda[out] 
-map [out] -vcodec h264_nvenc 
-rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4

I picked yuv420p because looking at the overlay code ( https://github.com/FFmpeg/FFmpeg/blob/4976b102d89787d59a3aaa438b76c62aec86ad5a/libavfilter/vf_overlay_cuda.c#L94 ) - you can only overlay an image with alpha on top of an image of this type.

However, this results in a very green image! - although the overlay (the date, in this case) seems to have been overlaid with the right alpha etc.

I was wondering if the blend mode or some other aspect of the overlay was causing a problem, so attempted to reduce the problem size. I thought that perhaps just processing the video - movie->decode->encode - might shed some light, so I tried this:

ffmpeg -y -report -hide_banner -loglevel verbose 
-hwaccel cuda -hwaccel_output_format yuv420p -i movie.mp4 
-f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
-filter_complex [0:v]hwupload[mp4_stream] 
-map [mp4_stream] -vcodec h264_nvenc 
-rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4

This is essentially the same (I hope) but just ignoring the overlay frames - still green.

However, replacing the yuv420p with nv12: -hwaccel cuda -hwaccel_output_format nv12 results in a correctly coloured image. This is no use for my actual use case, as nv12 does not support alpha overlays.

Versions:

ffmpeg version 4.4.2-0ubuntu0.22.04.1

Hardware: NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1)

Kernel: 5.15.0-71-generic #78-Ubuntu SMP

Driver: 525.105.17

How to get the output the correct colour, while still using yuv420p, so I can overlay properly on top of it.

I have an inkling that this is due to some planes of the YUV file being zero? But how to fix this in the command line ffmpeg - I'm not sure. Thanks!

Answer 1

So, after some (long) time - it is a problem with the old version of ffmpeg - which is the version bundled with Ubuntu 22.04.

Using a self-compiled version of ffmpeg version 6.0 with CUDA Extensions. I can run the following command line, and everything is working perfectly.

/home/richja/bin/ffmpeg -y -report -hide_banner -loglevel verbose 
-hwaccel cuda -hwaccel_output_format cuda 
-i movie.MP4 
-f rawvideo -framerate 10.0 -s 1920x1080 -pix_fmt rgba -i - 
-filter_complex [0:v]scale_cuda=format=yuv420p[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda 
-vcodec h264_nvenc 
-rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart overlay.MP4

The performance of this setup is frankly incredible, and can render overlays at more than 12x real-time.