How to extract a video frame using NVIDIA card

Question

I want to extract a frame from the h264 video stream using NVIDIA card.
First of all we need to decode this stream via h264_cuvid decoder:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i ...

As I understand currently we can't extract a video frame using only NVIDIA card in FFmpeg as this functional doesn't exist, like doesn't exist a codec like mjpeg_nvenc or something else. Please correct me if I wrong.

I think we should be able to decode a video stream on the NVIDIA card then extract a frame via CPU codec, for example, mjpeg:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vframes 1 /tmp/frame.jpg

In this case we will get the following error:

Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented

NVIDIA decode video to NV12(by default) or NV21 pixel format and it's not appropriate format for the mjpeg codec. Please also correct me if I wrong.
Let's try to change the output resolution and convert the pixel format via scale_npp to yuvj420p as mjpeg codes support it:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vf "hwupload_cuda,scale_npp=w=1024:h=70:format=nv12:interp_algo=lanczos,hwdownload,format=yuvj420p" -vframes 1 /tmp/frame.jpg

I got the same error:

Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented

According the FFmpeg documentation format conversation isn't supported: "Note that automatic format negotiation and conversion is not yet supported for hardware frames"

How to decode an input h264 stream via h264_cuvid decoder then convert the decoded video pixel format to yuvj420p and extract a frame via mjpeg codec?
Or please share your thoughts how to extract a frame from a video stream with decoding or decoding and encoding on NVIDIA card to any picture format.

Answer 1

Since the video bitstream is being decoded on the GPU, CUDA upload isn't required.

Command with format conversion is

ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 1024x70 -i video_source \
       -vf "scale_npp=format=yuv420p,hwdownload,format=yuv420p" \
       -pix_fmt yuvj420p -color_range 2 -vframes 1 -y /tmp/frame.jpg

Answer 2

Actually we can extract a video frame using only NVIDIA card via thumbnail_cuda filter. For it is necessary to configure FFmpeg with:

--enable-cuda-sdk --enable-filter=scale_cuda --enable-filter=thumbnail_cuda

We can resize frames at the decoding step then not necessary to use scale_npp filter. When we use the thumbnail_cuda filter we can set NV12 as the output format to prevent additional pixel format convertation. NV12 will be converted to yuvj420p without yuv420p in the middle.

The FFmpeg command to perform it for a h264 stream looks as follow:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 120x70 -i video_source \
       -vf "thumbnail_cuda=2,hwdownload,format=nv12" \
       -vframes 1 frame.jpg

If we need to deocde mpeg2, hevc, vp8, etc then we should choose an appropriate decoder from the list:

ffmpeg -decoders | grep cuvid

 V..... h264_cuvid           Nvidia CUVID H264 decoder (codec h264)
 V..... hevc_cuvid           Nvidia CUVID HEVC decoder (codec hevc)
 V..... mjpeg_cuvid          Nvidia CUVID MJPEG decoder (codec mjpeg)
 V..... mpeg1_cuvid          Nvidia CUVID MPEG1VIDEO decoder (codec mpeg1video)
 V..... mpeg2_cuvid          Nvidia CUVID MPEG2VIDEO decoder (codec mpeg2video)
 V..... mpeg4_cuvid          Nvidia CUVID MPEG4 decoder (codec mpeg4)
 V..... vc1_cuvid            Nvidia CUVID VC1 decoder (codec vc1)
 V..... vp8_cuvid            Nvidia CUVID VP8 decoder (codec vp8)
 V..... vp9_cuvid            Nvidia CUVID VP9 decoder (codec vp9)