0

I want to extract a frame from the h264 video stream using NVIDIA card.
First of all we need to decode this stream via h264_cuvid decoder:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i ...

As I understand currently we can't extract a video frame using only NVIDIA card in FFmpeg as this functional doesn't exist, like doesn't exist a codec like mjpeg_nvenc or something else. Please correct me if I wrong.

I think we should be able to decode a video stream on the NVIDIA card then extract a frame via CPU codec, for example, mjpeg:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vframes 1 /tmp/frame.jpg

In this case we will get the following error:

Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented

NVIDIA decode video to NV12(by default) or NV21 pixel format and it's not appropriate format for the mjpeg codec. Please also correct me if I wrong.
Let's try to change the output resolution and convert the pixel format via scale_npp to yuvj420p as mjpeg codes support it:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vf "hwupload_cuda,scale_npp=w=1024:h=70:format=nv12:interp_algo=lanczos,hwdownload,format=yuvj420p" -vframes 1 /tmp/frame.jpg

I got the same error:

Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented

According the FFmpeg documentation format conversation isn't supported: "Note that automatic format negotiation and conversion is not yet supported for hardware frames"

How to decode an input h264 stream via h264_cuvid decoder then convert the decoded video pixel format to yuvj420p and extract a frame via mjpeg codec?
Or please share your thoughts how to extract a frame from a video stream with decoding or decoding and encoding on NVIDIA card to any picture format.

6
  • scale_npp supports yuv420p.
    – Gyan
    Jun 14, 2018 at 14:37
  • @Gyan Thank you for the comment! I tried the following command: ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vf "hwupload_cuda,scale_npp=w=1024:h=70:format=nv12:interp_algo=lanczos,hwdownload,format=yuv420p" -vframes 1 /tmp/frame.jpg And received the same error: "Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented" Jun 15, 2018 at 6:53
  • The format in scale_npp needs to be changed to yuv420p
    – Gyan
    Jun 15, 2018 at 6:57
  • @Gyan Thanks! I changed the formats to yuv420p in the command: ffmpeg -hwaccel cuvid -c:v h264_cuvid -i video_source -vcodec mjpeg -vf "hwupload_cuda,scale_npp=w=1024:h=70:format=yuv420p:interp_algo=lanczos,hwdownload,format=yuv420p" -vframes 1 /tmp/frame.jpg Unfortunately, it didn't help and I got the same error. Jun 15, 2018 at 7:03
  • 1
    I don't have a compatible card to test with, but the decoded frames should already be on the GPU, so is hwupload required?
    – Gyan
    Jun 15, 2018 at 7:23

2 Answers 2

4

Since the video bitstream is being decoded on the GPU, CUDA upload isn't required.

Command with format conversion is

ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 1024x70 -i video_source \
       -vf "scale_npp=format=yuv420p,hwdownload,format=yuv420p" \
       -pix_fmt yuvj420p -color_range 2 -vframes 1 -y /tmp/frame.jpg
1

Actually we can extract a video frame using only NVIDIA card via thumbnail_cuda filter. For it is necessary to configure FFmpeg with:

--enable-cuda-sdk --enable-filter=scale_cuda --enable-filter=thumbnail_cuda

We can resize frames at the decoding step then not necessary to use scale_npp filter. When we use the thumbnail_cuda filter we can set NV12 as the output format to prevent additional pixel format convertation. NV12 will be converted to yuvj420p without yuv420p in the middle.

The FFmpeg command to perform it for a h264 stream looks as follow:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 120x70 -i video_source \
       -vf "thumbnail_cuda=2,hwdownload,format=nv12" \
       -vframes 1 frame.jpg

If we need to deocde mpeg2, hevc, vp8, etc then we should choose an appropriate decoder from the list:

ffmpeg -decoders | grep cuvid

 V..... h264_cuvid           Nvidia CUVID H264 decoder (codec h264)
 V..... hevc_cuvid           Nvidia CUVID HEVC decoder (codec hevc)
 V..... mjpeg_cuvid          Nvidia CUVID MJPEG decoder (codec mjpeg)
 V..... mpeg1_cuvid          Nvidia CUVID MPEG1VIDEO decoder (codec mpeg1video)
 V..... mpeg2_cuvid          Nvidia CUVID MPEG2VIDEO decoder (codec mpeg2video)
 V..... mpeg4_cuvid          Nvidia CUVID MPEG4 decoder (codec mpeg4)
 V..... vc1_cuvid            Nvidia CUVID VC1 decoder (codec vc1)
 V..... vp8_cuvid            Nvidia CUVID VP8 decoder (codec vp8)
 V..... vp9_cuvid            Nvidia CUVID VP9 decoder (codec vp9)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.