1

I am trying to improve the speed of decoding H.264 frame to RGBA frame using FFmpeg in C++. I have found a CLI command that can do that but I still do not fully understand the command to reproduce it in C++, so I hope you guys can give me an explanation of how the CLI command works.

The command that I found uses h264_nvenc encoder to improve the speed using GPU. If I understand correctly, FFmpeg will first decode the H.264 frame, and then encode it to mp4 frame. However, I cannot understand why a H.264 encoder would improve the speed of decoding H.264 video (or maybe encoding to mp4 frame). Can anyone please give me a detailed explanation?

Here is the CLI command that I found:

ffmpeg -hwaccel cuvid -r 120 -i <input_file> -c:v h264_nvenc -preset fast \
   -b:v 8M -vf scale=960:1080 -pix_fmt yuv420p <output_file> -y

Edited: Log of command with option "-c:v h264_nvenc"

C:\Users\Shared\test\test>ffmpeg.exe -hwaccel cuvid -r 120 -i C:\\Users\\Shared\\test\\test\\raw0.h264 -c:v h264_nvenc -preset medium -b:v 8M -vf scale=960:1080 -pix_fmt yuv420p C:\\Users\\Shared\\test\\test\\video0.mp4 -y
ffmpeg version 3.4.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray 
--enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy 
--enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 
--enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis 
--enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc 
--enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[h264 @ 0000027ecb2a6700] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from 'C:\\Users\\Shared\\test\\test\\raw0.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (CAVLC 4:4:4), yuv444p(progressive), 960x1080, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
Output #0, mp4, to 'C:\\Users\\Shared\\test\\test\\video0.mp4':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (h264_nvenc) (Main) (avc1 / 0x31637661), yuv420p, 960x1080, q=-1--1, 8000 kb/s, 120 fps, 15360 tbn, 120 tbc
    Metadata:
      encoder         : Lavc57.107.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/8000000 buffer size: 16000000 vbv_delay: -1
frame=  269 fps=261 q=25.0 Lsize=    2551kB time=00:00:02.23 bitrate=9356.5kbits/s speed=2.17x
video:2549kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.074325%

Log of command without option "-c:v h264_nvenc"

C:\Users\Shared\test\test>ffmpeg.exe -hwaccel cuvid -r 120 -i C:\\Users\\Shared\\test\\test\\raw0.h264 -preset medium -b:v 8M -vf scale=960:1080 -pix_fmt yuv420p C:\\Users\\Shared\\test\\test\\video0.mp4 -y
ffmpeg version 3.4.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[h264 @ 0000021e33556640] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from 'C:\\Users\\Shared\\test\\test\\raw0.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (CAVLC 4:4:4), yuv444p(progressive), 960x1080, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0000021e335589a0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000021e335589a0] profile High, level 4.2
[libx264 @ 0000021e335589a0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=8000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'C:\\Users\\Shared\\test\\test\\video0.mp4':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 960x1080, q=-1--1, 8000 kb/s, 120 fps, 15360 tbn, 120 tbc
    Metadata:
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/8000000 buffer size: 0 vbv_delay: -1
frame=  269 fps= 39 q=-1.0 Lsize=    2279kB time=00:00:02.21 bitrate=8423.8kbits/s speed=0.32x
video:2277kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.119200%
[libx264 @ 0000021e335589a0] frame I:2     Avg QP:25.09  size: 16508
[libx264 @ 0000021e335589a0] frame P:193   Avg QP:23.94  size:  9094
[libx264 @ 0000021e335589a0] frame B:74    Avg QP:23.92  size:  7332
[libx264 @ 0000021e335589a0] consecutive B-frames: 62.8%  1.5%  0.0% 35.7%
[libx264 @ 0000021e335589a0] mb I  I16..4: 38.2% 54.3%  7.6%
[libx264 @ 0000021e335589a0] mb P  I16..4:  9.2% 11.8%  0.8%  P16..4: 46.5%  3.4%  3.8%  0.0%  0.0%    skip:24.5%
[libx264 @ 0000021e335589a0] mb B  I16..4:  4.3%  3.7%  0.2%  B16..8: 35.0%  1.5%  0.2%  direct: 9.5%  skip:45.5%  L0:56.6% L1:40.9% BI: 2.5%
[libx264 @ 0000021e335589a0] final ratefactor: 19.31
[libx264 @ 0000021e335589a0] 8x8 transform intra:53.1% inter:88.1%
[libx264 @ 0000021e335589a0] coded y,uvDC,uvAC intra: 22.0% 41.2% 4.5% inter: 12.8% 39.4% 0.0%
[libx264 @ 0000021e335589a0] i16 v,h,dc,p: 18% 32% 19% 31%
[libx264 @ 0000021e335589a0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 17% 54%  2%  3%  3%  4%  1%  2%
[libx264 @ 0000021e335589a0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11%  9% 63%  3%  4%  3%  3%  2%  2%
[libx264 @ 0000021e335589a0] i8c dc,h,v,p: 63% 23% 13%  1%
[libx264 @ 0000021e335589a0] Weighted P-Frames: Y:86.0% UV:64.8%
[libx264 @ 0000021e335589a0] ref P L0: 53.6%  8.3% 17.1% 11.4%  9.6%
[libx264 @ 0000021e335589a0] ref B L0: 58.9% 14.4% 26.6%
[libx264 @ 0000021e335589a0] ref B L1: 84.8% 15.2%
[libx264 @ 0000021e335589a0] kb/s:8317.64
Improve this question

1 Answer 1

Reset to default
1

You are not using an encoder for decoding. What you are doing is telling FFMPEG to decode the input file using hardware acceleration.

NVDEC offers decoders for H.264, HEVC, MJPEG, MPEG-1/2/4, VP8/VP9, VC-1.

Codec support varies by hardware.

Note that FFmpeg offers both NVDEC and CUVID hwaccels. They differ in how frames are decoded and forwarded in memory.

Sample decode using NVDEC:

ffmpeg -hwaccel nvdec input output

Sample decode using CUVID:

./ffmpeg-git -hwaccel cuvid -c:v h264_cuvid -i input output

https://trac.ffmpeg.org/wiki/HWAccelIntro#NVENCNVDEC

Then you are encoding the file also using hardware acceleration (-c:v h264_nvenc).

NVENC can be used for H.264 and HEVC encoding. FFmpeg supports NVENC through the h264_nvenc and hevc_nvenc encoders

https://trac.ffmpeg.org/wiki/HWAccelIntro#NVENCNVDEC

In closing, the decoder does not improve speed. Using your GPU instead of your CPU does.

EDIT:

Now that you have supplied your console output and I have done a little more digging I can tell you:

In current versions of FFMPEG hardware acceleration is supported automatically if available. https://stackoverflow.com/questions/52765883/ffmpeg-using-hardware-acceleration-for-video-decoding

If it detects a stream which is decodable in hardware then it will attempt to delegate all significant processing to that hardware. https://trac.ffmpeg.org/wiki/HWAccelIntro

You can see in your output from

command without option "-c:v h264_nvenc"

That you are using the CPU to decode.

using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

Then when you specify

command with option "-c:v h264_nvenc"

It tells FFMPEG that hardware acceleration is available, so it uses "h264_nvenc" for the decode and encode (MP4 is a file container and H.264 is the video compression codec).

SideNote: I do not think -hwaccel cuvid is doing anything since you are not using a GPU based scaler and you do not specify -c:v h264_cuvid.

Improve this answer
4
  • (-hwaccel cuvid) is hardware acceleration, that I know. But in order to use that, you need to specify the codec for decoder (-c:v h264_cuvid ) or for encoder (-c:v h264_nvenc). My command does specify the encoder and it speeds up the encoding. However, I did not specify (-c:v h264_cuvid) (because this causes errors for me) so it should not use hardware acceleration for decoding.
    – Khoa Tran
    Commented Jul 23, 2019 at 1:10
  • I have also verified with Task Manager that only the GPU encoder is used to accelerate the encoding. The GPU decoder is never used when I run my command.
    – Khoa Tran
    Commented Jul 23, 2019 at 1:12
  • FFMPEG will always decode the input file before it encodes it again. You specify before the input file to use hardware acceleration so I am assuming that FFMPEG is choosing the decoder automatically. Can you show the console output from the command you ran?
    – GeekItTillItMhz
    Commented Jul 23, 2019 at 15:28
  • Sorry for the late reply. I have updated the question with console output.
    – Khoa Tran
    Commented Jul 26, 2019 at 1:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.