I got a new Nvidia GPU to speed up my video encoding. I am working with x264 files to x265 and wanted to go faster. Using the library libx265 I am getting about 85fps when working on a file that is 720p, but if I use the nvenc_hevc I get around 600-700fps

My input file is 269M and when I use the nvenc_hevc it grows to 394M. Doing the exact same command with libx265 gives me a file size of only 68M. Obviously I would like to keep the low file size, but take advantage of the GPU to process it 15X faster.

Anyone else have any experience with this video codec? My understanding is that the difference between libx265 and nvenc_hvec is not the algorithm that is used to do the compression, but that it utilizes the GPU to accelerate processing. With that understanding the resulting file sizes would be identical regardless of which I used, just the nvenc_hvec would be faster. Is that a faulty assumption?

My command is: ffmpeg -i infile.avi -c:v nvenc_hevc -rc vbr_2pass -rc-lookahead 20 -gpu any out7.mp4


ffmpeg -i infile.avi -c:v libx265 -rc vbr_2pass -rc-lookahead 20 -gpu any out7.mp4

When encoding I seem to only be using a small percentage of the GPU despite the huge performance increase:

nvidia-smi -l

| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|    0     21954    C   ffmpeg                                          93MiB |
Tue Nov  8 20:41:45 2016       
| NVIDIA-SMI 367.48                 Driver Version: 367.48                    |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|   0  GeForce GTX 960     Off  | 0000:42:00.0     Off |                  N/A |
|  1%   63C    P2    37W / 160W |     95MiB /  4037MiB |     11%      Default |

2 Answers 2


Just because you can get a hamburger at different restaurants doesn't mean they'll taste and look the same.

Apples and oranges

x265 and NVENC are two different encoder implementations that can output the same format, HEVC. What settings, options, and algorithms they use is up to the authors as long as the output conforms to the specs.

Choose the right tool for the job

These encoders are generally for different use cases. x265 may be slower, but it provides more quality per bit. NVENC is more suitable when speed is more important than quality or encoding efficiency, or when the CPU is not available or not up to the task, such as on a mobile platform where using the CPU would possibly consume more battery power and take a long time.

Option mixup

Also, you're mixing up options: -rc vbr_2pass -rc-lookahead 20 -gpu any are private options for nvenc_hevc, but you attempted to apply them to libx265, so they were ignored. There are two types of options: general and private. The private options are specific to each individual encoder. You can list private options such as with ffmpeg -h encoder=libx265.

You'll have to experiment with the encoding presets (see -preset option for both encoders) and rate control methods to get an output that is acceptable for you.

Waste of time?

Lastly, unless your input H.264 files were encoded very inefficiently (or are lossless which is doubtful), or if storage space is of utmost importance, then re-encoding from H.264 to H.265/HEVC is a waste of time and effort.

  • Thank you so much for your help @LordNeckbeard. I am setting up a service (prostabilizer.com) that will allow people to upload any kind of footage that they want stabilized. I am looking at bandwidth considerations, processing time optimizations, and so on. The faster I can process, and the smaller I can compress the better so I don't tie up the network more than needed. I can do a small server farm to distribute the processing locally, but eventually it all needs to get transmitted back to the users. When stabilizing a 4K video, it's at about 5fps, so optimization is key at each step.
    – Alan
    Nov 9, 2016 at 15:32
  • 1
    It seems as you said that the two authors of the two codecs have different default presets with libx265 being much more aggressive in file size. I can get similar file sizes if I use switches such as -b:v 3M -minrate 2M -maxrate 5M -bufsize 25M to control the bitrates. I like the more automated approach of libx265 for batch processing, but maybe there is a way to figure that out for nvenc_hevc.
    – Alan
    Nov 9, 2016 at 16:56

Just to add one specific point. The nVidia HEVC encoder doesn't support B-frames yet. So that crimps any hopes of achieviing the same quality at the same size, even if nvenc was as well-tuned as x265.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.