6

I have a web project I am working on that requires transcoding of multiple short video clips (6-10 seconds) uploaded by users into a web friendly H.264 format on my linux server. I have been able to enable NVIDIA NVENC hardware acceleration with ffmpeg and a 4 year old GeForce GTX 670, and I'm getting hardware encoding speeds twice that of my software encoding (Xeon E5-1620 v3). With an $800 video card budget, I'd like to be able to transcode these short video clips as fast as possible because I will have multiple users uploading them simultaneously.

The NVENC engine does have licensing limitations when implemented on a consumer level NVIDIA card: only 2 video transcoding threads can be run simultaneously, even if you have multiple cards. If I decide to go with one of the pricy quadro card line, then I am only limited by the other hardware elsewhere in my system in terms of how many threads I can run. However, with my specific project I'm a lot better off transcoding these clips in a series rather than parallel because the clips will be viewed in the order they're uploaded. Clips later in the queue can be transcoded as earlier clips are viewed. If clips are transcoded in parallel on the same card, performance is inversely proportional to the number of simultaneous threads.

Having stated this, my plan is to set up two NVIDIA cards and run a single thread on each one to maximize throughput. The NVIDIA codec SDK is vague on NVENC performance difference between various cards, but it seems that there is a major difference between GPU generations Maxwell Gen 2> Maxwell Gen 1> Kepler. I can find no reliable benchmarks for NVENC encoding (opposed to CUDA benchmarking, which is easy to find).

In the absence of hard benchmarking data comparing various cards, what features of the currently available cards would have to largest impact on single thread NVENC encoding speed? Since the actual GPU is not utilized fully during transcoding, does GPU and memory clock speed affect this function much? I have $400/card to spend, but if the entry level Maxwell Gen 2 GeForce GTX 960 is as good as the more recent cards, then I will put extra money into other aspects of the server (CPU/RAM/SSD etc). I know this might seem like a subjective question (what's best), but I am trying to make a self-educated guess based on an understanding of the transcoding hardware.

  • I just wanted to say that I love this post and hope for an equally interesting answer. I wonder why the GPU doesn't get utilized completely, is it lack of optimization? A bottle neck somewhere else in the pipeline? – kimgroth Jan 19 '16 at 10:58
  • From what I've read there is a completely separate NVENC chip doing most of the work, so my guess would be that it is bottleneck. NVIDIA is quite vague in explaining it for some reason. – user255406 Jan 20 '16 at 17:32
  • Will you 1+ this question to bump it up? – user255406 Jan 20 '16 at 17:33
3

There are two main features in NVIDIA NVENC encoding:

  1. Video memory if you need to transcode many streams.
  2. NVENC SIP - than better this SIP than better encoding performance is. It depends on GPU. You can find the best explanation related GPU and NVENC SIP generation on Wikipedia website. Maxwell Gen 2 is the best at the moment(for 2016).

According to the license limitation: it's possible to run only 2 encoding threads simultaneously on the consumer level NVIDIA cards (any GTX card). It's regulated on the driver level, however it’s possible to remove this limitation. In this case, the value of maximum transcoding threads will depend on video memory size and video engine utilization. Video memory size which is necessary for one transcoding stream is various and depends on a video card model. For encoding SD stream on QUADRO K4200(4GB) is necessary 100 MB of video memory, but for encoding the same stream on GTX 980TI(6GB) we need 170MB.

My results in transcoding SD sources in real time are:

  • QUADRO K4200(4GB): one transcoding thread costs 100 MB and we can run in parallel about 36 threads, but the bottleneck is the video engine utilization. I can run about 30 parallel threads with "-preset hp -vcodec nvenc_264"
  • GTX 980TI(6GB): the bottleneck is the video memory. I can run about 32 threads (32*170=5440) in parallel. I did it for my education, of course. But the video engine results is better then on K4200 in 2.5 times. These 32 threads with "-preset slow -vcodec nvenc_264 -vf yadif=0" and video engine utilization are only 80%.
  • I also tested it on GTX 660(2GB, Maxwell Gen 2). It was about 15 parallel threads due to video memory.

My conclusion(for 2016): If you need to transcode no more than 2 threads in parallel, then GTX 960 is a good variant. Also, you can save some money to another hardware and look for another video card with Maxwell Gen1. If you go to the hack way, then GTX 960 is a good variant but only with 4GB of video memory.

UPDATE FROM 2018: Nowadays the situation in this area has changed in better direction. The card with the best encoder is Tesla V100, but it's too expensive. The best work variant is the video cards based on Pascal CHIPs. If you prefer the hack way then GTX1050TI(4GB), it will be perfect for your budget in 800$. Otherwise, Quadro P2000 doesn’t have license limitations and also very comfortable for the same budget.

  • Thanks for your answer. After some testing I realized that the overhead time it takes to initiate the transcoding becomes a huge issue with multiple short clips. Have you experienced this? I'm seeing maybe 1s/clip overhead. When my clips take average 1s/clip to transcode using CPU only, the GPU benefit vanishes. Thoughts? – user255406 Mar 20 '16 at 14:45
  • Unfortunately, I don't have experience with multiple short clips. One idea: you need maintain maximum possible NVENC threads. It will be faster then on CPU. For this you need to use variant with many transcoding streams in parallel for this need to buy QUADRO video card or go to the hack way. – Ivan Kolesnikov Mar 21 '16 at 6:48
  • 1
    Then you can to find out a maximum possible NVENC threads and always maintain it or you can try to run new FFmpeg instance if the maximum number of streams will be exceeded then will be occurs the following error: "Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height". – Ivan Kolesnikov Mar 21 '16 at 6:49
  • Unfortunately the clips will be uploaded sporadically, sometimes a bunch, sometimes a few. I'm leaning towards a CPU only solution as this seems to be fool proof. – user255406 Mar 22 '16 at 11:44
  • 1
    Unfortunately, I haven't real work experience with QS, but some sources from Intel says what QT faster than NVENC in a four times. – Ivan Kolesnikov Mar 28 '16 at 7:17
1

At the very top of your $800 budget, you can get a Quadro M4000 that does NOT have the license limitation of 2 concurrent transcodes. We use these cards to transcode 5-10 incoming live streams to 2-3 output bitrates.

  • Can you comment on the time it takes to initiate a transcoding steam? On my consumer card it was about a full second. Since I have dozens of short clips coming in at the same time, a one second overhead per clip ends up costing me dearly. – user255406 Sep 17 '16 at 19:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.