1

Has 4 nvidia devices:

nvidia-smi topo --matrix
        GPU0    GPU1    GPU2    GPU3    CPU Affinity
GPU0     X      PHB     SYS     SYS     0-5,12-17
GPU1    PHB      X      SYS     SYS     0-5,12-17
GPU2    SYS     SYS      X      PHB     6-11,18-23
GPU3    SYS     SYS     PHB      X      6-11,18-23

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe switches (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing a single PCIe switch
  NV#  = Connection traversing a bonded set of # NVLinks

We want to re-code h264 videos to lower resolution. When we run 2-3 processes per 0 and 1 devices:

ffmpeg -v error -y -hwaccel cuvid -hwaccel_device 0 -c:v:0 h264_cuvid -gpu 0 -i /mnt/worker/6797/be2c96e8f0fd7/objects/133607/fb3/8279fb61a4d0bf66c3f871bdb30e5fb3/1530758904.34570.data -map_metadata -1 -map_chapters -1 -map 0:v:0 -map 0:a:0 -c:v:0 h264_nvenc -gpu 0 -b:v:0 650000 -filter:v:0 scale_npp=-2:360 -c:a:0 aac -gpu 0 -b:a:0 96000 -ac:a:0 2 -max_muxing_queue_size 9999 -movflags +faststart -f mp4 /home/uploads/tmp/out.mp4

all works fine

nvidia-smi dmon -c 1
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0   102    48     -    20     9    14   100  3304  1189
    1   105    54     -    25     9    17    99  3304  1201
    2    15    27     -     0     0     0     0   405   135
    3    15    27     -     0     0     0     0   405   135

but when works 2-3 processes per each device, then performance gets low. Summary compared to 2 devices:

nvidia-smi dmon
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    89    45     -     8     3     5    37  3304  1189
    1    92    50     -     9     3     6    30  3304  1201
    2    76    42     -     9     3     6    29  3304  1201
    3    87    44     -     9     3     6    33  3304  1189

htop,top,iftop,atop show nothing overloaded

Sometimes performance rises higher, or drops below (i`m suspect it depending on input files)

How to I can utilize all decoders ?

  • Realistically - get a faster machine. That's pretty much the fix. – Dr Mayhem Dec 26 '18 at 21:45
  • @DrMayhem what means faster? We have 2 CPU - Xeon(R) CPU E5-2620 v3. 20% loaded. – Hett Dec 27 '18 at 5:05
  • Faster/more CPUs. Faster pipeline. Greater concurrency. – Dr Mayhem Dec 27 '18 at 9:55
  • @dr-mayhem cpu is not loaded more than 15% when hwaccel is used – Hett Dec 27 '18 at 17:37
  • Hett - it's everything. cpu was one of many things worth looking at. – Dr Mayhem Dec 27 '18 at 20:49
1

NVidia boards has a limit number of concurrent encoding tasks, according number of encode engine chips on board. Look at specifications of boards you are using.

Even if you have a board with unrestricted number of sessions to it, you will have a speed degradation, because of some task pipelining on host side.

|improve this answer|||||
  • you are not right! I use full hardware transcode (learn about hwaccel) trac.ffmpeg.org/wiki/HWAccelIntro – Hett Dec 27 '18 at 8:23
  • processes outside of limits lead to an error – Hett Dec 27 '18 at 8:24
1

Problem was resolved by replace

scale_npp=-2:360

to

scale_npp=-2:360,hwdownload,format=nv12

now decoders fully loaded:

nvidia-smi dmon -c 1
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0   101    50     -    27     8    15   100  3304  1189
    1   105    54     -    25     8    12   100  3304  1201
    2   104    52     -    48    10    19   100  3304  1316
    3    97    50     -    23     9     8   100  3304  1189
|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.