8

On my machine, Adobe Media Encoder is very slow. It takes 12 hours to render 4 minutes of 1080p video for YouTube (the video is based on Adobe After Effects project with an original MOV file and four effects: curves, remove grain, unsharp mask and hue/saturation.

enter image description here

I noticed that it doesn't use all the memory available, nor does it try to use as much CPU as possible. Here's the actual usage during the encoding process:

enter image description here

enter image description here

Is there a way to speed up the encoding process by convincing the application to use all CPU and all memory available?

As a side note, is it expected to be that slow? I can't possibly imagine movie making companies using it to encode a movie which usually is slightly longer than 4 minutes and has slightly more than four basic effects, even with a whole data center. Removing "remove grain" effect speeds up the encoding from 12 hours to 3:30.

  • 1
    There is absolutely no reason it should need all memory available. Encoding video is a linear process of opening any input frames, applying processing and then saving the output frame. There is no reason this should take much memory at all because it works as a stream. Some memory is used for pattern matching, but even that is fairly limited. – AJ Henderson Oct 19 '14 at 20:36
  • try ffmpeg. The payoff for having to learn how to use it is that it is way faster than any proprietary (software) media encoder I've ever used. – stib Oct 20 '14 at 4:14
  • adding any type of effect that needs to be rendered will add time onto the export, some effects will render in real time as such but others like "remove grain" will have to reference every frame before to match them up, this will take time to complete. I found that when I was using PP just adding a LUT or simple effect even on my 8 core mach with 16gb ram would take time to render. Sometimes you just have to suck up to the render and leave it doing its thing, Ive had it say 12+ hours to do an export but then once it finished a section it dropped massively as the effect had finished rendering a – Adam Mann Pro Dec 10 '14 at 14:48
  • FYI if you're exporting an AE comp from AME you're not actually "encoding" the file so to speak, you are first rendering it THEN encoding it. The distinction is important because AME is presumably running a sub-process of AE to get those frames out, to then encode them. So the bottle-neck is actually AE's internal renderer which is horribly inefficient as I think we all know, and definitely does not hit 100% CPU. That is why degraining makes such a big difference! And FYI I believe Adobe moved that to the GPU in the last release... – Spencer Sep 27 '18 at 21:27
6

There might not be a way. Based on your description of the problem, it sounds like the processing is the slow part. While the video encoder itself is able to do multi-threaded processing, the image processing you are doing may not be able to. By default, Premiere has always done as much parallel processing as possible for me and I frequently see it hit 99% CPU usage on my hyper-threaded quad core desktop.

The problem is most likely that some of your effects require the result of the previous frame to begin processing on the next frame. If this is the case, there isn't an easy way to break it down across multiple threads and that would make it impossible to hit 100% CPU usage on a multi-core computer.

Similarly, RAM wouldn't be expected to cap on encoding as it is a stream operation. There are some things that may need to be tracked over time, but for the most part, it is a stream operation with data going out as fast as it comes in, so there isn't much accumulation of data.

There are two main things that can improve the performance. The first is GPU processing. General purpose CPUs aren't really ideal for many audio/video processing tasks. They are designed for doing a wide variety of general purpose operations, but aren't super efficient at basic operations. GPUs, on the other hand, are designed for doing simple operations very quickly. They can often reduce video processing tasks quite a bit.

A further improvement can be found in purpose built hardware. There are professional cards designed specifically for processing video and provide real time processing and encoding of video. These are often included in high end professional video editing workstations, but the price can be quite high. It isn't unheard of for a professional video editing workstation to be able to get up in the $15,000 to $20,000 range or even higher, just for hardware. They can go for a lot less if you don't mind waiting for renders (I work on a $2,350 system for my video work) but they can also get very high.

| improve this answer | |
6

I'm not happy when I'm rendering unless I can actually smell burning plastic, nothing is more frustrating than AE telling you that your render won't complete until the heat death of the universe, while at the same time your CPU is barely raising a sweat. However there is something you can do about it.

After effects comes with a command line renderer called aerender.exe plain ol' aerender on mac). It's in the AE program folder. Using it is a bit challenging for people unused to the -=awesome -=power of the command line, but it is definitely worth doing if you're doing some heavy lifting in After Effects, and you want to be able to harness all the power of your machine.

The beauty of it is that you can run as many instances as you like. What I normally do is keep adding instances until I start running low on memory. I have a 40 physical core machine and it takes a lot to max out the CPU but here's a screenshot where I finally managed it. Each one of those windows is a new instance of the AE render engine:

enter image description here

In that example the per-frame speed of each renderer barely decreased compared to a single instance, so I achieved a speed increase of roughly 4000%. YMMV, depending on how many cores and how much ram your machine has. I went to the effort because I had a project involving very long and and reasonably complex comps. The project would simply not have got done without this technique, or I would have had to farm it out to a commercial render farm.

Of course the speed-up comes at a price, which is a slight increase in complexity of your workflow. You need to set your comps to render as image sequences, and use the multi-machine settings so that each renderer will look for the next unrendered frame. At the end if you need a movie file you'll have to do a compression pass, but in my case I usually master to a png or tiff sequence, and then run off h.264 encoded copies for clients from the master.

And you need to know a little bit about the command line. Stop looking so glum—that's a GOOD thing, that's a GOOD thing.


Windows

The powershell command line I used in that scenario was:

for ($i = 0 ; $i -lt 40; $i++ ){
  Start-Process 'C:\Program Files\Adobe\Adobe After Effects <<version>>\Support Files\aerender.exe'-ArgumentList '-project', '"c:\path\to\project.aep"'
  sleep 5;
}

This is executed in a powershell window (type in WindowsR and then type powershell.exe). This is a shell scripting language integrated into modern versions of Windows that can be used for all kinds of stuff.

Mac Using Bash version > 3 (default shell in current versions of OSX), or indeed with cooler shells like zsh, you can use this script:

for i in {1..40}; do
  /Applications/Adobe\ After\ Effects\ CC\ 2017/aerender -project ~/Path/to/myproject.aep &
  sleep 5
done

The script works thus:

  • starts a loop which goes for 40 repeats, you can change the 40 to whatever you think is a sane amount.
  • Then it starts aerender as a new process either with the start-process command in PS or with the & in bash, Obviously change the path to aerender.exe / aerender and your project to whatever it is on your machine. A quick and accurate way of filling in paths with the command line is to drag files and folders into it. Also using the tab to auto-fill paths saves a lot of typing and typos.
  • Finally it sleeps for 5 seconds—I found that opening too many instances of aerender too quickly was a reliable way to bluescreen / kernel panic my computer.


If you have multiple comps you can render them as movies but start a new renderer for each one. If you invoke aerender.exe like this

aerender.exe -project "c:\path\to\proj.aep" -rqindex 3

it will render the third comp on the render queue. Putting it into a script you can do:

Windows

$numcomps = 12
$projpath= "C:\path\to\project.aep"
for($i=1; $i -lt ($numcomps+1); $i++){
  Start-Process 'C:\Program Files\Adobe\Adobe After Effects <<version>>\Support Files\aerender.exe' -ArgumentList '-project', '$projpath', '-rqindex', '$i'
  sleep 5;
}

Mac

for i in {1..12}; do
  /Applications/Adobe\ After\ Effects\ CC\ 2017/aerender -project ~/Path/to/myproject.aep -rqindex $i &
  sleep 5
done

That will start a new aerender instance for each comp–once again, change the 12 to however many comps there are in the queue. Note that this technique can run into problems if you have lots of comps in your queue–if you use all your physical memory things will come to a shuddering halt pretty quick. To keep the number of instances down you have to check how many aerender processes are running and only start a new one when a running one finishes. A simpler workaround is to make as many copies of your project as you want there to be instances, with the render queue divided up between them, and then render them all concurrently. So render thread one would be rendering comps 1,2,3, render thread two would be doing 3,4,5, and so on.

More details here (my blog).

| improve this answer | |
  • 1
    you are a legend! – Tony Sepia Jun 22 '17 at 12:07
  • that's what I keep telling people. – stib Jun 23 '17 at 14:37
  • @stib is there anything similar for Premiere Pro? I'm having similar problems as OP between AME and PP. Example from today: AME will take 1h 20m to export a video, PP will export it in 5-10 mins - all at same quality etc. – 5Diraptor Jun 10 at 16:03
  • Unfortunately there is no command line renderer for PP. – stib Jun 12 at 6:45
  • @5Diraptor I had the same issue, if you "render in AME" from PP it opens AME, allocates the leftover resources (after it's reserved them when you opened PP) to AME, which are now shared with PP, so you end up not utilising your full system. You have to close PP and re-open AME and it should (in my case, does consistently) then allocate the full amount of resources (according to your preferences for RAM allowance) to AME now and in turn be much quicker, I saw a 400% increase, huge. – Mullazman Aug 27 at 8:51
0

While this question has already been answered; my recommendation:

Use AfterEffects for your rendering if you have a system with a high number of cores and a lot of RAM.

My main workstation is a Dual E5-2697 v2 (24 Logical / 48 Hyper Threaded Cores); with 128 GB of RAM.

If you have to do simply a transcode of a file; or can use Bridge to import your PPro Project into AE (barring your edits will remain intact); the feature under Memory (Edit -> Preferences) allow for you to use Multi-threading.

That being said:

If I am doing just a straight render; or something simple; I can use AE to allocate 32 of my 48 HT cores; additionally; I can allocate 3 GB of ram to each Thread; which is depending on what type of encode I'm doing, anywhere between 2-100x faster in terms of rendering even with my Dual Titan Black GPUs.

This of course; only applies if you have a LOT of cores; and a LOT of ram...

Premiere; rendering through AME; will be limited to using only 1 Thread unfortunately; and will rely on your GPU unless you have that disabled to use software rendering under Project Settings.

Hope this helps!

| improve this answer | |
0

How to speed up Media Encoder to use 100% of your cpu? Here is an example: Create a file: settings.ini place inside of it

[Settings] mode=developer mthread=enabled mthreadcount=3 (for a 4 core 8 threaded machine say, 7) sli=false slimode=all priority=high scaling=low cloud=false loadwhenneeded=true speakers=2

Place in Adobe Premiere, After Effects, and Media Encoders executable folders.

For a quad core / 4 threadded machine like mine, this made Media Encoder jump from 1 core to using 96% of all four. You must watch the youtube video "Render 300X FASTER | Premiere / After Effects / Media Encoder " I'm successfully using this with CC 2018 products.

| improve this answer | |
  • Wow, now that's interesting! – Spencer Sep 27 '18 at 21:30
0

Close After Effects while rendering with Media Encoder. By doing so my CPU usage went from 30% to 60%. I guess because Adobe isn't sparing CPU power for working simulatiously in AE.

| improve this answer | |
  • I believe this is due to the dynamic way that Adobe allocates RAM in the background. It will limit the amount of RAM per adobe process, to prevent overflow and whatnot. So by closing one Adobe app you theoretically free up more RAM for another. – Spencer Sep 27 '18 at 21:29
0

Can I suggest this is still an issue 5 years on! You need to close all Adobe apps and then open Media Encoder (if that's where you're rendering from) FIRST, then it'll assign the full resources available to that app, I haven't seen it dynamically share resources, it simply allocates CPU-time according to the number of apps open when you start the Adobe app (this is repeatable, try opening LR for example before rendering, it'll only use 10% of the CPU - then open AME first then lightroom and you'll get 100%)

I got a 4x increase in render times because of this. Hope it helps. Adobe's "resource management" sucks.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.