9

On my machine, Adobe Media Encoder is very slow. It takes 12 hours to render 4 minutes of 1080p video for YouTube (the video is based on Adobe After Effects project with an original MOV file and four effects: curves, remove grain, unsharp mask and hue/saturation.

enter image description here

I noticed that it doesn't use all the memory available, nor does it try to use as much CPU as possible. Here's the actual usage during the encoding process:

enter image description here

enter image description here

Is there a way to speed up the encoding process by convincing the application to use all CPU and all memory available?

As a side note, is it expected to be that slow? I can't possibly imagine movie making companies using it to encode a movie which usually is slightly longer than 4 minutes and has slightly more than four basic effects, even with a whole data center. Removing "remove grain" effect speeds up the encoding from 12 hours to 3:30.

4
  • 1
    There is absolutely no reason it should need all memory available. Encoding video is a linear process of opening any input frames, applying processing and then saving the output frame. There is no reason this should take much memory at all because it works as a stream. Some memory is used for pattern matching, but even that is fairly limited.
    – AJ Henderson
    Oct 19, 2014 at 20:36
  • try ffmpeg. The payoff for having to learn how to use it is that it is way faster than any proprietary (software) media encoder I've ever used.
    – stib
    Oct 20, 2014 at 4:14
  • adding any type of effect that needs to be rendered will add time onto the export, some effects will render in real time as such but others like "remove grain" will have to reference every frame before to match them up, this will take time to complete. I found that when I was using PP just adding a LUT or simple effect even on my 8 core mach with 16gb ram would take time to render. Sometimes you just have to suck up to the render and leave it doing its thing, Ive had it say 12+ hours to do an export but then once it finished a section it dropped massively as the effect had finished rendering a Dec 10, 2014 at 14:48
  • FYI if you're exporting an AE comp from AME you're not actually "encoding" the file so to speak, you are first rendering it THEN encoding it. The distinction is important because AME is presumably running a sub-process of AE to get those frames out, to then encode them. So the bottle-neck is actually AE's internal renderer which is horribly inefficient as I think we all know, and definitely does not hit 100% CPU. That is why degraining makes such a big difference! And FYI I believe Adobe moved that to the GPU in the last release...
    – Spencer
    Sep 27, 2018 at 21:27

7 Answers 7

6

There might not be a way. Based on your description of the problem, it sounds like the processing is the slow part. While the video encoder itself is able to do multi-threaded processing, the image processing you are doing may not be able to. By default, Premiere has always done as much parallel processing as possible for me and I frequently see it hit 99% CPU usage on my hyper-threaded quad core desktop.

The problem is most likely that some of your effects require the result of the previous frame to begin processing on the next frame. If this is the case, there isn't an easy way to break it down across multiple threads and that would make it impossible to hit 100% CPU usage on a multi-core computer.

Similarly, RAM wouldn't be expected to cap on encoding as it is a stream operation. There are some things that may need to be tracked over time, but for the most part, it is a stream operation with data going out as fast as it comes in, so there isn't much accumulation of data.

There are two main things that can improve the performance. The first is GPU processing. General purpose CPUs aren't really ideal for many audio/video processing tasks. They are designed for doing a wide variety of general purpose operations, but aren't super efficient at basic operations. GPUs, on the other hand, are designed for doing simple operations very quickly. They can often reduce video processing tasks quite a bit.

A further improvement can be found in purpose built hardware. There are professional cards designed specifically for processing video and provide real time processing and encoding of video. These are often included in high end professional video editing workstations, but the price can be quite high. It isn't unheard of for a professional video editing workstation to be able to get up in the $15,000 to $20,000 range or even higher, just for hardware. They can go for a lot less if you don't mind waiting for renders (I work on a $2,350 system for my video work) but they can also get very high.

6

I'm not happy when I'm rendering unless I can actually smell burning plastic, nothing is more frustrating than AE telling you that your render won't complete until the heat death of the universe, while at the same time your CPU is barely raising a sweat. However there is something you can do about it.

After effects comes with a command line renderer called aerender.exe plain ol' aerender on mac). It's in the AE program folder. Using it is a bit challenging for people unused to the -=awesome -=power of the command line, but it is definitely worth doing if you're doing some heavy lifting in After Effects, and you want to be able to harness all the power of your machine.

The beauty of it is that you can run as many instances as you like. What I normally do is keep adding instances until I start running low on memory. I have a 40 physical core machine and it takes a lot to max out the CPU but here's a screenshot where I finally managed it. Each one of those windows is a new instance of the AE render engine:

enter image description here

In that example the per-frame speed of each renderer barely decreased compared to a single instance, so I achieved a speed increase of roughly 4000%. YMMV, depending on how many cores and how much ram your machine has. I went to the effort because I had a project involving very long and and reasonably complex comps. The project would simply not have got done without this technique, or I would have had to farm it out to a commercial render farm.

Of course the speed-up comes at a price, which is a slight increase in complexity of your workflow. You need to set your comps to render as image sequences, and use the multi-machine settings so that each renderer will look for the next unrendered frame. At the end if you need a movie file you'll have to do a compression pass, but in my case I usually master to a png or tiff sequence, and then run off h.264 encoded copies for clients from the master.

And you need to know a little bit about the command line. Stop looking so glum—that's a GOOD thing, that's a GOOD thing.


Windows

The powershell command line I used in that scenario was:

for ($i = 0 ; $i -lt 40; $i++ ){
  Start-Process 'C:\Program Files\Adobe\Adobe After Effects <<version>>\Support Files\aerender.exe'-ArgumentList '-project', '"c:\path\to\project.aep"'
  sleep 5;
}

This is executed in a powershell window (type in WindowsR and then type powershell.exe). This is a shell scripting language integrated into modern versions of Windows that can be used for all kinds of stuff.

Mac Using Bash version > 3 (default shell in current versions of OSX), or indeed with cooler shells like zsh, you can use this script:

for i in {1..40}; do
  /Applications/Adobe\ After\ Effects\ CC\ 2017/aerender -project ~/Path/to/myproject.aep &
  sleep 5
done

The script works thus:

  • starts a loop which goes for 40 repeats, you can change the 40 to whatever you think is a sane amount.
  • Then it starts aerender as a new process either with the start-process command in PS or with the & in bash, Obviously change the path to aerender.exe / aerender and your project to whatever it is on your machine. A quick and accurate way of filling in paths with the command line is to drag files and folders into it. Also using the tab to auto-fill paths saves a lot of typing and typos.
  • Finally it sleeps for 5 seconds—I found that opening too many instances of aerender too quickly was a reliable way to bluescreen / kernel panic my computer.


If you have multiple comps you can render them as movies but start a new renderer for each one. If you invoke aerender.exe like this

aerender.exe -project "c:\path\to\proj.aep" -rqindex 3

it will render the third comp on the render queue. Putting it into a script you can do:

Windows

$numcomps = 12
$projpath= "C:\path\to\project.aep"
for($i=1; $i -lt ($numcomps+1); $i++){
  Start-Process 'C:\Program Files\Adobe\Adobe After Effects <<version>>\Support Files\aerender.exe' -ArgumentList '-project', '$projpath', '-rqindex', '$i'
  sleep 5;
}

Mac

for i in {1..12}; do
  /Applications/Adobe\ After\ Effects\ CC\ 2017/aerender -project ~/Path/to/myproject.aep -rqindex $i &
  sleep 5
done

That will start a new aerender instance for each comp–once again, change the 12 to however many comps there are in the queue. Note that this technique can run into problems if you have lots of comps in your queue–if you use all your physical memory things will come to a shuddering halt pretty quick. To keep the number of instances down you have to check how many aerender processes are running and only start a new one when a running one finishes. A simpler workaround is to make as many copies of your project as you want there to be instances, with the render queue divided up between them, and then render them all concurrently. So render thread one would be rendering comps 1,2,3, render thread two would be doing 3,4,5, and so on.

More details here (my blog).

5
  • 1
    you are a legend!
    – Tony Sepia
    Jun 22, 2017 at 12:07
  • that's what I keep telling people.
    – stib
    Jun 23, 2017 at 14:37
  • @stib is there anything similar for Premiere Pro? I'm having similar problems as OP between AME and PP. Example from today: AME will take 1h 20m to export a video, PP will export it in 5-10 mins - all at same quality etc.
    – AutoBaker
    Jun 10, 2020 at 16:03
  • Unfortunately there is no command line renderer for PP.
    – stib
    Jun 12, 2020 at 6:45
  • @5Diraptor I had the same issue, if you "render in AME" from PP it opens AME, allocates the leftover resources (after it's reserved them when you opened PP) to AME, which are now shared with PP, so you end up not utilising your full system. You have to close PP and re-open AME and it should (in my case, does consistently) then allocate the full amount of resources (according to your preferences for RAM allowance) to AME now and in turn be much quicker, I saw a 400% increase, huge.
    – Mullazman
    Aug 27, 2020 at 8:51
1

While this question has already been answered; my recommendation:

Use AfterEffects for your rendering if you have a system with a high number of cores and a lot of RAM.

My main workstation is a Dual E5-2697 v2 (24 Logical / 48 Hyper Threaded Cores); with 128 GB of RAM.

If you have to do simply a transcode of a file; or can use Bridge to import your PPro Project into AE (barring your edits will remain intact); the feature under Memory (Edit -> Preferences) allow for you to use Multi-threading.

That being said:

If I am doing just a straight render; or something simple; I can use AE to allocate 32 of my 48 HT cores; additionally; I can allocate 3 GB of ram to each Thread; which is depending on what type of encode I'm doing, anywhere between 2-100x faster in terms of rendering even with my Dual Titan Black GPUs.

This of course; only applies if you have a LOT of cores; and a LOT of ram...

Premiere; rendering through AME; will be limited to using only 1 Thread unfortunately; and will rely on your GPU unless you have that disabled to use software rendering under Project Settings.

Hope this helps!

0

Close After Effects while rendering with Media Encoder. By doing so my CPU usage went from 30% to 60%. I guess because Adobe isn't sparing CPU power for working simulatiously in AE.

1
  • I believe this is due to the dynamic way that Adobe allocates RAM in the background. It will limit the amount of RAM per adobe process, to prevent overflow and whatnot. So by closing one Adobe app you theoretically free up more RAM for another.
    – Spencer
    Sep 27, 2018 at 21:29
0

Can I suggest this is still an issue 5 years on! You need to close all Adobe apps and then open Media Encoder (if that's where you're rendering from) FIRST, then it'll assign the full resources available to that app, I haven't seen it dynamically share resources, it simply allocates CPU-time according to the number of apps open when you start the Adobe app (this is repeatable, try opening LR for example before rendering, it'll only use 10% of the CPU - then open AME first then lightroom and you'll get 100%)

I got a 4x increase in render times because of this. Hope it helps. Adobe's "resource management" sucks.

1
  • Well, that seems to be a USP for AE. So they have every incentive to keep it that way. Unfortunately. Also, there is hardly any competition. And don't even mention the open source products... OpenShot has no proxy support, Blender cannot time stretch properly... and none seems to properly support multicore systems outside of the encoder/decoder: parallelizing video processing would be trivial - just use key frames to split the work into work units (more or less)... or on the app side this wouldn't even be necessary because the key frame distinction is often not relevant there. Mar 2, 2021 at 17:49
0

Its always best to use the command line Aerender. Personally, I have a single user account with adobe, which allows me to install and run AE GUI on one, while using AERENDER on the other. No longer need to put in a text file for render only. Here's the only issue you will run into: Some plugins require separate licensing for each machine, or even render only licenses. The other issue you will run into is that each set of license for adobe software with 2 running seats (1 gui\commandline, 1 commandline render only) will require its own email address. I miss the days when you could have unlimited command line renderers so long as you had the licesnses for your plugins, but they are long gone. Adobe has also forced us to maintain multiple email addresses for this purpose, which defeats the purpose entirely. If they sold off multi-render seating, which would allow more instances to be run concurrently under the same user, similar to their institutional licensing, it would allow them to retain the email address license model, bring in more revenue, and return a once powerful function to the user community. But hey, I'm just sayin... You might, and I wouldn't bet the farm on this, but might, get around it using a set of VM files to host your software on multiple machines with one vm copy holding the software and multiple machines with copies of that VM on a closed network. You only need to update one VM, then copy it over to other machines and run it all inside a closed network with no internet (no way to limit license use). Not perfect but it might work if you only utilize it on a per project basis, allowing the main vm to connect to the internet once every week or so to update etc, and then back to the closed network and copy the vm over... ...Just sayin... There are other ways around this. Look up online render farms. They will take in your project files and render them out quickly. Some even support Premiere Pro and other applications beyond adobe software, and their pricing is pretty fair for the most part. Much faster to run a render with 20 cores and pay $5 to $10 US, get an output file in your desired format and go to print. For 10 bucks, I'll save the headache and just do the upload. Of course, you're best off with a wired network and a ton of bandwidth. Option 2: You could find a boxed CS6 with a valid key, apply it to your licensing online, and install the full on your main box, then buy a bunch of i5\i3 machines slap some nvidia 1-2gb cards in them, add 4-16gb of ram, and network them together. Then you could hook them to RAID NAS drives, one input, one output, and use the Render only setting from cs6 to have your very own render farm. This isn't cheap. About 5 grand for all of it with a 20 core running farm. You then have to install something to initiate the farm on each machine. There are some good softwares for that too. Then, open premiere, save project and files to input RAID, send the entire sequence to after effects, save that file with a render queue, and put into your farm initiator, let the other 10 machines work on it, Have one machine set up with media encoder to watch the folder and compress the output to a final file on your main machine. You can have a coffee (or a beer), then start the next project, and since there is only one set of writes going to the input nas, but many reads, it will be slow going at first. I suggest a third device for cache files, and raid-ed ssds on all of them (get cheapos, it'll be okay, just factor in the wear and tear to your pricing for clients; for personal work or sidehustle, you're better off with used disks, as cheap as you can get them, and replace them when they begin to have problems).

0

How to speed up Media Encoder to use 100% of your cpu? Here is an example: Create a file: settings.ini place inside of it

[Settings]
mode=developer
mthread=enabled
mthreadcount=3  (for a 4 core 8 threaded machine say, 7)
sli=false
slimode=all
priority=high
scaling=low
cloud=false
loadwhenneeded=true
speakers=2

Place in Adobe Premiere, After Effects, and Media Encoders executable folders.

For a quad core / 4 threadded machine like mine, this made Media Encoder jump from 1 core to using 96% of all four. You must watch the youtube video "Render 300X FASTER | Premiere / After Effects / Media Encoder " I'm successfully using this with CC 2018 products.

1
  • Wow, now that's interesting!
    – Spencer
    Sep 27, 2018 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.