Does any software have the capability to create a mask/alpha value based on the volume of an audio channel

Question

I'm looking for an automagical way to create a Google Hangouts effect, ie. 1. I have a number of files that are recordings of individual conversation members. 2. I want to put together the video tracks into one project, and mask each one when the corresponding audio is silent (below some threshold), and to make it visible when there is audio from that audio track (so basically how Skype/Hangouts looks -> someone is talking, you see their webcam)

I haven't been able to find something like this using Google, or poking around in Premiere and Lightworks, but if it's available in another program, that would be great to know.

The mask doesn't need any motion tracking - either the entire screen is entirely transparent (audio), or entirely opaque (no audio).

Is there any way to do this automatically, or is it something that has to be manually put together?

Answer 1

You could certainly do it in After Effects. If you convert the audio levels to keyframes you can then use expressions / the pickwhip to drive the opacity of the layers. After Effects is not free. I wouldn't be surprised if it was possible in Fusion (freemium) and certainly other compositing tools like Nuke or Apple Motion. Natron, (free) may have the capability. There was an audio to ascii plugin, I don't think it still works though.

You could do it in Blender, because of course you could do it in Blender. Blender is free.

For a more programmatic approach you could use something like Processing or OpenFrameworks (free). You need Java or C++ knowledge to use these.

It is definitely possible in all of these programs, though in none of them is it a beginner-level operation. The simplest workflow would probably be After Effects, though that might be my bias because it's the one I know best.