1

I have 4 video streams:

  • a color stream A1
  • a black and white stream A2, representing the distance of objects in the color stream A1
  • a color stream B1
  • a black and white stream B2, representing the distance of objects in B1.

Now I want to key between either stream A1 or B1 depending on which is closer to the camera. Being a programmer myself I thought of solving this by running an operation on every individual pixel of the separate streams in order to generate the output video:

if luminosity of A2 is smaller than luminosity of B2, use the color of A1, otherwise use the color of B1

Is it possible to do this kind of operation in After Effects, or is it possible to write expressions like these that run on every pixel?

  • Expressions aren't low level like that. There's all kinds of values you can pull from but if you want a pixel specific solution, you'll have to write a plugin in C++ with their SDK. – LetTheWritersWrite May 20 '17 at 23:09
1

There is a way of doing per-pixel logic such as what you want, by using Blending Modes. Basically the different transfer modes in AE can be though of as Boolean logic operations that go on at the pixel level. The different transfer modes are spelled out here.

So I think I've worked out how you'd do it.

Here are the images I'm using: two depth maps and two colour maps:

Picture 1 and depth matte

enter image description hereenter image description here

Picture 2 and depth matte

enter image description hereenter image description here

As you can see, no expense was spared with the render.

OK so in AE I make a comp with just the depth mattes in it. I set the blending mode of the top layer to Darker Color.

enter image description here

The darker color blending mode does this logic on each pixel:

(luminosity of this layer < luma of layer(s) below)? This layer: lower layer

This means the darker object will appear wherever two objects overlap That gives me this result:

enter image description here

Now we have to turn this into a matte for picture 1, where the black will be opaque and white transparent. To get rid of the parts of the comp that aren't relevant to picture 1 I put this comp into a new comp, along with another copy of depth matte 1, which I put on top.

enter image description here

I set the blending mode of the top one to Difference which can be though of as:

(this layer == lower layer)? 0: abs(this layer - lower layer)

and this is the result:

enter image description here

OK, now I put this comp in a new comp with my colour images. This comp goes above image 1.

enter image description here

In this final comp I set the depth map comp as a Luma inverted matte for picture 1. Note that when you do that it turns off the visibility for that layer (see the eye icon is disabled on the top layer). This is the result:

enter image description here

Obviously there's a bit of a problem in that there are semi-opaque areas. So I need some sort of threshold. I could do this a few ways, but the way I like is with the curves effect. I apply curves to the top layer (the depth matte layer) like so:

enter image description here

I originally set this to be a hard step at 50%, but that makes the edges of the matte super sharp, so I made it slope a bit around the mid-point. You can play with this to get it right. That's the advantage of using Curves instead of say the Threshold effect.

Finally I got this result, with the figures correctly sorted on the "z" axis:

enter image description here

I hope this all makes sense, and that it works in real life. Thanks for the question, it was an interesting exercise.

-1

I can't take the time to write the expression script now, but that would be your best bet. Writing an expression which could, perhaps reference a totally separate composition; which includes a key (color key/chart or luma key) with the values you want it to identify, then have the expression perform the action based on matching results.

You'd have to play with it to get it to work, but definitely possibly with some tinkering.

  • I'm close to flagging this as "not an answer", but didn't since there is an attempt to answer. Only problem is the answer is wrong: expressions can't do what the OP wants - they can't alter the pixels, only control the properties of a layer. – stib Jul 20 '17 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.