3

I have some videos with a speaker looking at the side of a camera or directly to the camera, but his or her pupils were running from side to side as speaker reads from teleprompter.

Is there any possibility to move pupils slightly to make them look exactly at the viewer?

Improve this question

1 Answer 1

Reset to default
3

That would be a task of compositing using something like Nuke or After-Effects. Here's how I'd tackle something like this:

  • Grab a frame of the person looking directly in the camera
  • Track the positions of the eyes ("track_left" and "track_right" as seperates)
  • Mask out the eyes from the still-frame and separate them into eye_left and eye_right
  • Link each eye to the corresponding tracking, so that it follows the original eyes.
  • Perhaps apply a subtle slow flickering (add curves and in the effect-opacity type in the expression "wiggle(0.5, 15)" or something in that area, using After-Effects)
  • When the person is looking down, try to keyframe the opacity of the eyes so the real eyes are revealed from time to time, depending on how convincing this turns out.

If the tracker doesn't stick very well you can try the mocha-tracker which comes natively with After-Effects for the latest versions. Make sure to only mask out the inner area of the eyes - don't patch out too much of the eyes and only the bare minimum to remain realistic shadows, wrinkles, etc.

I hope this works out for you - if not, please post your results and we'll see how to improve it.

Improve this answer
5
  • Is there any automated solution?
    – Roman Matveev
    Commented Jul 13, 2020 at 13:33
  • 1
    The only automated solution I could think off would be developing a method involving deepfakes. I can help you get started on deepfakes but an application such as this would require a lot of research-time. You'd be better of spending a few days compositing if you're asking me.
    – Florian Claaßen
    Commented Jul 13, 2020 at 19:15
  • I read somewhere that Apple's going to use AI to auto-correct your gaze on facetime, so it looks like you're looking into the lens, and not staring at yourself on the screen. The implications of eye-tracking from an always-on, front-facing camera are a tad Orwellian if you consider that it'll also be possible to know which on-screen element has your attention at all times.
    – Jason Conrad
    Commented Jul 15, 2020 at 22:16
  • BTW. The best solution is to have your teleprompter further from your talent, or to make the text as narrow or small as possible, while still being legible to the talent.
    – Jason Conrad
    Commented Jul 15, 2020 at 22:20
  • 1
    ... or, in situations where you can get away with it, don't use a teleprompter. Instead, have someone off-screen read the lines to the talent, one-by-one, and say, "repeat after me." I like this method because it alleviates the cognitive dissonance associated with reading, and allows the talent to focus on performance.
    – Jason Conrad
    Commented Jul 15, 2020 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.