How does green screening work with green objects?

Question

A question that has been on my mind for so long is: How exactly does green screening work with an object to vanish?

Short example: Normal green screens as a background are replaced with an image the viewer sees, like a news show where they show the weather. The weather-man stands infront of the greenscreen and presents the weather - while the greenscreen is replaced with actual information, kinda simple.

But how does it work when a real scene is filmed and only an object should be replaced with the actual background staying intact? I saw a still-image from a Harry Potter film where Hermione got a book in her hand which is being picked up by a green hand, so the hand could be removed later so the book looks like it is floating. How does the computer 'know' the background behind the hand?

Its kinda hard to explain and I cant draw or show you what I mean since i'm posting via phone.

Imagine the camera shot as a 2D Image - Hermione with the book in her hand and the green hand receiving and grabbing the book. As you film this, the hand is opaquing the background it is infront from. Since the hand is removed later, the background will be shown since there is no more hand to see (removed because of the effect for the movie)

How is it possible to see the background? Is the scene once filmed without the hand and then again with the Hand to forge two images together in the end?

I hope I explained it right so you can help me with that, it's really interesting and bugging me at the same time.

Answer 1

For static shots they shoot what's called a background plate: the same scene shot from the same angle but without the foreground action. Then where the green is taken out the empty background shows through.

If the camera is moving this presents a problem. This can be solved by using motion control - basically a robot arm with the camera on it that can repeat the same camera movement with and without the foreground action. This technique is being used less and less, because today even a modest laptop running off-the-shelf software is now capable of tracking moving cameras in 3D. So that the computer works out where the camera was in 3D space, then the background–which can be real or CGI–can be composited with the camera footage.