How to use chroma key to replace design of complex object?

Question

For an upcoming shoot I need to create a classroom set, but I don't have access to the school tables I need to create the desired effect; instead, I was thinking about filming with simply IKEA tables painted in blue/with tracking markers and replacing with a 3D model. I've never tried anything like this before, nor do I have PP/AE (I mostly use Hitfilm basic and Blender, + Fusion 360 for 3D modelling (my only paid software)), but I need the actors to interact with the table, placing objects on it, etc. so I can't just superimpose the model over a blank plate.

I'm mostly trying to find the most fail-safe way over necessarily the easiest, although I don't exactly want to spend much money, hence why the chroma key solution looks appealing. The end goal is to map the model onto the real table (which is of the same shape), not to use chroma key necessarily.

I was wondering if anyone has any experience with this sort of thing, and if so, I'd like to hear any potential pitfalls or advice about full-scale object replacement of this form (maybe chroma key is not the simplest method?). Advice on lighting the complex shape sufficiently, or placement of tracking markers, is also appreciated.

Answer 1

What you are thinking about is a very great way to save production costs and is used in many cases on many sets. Here is what your pipeline would look like:

• Shoot the footage with green tables
• Track the movement of the camera (using pfTrack preferably, but after effects can do the trick too)
• Importing the tracked movement into any 3D-software such as blender
• modeling the table so it fits the dimensions
• moving the 3D-camera using the tracked data, so the 3d-camera matches the real camera.
• rendering the sequence (preferably as an image sequence, such as .exr or .tiff)
• importing the 3D-asset into the compositing-program (such as After Effects)
• Keying out the green table and "replacing" it with the cgi table.

These steps are requiered only if you have a moving camera. If you don't, you can scrap points 2, 3 and 5. So if you want to save effort in the post-production, you can try to lock down your camera and put some artificial movement on your footage once it's composited.

Now, the tricky part is getting the shadows and the perspective right. Make sure to pack something like a grey-ball, or something matte and something shiny, so you can hold them in the frame to have a reference for later. You might need to fake the contact-shadows of the arms and hands when they are placed on the table. All of this is manageable from inside After-Effects though, no need to fire up heavy artillery like Nuke.

Answer 2

It might be false economy to try to do this in post. If you cost the time you will spend getting it to look halfway decent, it might work out a lot more expensive than going to your local furniture dealer or antiques shop and asking if you can borrow / rent a desk that looks like what you actually want.

A lot of smaller shop owners are happy for the publicity and will loan stock items for short periods in return for credits, or let you rent items, particularly things like antiques and second-hand goods. This is a pretty standard thing to do for indie film-makers, and I think will probably work out a lot more convincing than trying to do it with 3D.