My question is related to the technicality behind video overlaying. When a 3D object or an image is added to a video sequence as an overlay to render, does ffmpeg fully decode the original sequence to a raw file, then mix the object or image to the desired position and re-encode back to the desired format?. I'd like to know if this is the process followed by the industry to mix 3D objects, images or other material with generic video contents. Or is there another approach to perform this task. Your assistance is highly appreciated. Thanks.
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Yes, compositing, as it is known in the industry, involves decoding two or more assets in to video streams and them combining those streams in some way. The end result is then saved as a new file on render. There is some variation in when rendering is done though. Some software can put off rendering for a while to reduce the number of re-encodes required.
For example, if combining two layers of video in After Effects, you can embed the entire composition in to Premiere rather than re-encode the video to a lossy format. This prevents further generations of loss without having to store the lossless render while working on the video (which would be very large).
Alternately, lossless file formats such as TGA sequences can be used for storing the result of a render without an additional generation of loss (since the render is preserved perfectly.)
The final render still uses the same basic process of using some blending algorithm to combine the pixel data from two or more video streams in to a single stream though and that stream eventually has to be stored again.