I've been reading up on video compression. I understand that 'raw' media file sizes are to big for practical usage so they have to be compressed, but I'm wondering what happens when the video is being played back.

I mean is the whole video decoded (decompressed) and stored somewhere in memory (this doesn't seem possible) or are the individual frames decoded one at the time and then forgotten, so that when you want to replay a portion of the video you need to decode it again?

It's probably a very noob question and it's because I am one, so would appreciate any info.

  • 1
    Yes, the player only decodes frames as needed. The decoder's buffer stores all frames needed for the current queue of frames being played. When you replay a portion you played some time ago, it has be re-decoded. However, players may maintain a cache so a portion replayed very soon after its last play may still be in that cache. – Gyan Oct 8 '18 at 5:04

In most common compression schemes, there isn't a 1::1 correspondence between "video" frames and "file" (or stream) frames **. Any particular video frame may require the contents of several file frames to become a complete viewable image. So a player will read and buffer as many file frames as needed to reconstruct the video frame in question. The player may buffer much more, but it's not required to.

For example, if some video frame ends up as a P frame when encoded in the file, the player will need to look back to find the previous I frame, decode that, then apply the differences encoded in the intervening P frames to get the complete result.

** EDIT: There is of course a file frame for each video frame, but the point is that you can't usually read one frame from the file and get a complete video frame: that's only true for I frames.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.