3

As far as I understand, the motion estimation is a process, where when we have two frames t and t+1 we search for the blocks/pixels that changed position and then encode that change with a motion vector. So in the end we have only the frame t and its encoded successor.

After that we use compensation in order to use these motion vectors, and further encode only the difference between the "vectorized" frame t+1 and the frame t.

Firstly, am I right? Secondly, when the prediction issue comes into play? I'm reading about prediction all the time, but in the sequence I've described, I don't see any prediction happening anywhere.

2

The prediction comes in when actually adding the motion estimation to both frames t and t+1. Consider a video of a ninja cutting a melon in half, but shot at a very short shutter speed, so as to freeze motion in each frame. The motion estimator notices that the sword is moving between frame t and frame t+1. (It also moved between frame t-1 and t, and also frame t+1 to t+2). Frame t gets half the motion from t-1 to t and half the motion from t to t+1. The latter half is the prediction. Frame t+1 gets half the motion from t to t+1 and half the motion from t+1 to t+2. In this way, the video does not appear to shift half a frame later than the audio.

| improve this answer | |
  • So by estimating the motion of the frame t+1 (based on frame t), we're also "predicting" half the motion of the frame t+2 (the "starting points" of its own motion vectors)? – Arkoudinos Nov 29 '16 at 1:53
  • Nothing in t+2 contributes to the estimation of frame t. Frame t gets half its motion estimate from the immediate past (frame t-1) and the immediate future (frame t+1). It doesn't use the full motion difference between t-1 and t or t and t+1. It should use at most half of those motion vectors (minus it's own instantaneous motion). It could use less than half, depending on how much blurring you want. – Michael Tiemann Nov 29 '16 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.