2

I recently bought Google Pixel and have gotten into blogging. Occasionally I'll turn off image stabilization when using a tripod, but if I forget to turn it back on it drastically affects the quality of videos.

I recently filmed an entire evening worth of un-stabilized video and tried to fix it in iMovie with the "stabilize shaky video" feature, but it did a bad job and couldn't match what the Pixel is ordinarily capable of.

How is it a tiny smartphone can do such a better job in real time compared to post production? Is it all just algorithms, or is something else going on in hardware?

3

The smartphone has acceleration sensors it needs for various things: knowing its orientation (so that it can keep the display content upright), interpolating between GPS fixes in order to give better navigation (and compass readings) than GPS alone would. These acceleration sensors tell it how much it needs to compensate. Dedicated cameras tend to use the acceleration signals for then either moving either a compensation lens (optical image stabilisation) or the sensor appropriately which means that they can still work with long continuous exposures on the same pixel sensor and compensate for sub-pixel shakes. I don't know whether smartphones do that kind of thing: they probably just shift by multiples of whole pixels and read out faster.

At any rate, they don't need to rely on image processing in order to get reliable data for how much should be shifted where in case both smartphone and observed scene are in movement (and if they weren't, why would we be recording a movie rather than a still image in the first place?). They can just use their motion sensors.

Even if they do the compensation purely digitally, they will do it at a faster rate than the final frame rate. And since you are recording not raw but with a motion-optimized lossy encoder, the encoder can well decide "I'll just record a smear here since people will not be able to tell the difference to the details of the blur I have". Except that after motion compensation, the details of the blur might become the details of a static scene and if they have been thrown away, you cannot reconstitute them just because they would be more visible than the encoder reasonably expected them to be.

So for one, the smartphone has better information what motion it should compensate and what not. And for another, it has the raw input before reduction to the frame rate and lossy perception-based encoding to work with.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.