1

So I had an idea. Almost everyone has smartphone equipped with gyroscope and accelerometer. It wouldn't seem too far fetched to me to strap this smartphone to DSLR and have some app record data from these sensors while shooting video. Then you would load both the video from DSLR and data from your smartphone into computer and use the sensor data to stabilise your footage. Of course you would have to time-sync your data and video and you would probably get just 720 stabilised video out of 1080 original footage because of cropping, but you would get stable video for good coin (if you already have both smartphone and dslr) and in very portable package (unlike 3axis brushless gimbals and seadicams).

Now I am pretty sure I am not first person to think of that. Is there any solution like this? or do you see some obstacle I've missed?

6
  • Cool idea! @AJHenderson's answer below seems to dispute it (he's a really smart guy!), but I like how you were thinking: use the stabilization information from one device and apply it to footage from another, provided they were locked together during the shot. Oct 16, 2014 at 16:35
  • @BrettFromLA - I made some additional updates to my answer. My original post was a bit too harsh on the idea as it does provide some simplification to processing, it just doesn't provide anything you can't theoretically extract with sufficient software processing given the right kind of footage.
    – AJ Henderson
    Oct 16, 2014 at 19:59
  • @AJHenderson Right; that makes sense. It seems as though the OP's idea would gather the stabilization data BASED ON THE VISUALS during shooting, whereas the stabilization data could be generated BASED ON THE VISUALS during post-production and produce exactly the same results. Oct 16, 2014 at 20:39
  • Gyroscopes, gimbals and springs would eliminate the unavoidable (??) blur that comes from stabilizing in post. Oct 16, 2014 at 20:40
  • 1
    @BrettFromLA - not quite exact in every case, but in enough to make the value proposition dubious, yes. As for the blur, yes, that blur is from two things, camera movement during a frame exposure and trying to warp to correct for the perspective of the camera in the wrong place. Physical stabilization addresses both of those, which is why it is pretty much always preferable to stabilization in post unless space constraints prevent it from being an option. Mesh clouds (3d models) like MS Research is doing help reduce warp with more accurate guesses, but are still not perfect.
    – AJ Henderson
    Oct 16, 2014 at 20:49

3 Answers 3

2

This doesn't work, at least not exclusively (more on that after some background). With stabilization, there are two main ways to do it. You can either stabilize during shooting by keeping the camera steady or after by taking unstable video, counteracting some of the motion and adjusting the image to account for other types of motion.

When you stabilize during shooting, gyroscopes, gimbals and springs are used to cancel out forces trying to move the camera to actually keep it stable. This works because the camera doesn't actually move. Things like optical image stabilization and steadicams perform this function.

Correction after the fact is possible, but harder. When a camera moves, the problem is that it changes what the shot looks like. Both angular movement and perspective can vary based on the actual settings of the camera and not just the movement, so there isn't a direct mapping from movement to correction.

There are two main types of camera movement, the camera can move through space, which results in near objects appearing to move more than far away objects as the relative change is greater. (Think of a block directly in front of the lens vs a mountain in the distance, the block can clear the screen entirely while the mountain doesn't appear to have moved.)

The other form of movement is rotation. This makes the rate of change depend on how zoomed in we are (the focal length of our lens). If we have a very wide shot, then slight vibrations don't make much difference. If we zoom in a long ways to a tight shot however, even the smallest vibrations have a major impact on the shot.

None of this information would be available to the data logging on the smartphone, so a lot still needs to be figured out by analyzing the footage. You additionally would need to somehow keep the smartphone and camera together which a) adds weight, b) adds bulk and c) decreases the ergonomics. The data really isn't that valuable as that information and all the other information we need can easily be gained simply from looking for high contrast areas of the image.

Stabilizing after the fact works by doing edge detection on each frame and looking at how the edges move from frame to frame. These edges provide details about where things are in the scene and allow us to actually recreate the motions of the camera and the objects within the scene by looking at all the changes. If they are consistent overtime with a stationary scene, we know they are static objects, if they don't, we know they are moving objects.

So, basically, the data the smartphone would gather isn't needed, isn't that helpful and adds too much disadvantage in trying to capture it to justify the gathering in the first place. It could possibly save on some post processing load to simplify calculations from edges or provide improvements in cases where the footage lacks good trackable subjects, but generally, I wouldn't expect it to be worth much. If it costs more than $100 or so, you are better off going with hardware stabilization that will provide far more benefit.

For more of an idea about what can be determined about video just from analysing the footage itself, I suggest taking a look at Microsoft Research's Hyperlapse Project. Pay particular attention to the path they are able to derive as that is the same thing a device lack this would provide the rotational and acceleration data would be helpful towards determining.

1

SteadXp works like you describe, although it is still a product in development. Also, it uses its own accelerometers, rather than using your cell phone's. Presumably, this is simpler than relying on the myriad protocols of various cell phone manufacturers.

1
  • Data gathering on the phone wouldn't be that hard. It's available to the OS on most phones and can be accessed in a standard way through that. The bigger trick would be reliable mounting.
    – AJ Henderson
    Oct 16, 2014 at 19:53
0

The Sony a7c, FX9, RX0, a7siii and ZV-1 put the gyro data in metadata. Sony provides free software (Sony Catalyst) to use the gyro data to stabilize video in post where you can choose how much of the frame is cropped for stabilization.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.