Why don't phones correct vertical filming?

Question

Many people get really mad at filmers suffering from the Vertical Video Syndrome and I find it hard to disagree. Phone-shot videos filmed vertically are really annoying to watch.

But there's one thing I really don't get; why doesn't the phone correct that? The lens itself is always perfectly round and obviously doesn't really have an up- or downside. As far as I can imagine, shooting every video as if the phone was held horizontally would just be a formality in the software.

Is there any reason why that's never implemented? Is there any use in vertical filming?

Answer 1

There is indeed no technical limitation that would prevent us from preventing vertical videos in software. The issue is more of an UX/usability problem. While the sensor would allow us to capture virtually any part of it (video has usually always a lower resolution than the sensor can provide, when making videos the phone just records a small portion of the sensor area) we kind of don't want an auto correcting orientation as its rather unintuitive. The emphasize is on kind of.

Your phone screen is 9:16 so in order to show 16:9 video on your phone screen you either need to turn your phone by 90° or show the video with giant black bars on the top and bottom, which would be very annoying when recording a video. It's quite unintuitive to record horizontally while in a vertical position, so from a UX design perspective you would say the user wants to shoot vertically when holding its phone vertically because thats what you would usually expect your phone to act like. Hold it vertical you shoot vertical, hold it horizontal you shoot horizontal, a pretty intuitive concept.

Though its still no reason to not at least implement an option to auto correct this and my guess why phone manufacturers aren't doing this is probably largely due to increased engineering costs in the camera module (afaik it can't be done purely in software, you would probably need to alter the functionality of your camera module a bit).

Users who shoot vertically don't really care about the fact that they have a horrible aspect ratio as they mostly just watch their video on a phone anyway. Vine is a good example for that, it really promotes doing vertical video because the aspect ratio of the device that your video will be watched on most of the time has the same vertical aspect ratio. So the demand isn't really that huge so far to warrant such a feature but I'm certain we will see this at some point.

Answer 2

To start answering a question like this you need to first look at why we shoot video horizontally. Human vision is not a square, we see much wider than we see tall. Early TV was more or less square due to limitations of technology at the time (it's hard to make an electron beam go really far to one side without lots of extra cost or space.) Theater however didn't have this problem and so they stuck with something that more fits human vision to make the most use of our ability to see.

Portrait video doesn't make much use of the range of our vision, but it is used in portraiture because it more naturally captures the human form (though not normally in such a narrow 9 by 16 ratio.) Thus, some people end up naturally holding their phone to shoot video in portrait since it is both how they hold a phone and how they are used to seeing photos of individual people.

To an extent, this even makes sense for videos that will be played back on the small device where the screen size is a bigger limitation than the visible area of our vision. For a situation like a video conference for example, portrait video makes a lot more sense, but for anything that will be seen at a larger size, portrait doesn't make sense as it drastically limits the amount of visual field we can fill.

With that established, there are two things that could be done to help with not shooting portrait. One, you could use the accelerometer to detect vertical filming and simply prevent it (or give a warning), but this is a frustrating artificial limitation to put on users, particularly since there are some situations where it could be valid.

The second potential solution is to simply record a 16x9 image regardless of phone orientation when recording video. Most modern smart phones have sufficient resolution to expose a 1920 pixel wide image in either direction since a 1080p image is really only a 2MP or so image. The problem comes down to optimization. In general, image sensors take a while to read out data from the sensor. This is why, for example, a high end DSLR may be able to shoot 60p video but can't take more than 7 or 8 full resolution photos at a time.

To get around this readout speed problem for video (which needs 24 or more frames per second), only a portion of the sensor is actually read. This is often hooked up to special hardware that will only read the lines of the sensor that are needed for the video and skip the others. In sensors of this type, it would need additional hardware to skip the appropriate lines in both orientations and would increase the cost and complexity of the sensor.

Additionally, you then have a display problem. If you are shooting a 1080p video while the camera is vertical, no phone that I am aware of has sufficient resolution for a 1:1 display of the captured video horizontally across the screen. You could scale it down, but the aspect ratio of the phone would still make it use a very small portion of the screen when recording, making it hard to see what you are doing clearly.

So there are a lot of factors why a phone can't simply shoot horizontally regardless of orientation, ranging from the fact there are some good reasons to use the camera for video in a portrait orientation on occasion, the sensor itself may not support it and the user interface when doing so would be limiting.

Answer 3

I think you need to think outside the (16x9) rectangle a bit. While a lot of video filmed in portrait mode on a phone looks rubbish, that doesn't mean that filming in portrait mode is necessarily wrong — after all a lot of landscape video filmed on a phone looks rubbish too.

At my work we use 9x16 video a lot. While the format may not lend itself to the type of content you see in a cinema, there are times when portrait video works really well. For instance we have an installation comprising a projection filling the rear wall of a corridor, where full-size standing figures react to the viewer as they walk toward the screen. We put a Red One on its side and filmed it portrait because it suited the content, and we were able to use the full resolution of the format by doing it that way.

Internationally renown visual artist Bill Viola uses portrait mode video a lot, and his work is some of the most polished cinematography you'll see. A lot of his works again involve standing figures, so the ratio suits the content.

While landscape mode matches the way humans see, we don't have to limit ourselves to trying to recreate reality. Using portrait mode video creates visual tension because of the narrow field of view. It allows a full body shot to be much closer than in landscape mode, allowing for more intimate portraits of standing figures. I'm sure there are countless other examples of good uses of portrait mode video, the point being that it's the content that determines whether the form is appropriate.

Answer 4

While I mostly agree with what Professor Sparkles says above, note that Google is attempting to convince people not to film vertical video. In this YouTube video they show the new overlay that is displayed while filming vertically on Android phones, gently encouraging the user to turn their camera.

Answer 5

It is supposed to be that way!!! It is recorded in vertical orientation and designed to be viewed in vertical orientation on a similar device to which it was recorded. This "problem" is not a technical limitation as has been pointed out, it is a user education issue. You are watching it wrong, and the fact that it looks terrible is a reminder of your mistake.

Answer 6

In 2014, when this question was first asked, it made sense to decide the viewing orientation of the recorded video up front, and to discard (or ignore) extra pixels in order to reduce the data storage requirement of the saved video. As compression algorithms improve, and storage costs fall, this practice will become less of a technological imperative, and more of a stylistic choice or artifact of historic convention.

Camera lenses almost* always cast a round image circle on a focal plane. When sensor manufacturers decide to make sensors non-square, they are attempting to maximize the utility of the sensor's surface area based on popular trends in video consumption. Because a screen's viewing orientation is never guaranteed, and as users expect devices to adapt orientation on-the-fly, we will gradually see more cameras recording 1:1 aspect ratios, and displays cropping as necessary.

In a sense, we already see this with 360º videos, which capture a spherical image, and selectively crop that image based on accelerometer input during playback. As machine learning and computer vision techniques improve, this trend will continue beyond "panoramic" imagery (where images are mapped to a sphere, cylinder, or other 3-D manifold) and progressively include true depth information. This field is rapidly advancing in 2020. Here are a few links to recent progress, which I'm sure will become outdated very quickly: Neural Radiance Fields for View Synthesis, and Consistent Video Depth Estimation

*The exception here are anamorphic lenses, which are more expensive to produce than normal lenses.

Answer 7

There might be a technical limitation: The CCD sensor.

If the CCD is not square, a mechanism needs to be created so that it is always stabilized to the ground level (something more elaborated but like the naval compasses that keep their horizontal level no matter the sea waves). If it is not the CCD, then there should be a window that makes the 16:9 or whatever aspect ratio that the camera offers to limit the CCD sensor reception area, and in this case that window is what needs to move or part of the captured image is discarded to adjust that image to the aspect ratio. The truth is that I'm more inclined to not squared CCD sensors, otherwise part of the CMOS memmory will be lost which is a waste of money, and in that case seems more logical to have a squared picture or movie that you can modify to the aspect ratio you like (being you the one that discards the part of the video to match the aspect ratio).

Answer 8

To answer your question, this could be done with software quite easily. A better solution would be to darken the screen like web pages do when they want your attention. Then nag the user to death about how much nicer a video would be horizontally. Then say, are you sure you still want to hold the phone vertically? Are you really, really sure? For those of you who defend vertical filming, I ask you, would you watch a movie in your theater if it were filmed vertically. Why not? Videos shown from phones are fine vertically, but people usually share them to other media including news footage.