4

To my understanding there are basically 4 types of video:

  1. Ordinary 2D videos - these can be viewed by pretty much any device that has a screen
  2. 360 videos - these can be viewed by any smartphone that has a gyroscope
  3. 3D videos - these can be viewed by 3D glasses along with the right kind of screen, or by using a VR headset
  4. VR videos - these are a combination of types 2 and 3, and can be viewed by any VR headset, like a Samsung Gear VR headset or even a Google Cardboard headset

But I'm interested in whether there's a 5th type of video. My question is, is it possible to make a VR video that isn't just 360 + 3D, but also makes use of positional tracking? Such a video, if it existed, would only be viewable on a high-end VR headset like the Oculus Rift and the HTC Vive, which utilize both rotational tracking and positional tracking. They wouldn't be viewable on low-end headsets like Google Cardboard, or mid-tier headsets like Samsung Gear VR, since they only utilize rotational tracking.

If it is possible, how would you go about creating such a video? It's the equivalent of shooting an ordinary (360+3D) VR video not just from one point in a room, but from every point in the room. I imagine that it would be useful for creating virtual tours. In any case, even if you managed to create such a video, how exactly would you make that video by viewable by others? Are there any apps for Oculus Rift or HTC Vive that would allow you to play a positional-tracking VR video?

EDIT: I found out that there is such a type of video. It's called a "light field video" or "holographic video", and it can be captured with a special light field camera. But I still don't know how you would show that video to someone.

1
  • There is a lot of ongoing basic and applied research into light field capture, display, and the technology to get from one to the other. There are similarities to what you're describing, but nothing commercially viable yet. Jul 3, 2017 at 17:20

2 Answers 2

1

You can use photogrammetry if you had enough different viewpoints to build a 3d representation of the scene and apply all necessary texturing and lighting. The video would then play as an animated 3d scene rather than a traditional movie.

The amount of processing involved is high but it is possible. Microsoft's Hyperlapse video stabilization actually does some of this by determining a point cloud of the scene from some basic photogrammetry.

Software like Destinations on the Vive already use photogrammetry for reproducing landmarks in a fully explore-able 3D world. There are also several animated projects which are composed of scenes that you can fully explore. 8i also has done some early work in video like you describe. The quality level still leaves a lot to be desired, but work is being done in trying to accomplish what you are talking about.

As an early Vive owner who has tried all these things first hand, I'd say "volumetric video" is at the interesting experimental phase, but not really that great yet. I'm sure we'll see significant improvements in the future though as the technology is refined, particularly since it has implications outside of just VR and AR for things like computer vision.

7
  • I found out that these videos are called "light field videos" or "holographic videos". I guess "volumetric videos" is yet another word for them. In any case, I have an Oculus Rift, not an HTC vive. Do you know any apps or experiences for Oculus Rift that show such videos? Jul 1, 2017 at 4:07
  • Light field is something different. That's what the Lytro shoots and is still single direction but contain focusing and depth information as it records the direction of light rather than just the intensity.
    – AJ Henderson
    Jul 1, 2017 at 4:25
  • I think some people are using the term "light field video" as a synonym for "360 light field video", i.e. the sort of thing that the Lytro Immerge shoots, which is synonymous with "volumetric". In any case, it looks like that "8i" app you were referring to is available for Oculus Rift: store.steampowered.com/app/462530/8i__Make_VR_Human Do you know any other Oculus Rift apps or experiences that showcase volumetric video? Jul 1, 2017 at 4:37
  • Volumetric means it has volume, ie geometry, light field is significantly different in how it works since light field is still relatively fixed perspective.
    – AJ Henderson
    Jul 1, 2017 at 4:52
  • I'm seeing people use light field to describe what 8i is doing, and volumetric to describe Lytro Immerge (and vice versa). Are 8i and Lytro Immerge (as opposed to regular Lytro) doing fundamentally different things? Jul 1, 2017 at 5:05
1

For this you are going to have to build a 3d-world. This is because if you want to move position you are going to have to be moving in 3d. The easiest way to build up the 3d scene is with photogrammetry like Aj Henderson said.

I would load it into a game engine because a game engine is designed to render 3d scenes read time and already lots of them have vr features built in. For example Unreal Engine or Unity.

2
  • But then how would you show such a video to someone else? Are there any apps for Oculus Rift besides 8i which show such videos? Jul 1, 2017 at 19:09
  • 1
    To show the video you would have to have a full game set up in the game engine which you can export. In 8i case they used the unity game engine as the base for their 3d environments.
    – Tim
    Jul 1, 2017 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.