This technique could radically change how VR media is captured
Thanks to breakthroughs in technologies like virtual reality (VR), video is becoming more immersive than ever. Google is now showing off a new technique that turns 3D video into an almost lifelike experience.
Using an array of 46 cameras to create “light field videos,” it allows viewers to change their perspective within the scene. They can also peer around objects in the frame to get a better view. Although it’s a complicated setup, it could redefine the way that media is recorded.
One look at Google’s light field video camera array is enough to scare away most people. The globe-shaped setup contains 46 individual cameras that work in unison to capture footage of a scene. After recording, the footage from every camera is merged into one video.
When it is played back, viewers are able to adjust their perspective within the video as it responds in real-time. The combination of high-definition video quality and being able to move freely creates a truly immersive scene.
Notably, the technique differs from existing methods of capturing video for VR applications. They typically use stereoscopic 3D. This allows users to look around a scene from a stationary point but doesn’t allow them to change perspectives. The additional element added by Google’s new rig does a great job of making videos more realistic. It’s also eerily similar to the “enhance” scene in 2007’s “Blade Runner.”
1. How to use subtle AR filters to survive your Zoom meetings?
2. The First No-Headset Virtual Monitor
Interestingly, the capture rig is so large that changing perspective within a video can actually reveal new parts of a given scene. For instance, viewers can move over to look around a corner or examine another side of an object. Since the video is rendered in 3D, slight changes in the viewpoint are accurately displayed.
Google plans to present its new technique at SIGGRAPH 2020 later this summer. Obviously, the conference will take place online due to the COVID-19 pandemic.
Technically, Google’s new video capture technique can make videos that are playable in 2D. Those who browse with Chrome and who have experimental web platform features enabled can actually see a few example videos on the project’s website.
While the demos are certainly captivating, they are even better while watching them through a VR headset. Imagine being able to not only move your head in a swivel motion to change the scene but to move it in almost any direction. With high-definition video that responds in real-time, VR will be much more realistic.
If the rig looks familiar, that’s because it somewhat resembles another Google project that emerged last year. The tech company unveiled a giant LED egg that used a similar camera array to capture human motion and extremely accurate 3D models.
These recent breakthroughs make it clear that Google, specifically its AI division, is eager to make traditional media more immersive. It will be interesting to see how similar tech is integrated into smaller cameras in the coming days.
