Facebook published new research today which the company says shows the “thinnest VR display demonstrated to date,” in a proof-of-concept headset based on folded holographic optics.
Facebook Reality Labs, the company’s AR/VR R&D division, today published new research demonstrating an approach which combines two key features: polarization-based optical ‘folding’ and holographic lenses. In the work, researchers Andrew Maimone and Junren Wang say they’ve used the technique to create a functional VR display and lens that together are just 9mm thick. The result is a proof-of-concept VR headset which could truly be called ‘VR glasses’.
The approach has other benefits beyond its incredibly compact size; the researchers say it can also support significantly wider color gamut than today’s VR displays, and that their display makes progress “toward scaling resolution to the limit of human vision.”
Let’s talk about how it all works.
Why Are Today’s Headsets So Big?
It’s natural to wonder why even the latest VR headsets are essentially just as bulky as the first generation of headsets that launched back in 2016. The answer is simple: optics. Unfortunately the solution is not so simple.
Every consumer VR headset on the market uses effectively the same optical pipeline: a macro display behind a simple lens. The lens is there to focus the light from the display into your eye. But in order for that to happen the lens need to be a few inches from the display, otherwise it doesn’t have enough focusing power to focus the light into your eye.
That necessary distance between the display and the lens is the reason why every headset out there looks like a box on your face. The approach is still used today because the lenses and the displays are known quantities; they’re cheap & simple, and although bulky, they achieve a wide field of view and high resolution.
Many solutions have been proposed for making VR headsets smaller, and just about all of them include the use of novel displays and lenses.
The new research from Facebook proposes the use of both folded optics and holographic optics.
What are folded optics? It’s not quite what it sounds like, but once you understand it, you’d be hard pressed to come up with a better name.
While the simple lenses in today’s VR headsets must be a certain distance from the display in order to focus the light into your eye, the concept of folded optics proposes ‘folding’ that distance over on itself, such that the light still traverses the same distance necessary for focusing, but its path is folded into a more compact area.
You can think of it like a piece of paper with an arbitrary width. When you fold the paper in half, the paper itself is still just as wide as when you started, but it’s width occupies less space because you folded it over on itself.
But how the hell do you do that with light? Polarization is the key.
It turns out that beams of light have an ‘orientation’. Normally the orientation of light beams at random, but you can use a polarizer to only let light of a specific orientation pass through. You can think of a polarizer like the coin-slot on a vending machine: it will only accept coins in one orientation.
Using polarization, it’s possible to bounce light back and forth multiple times along an optical path before eventually letting it out and into the wearer’s eye. This approach (also known as ‘pancake optics’ allows the lens and the display to move much closer together, resulting in a more compact headset.
But to go even thinner—to shrink the size of the lenses themselves—Facebook researchers have turned to holographic optics.
Rather than using a series of typical lenses (like the kind found in a pair of glasses) in the folded optics, the researchers have formed the lenses into… holograms.
If that makes your head hurt, everything is fine. Holograms are nuts, but I’ll do my best to explain.
Unlike a photograph, which is a recording of the light in a plane of space at a given moment, a hologram is a recording of the light in a volume of space at a given moment.
When you look at a photograph, you can only see the information of the light contained in the plane that was captured. When you look at a hologram, you can look around the hologram, because the information of the light in the entire volume is captured (also known as a lightfield).
Now I’m going to blow your mind. What if when you captured a hologram, the scene you captured had a lens in it? It turns out, the lens you see in the hologram will behave just like the lens in the scene. Don’t believe me? Watch this video at 0:19 at look at the magnifying glass in the scene and watch as it magnifies the rest of the hologram, even though it is part of the hologram itself.
This is the fundamental idea behind Facebook’s holographic lens approach. The researchers effectively ‘captured’ a hologram of a real lens, condensing the optical properties of a real lens into a paper-thin holographic film.
So the optics Facebook is employing in this design is, quite literally, a hologram of a lens.
Facebook’s Prototype Photoreal Avatars Now Have Realistic Eyes
Researchers at Facebook figured out how to add natural looking eyes to their photorealistic avatar research.
Facebook, which owns the Oculus brand of VR products, first showed off this ‘Codec Avatars’ project back in March 2019. The avatars are generated using a specialized capture rig with 132 cameras. Once generated, they can be driven by a prototype VR headset with three cameras; facing the left eye, right eye, and mouth. All of this is achieved with machine learning.
While the graphics and face tracking of these avatars are impressive, the eyes tended to have an uncanny feeling, with odd distortions and gaze directions that didn’t make sense.
In a paper titled ‘The Eyes Have It: An Integrated Eye and Face Model for Photorealistic
Facial Animation‘, the researchers present a solution to this problem. Part of the previous pipeline involved a “style transfer” neural network. If you’ve used a smartphone AR filter that makes the world look like a painting, you already know what that is.
But while this transfer was previously done at the image stage, it’s now done on the resulting texture itself. The eye direction is explicitly taken from the eye tracking system, rather than being estimated by the algorithm.
The result, based on the images and videos Facebook provided, is significantly more natural looking eyes. The researchers claim eye contact is critical to achieving social presence, and their system can now handle this- a feature you won’t get in a Zoom call.
Our goal is to build a system to enable virtual telepresence, using photorealistic avatars, at scale, with a level of fidelity sufficient to achieve eye-contact
Don’t get too excited just yet- this kind of technology won’t be on your head any time soon. When presenting codec avatars, Facebook warned the technology was still “years away” for consumer products.
When it can be realized, however, such a technology has tremendous potential. For most, telepresence today is still limited to grids of webcams on a 2D monitor. The ability to see photorealistic representations of others in true scale, fully tracked from real motion, with the ability to make eye contact, could fundamentally change the need for face to face interaction.
Iron Man VR Review Discussion (SPOILER-FREE)
Zeena is UploadVR’s video specialist with experience in the VR and gaming industries. Her love for VR started back when she was reporting on the launches of major headsets in 2016, and is flourishing now that she can shine a spotlight on how far the industry has come as our Video Producer. She will be bringing you tons of content throughout the week, including the VRecap, regular gaming and culture showcases, and more – subscribe to our [YouTube] channel for the latest.
SideQuest Teases Revamped Design Coming Soon
Oculus Quest sideloading platform, SideQuest, is getting a bit of a redesign soon, and we’ve just got a first glimpse at it.
The below teaser image was recently posted on Reddit and verified by SideQuest’s Shane Harris. “Those screenshots are accurate since we just took them from the current functional system ( locally at least),” Harris wrote. “We have some new discovery tools too we are working on that will boost discovery for developers & users.”
Overall it doesn’t look too different from what we have now. Harris said this design will still work similarly to the current layout, but you can see some options for filtering on the sidebar and smaller thumbnails will fit more apps onto the front page.
SideQuest celebrated its first anniversary back in May, confirming there were now over 500 titles available on the platform, so those additional tools definitely seem necessary. With SideQuest, content creators can bring their VR experiences to Quest without having to be approved by Facebook’s own strict curation guidelines, provided users opt into content from unknown sources on the headset.
Facebook recently announced that it will be launching a new distribution channel for Quest in 2021. This new method won’t require store approval nor sideloading content, to which Harris said he was delighted to hear.
“We hope that we have played a part in showing the real value of the indie developer ecosystem,” Harris said. “We will be working to integrate with the new direct link system as soon as we can and will be encouraging developers to explore the benefits of leveraging the Oculus ecosystem for a smoother user experience.”
Delta expects to see an 80% decline in travelers over the 4th of July weekend, and it shows how abysmal things are right now in the airline industry (DAL)
An app helping families save for college used this pitch deck to raise $9 million from investors like Anthos Capital and NBA all-star Baron Davis
Crypto Market Analysis: 1st July 2020
The Trump administration has hired a military contracting firm backed by Trump adviser Peter Thiel to build a virtual border wall
H.I.G. Carves Out Piece of Genuine Parts
19 TV shows Netflix canceled even though critics loved them
How much money TikTok influencers get paid to promote songs in their videos, according to music marketers, creators, and managers
Second Lawsuit Emerges: Why Are Hackers Targeting AT&T Crypto Investors?
I tried Daily Harvest’s new ‘Scoops’ frozen desserts — the chocolate flavor is the best vegan ice cream I’ve ever tasted
Buying a house during the pandemic was a little tricky, but worth it for the rock-bottom rate we got on our mortgage
High-yield savings accounts aren’t earning much interest right now, but keeping my money at Ally Bank is still the smartest choice
Here are all the famous people Jeffrey Epstein was connected to
Bitcoin Breaks Below $9,000 as Sellers Invalidate Bullish Technical Pattern
Lemonade, a tech-driven insurance company, soars 132% in trading debut (LMND)
PiGirrl Zero w/ Audio #3DThursday #3DPrinting
A Massive Star Has Seemingly Vanished from Space With No Explanation
A Russian TV star denied holding racist views after she was fired as a brand ambassador to Audi after Instagram post controversy
Crypto Trading Woes: What They Are and How UpBots Solves Them
Centerbridge Sells KIK Piece to Voyant
Why Isn’t BCH Way More Popular on the Dark Web?
Leaked emails show Amazon is delaying Prime Day again to October as concerns grow that a new COVID-19 demand spike may hit supply chains (AMZN)
Trump’s favorite trade scorecard worsened in May as exports hit lowest level since 2009
Review: Calculator Kit is Just a Few Hacks From Greatness
EYE on NPI – Alorium Evo M51 #EyeOnNPI #DigiKey #Adafruit @digikey @Adafruit @AloriumTech @Intel
Fursuit- or puppet-head base – version 71 – Toon dragon #3Dprinting #3DThursday
St Paul’s bomb plotter now denies she got cold feet, court hears
Twitter Adds Crypto Emoji for Binance, Following Bitcoin (BTC) and Crypto.com Coin (CRO)
Will This Year’s Third Quarter Be Negative for Bitcoin?
Fortnite Fireworks Locations: Where To Light Fireworks For Captain America Challenge
Evolution Managers Backs Another New Manager
Dominus Capital raises up to $350m for Fund 3 but the target size for the fund looks to have dropped down to $525m
Bitcoin Price Prediction: BTC/USD Couldn’t Push Higher; Price Remains Below $9,200
Unknown Cybercrime Gang Holds Thousands of Databases For Ransom
Peter Schiff: Bitcoin HODLers Remain Delusional
EARN IT Act amendments transfer the fight over Section 230 to the states
Fable Trademark Renewed Ahead Of Xbox Series X First-Party Showcase
David Starkey widely criticised for ‘slavery was not genocide’ remarks
Lithuania to Launch CBDC This Month — Intended for Collection Purposes, Not Trade
Facebook’s Prototype Photoreal Avatars Now Have Realistic Eyes
Sun Sets on Offshore Banking as Assets Worth $11 Trillion Uncovered
New York Times1 week ago
Gen Z Will Not Save Us
Gaming1 week ago
All safe codes – The Last of Us Part 2
BBC1 week ago
Ron Jeremy: Adult star charged with rape and sexual assault
New York Times1 week ago
The Boy Who Cried Fake News
Gaming1 week ago
Valorant update 1.02 Patch notes add ranked mode and surrender option
Gaming1 week ago
The Last of Us Part 2 voice actors and cast
Blockchain1 week ago
New Class of Crypto Assets Will Outshine Bitcoin (BTC) in Next Crypto Craze, Says Polyient Games Executive
zephyrnet7 days ago
Compound (COMP) is now available on Coinbase Earn