Connect with us

AR/VR

Facebook Reality Labs continues to focus its efforts on EMG-based neural interfaces to control AR/VR devices

Published

on

In Augmented Reality and Virtual Reality News

July 16, 2021 – Facebook Reality Labs (FRL), Facebook’s Augmented and Virtual Reality (AR/VR) research team, is celebrating new results published by its University of California San Francisco (UCSF) research collaborators in The New England Journal of Medicine, that demonstrate how someone with severe speech loss has been able to type out what they wanted to say almost instantly, simply by attempting to speak. In other words, UCSF has restored a person’s ability to communicate by decoding brain signals sent from the motor cortex to the vocal tract. FRL stated that the study marks an important milestone for the field of neuroscience, and it concludes Facebook’s years-long collaboration with UCSF’s Chang Lab.

Established in 2017, Facebook Reality Labs’ (FRL) Brain-Computer Interface (BCI) project began with an ambitious long-term goal: to develop a silent, non-invasive speech interface that would let people type just by imagining the words they want to say.

The FRL team has made progress on this mission over the course of four years, investing in the exploration of head-mounted optical BCI as a potential input method for the next computing platform — essentially, a way to communicate in AR/VR with the speed of voice and the discreteness of typing. 

However, in a lengthy blog post, FRL stated that Facebook has no interest in developing products that require brain implanted electrodes, and that while it still believes in the long-term potential of head-mounted optical BCI technologies, it has now decided to focus its immediate efforts on a different neural interface approach that has a nearer-term path to market: wrist-based devices powered by electromyography (EMG). 

FRL explained how EMG works: When you decide to move your hands and fingers, your brain sends signals down your arm via motor neurons, telling them to move in specific ways in order to perform actions like tapping or swiping. EMG can pick up and decode those signals at the wrist and translate them into digital commands for a device. In the near term, these signals will allow users to communicate with their devices with a degree of control that’s highly reliable, subtle, personalizable, and adaptable to many situations. The company added that as this area of research evolves, “EMG-based neural interfaces have the potential to dramatically expand the bandwidth with which we can communicate with our devices, opening up the possibility of things like high-speed typing.”

We’re still in the early stages of unlocking the potential of wrist-based electromyography, but we believe it will be the core input for AR glasses, and applying what we’ve learned about BCI will help us get there faster.”

FRL Research Director, Sean Keller, commented: “We’re developing more natural, intuitive ways to interact with always-available AR glasses so that we don’t have to choose between interacting with our device and the world around us. We’re still in the early stages of unlocking the potential of wrist-based electromyography, but we believe it will be the core input for AR glasses, and applying what we’ve learned about BCI will help us get there faster.”

Speech was the focus of FRL’s BCI research because it’s inherently high bandwidth — you can talk faster than you can type. But speech isn’t the only way to apply this research, according to FRL. BCI team’s foundational work can also be leveraged to enable intuitive wrist-based controls. As a result, FRL is no longer pursuing a research path to develop a silent, non-invasive speech interface, and is instead going to pursue new forms of intuitive control with EMG as it focuses on wrist-based input devices for AR/VR.

“As a team, we’ve realized that the biofeedback and real-time decoding algorithms we use for optical BCI research can accelerate what we can do with wrist-based EMG,” said FRL Neural Engineering Research Manager, Emily Mugler. “We really want you to be able to intuitively control our next-generation wristbands within the first few minutes of putting them on. In order to use a subtle control scheme with confidence, you need your device to give you feedback, to confirm it understands your goal… Applying these BCI research concepts to EMG can help wrist-based control feel intuitive and useful right from the start.”

FRL concluded by saying that it will continue sharing more as work progresses, and that later this year, it will share more about how haptic wearables will add another dimension to the next computing platform and will “enhance our ability to establish presence and learn new interaction paradigms.”

For more information on Facebook Reality Labs and its research into Augmented and Virtual Reality, click here.

Image credit: Facebook Reality Labs

About the author

Sam Sprigg

Sam is the Founder and Managing Editor of Auganix. With a background in research and report writing, he covers news articles on both the AR and VR industries. He also has an interest in human augmentation technology as a whole, and does not just limit his learning specifically to the visual experience side of things.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.auganix.org/facebook-reality-labs-continues-to-focus-its-efforts-on-emg-based-neural-interfaces-to-control-ar-vr-devices/

AR/VR

Warplanes: WW1 Fighters to See Official Oculus Quest Store Launch This Week

Published

on

Back in March, Home Net Games launched the popular Warplanes: WW1 Fighters for PC VR headsets as well as Oculus Quest via App Lab – making it one of the earliest. The studio has now announced that Warplanes: WW1 Fighters will be officially released on the Oculus Store at the end of July.

Warplanes: WW1 VR

Whilst App Lab has been a great way for indie developers to get their projects onto Oculus Quest finding them is another matter, with services like SideQuest or App Lab DB simplifying discovery. The end goal for all developers is to get their videogame onto the proper Oculus Store and that’s exactly what Home Net Games is going to achieve this month.

Through App Lab Warplanes: WW1 Fighters has received lots of positive feedback for its World War 1 aerial combat, where you can jump into the cockpit of authentic planes from the era. With single-player and multiplayer modes where you can complete campaign missions or go head-to-head with other pilots, multiplayer battles include Deathmatch, Team Deathmatch and Last Man Standing or team up with friends on co-op missions. Warplanes: WW1 Fighters features 18 planes; from light scout planes to heavy bombers. Progression will allow you to build your own squadron, upgrading and customizing your planes.

These can realistic or simple to control thanks to Warplanes: WW1 Fighters‘ customizable flight models. Set them for a more arcade-like experience or go full simulation with real-life aerodynamics. Gameplay options allow further in-depth tweaking such as enabling manual takeoffs and landing, aggressive AI, disabling HUD and realistic damage which affects flight.

Warplanes: WW1 VR

Warplanes: WW1 Fighters supports Oculus Quest 2’s 90Hz mode for improved performance, great for those dogfighting exchanges. The studio also confirms that: “If you purchased the game on App Lab, the game will be upgraded to the Oculus Store version with no extra cost. All saves and settings will be compatible with the new version. Of course, you’ll be getting all future updates as well.”

Home Net Games will be officially launching Warplanes: WW1 Fighters for Oculus Quest on 29th July 2021. For further updates on the latest App Lab titles, keep reading VRFocus.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.vrfocus.com/2021/07/warplanes-ww1-fighters-to-see-official-oculus-quest-store-launch-this-week/

Continue Reading

AR/VR

Facebook is Unlocking AR Capabilities for Developers on Quest 2

Published

on

Facebook announced today that an upcoming update to the Quest development SDK will include experimental support for a Passthrough API which will allow Unity developers to build AR experiences and features into apps on Quest 2.

Although Quest wasn’t launched as an AR headset initially, its impressive passthrough camera capability showed that it could handle AR functions surprisingly well. In fact, if you use the passthrough view as your default background, the Quest boundary and main menu floats against the backdrop of your playspace—effectively making it an AR experience that’s been available on the headset for some time.

Until now Facebook kept the tools for building AR apps on the headset to itself, but today it announced it’s unlocking the same capabilities for third-party developers too.

Initially rolling out as an “experimental” feature, the so called ‘Passthrough API’ will become available in the v31 update to the Quest development SDK. The Passthrough API is only available on Quest 2 for now; it isn’t clear if Facebook plans to extent it to the original Quest.

Facebook expects the new capabilities will enable a brand new class of applications on Quest 2 that take advantage of AR in interesting ways. Here’s some examples and explanations of the features developers can use:

Composition: You can composite Passthrough layers with other VR layers via existing blending techniques like hole punching and alpha blending.

Styling: You’ll be able to apply styles and tint to layers from a predefined list, including applying a color overlay to the feed, rendering edges, customizing opacity, and posterizing.

Custom Geometry: You can render Passthrough images to a custom mesh instead of relying on the default style mesh—for example, to project Passthrough on a planar surface.

Although transparent AR headsets—like HoloLens and Magic Leap—give a much clearer (and color) view of the real-world, passthrough AR headsets like Varjo XR-3 (and now Quest 2), tend to offer a much more immersive field-of-view and more convincing virtual imagery thanks to complete opacity control and the potential for perfect latency between  real world imagery and virtual imagery. Though Quest 2’s AR view is still fairly low resolution and black & white, it’s expected that future headsets from Facebook will focus on improving the passthrough AR view.

While developers will be able to start experimenting with AR on Quest 2 with the v31 SDK via Unity, support for other engines—and the ability to actually publish AR apps to users—is expected later this year.

With a device like Quest 2 loaded full of cameras, it’s definitely worth thinking about privacy, especially now that third-party apps can make use of AR capabilities. On that front, Facebook says that “apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors,” and that “raw images from device sensors are processed on-device,” meaning that the images the camera sees don’t get sent to Facebook or to third-party developers.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

Click here to access.

Source: https://www.roadtovr.com/facebook-unlocking-ar-quest-2/

Continue Reading

AR/VR

Two New ‘Harry Potter’ VR Experiences Launch at Harry Potter New York

Published

on

Two new Harry Potter VR experiences have launched at the Harry Potter New York retail location. Wizards Take Flight and Chaos at Hogwarts were developed in collaboration with Dreamscape, Wevr, and Keylight.

The Harry Potter New York store, which launched last month as a retail destination for fans of the series, is now home to two new VR experiences offering visitors a chance to cast spells with wands at Hogwarts and ride flying brooms through London.

Both experiences launched last week, supporting up to six players at at time, and have a total run time of about 30 minutes (including gearing up and down).

Chaos at Hogwarts

After missing the Hogwarts Express at King’s Cross Station, Dobby helps get us to Hogwarts School of Witchcraft and Wizardry. Students! Dobby needs your help. Journey through Hogwarts Castle, cast spells with your wand, round up Pixies, discover hidden secrets, and find whether you and your friends can manage the Chaos at Hogwarts.

Wizards Take Flight

Take flight into the wizarding world on your own broom. Fly freely in the skies above Hogwarts before meeting Hagrid at Knockturn Alley. Then battle Death Eaters over the city of London, casting spells with your wand as you try to escape through the clouds to the safety of Hogwarts Castle.

For Chaos at Hogwarts, players will wear a VR backpack, headset, foot-trackers, and hand-trackers. In Wizards Take Flight players will each be seated on a physical broom accessory and wear a VR headset and hand-trackers. Both experiences are said to make use of “various haptics and special effects.”

Both Harry Potter VR experiences are priced at $34, and while tickets are technically available on the Harry Potter New York website, it appears that both VR experiences are completely sold out through August 15th.

Chaos at Hogwarts and Wizards Take Flight were made in collaboration with VR LBE, development, and production companies Dreamscape Immersive, Wevr, and Keylight.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

Click here to access.

Source: https://www.roadtovr.com/harry-potter-vr-new-york-chaos-at-hogwarts-wizards-take-flight/

Continue Reading

AR/VR

The VR Drop: A Breezey RPG Summer

Published

on

It’s hot, it’s sunny so why stay inside playing virtual reality (VR) videogames? Because it’s hot, too sunny and quite frankly July has been filled with awesome releases. And that’s set to continue as the month comes to a close next week with more VR titles than you can shake a stick at gearing up for launch. Here are five that VRFocus is looking forward to in the coming days.

Winds & Leaves

Winds & Leaves – Trebuchet

After previously releasing Prison Boss VR Canadian developer Trebuchet returns with a far more open-air experience. In Winds & Leaves you become a virtual gardener bringing life back to a barren planet. Using a unique connection to the trees and plants around you, the only way you can explore the world is by planting forests that provide life-giving energy whilst offering a safe haven to return to. A VR experience for nature lovers.

  • Supported platforms: PlayStation VR
  • Launch date: 27th July

Arcsmith – Bithell Games

In Arcsmith you become a space engineer guided by a rather reluctant master arcsmith Korith Dinn. Onboard his usually quiet space station you’ll learn how to construct a variety of space-based items and machinery, fitting parts together in your own way. Whilst these three-dimensional engineering puzzles have specifications to work to, the modular design of the components means you can get creative with each assembly.

Arcsmith

Zombieland VR: Headshot Fever – XR Games

Initially due for release earlier this month on Steam, Zombieland fans can get in on the action in a few days. An official franchise tie-in with the films, Zombieland VR: Headshot Fever is an arcade-style shooter testing your aim and speed across a variety of zombie-filled courses. Get two headshots in a row to activate slow-mo, giving you more time to rack up those kills and points to unlock more goodies.

Vengeful Rites – Deep Dive Interactive

A Steam Early Access title that arrived back in 2018, Vengeful Rights is a big, story-driven role-playing game (RPG) set for an official launch next week. Filled with puzzles to solve and monsters to fight you’ll be able to wield swords, bows and magical abilities as you seek to save the world in this classic fantasy adventure.

Neon Hat

Neon Hat – Entalto Studios

From Spanish indie team Entalto Studios, Neon Hat is a very vibrant, cyber racer designed for use with PlayStation Move controllers, each one serves as a rocket booster allowing players to fly around corners and through checkpoints. Featuring its own original synthwave soundtrack, Neon Hat features ten courses across three gameplay modes. 

  • Supported platforms: PlayStation VR
  • Launch date: 29th July

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.vrfocus.com/2021/07/the-vr-drop-a-breezey-rpg-summer/

Continue Reading
Esports4 days ago

How to reduce lag and increase FPS in Pokémon Unite

Esports4 days ago

Coven skins for Ashe, Evelynn, Ahri, Malphite, Warwick, Cassiopeia revealed for League of Legends

Esports4 days ago

Will New World closed beta progress carry over to the game’s full release?

Esports4 days ago

How to add friends and party up in New World

Esports4 days ago

Can you sprint in New World?

Esports4 days ago

How to claim New World Twitch drops

Esports4 days ago

Twitch streamer gets banned in New World after milking cow

AR/VR4 days ago

Moth+Flame partners with US Air Force to launch Virtual Reality sexual assault prevention and response training

Blockchain5 days ago

Uniswap (UNI) and AAVE Technical Analysis: What to Expect?

Blockchain5 days ago

Rothschild Investment Purchases Grayscale Bitcoin and Ethereum Trusts Shares

Esports4 days ago

Konami unveils Yu-Gi-Oh! Master Duel, a digital version of the Yu-Gi-Oh! TCG and OCG formats

Esports4 days ago

How to change or join a new world in New World

Esports4 days ago

Best Akshan builds in League of Legends

Esports4 days ago

Team BDS adds GatsH to VALORANT roster as sixth man before EU Stage 3 Challengers 2

Esports4 days ago

Overwatch League 2021 Grand Finals to be held in Los Angeles, playoff bracket in Dallas

Esports4 days ago

Here are all the servers in the New World closed beta

Esports4 days ago

How to turn off and on PvP in New World

Blockchain5 days ago

NexWEB Technologies Chooses Butterfly Protocol for Powering its Blockchain Domain-Based NFT Platform

Gaming4 days ago

Why Is It Better to Play Slots Using Cryptocurrency?

Blockchain5 days ago

WiV Signs Agreement with Georgian Government To Develop Georgian Wine’s Global Presence And Quality Assurance

Trending