The Importance of Multisensory Feedback in Spacial Computing
The improvement of immersion is one of the key issues in VR and AR. In a multisensory VR experience, a virtual body can be perceived as one‘s own, which is also called “Sense of Embodiment” (SoE). Through appropriate visual and tactile stimulation, this technology can be used to manipulate the sense of body possession. “The illusion of a virtual hand” and the “rubber hand” are experiments that have already proven this technology.
Embodiment is a concept that refers to cognitive psychology and translates to thoughts, feelings, behaviors, and perceptions based on our sensory experiences and our body positions.
To achieve this illusion, there are lots of different tools like VR gloves, bracelets, exoskeletons, or suits. These devices have been built to fill the gap between reality and virtuality. However, it cannot be denied that integration into the consumer market is difficult for various reasons (size, price, etc.). Accordingly, user experience (UX) and user interface (UI) design in applications with multisensory feedback has not been explored enough.
Therefore, the collaboration with Simon Frübis will cover the integration of sound to haptics with Interhaptics, as well as supporting the design of haptic feedback for applicable haptic devices.
Simon Frübis is an interaction design student at the University of Applied Sciences Magdeburg-Stendal. In his thesis, he investigates multisensory feedback for Augmented Reality (AR) and Virtual Reality (VR). By creating virtual wearables and including haptics and audio, Simon evaluates the effects of multisensory feedback on the user experience (UX).
“I see strong potential in the combination of AR with our most natural input device — our Hands. Multisensory feedback allows us to interact quickly and precisely to enable more immersive handinteractions. We still need to explore working solutions by creating a new design pattern language. Interhaptics offers a great platform and foundation to design such patterns and to identify best-practices.” Simon Frübis — passionated about interaction design, immersive technologies, spatial and physical computing.
Interhaptics is a software company specializing in haptics. Interhaptics provides hand interactions and haptic feedback development and deployment tools for extended reality and mobile applications. Interhaptics’ mission is to enable the growth of a scalable haptics ecosystem. Interhaptics strives to deliver top-notch development tools for the XR, mobile, and console developer community, and the interoperability of haptics-enabled content across any haptics-enabled platform.
The first prototype shows how Simon integrated low-budget haptic devices into VR.
With the help of Interhaptics’ Haptic Composer, Simon will be able to design custom haptics for sound-driven actuators. A device like this separates low-frequencies from audio to transform them into haptic feedback. Once placed on certain body parts, it can convince the brain that it is exposed to high acoustic energy by using the auto-completion principle. It works the same way as when you look at a picture of a sunny beach; you can almost smell sunscreen and hear the waves clashing. This is because our brain automatically retrieves the different aspects of the experience to complete it by itself.
“Sound to haptic devices are the easiest entry point for OEM manufacturers in the haptics market. They are a powerful and expressive category, and we are looking forward to exploring with Simon, their ability to enhance VR experience”
Today, to create haptics with Interhaptics, one creates an asset (haptic material) within the Haptic Composer which stores the representation of haptics in 4 perceptions (Vibration, Textures, Stiffness, and Thermal conductivity).
The collaboration will cover the integration of sound to haptics with Interhaptics.
GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.How will you do that? Membership includes access to:
Newsletters, such as DeanBeat
The wonderful, educational, and fun speakers at our events
Special members-only interviews, chats, and “open office” events with GamesBeat staff
Chatting with community members, GamesBeat staff, and other guests in our Discord
Solaris: Offworld Combat is coming to PSVR this June with a physical release from Perp Games. According to a First Contact Entertainment representative, the digital version is planned to release a month earlier in May.
Perp Games on Twitter: “We’re not finished just yet. Solaris Offworld Combat is the next game to be getting a physical box release. Coming in June to global markets! Will you be buying it? https://t.co/5sphrqsh10” / Twitter
Solaris: Offworld Combat on PSVR
Originally, Solaris was coming to PSVR late last year around the same time as the Quest and PC VR version of the game but got delayed. Now, it’s slated for release in just a couple of months.
The latest VR shooter from First Contact Entertainment (creators of Firewall Zero Hour) is a sci-fi competitive VR shooter that feels a bit like Quake in VR due to its speed and intense arena levels. It’s a very breezy, fast-paced game that’s accessible and easy to quickly jump in and out of. The closest comparison is probably Hyper Dash.
Soalris is a notable release because other than Firewall Zero Hour, there really haven’t been many options for shooter fans on PSVR. Alvo is coming soon too, but the headset is on its last legs at this point.
The PS Aim Controller continues to be one of the best things about the PSVR platform, so I’m all for seeing more games support it, but it’s a shame games like this didn’t hit PSVR earlier in its life cycle. Hopefully PSVR 2 on PS5 is backwards compatible and it can give late-life cycle games like this one new life when it releases.
Solaris is coming to PSVR very soon with a planned digital release in May and physical release from Perp Games in June. For more on this game make sure and read our Solaris: Offworld Combat review and stay tuned for all the latest in VR.
The VR fitness genre is still relatively new, all things considered. Even though the space is still somewhat niche, it’s absolutely growing at a fast pace as seen by the number of games and players that continue to spring up. How did we get to this point though, you might ask?
Microsoft won a large US Army contract to supply advanced AR headsets for frontline soldiers, based on the HoloLens platform.
The US Army’s Integrated Visual Augmentation System (IVAS) program aims to equip infantry with AR helmets for situational awareness and convenient display of sensor outputs.
The contract is worth up to $21.88 billion over 5-10 years. While the order has been widely reported as 120,000 units, a US Army statement to Breaking Defence suggests that is the maximum, not a fixed quantity.
Early evaluation units based on HoloLens 2
In 2018 Microsoft won the $480 million evaluation contract for just over 2500 units, based on HoloLens 2 with some modifications and an extra sensor.
The current, ruggedized, upgraded IVAS
The evaluation found the hardware not rugged enough for military use, and identified problems with the sensors at night. Since then the hardware has been significantly upgraded. It’s more ruggedized and houses many more sensors.
The field of view has been significantly increased from roughly 40°x30° to 80°x40°. That’s significantly wider than any other see-through AR headset on the market.
Reported use cases for the headset include:
overlaying icons on friendly units, objectives, threats, and points of interest
built-in night vision & thermal view modes
live picture-in-picture feeds from drones, including the Soldier Borne Sensors (SBS) personal drone
simulated weapons & enemies for training exercises
scanning nearby people for high temperature (COVID-19)
facial recognition for hostage rescue situations
The Army is also testing integrations with vehicles, such as soldiers being able to see-through the walls of the armored vehicle carrying them. That means on dismounting they’ll be situationally aware.
Some Microsoft employees have protested providing technology for the military, but that’s unlikely to have any effect given the enormous potential value of the contract.
IVAS is still in the late testing & evaluation stage, and the scale of deployment will depend on future budgets. But if things go to plan, frontline soldiers could be equipped with these game-changing AR capabilities by the end of the decade.