Connect with us

AR/VR

Pinterest: How AR elevates our data strategy

Avatar

Published

on

Pinterest is announcing plans to expand its product-tagging beta for shoppable Story Pins, allowing advertisers and creators to tag products in photos. The company also added to its Try on platform today with an AR eyeshadow launch. These build on a wider series of features that include the introduction of Pinterest’s facial AR technology exactly one year ago.

Pinterest senior VP Jeremy King talked with VentureBeat about how the company’s data strategy made these technical changes possible. He explained how Pinterest is powered by computer vision technology and focused on advancing its AR capabilities to make the site more shoppable.

This interview has been edited for clarity and brevity.

VentureBeat: Could you tell me about Pinterest’s decision to expand Try on with eyeshadow and add shoppable Story Pins? You mentioned that this technology has been developing for about a year. I’m wondering if there are any considerations at the data level that drive this innovation.

Jeremy King: One of the hardest things about these kinds of AR technologies is really making sure that you don’t introduce bias into the data. So we have some wonderful technologists, both on the machine learning side and the computer vision side, that can really help us test all kinds of different skin tone ranges and different lipstick colors.

And it’s funny, there has been a huge amount of face mask Try on during the pandemic, and you can imagine that adds its own bias into the computer because the computer doesn’t know whether it’s a regular face or if somebody’s got a face mask on. And so it changes. It has changed the algorithm pretty dramatically. We have billions of pictures of people in our system and so can use that data to make sure that we shake out all the bias in the system.

The No. 1 request on Pinterest is, “Once I found this beautiful thing and [I] want to be able to get it.” We call it inspiration to action. It has a lot to do with sometimes seemingly boring technology, ingesting millions of catalogs from thousands of retailers, making sure you’ve got pricing and inventory correct. Making sure that the latest images are there and you can understand, from a hero image — you call it a lifestyle-type image — you want to identify 20 different items that are in a lifestyle picture. As you take a picture of your living room, there are probably 20 or 30 items sitting in that picture. And VMO identified each one of those items because when people add — that picture may be added to dozens, if not hundreds of boards. And it was added to those boards for different reasons. Sometimes we think of Pinterest as a giant human labeling system, where people use board games and they tag items inside of boards. That empowers our computer vision technology to help us get better and better at item identification.

VentureBeat: How do you get to that inspiration point with the data? Are there any particular types of frameworks or languages that you use?

King: We use a lot of open source and machine learning technology that we’ve enhanced over the years. Our core technology that powers Pinterest is a system called graph stages, a giant graph database that uses a number of techniques to identify pictures and images that are like each other. The more that you use Pinterest, the more that you add things to a board, the better we get, of course. We’ve also open-sourced a number of those technologies.

If you do a search on Pinterest, oftentimes you’re very open. You don’t use very specific queries. Like on Google it’s not uncommon to have 7- to 10-word queries. In Pinterest, oftentimes you’re saying “inspiration” or “inspiring living room” or “shabby chic bedroom set,” and so we have lots of opportunities to show many different things. We can start pretty quickly into several different fingers on the inspiration track and then get people to narrow down the results with images, which is what we’re after.

VentureBeat: Pinterest builds and releases new features often. How has your existing tech infrastructure allowed your engineers to add on extra capabilities like the ones launched today?

King: The underlying framework of Pinterest not only includes this graph database, but it also includes an experimentation platform that we built from scratch. And so we’re running hundreds of experiments at a time and slicing the user basis to have different users try different areas. So, as a result, we can rapidly iterate features and then launch the things that actually do well. We have about a 30% to 40% success rate for features as we launch them. And that’s pretty common, to throw away 70% of the work that you’re doing because it doesn’t work.

VentureBeat: You mentioned that with the experimentation system you have a 30% to 40% success rate. How do you measure success?

King: We have hundreds of different metrics that we’re tracking. Sometimes it can be user engagement. Time on site isn’t a metric that we really are driving people toward. We want you to discover what you want and actually go do it — whether it’s to go paint your walls or to find something creative for your kids to do or what to cook for dinner, we want you to go find the thing and actually go out and do it.

VentureBeat: And how do you adjust your tech and that backend data to make Pinterest more inspirational and relevant with the right advertisements?

King: Our computer vision technology really allows us to make sure that even when you’re floating through an organic experience, where you’re throwing billions of pins on your board, that when we’re injecting ads, they’re relevant ads. So we have hundreds or thousands of ads, and we can show ads that are very relevant to what you’re thinking. And what we find is most of the time, when we get it right, you don’t even know which ones are bad and which ones are not. We get a lot of feedback that we’re deliberately engineering Pinterest in a way that leaves people feeling positive and inspired. We have the manual override features where you can say “hide this pin” or “hide this ad” or “don’t ever show me this again,” and that sort of thing. And that feedback loop is very deliberate.

VentureBeat: How would you describe Pinterest’s overall data strategy? And what has that journey looked like, maybe since when you joined Pinterest a couple of years ago?

King: At a high level, Pinterest used to be completely about an image signature. So everything we did was built around a pin or an image. And it turns out that it’s not always the best when you’re doing something like a catalog, right? A canonical catalog. Because you might have 20 different colors of a T-shirt or 50 different colors of a vase or hundreds of different colors of lipstick. And so when you identify a particular item, you actually want to nail that item. And so it turns more into a canonical database. And so we’ve built really two parts of Pinterest now, and I was describing the inspiration part, which uses our image technology, and then our shopping cart, where we’re using more traditional data structures. And what we’ve been doing is really spending time ingesting hundreds of millions of items from Etsy and eBay and so forth, from all these large and reputable retailers, to make sure that when you find an item, whether it’s a table or couch or a lamp or something, that you can find not only that exact item, but items that are very similar to that. And that’s really what’s made the shopping experience so much better in the last 18 months.

We still have a long ways to go. You know, there are hundreds of billions of pins. And you know, so many of them have items in them, and so we’re rapidly going through as many companies and catalogs as we can in order to product-tag. This is part of the reason that we’re announcing product-tagging with Story Pins, because we really feel like the creator can actually identify items.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform
  • networking features, and more

Become a member

Source: https://venturebeat.com/2021/01/22/pinterest-how-ar-elevates-our-data-strategy/

AR/VR

‘Farpoint’ Studio Impulse Gear Announces a New VR Game Coming This Year

Avatar

Published

on

Impulse Gear, the studio behind the stellar PSVR exclusive Farpoint (2017), announced today that it has been developing a new VR game which it plans to release this year.

Following a handful of post-launch updates to Farpoint, we’ve heard very little from studio Impulse Gear on what’s next. Today, more than three and a half years after the launch of Farpoint, the studio confirmed that it’s been in development of a new VR game. And, what’s more, the game is expected to launch this year! UploadVR first spotted the subtle announcement over at the studio’s blog.

Given the quality of Farpoint and its innovative support of PS Aim, it’s great to hear that Impulse Studio will be following the game with another VR title, but so far we have no idea what it will be except for two tiny clues: a sleek looking ‘L’ logo which links to a ‘Decrypted Message’ audio file.

Image courtesy Impulse Gear

The Decrypted Message sounds like a garbled alien voice which reads (to the best of our hearing):

You having trouble in violence to bring peace… leeches of [unintelligible] [unintelligible]… we know the way. It is this way… to the left and then in words.

A sci-fi setting seems assured, but going with the minimal hints alone, it’s hard to say if we’re looking at a direct sequel to Farpoint or not.

Our guess is that Impulse Gear’s next game will not be a direct sequel, largely because of the current VR landscape. Farpoint was a PSVR exclusive title, and while Sony recently announced it’s working on a next-gen PSVR headset, the company said it wouldn’t come in 2021. Meanwhile, Impulse Gear says its upcoming VR game will indeed launch in 2021.

So that suggests the studio is either planning to launch a new VR game on some headset other than PSVR… or it plans to launch a new game at the tail-end of the original PSVR’s lifespan.

In any case, Impulse Gear says it will offer a “full reveal and announcement soon,” which we’ll be looking forward to.

The post ‘Farpoint’ Studio Impulse Gear Announces a New VR Game Coming This Year appeared first on Road to VR.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.roadtovr.com/farpoint-studio-impulse-gear-new-vr-game-teaser/

Continue Reading

AR/VR

Quest 2 Now the Most Used on Steam, Monthly-connected Headsets Hit Record High of 2.8 Million

Avatar

Published

on

Another month, another record high count for the VR usership on Steam. Quest 2, despite being a standalone headset, has helped to drive much of the growth thanks to its ability to connect to a PC to play PC VR games on Oculus PC or Steam. It has become the most used headset on Steam for the first time.

Monthly-connected VR Headsets on Steam

Each month Valve collects info from Steam users to determine some baseline statistics about what kind of hardware and software is used by the platform’s population, and to see how things are changing over time, including the use of VR headsets.

The data shared in the survey represents the number of headsets connected to Steam over a given month, so we call the resulting figure ‘monthly-connected headsets’ for clarity; it’s the closest official figure there is to ‘monthly active VR users’ on Steam, with the caveat that it only tells us how many VR headsets were connected, not how many were actually used.

While Valve’s data is a useful way see which headsets are most popular on Steam, the trend of monthly-connected headsets has always been obfuscated because the data is given exclusively as percentages relative to Steam’s population—which itself is an unstated and constantly fluctuating figure.

To demystify the data Road to VR maintains a model, based on the historical survey data along with official data points directly from Valve and Steam, which aims to correct for Steam’s changing population to estimate the actual count—not the percent—of headsets being used on Steam.

The latest shows that the holiday surge has not relented, with February bringing Steam’s monthly-connected headset count to a new record high of 2.8 million. The record figure comes immediately after January had set the prior record at 2.6 million.

Quest 2 was the most significant driver of the increase seen in February, with Quest 2 becoming the single most used VR headset on Steam.

Share of VR Headsets on Steam

Looking at the breakdown of individual headsets in use on Steam, we can see that Quest 2 is continuing to take a large bite out of the pie. While the headset saw an inflated gain last month (because it was the first month the headset got its own category Valve’s data), this month shows the headset’s true pace, picking up a whopping +5.51% in share for a total of 22.91%. This makes Quest 2 the most used VR headset on Steam in February, besting the 21.58% share held by Rift S.

Though the gains in Quest 2 appear to have come largely from other Facebook headsets; even though Quest 2 picked up +5.51% share, the share of all Facebook headsets on Steam only rose by +1.58% (suggesting existing Oculus customers are trading up to a Quest 2). Still, this furthers Facebook’s position overall, with 57.98% of headsets in use on Steam made by Facebook.

Other winners this month was Valve’s own Index headset, which has reached 16.0% (+0.17%), and Windows Mixed Reality which, despite adding a few thousand headsets, actually dropped in share down to 6.15% (−0.23%) as a result of bigger proportional gains by others (like Quest 2).

The largest losses in the share of headsets on Steam in February came from Rift S (−1.78%), Quest (−1.58%), and HTC Vive (−1.01%), with the share of all HTC headsets on Steam dropping to 17.24% (−1.29%).

The post Quest 2 Now the Most Used on Steam, Monthly-connected Headsets Hit Record High of 2.8 Million appeared first on Road to VR.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.roadtovr.com/quest-2-most-used-vr-headset-steam-record-high/

Continue Reading

AR/VR

Microsoft Mesh to Enable Shared Experiences Across XR Platforms

Avatar

Published

on

Microsoft Mesh

Today see’s the start of Microsoft Ignite, its online virtual event which has started with an XR bang. Taking to AltspaceVR’s virtual stage was Microsoft Technical Fellow Alex Kipman to announce Microsoft Mesh, its new mixed-reality (MR) platform which aims to make shared holographic experiences effortless across multiple devices.

Microsoft Mesh

Showcasing Mesh by hosting the keynote in the social app, Kipman welcomed various speakers including Microsoft CEO Satya Nadella, director James Cameron, Niantic CEO John Hanke and Cirque du Soleil co-founder Guy Laliberté as viewers tuned in from around the world, in both VR and via other devices.

The platform is powered by Microsoft Azure, its cloud-computing service, benefiting from its enterprise-grade security and privacy features. The core focus of Microsoft Mesh is to enable multi-user XR, where companies and consumers can take a device with a Mesh-enabled application and swap ideas, learn or simply socialise. It’ll support 3D models for users to interact with, whilst a full suite of AI-powered tools will enable avatar creation. spatial rendering and more.

“This has been the dream for mixed reality, the idea from the very beginning,” said Kipman in a blog post. “You can actually feel like you’re in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.”

Microsoft Ignite, Alex Kipman and John Hanke
Alex Kipman and John Hanke at Microsoft Ignite

“Our part of this is the work of stitching the digital and physical worlds together, connecting the bits and atoms so these experiences can be possible using the Niantic platform,” Hanke said. “But social connections are really at the heart of everything we do, and Microsoft Mesh innovations just enrich that.”

Microsoft Mesh will work on HoloLens 2, Windows Mixed Reality, Oculus headsets, PCs, Macs and smartphones so its not restricted to one particular platform. While an official launch date has yet to be confirmed, a collaborative preview of the Microsoft Mesh app for HoloLens is available and access can be requested for a new version of Mesh enabled AltspaceVR. Eventually, Mesh will be integrated within Microsoft Teams and Microsoft Dynamics 365.

As further details are released for Microsoft Mesh, VRFocus will keep you updated.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.vrfocus.com/2021/03/microsoft-mesh-to-enable-shared-experiences-across-xr-platforms/

Continue Reading

AR/VR

Pokémon GO Demo Shown on HoloLens 2 at Microsoft Ignite

Avatar

Published

on

Pokemon Go HoloLens 2

During today’s Microsoft Ignite event, in collaboration with Niantic Labs, the companies showcased what the possible future of Pokémon GO could be. They demoed a prototype of the augmented reality (AR) videogame on HoloLens 2 for the first time.

Pokemon Go HoloLens 2

Currently just a proof-of-concept, the demo was part of Microsoft’s Mesh announcement to build shared experiences in mixed reality (MR). In the video which firmly pointed out that what was being shown wasn’t for consumer use, Niantic CEO and Founder John Hanke strolled through a park with various Pokémon running around his feet.

Using HoloLens 2’s hand tracking he then brings up his left hand which activates a menu UI, giving three options; a Poké Ball, Fruit and Pokémon, the latter offering a list of his available creatures. Selecting Pikachu, Hanke goes onto feed the famous yellow character some fruit before continuing with the presentation which doesn’t involve a battle, unfortunately.

It’s a tantalising look at the future of arguably the most popular AR title, where players no longer have to look at their phone screen because it’ll all appear in a set of AR glasses. That’s going to be some way off as devices like HoloLens 2 aren’t for consumer use, yet the next generation of smart glasses could well achieve that.

Pokemon Go HoloLens 2

“We’re committed to leveraging and expanding our platform to build real-world AR experiences for as many devices as possible and reaching a diverse set of players no matter their physical location,” says Hanke in a blog post. “We’ve only scratched the surface. We know the years ahead to be filled with important achievements which will serve as waypoints in AR’s journey to become a life-changing computing platform.”

AR is viewed by many companies as the future of mobile computing, offering hands-free access to a digital world that interacts with the real one, whether that’s for gaming, work or any other use case. And it’s these kinds of partnerships that help to build that groundwork. As Niantic and Microsoft continue to announce more from their collaboration, VRFocus will keep you update.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.vrfocus.com/2021/03/pokemon-go-demo-shown-on-hololens-2-at-microsoft-ignite/

Continue Reading
Bioengineer5 days ago

Bioinformatics tool accurately tracks synthetic DNA

Bioengineer5 days ago

Unburdening China of cancer: Trend analysis to assist prevention measures

Nano Technology4 days ago

CEA-Leti & Dolphin Design Report FD-SOI Breakthrough that Boosts Operating Frequency by 450% and Reduces Power Consumption by 30%: Joint Paper Presented at ISSCC 2021 Shows How New Adaptive Back-Biasing Technique Overcomes Integration Limits in Chip Design Flows

Energy5 days ago

New Columbia Solar raises $75 million, looks to develop 50 projects in D.C.

Energy5 days ago

Washington State awards low-income solar project grants

SPAC Insiders5 days ago

Matt’s Top 3 SPAC Targets – Advanced Materials

AR/VR5 days ago

Use the Force in Star Wars Pinball VR This April

Start Ups5 days ago

When Should Software Be Patented?

Cleantech5 days ago

Water recycling technology delivers cash savings

Energy5 days ago

Watch: IKEA completes 1.35 MW solar carport, with seven more underway

Automotive5 days ago

Teslas and other EVs could enter a golden decade with newly-introduced US bill

Energy5 days ago

Cleantech Roundup: SolarEdge, Tigo Energy, MIT, and more

AI5 days ago

MyHeritage now lets you animate old family photos using deepfakery

Bioengineer5 days ago

Pioneering prehistoric landscape reconstruction reveals early dinosaurs lived on tropical islands

Cleantech5 days ago

Water recycling technology delivers cash savings

Aerospace5 days ago

NASA and Roscosmos leaders speak as plans finalized for flying astronaut on upcoming Soyuz flight

AI5 days ago

7 Remarkable IVR Trends For the Year 2021 And Beyond

AI5 days ago

Top Chatbot Use Cases / Applications in Business in 2021

AR/VR5 days ago

STYLY Creator Psychic VR Lab Completes $8.5m Funding Round

AR/VR5 days ago

STYLY Creator Psychic VR Lab Completes $8.5m Funding Round

Trending