Google today announced changes to ML Kit, its toolset for developers to infuse apps with AI, designed to make it easier to use offline. While the original ML Kit was tightly integrated with the web development platform Firebase, the refreshed ML Kit makes available on-device APIs in a standalone SDK that doesn’t require a Firebase project.
Google notes that more than 25,000 applications on Android and iOS now make use of ML Kit’s features, up from just a handful at its introduction in May 2018. Much like Apple’s CoreML, ML Kit is built to tackle challenges in vision and natural language domains, including text recognition and translation, barcode scanning, and object classification and tracking.
With the transition from ML Kit for Firebase’s on-device APIs to the ML Kit SDK, a face contours model — which can detect over 100 points in and around a user’s face and overlay masks and beautification elements atop them — has been added to the list of APIs shipped through Google Play Services, Google’s background service and API package for Android devices. (This should result in smaller app footprints and enable the model to be reused between apps.) Beyond this, Android Jetpack Lifecycle support has been added to all ML Kit APIs, making integration with the CameraX support library ostensibly easier than before.
Lastly, two new APIs are now available as part of the ML Kit early access program: entity extraction and pose detection. Entity extraction spots items in text including phone numbers, addresses, payment numbers, tracking numbers, date and time, and more and makes them actionable, while pose detection supports 33 skeletal points like hands and feet tracking.
In a related development, ML Kit now supports custom TensorFlow Lite image labeling, object detection, and object tracking models. Support will expand to additional types of models in the future, Google says.
The updates come after Google added new natural language processing services for ML Kit, including Smart Reply. (Smart Reply suggests text responses based on the last 10 exchanged messages and runs entirely on-device, and it’s been incorporated into Gmail, Hangouts Chat for G Suite, and Google Assistant on smart displays and smartphones.) Last year during Google’s I/O 2019 developer conference, ML Kit gained three new capabilities in beta, starting with a translation API supporting 58 languages and a pair of APIs that let apps locate and track objects of interest in a live camera feed in real time.
Google says that for the time being, ML Kit will continue to work with Firebase features like A/B testing and Cloud Firestore and that the cloud-based APIs and model deployment will remain available through Firebase Machine Learning. However, developers who wish to take advantage of the new features and updates will have to switch to the SDK.
Last night, Tesla CEO Elon Musk spoke at the virtual World AI Conference 2020 in Shanghai. He shared just how close Tesla is to reaching “Level 5” autonomy. He also said that Tesla China will get a chance to create original designs and engineering in the future.
Tesla China & Autopilot
Elon Musk said that in China, Tesla’s Autopilot worked “reasonably well,” and that Tesla is building up its engineering team in China. “If you are interested in working at Tesla China as an engineer, we would love to have you work there. That will be great.”
One thing that Elon wanted to really emphasize was that Tesla is going to be doing original engineering in China. It won’t be converting things from American designs into Chinese, but will crate actual, original Chinese designs.
Level 5 Autonomy
“I’m extremely confident that Level 5 or essentially complete autonomy will happen, and I think will happen very quickly. I think at Tesla, I feel like we are very close to Level 5 autonomy. I think — I remain confident that we will have the basic functionality for Level 5 autonomy complete this year,” he said.
Elon reiterated his April 22, 2019 Autonomy Day remarks:
➡️ Level 5 autonomy is possible
➡️ Tesla is getting close to achieving it
➡️ He’s absolutely confident it can be accomplished with the sensor hardware in all Tesla vehicles
Elon Musk also said that he thought there were no fundamental challenges remaining for Level 5 autonomy and dove into the small problems Tesla has encountered.
“There are many small problems. And then there’s the challenge of solving all those small problems and then putting the whole system together and just keep addressing the long tail of problems. So you’ll find that you’re able to handle the vast majority of situations. But then there will be something very odd. And then you have to have the system figure out a train to deal with the very odd situations. This is why you need a kind of real-world situation. Nothing is more complex and weird than the real world. Any simulation we create is necessarily a subset of the complexity of the real world.”
Elon Musk’s words are similar to his recent Autopilot rewrite update that he shared on Twitter earlier this month. In that update, he shared that it was going well and that a lot of functionality would be ready to release in 2–4 months, but that it still needs to be proven safe for owners to use.
Going well. Team is kicking ass & it’s an honor to work with them. Pretty much everything had to be rewritten, including our labeling software, so that it’s fundamentally “3D” at every step from training through inference.
A lot of functionality will happen all at once when we transition to the new software stack. Most likely, it will be releasable in 2 to 4 months. Then it’s a question of what functionality is proven safe enough to enable for owners.
“So yes, I think there are no fundamental challenges remaining to Level 5 autonomy,” Musk said, explaining all the little details Tesla will still need to focus on to reach Level 5 autonomy. “So we are really deeply enmeshed in dealing with the tiny details of Level 5 autonomy. But I’m absolutely confident that this can be accomplished with the hardware that is in Teslas today and simply by making software improvements.”
Autopilot AI Chips, Dojo, Vector Space
“In developing AI chips for Autopilot, what we found was that there was no system on the market that was capable of doing inference within a reasonable cost or power budget. So, if we had gone with conventional GPUs, CPUs, and that kind of thing, we would have needed several hundred watts and we would have needed to fill up the trunk with computers and GPUs and big cooling systems. It would have been costly and bulky and have taken up too much power, which is important for range for an electric car. So we developed our own AI chip, the Tesla Full Self-Driving computer with dual systems on chips with the eight-bit and accelerators for doing the dot products.”
Elon explained that AI consisted of doing many dot products. For those (such as myself) who know nothing about this, a dot product is the sum of the products of the corresponding entries of two sequences of numbers. Keeping that in mind, what Elon said next was aimed at highlighting how powerful the human brain truly is: that “effectively means that our brain must be doing a lot of dot products.”
Elon noted that Tesla still hasn’t fully explored the power of Tesla’s FSD computer. They’ve only turned on the second system on the chip “a few months ago.” Elon said that making full use of the FSD computer could take at least another year or so.
Tesla Dojo System
Elon spoke about Tesla’s Dojo system and said that it was a training system that is intended to be able to process fast amounts of video data to improve the training for the AI system. “The Dojo system — that’s like an FP16 training system and it is primarily constrained by heat and by communication between the chips.”
Tesla is developing new buses (subsystems used to connect computer components and transfer data, not transit buses) and sort of heat projection or cooling systems to help solve these challenges. “We are developing new buses and a new sort of heat projection or cooling systems that enable a very high operation computer that will be able to process video data effectively.
“How do we see the evolution of AI algorithms? I’m not sure how’s the best way to understand it, except what neural net seems to mostly do is take a massive amount of information from reality, primarily passive optical, and create a vector space, essentially compress a massive amount of photons into a vector space.”
He shared that earlier that very morning he was wondering, “Have you ever tried accessing the vector space in your mind? Like, we normally take reality just for granted in a kind of analog way. But you can actually access the vector space in your mind and understand what your mind is doing to take in all the world data.”
He explained that what we are actually doing is trying to remember the least amount of information possible. “So it’s taking a massive amount of information, filtering it down, and saying what is relevant. And then how do you create a vector space world that is a very tiny percentage of that original data. Based on that vector space representation, you make decisions.”
In essence, it’s a compression and decompression that is going on on a massive scale, which, Elon says, “Is kind of how physics is like.” Just after this part, there was some type of audio error or upload issue where the audio of what Elon was explaining became muted and the clip cut out to explain the Giga Shanghai updates. However, you can hear Elon musing about how we are all made from hydrogen and that the universe is sentient. (I believe that as well. I personally found that fascinating and would love to hear Elon talk more about that!!)
Giga Shanghai Updates
Elon shared that things at Giga Shanghai are going really well, and that he is incredibly proud of the Tesla team. “They’re doing an amazing job. I really can’t say enough good things. Thank you to the Tesla China team. And I look forward to visiting Giga Shanghai as soon as possible.”
“It’s really an impressive work that’s been done. I really can’t say enough good things. Thank you to the Tesla China team. We expect over time to use more AI and essential smarter software in our factory. But I think it will take a while to really employ AI effectively in a factory situation. You can think of a factory as a complex, cybernetic collective involving humans and the machine. This is actually how all companies are really.”
Johnna Crider is a Baton Rouge artist, gem, and mineral collector, member of the International Gem Society, and a Tesla shareholder who believes in Elon Musk and Tesla. Elon Musk advised her in 2018 to “Believe in Good.” Tesla is one of many good things to believe in. You can find Johnna on Twitter
The highlight of almost any sushi platter is the fatty tuna. Finding that perfect cut of tuna that melts in your mouth is something that fish buyers spend years of their life learning how to do. But now a Japanese advertising agency named Dentsu Inc has developed an app called Tuna Scope that allows someone to do the same with little to no training (via The Verge).
The firm trained the machine learning algorithm that powers the software using thousands of images of tuna tail cross-sections. The cut can tell human buyers a lot about the quality of fish they’re about to purchase. In testing against human experts, Dentsu claims it found the app gave the same grade more than four out of five times.
The app is currently in use by one company, conveyor belt sushi chain Kura Sushi. The restaurant buys the majority of its tuna outside of Japan. Part of the reason the company started using the app is that it allows its employees to grade tuna without traveling. That’s a significant perk during the current pandemic. Moreover, conveyor belt restaurants in Japan tend to offer the least expensive sushi, so there’s a cost-saving aspect at play as well.
As you might have already guessed, traditionalists are skeptical of the app. Keiko Yamamoto, a sushi chef who teaches in London, told The Verge it’s challenging to convey the exact qualities tuna buyers look for when they see a fresh catch. There’s also the question of whether the app can scale to meet the demands of high-end sushi restaurants and the exacting chefs that run them. Tuna Scope examines images of frozen tuna tail cross-sections to deliver its quality verdict. However, high-end restaurants tend to purchase their fish from suppliers that deal with freshly caught tuna. As they cut the fish, they give a variety of grades to different parts of the fish. Like with most instances of new technology, we’ll probably see some businesses continue to do things the way they’ve always done them.
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Following Canada’s lead earlier this week, privacy watchdogs in Britain and Australia today launched a joint investigation into how Clearview AI harvests and uses billions of images it scraped from the internet to train its facial-recognition algorithms.
The startup boasted it had collected a database packed with more than three billion photos downloaded from people’s public social media pages. That data helped train its facial-recognition software, which was then sold to law enforcement as a tool to identify potential suspects.
Cops can feed a snapshot of someone taken from, say, CCTV footage into Clearview’s software, which then attempts to identify the person by matching it up with images in its database. If there’s a positive match, the software links to that person’s relevant profiles on social media that may reveal personal details such as their name or where they live. It’s a way to translate previously unseen photos of someone’s face into an online handle so that person can be tracked down.
Now, the UK’s Information Commissioner (ICO) and the Office of the Australian Information Commissioner (OAIC) are collaborating to examine the New York-based upstart’s practices. The investigation will focus “on the company’s use of ‘scraped’ data and biometrics of individuals,” the ICO said in a statement.
AWS won’t sell facial-recog tool to police for a year – other law enforcement agencies are in the clear
“The investigation highlights the importance of enforcement cooperation in protecting the personal information of Australian and UK citizens in a globalised data environment,” it added. “No further comment will be made while the investigation is ongoing.”
In response, Clearview AI told us it “searches publicly available photos from the internet in accordance with applicable laws. It is used to help identify criminal suspects. Its powerful technology is currently unavailable in UK and Australia. Individuals in these countries can opt-out. We will continue to cooperate with UK’s ICO and Australia’s OAIC.”
The move comes days after the Office of the Privacy Commissioner of Canada announced that Clearview will stop operating in Canada. The agency has been probing the startup since February to see whether its methods complied with the country’s privacy laws.
“In response to the commissioner’s request, Clearview AI has ceased its operations in Canada,” the AI biz told The Register today.
“We are proud of our record in assisting Canadian law enforcement to solve some of the most heinous crimes, including crimes against children. We will continue to cooperate with OPC on other related issues. In addition, Canadians will be able to opt-out of Clearview’s search results.”
Clearview’s last Canadian customer was the Royal Canadian Mounted Police (RCMP), which has suspended its contract indefinitely with the biz. The Privacy Commissioner of Canada also has a separate ongoing investigation into the RCMP’s use of Clearview’s facial-recognition technology.
In May, Clearview was sued in the US by the American Civil Liberties Union. At the time, the startup argued that since the images were all publicly available, it should be, somehow, protected under The First Amendment. Clearview’s lawyer Tor Ekeland told us: “Clearview AI is a search engine that uses only publicly available images accessible on the internet. It is absurd that the ACLU wants to censor which search engines people can use to access public information on the internet. The First Amendment forbids this.” ®