By AI Trends Staff
In the black box problem in machine learning, data goes in, suggested decisions come out, and how the model arrived at its suggestions may or may not be explainable. This problem intrigued Prof. Anupam Datta of Carnegie Mellon University, who in 2014 with his PhD student Shayak Sen began researching explainable AI. At the same time, Will Uppington was among the founders at Bloomreach, which was trying to make black box machine learning models into a commercial product, and was running into similar issues around visibility into how models produce their answers.
Meeting each other in 2018, they decided to form the startup Truera in early 2019 to address the challenge. The company offers a Model Intelligence Platform, using AI explainability to power it. CEO Uppington recently responded via email to queries from AI Trends about the startup.
AI Trends: What business problem are you trying to solve?
Will Uppington: Machine Learning (ML) is exploding, but ML has a big flaw: it’s a black box. That means even when models work, data scientists don’t necessarily know why. This hinders data scientists from building high quality ML applications quickly and efficiently. It also becomes a problem when non-data scientists, such as business operators, regulators, or consumers ask questions about a result or when models break once they go live.
The science behind what drives outputs of ML models is called AI Explainability. AI explainability is the breakthrough technology that can address these challenges but it’s not enough on its own. You need a new class of software that analysts are calling Model Intelligence Platforms that leverage AI explainability, to address these problems throughout the model lifecycle: development, evaluation/testing, and monitoring.
AI Trends: How does your solution address the problem?
Uppington: Truera’s model intelligence software removes the “black box” surrounding Machine Learning (ML) and provides intelligence and actionable insights throughout the ML model lifecycle—enabling companies to improve the quality and accuracy of their models, boost stakeholder collaboration, and address responsible AI concerns including explainability and bias. Truera’s technology builds on six years of AI Explainability research performed at Carnegie Mellon University (CMU), which pioneered many of the methods that are becoming the standard for explaining popular ML models such as Tree models and Neural Networks.
AI Trends: How are you getting to the market?
Uppington: Truera is helping enterprise customers across all industry verticals including financial services, insurance, healthcare, pharmaceutical, manufacturing, and retail. Headquartered in the US, Truera sells through a direct sales force and has recently expanded its sales team globally to enter markets in Europe and Asia.
AI Trends: Do you have any users or customers?
Uppington: The Truera platform is already deployed at and delivering value to a number of early Fortune 100 customers.
AI Trends: Any anecdotes/stories?
Uppington: Here’s a good example from a recent case study: Standard Chartered is a leading international banking group, with a presence in 60 of the world’s most dynamic markets and serving clients in a further 85.
To see how Truera could help Standard Chartered ensure responsible AI, the teams worked with a number of credit decisioning models (and associated training data) built on multiple development platforms, including open source. The Truera Model Intelligence software helped data scientists and other stakeholders build trust in the quality of the model by explaining the model, assessing it for unfair bias, examining its stability over time, and surfacing other relevant intelligence about its inner workings.
This allowed Standard Chartered’s data scientists and business stakeholders to gain greater confidence in the quality of the model. Data scientists used the Truera platform to identify the most important features that drove individual credit decisions; e.g. identifying that Jane may have been denied credit because of her low income and high debt-to-income ratio. They could also understand how individual features contributed to the model’s assessment of risk; e.g. how increasing income affects the model’s risk scores. Crucially, they could assess that the model remained stable over time and that it was possible to identify and mitigate unjust bias.
Collectively, these capabilities provided all of the relevant Standard Chartered teams, including first line development, second line validation, third line audit/compliance and other stakeholders with the level of visibility into the inner workings of the model that they needed to build trust in the quality of the model and move it to production.
AI Trends: How is the company funded?
Uppington: Truera is a venture-funded startup. The company has raised over $17.3M over two rounds of funding led by Greylock and Wing VC with participation from other investors including Conversion Capital, Data Community Fund, B Capital Group via the firm’s Ascent Fund, and Harpoon Ventures.
Truera has some competition, according to an account in VentureBeat. Fiddler is a Mountain View, California-based startup developing an “explainable” engine that is designed to analyze, validate, and manage AI solutions. Also, Kyndi last June raised $20 million for an explainable AI product that analyzes documents.
Vishu Ramachandran, Group Head, Retail Banking at Standard Chartered, based in London stated that the effective use of data and analytics at the bank is a “key enabler” of its business strategy to better serve its customers. Ramachandran added that as the bank continues to scale its operations, it will continue to leverage AI and algorithmic decision-making, but plans to use these technologies in a “fair, transparent and responsible way.”
He stated, “We see Truera as an essential partner in how we do this and in how we build and operationalize higher quality, trusted AI models faster and more efficiently.”
Jerry Chen, a Partner at Greylock, stated, “Truera’s Model Intelligence platform and its AI.Q technology are fundamental breakthroughs in AI. Removing the black box problem of machine learning is essential to build effective and responsible ML applications.”
Learn more at Truera.
