A European safety assessment rated the Tesla sixth of ten driver assistance systemsin its ability to keep drivers engaged, meaning actively engaged in the driving task as automation assists to some degree.
The Tesla Model 3’s Autopilot scored just 36 when assessed on its ability to maintain a driver’s focus on the road, according to a recent account from Reuters. The Tesla did receive high marks for performance and its ability to respond to emergencies, receiving an overall score of 131 and a rating of ‘moderate’.
The Mercedes GLE’s system had the highest overall score of 174, the top rating of ‘very good’ and a score of 85 for driver engagement. Most other vehicles had scores of 70 or above for driver engagement.
The European New Car Assessment Program (NCAP) worked with UK insurance group Thatcham Research to perform the assessment, which they called the first consumer ratings specifically focused on driver assistance systems, technology that automates some tasks, including acceleration, braking and steering support.
Safety and insurance researchers have frequently warned of the risks of consumers overestimating the systems’ abilities, a misconception increased by some automakers calling their products Autopilot (Tesla), ProPilot (Nissan) or CoPilot (Ford). (Others are Super Cruise (Cadillac), Drive Pilot (Mercedes Benz), Traffic Jam Pilot (Audi), Active Driving Assistant Professional (BMW), Highway Driving Assist (Kia) and Eyesight (Kia).)
The US National Transportation Safety Board (NTSB) has criticized Tesla’s Autopilot for enabling drivers to turn their attention from the road. US regulators have investigated 15 crashes since 2016 involving Tesla vehicles equipped with Autopilot.
“Unfortunately, there are motorists that believe they can purchase a self-driving car today. This is a dangerous misconception that sees too much control handed to vehicles that are not ready to cope with all situations,” stated Matthew Avery, a Euro NCAP board member and research director at Thatcham Research.
Europeans Ahead on Testing of Driver Assistance Systems
The US lags behind Europe in the testing of driver assistance systems, according to a recent account in Claims Journal, serving the insurance industry. The acting head of the US National Highway Traffic Safety Administration (NHTSA) announced recently that the agency would be making changes this year to a testing program that assigns safety grades to vehicles.
“We’re raising the bar for safety technologies in our new vehicles,” stated acting NHTSA chief James Owens. The agency in December 2015 issued proposed rules for testing procedures that would be similar to more comprehensive testing done by European regulators. But no rules have been put forward since then. The NTSB has criticized NHTSA for its hands-off approach to overseeing driver assistance programs. The NTSB has compared NHTSA’s testing and rating proposals unfavorably to consumer safety systems put in place by European agencies.
Euro NCAP began rating automatic braking systems in 2014. It has been testing the performance of advanced cruise control, lane-centering systems and blind spot detection since 2018. Beginning this May, it began to grade how well a car’s system keeps the driver engaged.
The group is a non-governmental body but funded by some EU countries and also receives money from national motor clubs and insurers. The group shares testing methods with NHTSA and the NTSB on a regular basis.
In 2018, EU regulators required the installation of acoustic and visual warning signals for lane-keeping systems every 15 seconds if drivers take their hands off the wheel. As a result, Tesla had to issue a software update to its Autopilot system in the EU. A regulatory body is currently working on rules for more advanced hands-off systems that can control braking, acceleration, and lane changes at speeds of up to 60 km/h (37 mph).
Under draft EU rules, carmakers among other things need to show how the system safely hands control back to the driver, how the car monitors the road, and how it reacts in emergency situations.
The US currently has no rules for automated driver assistance systems. Automakers are allowed to self-certify that their vehicles comply with existing rules, according to University of South Carolina law professor Bryant Walker Smith, who focuses on automated driving.
AAA Testing Finds Automated Driver Assistance Systems to be Unreliable
A study by the American Automobile Association in the US found driver assistance systems to be unreliable, according to a recent account in Car and Driver.
AAA tested five 2019 and 2020 vehicles equipped with the most advanced technology each automaker had to offer. These included a 2019 BMW X7 with “Active Driving Assistant Professional,” a 2019 Cadillac CT6 with “Super Cruise,” a 2019 Ford Edge with “Ford Co-Pilot360,” a 2020 Kia Telluride with “Highway Driving Assist” and a 2020 Subaru Outback with “EyeSight.” All of these systems are regarded as Level 2 autonomous systems, meaning the driver is expected to remain aware while the system is in use.
The AAA testing showed that all five vehicles experienced on average one issue—such as the need for the driver to act quickly to keep the vehicle centered in a lane—every eight miles.
The safety benefits of such systems, the study concluded, are not reliable. The systems become dangerous when drivers over-rely on the technology and do not notice when the systems disengage—which they often do with little notice, AAA noted. Of all the errors that the systems made on open-road testing, 73% involved instances of lane departure or erratic lane position.
“Manufacturers need to work toward more dependable technology, including improving lane keeping assistance and providing more adequate alerts,” stated Greg Brannon, director of automotive engineering and industry relations at AAA, in a statement. “Active driving assistance systems are designed to assist the driver and help make the roads safer, but the fact is, these systems are in the early stages of their development.”
In the AAA study, the Cadillac CT6 experienced the fewest number of issues over the roughly 800 miles the vehicles each traveled, followed by the BMW X7, Subaru Outback, Kia Telluride, and Ford Edge. On the closed course portion of the test, the vehicles had difficulty when approaching a simulated disable vehicle, with a collision occurring two-thirds of the time.
“We know human error contributes to 94% of all crashes, which is why we are focused on advancing driver assist technologies that can help significantly enhance safety,” stated Wade Newton, the VP of communications at the Alliance for Automotive Innovation, to Car and Driver.“However, as we integrate these increasingly advanced driver assistance features into more vehicles, it is critical that drivers fully understand the system’s capabilities and limitations as well as their responsibilities.”
Facial recognition differs from the conventional camara surveillance, as it is not a mere passive recording, but rather it entails identification of an individual by comparing newly capture images with those images saved in a data base.
The status in Europe
Although facial recognition is not yet specifically regulated in Europe, it is covered by the General Data Privacy Regulation – GDPR – as a means of collecting and processing personal biometric data, including facial data and fingerprints. Therefore, facial recognition is only possible under the criteria of the GDPR.
Biometric data provides a high level of accuracy when identifying an individual due to the uniqueness of the identifiers (facial image or fingerprint) and a great potential to improve business security.
The processing of biometric data, which is considered sensitive data, is in principle prohibited with some exceptions, such as, for reasons of substantial public interests, to protect the vital interest of the data holder or another person, or if data holder has given its explicit consent, to name some.
Moreover, other factors such as proportionality or power imbalance are considered to determine if it is a valid exception, for instance, facial recognition can be considered disproportionate to track attendance in a school, since less intrusive options are available. Also even when the data holder has explicitly consented to the processing of biometric data, consideration should be given to potential imbalance of power dynamics between the individual data holder and the institution processing the data. For instance in a student and school scenario, there could be doubts as to whether the consent of the parents of a student to the use of facial recognition techniques, is freely given in the manner intended by the GDPR and therefore, a valid exception to the prohibition of processing.
One of the challenges in this field is that the underlying technology used for facial recognition, for instance AI, can present serious risks of bias and discrimination, affecting and discriminating many people without the social control mechanism that governs human behaviour. Bias and discrimination are inherent risks of any societal or economic activity. Human decision-making is not immune to mistakes and biases. However, the same bias when present in AI could have a much larger effect.
Authentication vs. identification
Obviously biometrics for authentication (which is described as a security mechanism), is not the same as remote biometric identification (which is used for instance in airports or public spaces, to identify multiple persons’ identities at a distance and in continuous manner by checking them against data stored in a database).
The collection and use of biometric information used for facial recognition and identification in public spaces carries specific risks for fundamental rights. In fact, the European Commission (EC) has warned that remote biometric identification is the most intrusive form of facial recognition and it is in principle prohibited in Europe.
So where is all this going?
What should prevail: the protection of fundamental rights, or the advancement that comes with invasive and overpowering new technologies?
New technologies, like AI, bring some benefits, such as technological advancement and more efficiency and economic growth, but at what cost?
Using a risk-based approach the EC has considered the use of AI for remote biometric identification and other intrusive surveillance technologies to be high-risk, since it could compromise fundamental rights such as human dignity, non-discrimination and privacy protection.
The EU Commission is currently investigating whether additional safeguards are needed or whether facial recognition should not be allowed in certain cases, or certain areas, opening the door for a debate regarding the scenarios that could justify the use of facial recognition for remote biometric identification.
Artificial intelligence entails great benefits but also several potential risks, such as opaque decision-making, gender-based or other kinds of discrimination, intrusion in our private lives or being used for criminal purposes.
To address these challenges, the Commission in its white paper on AI, issued in February this year, has proposed a new regulatory framework on high risk AI, and a prior conformity assessment, including testing and certification of AI facial recognition high risk systems to ensure that they abide by EU standards and requirements.
The regulatory framework will include additional mandatory legal requirements related to training data, record-keeping, transparency, accuracy, oversight and application-based use, and specific requirements for some AI applications, specifically those designed for remote biometric facial recognition.
We should then expect new regulation coming, with the aim to have an AI system framework, compliant with current legislation and that does not compromise fundamental rights.
Opportunities for startups?
Facial recognition technologies are here to stay, therefore, so if you are thinking about changing your hair colour, watch out as your phone might not recognize you! With the speed in which facial recognition is growing, we should not wait too long for new forms of ‘selfie payment’.
Facial recognition is already been used quite successfully in several areas, among them:
Health: Where thanks to face analysis is already possible to track patience use of mediation more accurately;
Market and retail: Where facial recognition promises the most, as ‘knowing your customer’ is a hot topic, this means placing cameras in retail outlets to analyze the shopper behavior and improve the customer experience, subject of course to the corresponding privacy checks; and,
Security and law enforcement: That is, to find missing children, identify and track criminals or accelerate investigations.
With lots of choices on the horizon for facial recognition, it remains to be seen whether European startups will lead new innnovations in this area.
Combine Python and Tableau with TabPy; Learn Deep Learning with this Free Course from Yann LeCun; Find 15 Exciting AI Project Ideas for Beginners; Read about the Rise of the Machine Learning Engineer; See How to Incorporate Tabular Data with HuggingFace Transformers
This week on KDnuggets: Combine Python and Tableau with TabPy; Learn Deep Learning with this Free Course from Yann LeCun; Find 15 Exciting AI Project Ideas for Beginners; Read about the Rise of the Machine Learning Engineer; See How to Incorporate Tabular Data with HuggingFace Transformers; and much, much more.
I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:
I had a long conversation with one of my colleagues about imperfect information games and deep learning this weekend and reminded me of an article I wrote last year so I decided to republish it.
Poker has remained as one of the most challenging games to master in the fields of artificial intelligence(AI) and game theory. From the game theory-creator John Von Neumann writing about poker in his 1928 essay “Theory of Parlor Games, to Edward Thorp masterful book “Beat the Dealer” to the MIT Blackjack Team, poker strategies has been an obsession to mathematicians for decades. In recent years, AI has made some progress in poker environments with systems such as Libratus, defeating human pros in two-player no-limit Hold’em in 2017. Last year, a team of AI researchers from Facebook in collaboration with Carnegie Mellon University achieved a major milestone in the conquest of Poker by creating Pluribus, an AI agent that beat elite human professional players in the most popular and widely played poker format in the world: six-player no-limit Texas Hold’em poker.
The reasons why Pluribus represents a major breakthrough in AI systems might result confusing to many readers. After all, in recent years AI researchers have made tremendous progress across different complex games such as checkers, chess, Go, two-player poker, StarCraft 2, and Dota 2. All those games are constrained to only two players and are zero-sum games (meaning that whatever one player wins, the other player loses). Other AI strategies based on reinforcement learning have been able to master multi-player games Dota 2 Five and Quake III. However, six-player, no-limit Texas Hold’em still remains one of the most elusive challenges for AI systems.
Mastering the Most Difficult Poker Game in the World
The challenge with six-player, no-limit Texas Hold’em poker can be summarized in three main aspects:
Dealing with incomplete information.
Difficulty to achieve a Nash equilibrium.
Success requires psychological skills like bluffing.
In AI theory, poker is classified as an imperfect-information environment which means that players never have a complete picture of the game. No other game embodies the challenge of hidden information quite like poker, where each player has information (his or her cards) that the others lack. Additionally, an action in poker in highly dependent of the chosen strategy. In perfect-information games like chess, it is possible to solve a state of the game (ex: end game) without knowing about the previous strategy (ex: opening). In poker, it is impossible to disentangle the optimal strategy of a specific situation from the overall strategy of poker.
The second challenge of poker relies on the difficulty of achieving a Nash equilibrium. Named after legendary mathematician John Nash, the Nash equilibrium describes a strategy in a zero-sum game in which a player in guarantee to win regardless of the moves chosen by its opponent. In the classic rock-paper-scissors game, the Nash equilibrium strategy is to randomly pick rock, paper, or scissors with equal probability. The challenge with the Nash equilibrium is that its complexity increases with the number of players in the game to a level in which is not feasible to pursue that strategy. In the case of six-player poker, achieving a Nash equilibrium is computationally impossible many times.
The third challenge of six-player, no-limit Texas Hold’em is related to its dependence on human psychology. The success in poker relies on effectively reasoning about hidden information, picking good action and ensuring that a strategy remains unpredictable. A successful poker player should know how to bluff, but bluffing too often reveals a strategy that can be beaten. This type of skills has remained challenging to master by AI systems throughout history.
Like many other recent AI-game breakthroughs, Pluribus relied on reinforcement learning models to master the game of poker. The core of Pluribus’s strategy was computed via self-play, in which the AI plays against copies of itself, without any data of human or prior AI play used as input. The AI starts from scratch by playing randomly, and gradually improves as it determines which actions, and which probability distribution over those actions, lead to better outcomes against earlier versions of its strategy.
Differently from other multi-player games, any given position in six-player, no-limit Texas Hold’em can have too many decision points to reason about individually. Pluribus uses a technique called abstraction to group similar actions together and eliminate others reducing the scope of the decision. The current version of Pluribus uses two types of abstractions:
Action Abstraction: This type of abstraction reduces the number of different actions the AI needs to consider. For instance, betting $150 or $151 might not make a difference from the strategy standpoint. To balance that, Pluribus only considers a handful of bet sizes at any decision point.
Information Abstraction: This type of abstraction groups decision points based on the information that has been revealed. For instance, a ten-high straight and a nine-high straight are distinct hands, but are nevertheless strategically similar. Pluribus uses information abstraction only to reason about situations on future betting rounds, never the betting round it is actually in.
To automate self-play training, the Pluribus team used a version of the of the iterative Monte Carlo CFR (MCCFR) algorithm. On each iteration of the algorithm, MCCFR designates one player as the “traverser” whose current strategy is updated on the iteration. At the start of the iteration, MCCFR simulates a hand of poker based on the current strategy of all players (which is initially completely random). Once the simulated hand is completed, the algorithm reviews each decision the traverser made and investigates how much better or worse it would have done by choosing the other available actions instead. Next, the AI assesses the merits of each hypothetical decision that would have been made following those other available actions, and so on.The difference between what the traverser would have received for choosing an action versus what the traverser actually achieved (in expectation) on the iteration is added to the counterfactual regret for the action. At the end of the iteration, the traverser’s strategy is updated so that actions with higher counterfactual regret are chosen with higher probability.
The outputs of the MCCFR training are known as the blueprint strategy. Using that strategy, Pluribus was able to master poker in eight days on a 64-core server and required less than 512 GB of RAM. No GPUs were used.
The blueprint strategy is too expensive to use real time in a poker game. During actual play, Pluribus improves upon the blueprint strategy by conducting real-time search to determine a better, finer-grained strategy for its particular situation. Traditional search strategies are very challenging to implement in imperfect information games in which the players can change strategies at any time. Pluribus instead uses an approach in which the searcher explicitly considers that any or all players may shift to different strategies beyond the leaf nodes of a subgame. Specifically, rather than assuming all players play according to a single fixed strategy beyond the leaf nodes, Pluribus assumes that each player may choose among four different strategies to play for the remainder of the game when a leaf node is reached. This technique results in the searcher finding a more balanced strategy that produces stronger overall performance.
Facebook evaluated Pluribus by playing against an elite group of players that included several World Series of Poker and World Poker Tour champions. In one experiment, Pluribus played 10,000 hands of poker against five human players selected randomly from the pool. Pluribus’s win rate was estimated to be about 5 big blinds per 100 hands (5 bb/100), which is considered a very strong victory over its elite human opponents (profitable with a p-value of 0.021). If each chip was worth a dollar, Pluribus would have won an average of about $5 per hand and would have made about $1,000/hour.
The following figure illustrates Pluribus’ performance. On the top chart, the solid lines show the win rate plus or minus the standard error. The bottom chart shows the number of chips won over the course of the games.
Pluribus represents one of the major breakthroughs in modern AI systems. Even though Pluribus was initially implemented for poker, the general techniques can be applied to many other multi-agent systems that require both AI and human skills. Just like AlphaZero is helping to improve professional chess, its interesting to see how poker players can improve their strategies based on the lessons learned from Pluribus.
The pandemic has changed almost everything about our everyday lives. While you may still be adjusting to a new remote work environment and battling productivity and overpowering procrastination, your children are likely enduring the same waves of uncertainty.
As a result, parents must help young students overcome new challenges as the new medium of online learning sets the pace for youth worldwide.
Suppose you are wondering how to keep your kids motivated to learn during the lockdown. In that case, these top tips will help you define a suitable learning environment that compliments homeschooling and virtual classrooms.
Create A Space For Learning
Your child will need their own space to learn, and while you can create a learning environment that is similar to a home office space, your child’s learning area must not be shared with any adults that may be working from home.
You will need to stock up on specific supplies to ensure the at-home learning environment is optimally functional. Besides schooling supplies, suitable furnishings, and a PC or laptop, it is also a great idea to decorate the background to complement learning. Therefore, calming colors are best.
Define Learning Times
Overcoming procrastination and lack of motivation are challenges the world’s workforce currently faces. What’s more, students worldwide are facing the same issues. To encourage your child, you will need to define learning times and help them craft a strict daily schedule. In addition to this, you should also lead by example by sticking to your work schedule.
Explain The New Learning Dynamic
Regardless of your child’s age, the pandemic’s specifics, and how it has altered our lives is undeniably overwhelming for the average person. That said, these changes are even more frightening and confusing for younger minds who were previously conforming to entirely different dynamics of learning and socializing.
To best assist your child with the adjustment, parents must explain what is happening in the world. Your child should understand why their learning environment is different. Furthermore, you should also explain ongoing changes regarding the pandemic with your child to keep them up to date. This effort will prevent your child from feeling uncertain during this challenging time.
Be Present And Involved
Parents are required to support online learning as the parent-teacher collaboration dynamic has suddenly become vitally important. While previously, you may not have been as involved in your child’s schooling as education took place in controlled learning environments. As a result, you should make efforts to be present during your child’s online classes. Being involved is crucial to ensure your child maintains the ideal discipline that is required for learning. You will now need to participate in your child’s schooling in a manner that assists educators.
Education standards may have changed dramatically, although not all the changes should be seen as unfavourable. Several studies suggest that even social distancing requirements are unlikely to affect youngsters negatively. The best way to keep your child motivated is to be involved in their education and provide them with an ideal environment that supports learning.