Complexities Of Driving Controls And AI Autonomous Cars

Road Trip — The AI system of self-driving cars needs to be properly aligned with the core driving controls: the steering wheel, brakes, and accelerator. (GETTY IMAGES)

By Lance Eliot, the AI Trends Insider

Which is better, a lead foot on the brakes and a light-foot on the gas, or a lead foot on the gas and a featherweight foot on the brakes?

Hard to say.

If you are trying to drive onto the freeway, you usually need to double down on the gas pedal and make sure you enter into traffic at a fast and equitable speed.

If you are driving in a busy mall parking lot, probably best to keep your foot leaning on the brakes so that you don’t hit anyone.

I remember when I was guiding my children on how to drive a car that it seemed like they would inevitably drift toward having a heavy foot and a light foot on each of the respective pedals. Over time, they became proficient in judging how much pressure to apply for the gas and the brakes, doing so as based on the situation and the nature of the driving circumstances involved.

Today, they put little conscious thought into the matter and are seasoned drivers.

Novice drivers though aren’t quite sure how to treat the car controls. Besides my own children, I’ve seen the teenagers of other parents that were also apt to misjudge the controls when first learning to drive. It was somewhat comical one day to watch as a teenager drove down our street and his car seemed to start and stop. One moment the accelerator was being pushed, the next moment the teenager plied on the brakes. This makes sense in that he was concerned once his momentum got going that he was perhaps barreling too fast, so he wanted to slow down, but once he slowed down it became apparent that he needed to add some gas to get going again.

I’m sure we’ve all had the same experience when trying to learn to drive.

I’d also bet that sometimes you’ve found yourself thrown a kilter when trying to drive someone else’s car, and you were unsure of how sensitive the car controls were.

Dealing With Car Driving Controls

Whenever I rent a car, which I do a lot of the time due to my work travel, I often discover that during the first few minutes of driving the rental car that I am over-controlling it.

I need to initially get used to how the brakes react, how the accelerator reacts, and how the steering reacts. It doesn’t take long. It does though give you a pause for thought and perhaps allow you to reminisce about the old days of when you first learned to drive.

You might be so familiar with driving your car that it seems nearly unfathomable to imagine that anyone could drive it in a stuttered kind of way. For you, the reaction of the pedals and the steering is now considered “natural” and flows with an ease that you don’t likely think about. If someone else tries to drive your car, and they have troubles doing so, perhaps mentioning that the pedals are slow to react or overly fast to react, the odds are that you’d be surprised at this response. For you, the pedals and steering are “just right” and have reached a vaunted Goldilocks stage, namely not too easy and not too hard to use.

In referring to the car controls, it is simplest to focus on the brakes, the accelerator, and the steering wheel.

Those of course aren’t the only car controls you deal with.

You need to start the car. You need to put the car into gear, perhaps using reverse to back out of your garage, and then place the car into drive to head down the street that you live on. Besides the gears, you might have also been using your parking brake. I swear that at least once per month or so that I somehow forget that I had used the parking brake and then upon backing out of my garage, and realizing the car is moving really sluggishly, I sheepishly realize I had forgotten to release the parking brake. Hope it’s not a sign of dementia setting in.

You could suggest that your turn indicators or blinkers are also part of your car controls. They don’t likely cause the car to do anything in terms of its motion, but they are an important means of informing other drivers and pedestrians about what way you are going to go. I’m guessing you also long ago learned the arm signals that you can also use, placing your arm out the driver’s side window to use it as a turn indicator. It is a rarity to do so today. When I see someone use their arm in this manner, I often do a little laugh at the charm of using such archaic ways (also, does this mean their blinkers are broken, or are they just trying to be a friendly driver, or what is the story?).

Would you say that your headlamps are part of your car controls?

It seems a bit of a stretch. You could though argue that they are important to being able to see the road ahead, especially at nighttime. They also can forewarn other drivers of your presence. The headlamps and the fog lamps can be an important element of driving. We can nonetheless debate whether you consider them as a car control or not, since the headlamps don’t directly control the movement of the car.

I’m going to herein concentrate on the brakes, the accelerator, and the steering, claiming that those are the core controls of the car. I’ve mentioned the other control related aspects to make sure that no one accuses me of having forgotten about those other means of controlling the car. Yes, the parking brake can be quite important. Yes, choosing the gear for the car is quite important. And so on. Please just assume that I acknowledge and appreciate their role in the driving task.

Parking Brake Usage

Speaking of the parking brake, I learned a handy trick from a college buddy during my wild driving days as a college student.

One day, he was driving his car and I was sitting in the front passenger seat. We were rocketing along and headed from Los Angeles up to the Bay Area, going to a college football game up there. Admittedly, he was going faster than the speed limit. Wrong of us. You have to though realize that there is a lengthy freeway, designated as the 5 freeway, requiring about a 6- or 7-hour drive to get up to the Bay Area from Southern California, and I dare say that everyone driving this highway goes over the speed limit (well, okay, maybe not the truck drivers). You can cut down the drive to around 5 hours if you are willing to inch above the posted speed limit (probably more than just an inch!).

All of a sudden, my friend reached for the parking brake and began to make use of it. This didn’t make any sense to me. A parking brake was intended for use when you are parked, or, so I had thought. Why in the world was he toying with the parking brake? Had he lost his mind? Did he have some beers before we headed-out, doing a bit of pre-gaming before we reached our destination?

I hoped not.

Upon seeing my quizzical look, he explained that there was a California Highway Patrol (CHP) police car that had gotten onto the freeway and was now behind us, about a quarter mile or so. We were going over the speed limit and a handy target for the CHP to pull us over. My buddy told me that if he used the actual brakes, the brake lights would illuminate. This would be a sure sign that he was trying to bleed off speed and it might spark the CHP to come and give him a speeding ticket.

By using the parking brake, he was able to gradually reduce the speed of the car. This also did not provide any kind of overt clue of what was taking place. Clever? Or, a misguided use of a crucial feature of the car? You decide. In any case, we ended-up going the actual speed limit for much of the rest of the trip, doing so as long as the CHP officer was behind us.

So, as I say, the parking brake is indeed a car control.

The Gears Count Too

Likewise, the use of the gears.

You likely use the gears in a perfunctory way. Put the car into reverse to back out of your garage, put it into forward or drive to move ahead. You do this similarly when parking in a parking lot. Most of the time, there’s not much else involved in dealing with gears, at least in an automated transmission equipped car.

During a camping trip with my son’s Boy Scout troop, we drove up into the mountains, and I suddenly became more aware of the gears in my car. On some steep roads, I had to put the car into a lower gear. Also, while driving along the winding roads that dipped and rose, I could sense the car doing a lot of shifting among the gears. Usually, the shifting was done without any noticeable indication. On these tougher roads, the shifting became more obvious, both the sounds of the gears shifting and the engine noise, along with the feeling of the car as each gear shift took place.

A good driver is aware of the full gamut of ways in which they can control their car. No tool or feature or driving control should be ignored or forgotten and left to being unused if it can be put to appropriate use (and sometimes inappropriate use, I suppose, such as the parking brake to save us from a speeding ticket).

Evolution Of Driving Controls

Focusing on just the brakes, the accelerator, and the steering wheel, let’s consider how you make use of those driving controls.

At a tactical level, it’s apparent that you use the brakes to slow down the car. You use the accelerator to speed-up the car. You use the steering wheel to redirect the direction of the car. Novice drivers aren’t at first sure of which pedal is the brake and which is the gas. They often get confused about which is which. They are also unsure about whether to use their left foot, their right foot, or maybe both feet to control the pedals.

What’s interesting about the history of car controls is the evolution to what we have today.

In the United States, for example, we earlier in our history had the driving controls on the right side of the car, rather than the left side. This is surprising to most people here in the U.S. What, the driving controls were the “wrong” way, some ask? Note, wrong meaning that those with driving controls on the left side tend to think that’s the proper placement, while those that have their driving controls on the right side tend to think they have the proper placement.

In any case, we have an estimated several hundred million licensed drivers in the United States. California has the most licensed drivers, coming out to around 27 million or so. I mention these rather large numbers to point out that it is quite amazing that all of those people have learned how to drive a car. In fact, it likely didn’t take them very long to learn how to drive a car. Furthermore, they drive a car with relative ease, meaning that they don’t routinely struggle to control the car and make it go.

Imagine if driving a car was akin to piloting a plane or a spaceship.

Think about how much training everyone would need to have.

The odds too are that they would need refresher training from time to time. The complexity of the driving controls would cause us all to struggle when driving a car. It would be a chore to have to drive your car, even for a short trip to the grocery store.

Overall, in a kind of Darwinian process, we have landed upon a set of car controls that seems to work for us all. We have evolved cars to a point that the everyday person can drive a car. Plus, they can drive a car again and again. It’s almost as easy as riding a bike. Some might say easier since when riding a bike you need to first learn how to balance the bike.

It has all boiled down to a pedal to make the car stop, a pedal to make the car go, and a wheel that you can twist and turn to steer the car. That’s about as basic or fundamental as you can be. We could have instead evolved to a device on the dashboard of the car that does the braking and one that does the acceleration, or maybe have knobs on the steering column to do so. We could have evolved to turn the direction of the car via a knob or other contraption, rather than using a steering wheel.

The beauty of the pedals and the steering wheel are their simplicity. They are easy to learn how to use. Not only are they easy to use, they are devised to be used in a timely fashion. When you need to suddenly turn the car, you can grasp and twist the steering wheel, doing so almost instinctively, and using your arms and hands in a fluid motion as you do so.

The physical act of controlling the car is crucial. If the steering was cumbersome, imagine how bad things would be when you got into a panic situation. Having only a split second available to steer the car, any kind of added effort involved to physically do so would lead to more deaths and destruction. Humans need to have an ability to quickly make use of the controls. That’s why the controls are placed where they are.

You might find of interest that there have been studies done about moving or changing the nature of the driving controls.

I’ve already mentioned some ways that they could be physically repositioned. Some suggest that maybe we should just speak to the car controls, using our voice, and not need to do anything physical with our appendages. As you can guess, this would have both advantages and disadvantages (suppose you lost your voice, suppose you uttered a command inadvertently, etc.).

There are numerous studies about how long it takes to make use of the car controls.

Length of time to invoke car controls can be a life-or-death matter.

Contemplate for a moment what happens when you see that a dog is racing across the street in front of your car. You must first see the dog doing this, and thus your eyes need to be watching the road. Your brain has to process the aspect that the dog is possibly going to get hit. Your mind needs to decide what to do, determining whether to steer away, or hit the brakes, or speed-up to go around the dog. Once your mind decides what to do, your body needs to physically take the action required.

Even once you’ve grabbed the steering wheel or jammed your foot onto the brakes, it will take some additional time for the car to respond. There is not an instantaneous reaction to your invoking the car controls commands and having the car do what you want. When you turn the steering wheel, this is conveyed to the wheels of the car, and those wheels need to physically be turned, all of which takes time. Braking the car is the same kind of time delayed facet, meaning that once you’ve slammed on the brakes, this needs to be enacted by the tires and the braking system, along with the time involved in the physics of the car coming to a halt.

In our minds, we often blur the distinction between acting upon the car controls and the action of the car complying with those controlling actions. Only when you are in a dire situation do you at times become aware of the difference. Within your mind, you might be thinking that the car can stop on a dime, but by the time you move your body and get your foot to pressure the brake pedal, followed by the brakes being actually applied, followed by the tires being engaged by the brakes, followed by the physics of the road and the tires bringing the car to a halt, it can be much longer than you think.

If you’ve ever slid into an object that you thought you could brake the car before hitting, you know what I mean by this time delayed reaction aspect.

Another way in which you might have become aware of the indirect aspects of using the car controls and seeing what the car does would be in situations like sliding on a wet rain-soaked road or driving in the snow. You turn the wheels in one direction. The car though decides to slide in another direction. You turn the wheel again, hopeful of trying to get the tires to catch onto the roadway surface. You try to accelerate, but perhaps the wheels just spin without getting traction. And so on.

Have you ever had a moment whereby you did one action with the car controls and regretted it, and so immediately tried to undo the action? I was sitting at a red light waiting to make a right turn, and decided it was safe to go, so I pressured the accelerator to move ahead. All of a sudden, another car came barreling down the street and came into the lane that I was turning into. I realized that my best bet was to halt my right turn. I quickly shifted my foot from the accelerator pedal to the brake pedal.

Fortunately, I came to a halt and avoided colliding with the barreling car. I wondered though what my car must have thought of my actions. Suppose the car was actually alive, maybe akin to a pet, like your pet dog. You jerk the dog (or car) to move forward, but then you just as quickly jerk the dog (or car) to come to a halt. What kind of messed up owner are you? Can’t decide what to do, and cause the dog (or car) to undergo strain and stress, presumably needlessly if you had better planned out your actions.

Tactical and Strategic Uses

This brings up another point about the use of the car controls.

There are the tactical aspects of activating and using the controls, such as putting your foot onto the pedals and using your hands to turn the steering wheel. You normally though are making use of the controls in a more macroscopic way too, at least hopefully you are doing so.

Suppose you want to drive down the street and make a right turn at the corner. There are a series of tactical car control motions that you would undertake. You accelerate to get the car moving down the street. You steer the car to the right side of the road. If there’s a parked car at the curbside, you might slightly steer your car to the left to avoid brushing against the parked car. When you near the corner, you likely are steering the car towards the corner itself, and perhaps applying the brakes to slow down so you can readily make the turn.

You’ve put together a series of tactical car control commands.

They are each of their own merit. Yet, they also are part of a larger perspective on what you are trying to achieve.

You want to drive down the street, reach the corner, make the turn, and do this without hitting other cars. Doing so has required a series of back-to-back tactical car control command efforts.

Novice drivers often struggle with this overarching aspect. They are at first overwhelmed by the act of merely using the driving controls. That’s why most people will have a novice driver initially practice in an empty parking lot. Let the human become familiar with the itsy-bitsy aspects of how to use the controls. Once you’ve mastered that ability, you can then begin to consider how to dovetail those tactical elements into an overall driving plan.

I know that most of you are likely seasoned drivers and perhaps take for granted how easy it is to not only use the car controls, but also tie them together into a series of efforts to achieve a larger goal such as driving down the street to make a turn. As mentioned before, I think it is a miracle that we have hundreds of millions of people that do this each and every day, and yet we don’t have pure chaos and pandemonium. Could you get that many people to do something like balance their checkbooks? Driving a car is much harder, and astoundingly we’ve gotten everyone to do it reasonably well for most of the time.

AI Self-Driving Cars

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. An essential aspect of AI self-driving cars is the use of the driving car controls by the AI system. There is more to this than perhaps meets the eye.

Allow me to elaborate.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the automakers are even removing the gas pedal, brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5 and Level 4, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

For my overall framework about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

For the levels of self-driving cars, see my article: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For why AI Level 5 self-driving cars are like a moonshot, see my article: https://aitrends.com/selfdrivingcars/self-driving-car-mother-ai-projects-moonshot/

For the dangers of co-sharing the driving task, see my article: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 and Level 4 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task:

Sensor data collection and interpretation
Sensor fusion
Virtual world model updating
AI action planning
Car controls command issuance

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a Utopian world in which there are only AI self-driving cars on the public roads. Currently there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other.

For my article about the grand convergence that has led us to this moment in time, see: https://aitrends.com/selfdrivingcars/grand-convergence-explains-rise-self-driving-cars/

See my article about the ethical dilemmas facing AI self-driving cars: https://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/

For potential regulations about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/assessing-federal-regulations-self-driving-cars-house-bill-passed/

For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article: https://aitrends.com/selfdrivingcars/gen-z-and-the-fate-of-ai-self-driving-cars/

Returning to the aspects of the car controls, let’s consider how the AI is involved in making use of the car controls.

I’ll focus on the use of the car controls by the AI system solely, and not cover much about what happens when the AI and the human are trying to both deal with the car controls.

As mentioned earlier, this co-sharing is especially problematic.

Imagine that you were driving a car and another human sat next to you in the front seat, having another set of driving controls, and you both could each drive the car in terms of opting to use any of the pedals and turn the steering wheel as you wished. Mind boggling. The two of you would need to really be on the same wavelength and seek to avoid undermining each other. Consider too what would happen when an emergency arose. That makes any kind of coordinated effort even more arduous.

Shifting though to the use of a true Level 5 AI self-driving car, I’m going to walk you through some salient facets of what the AI must do about the car controls and issuing of car control commands.

Six Major Steps Involved

There are six major steps involved in generating and enacting of the car controls commands:

Determine car control commands to emit
Emit car control commands to ECU
Verify that car control commands were received and are viable
ECU instructs the automotive elements
Automotive elements physically enact the received commands
Ascertain that car has reacted to the commands

For ease of discussion, we’ll assume that there is an ECU (Electronic Control Unit) that translates commands given to it from the AI system and converts those commands into some set of specific operational activities for the car.

The ECU has the task of conveying the operational activities to a myriad of other subsystems that are in the car, including the Brake Control Module (BCM), the Central Timing Module (CTM), the Transmission Control Module (TCM), the Powertrain Control Module (PCM), the Engine Control Module (ECM), etc.

There could easily be one hundred or more such sundry control subsystems that each needs to be properly communicated with and instructed on what needs to be done. You could have all of the protocols and capabilities to do so within the AI system, but this seems unwieldy and generally ill-advised. It makes more sense to hide that detail from the AI system and allow the AI system to be providing commands at a high-level of abstraction.

One important point about this notion of having the AI issues commands at a high-level of abstraction is that you can potentially port the AI system over to other brands and models of cars. If you embed directly into the AI system the specific protocols of a particular model and brand of car, it will likely make it much harder to port the AI system to other cars.

By modularizing these aspects and keeping the AI above the fray, you are usually able to more readily port over the AI system.

That being said, let’s not kid ourselves. If the AI is far removed from the nature of the underlying car brand and model, it is possible that the AI system won’t be able to issue commands that might be feasible for the particular car that the AI is working on. Suppose the AI system emits a car control command that basically asks the car to accelerate from 0 to 60 miles per hour and the AI assumes that this can be done in let’s say 3 seconds. That’s the speed for sports-oriented cars and it isn’t going to work out well for more everyday cars.

Thus, the AI system will likely need to be versed in some aspects of the brand and model car that the AI is running on. Parameters about the specific performance capabilities of the car, and the overall kinds of expected car controls commands, all need to be cooked into the AI action planner that is seeking to ultimately get the self-driving car to do driving actions. It does little good, and actually great potential harm, for the AI system to be expecting the car to perform in ways that the self-driving car is unable or incapable of doing.

Another consideration for the AI system involves the type of network into which the car control commands is going to be conveyed.

Typically, most cars use the Controller Area Network (CAN) vehicle bus as the standard for electronic communications within the car and between the myriad of subsystems.

This message-based protocol is both loved and reviled. First conceived of and released as a formal protocol in the mid-1980s, it has expanded and adapted over the years. There are numerous complimentary protocols that emerged to deal with facets such as device addressing issues, flow control capabilities, and other matters. Weaknesses and qualms often center around CAN’s lack of robust security features and difficulties that can ensue when doing troubleshooting of CAN-related problems.

Generally, it is best to try to keep the AI system above the fray about the CAN network, though there needs to be a healthy dose of skepticism built into the AI about what happens once messages are flowing in the CAN and throughout the self-driving car. The AI cannot assume that there will be a perfect conveyance of messages. The AI cannot assume that the conveyance will necessarily happen in as timely a manner as might be otherwise expected. The real-world limitations need to be encompassed by however the AI is going to be expecting the car controls commands to be carried out.

In fact, let’s look briefly at each of the six major steps and consider the types of errors or problems that can arise.

In the first step, determining the car controls commands to emit, it is conceivable that the AI might fail to arrive at a set of car controls commands that it wants to have performed.

Perhaps the AI gets gummed up trying to decide what car control commands to use. Maybe the AI hits a snag in the processing, or maybe there’s a bug in the system, or maybe the circumstance of the status of the self-driving car has baffled the AI.

Those are obviously bad possibilities.

This is why my framework for AI self-driving cars includes the need for the AI to have a self-aware capability that is double checking what the AI system itself is doing. There also needs to be fail-safe features. One of the more intriguing aspects that some are pursuing includes the idea of using AI arguing machines to try and catch these moments when the AI is not seeming to get the job done.

For my article about self-awareness of AI self-driving cars, see: https://www.aitrends.com/selfdrivingcars/self-awareness-self-driving-cars-know-thyself/

For the fail-safe aspects, see my article: https://www.aitrends.com/selfdrivingcars/fail-safe-ai-and-self-driving-cars/

For how AI arguing machines might be useful, see: https://www.aitrends.com/features/ai-arguing-machines-and-ai-self-driving-cars/

For my article about ghosts or bugs in AI self-driving cars, see: https://www.aitrends.com/selfdrivingcars/ghosts-in-ai-self-driving-cars/

For the importance of safety and AI self-driving cars, see my article: https://www.aitrends.com/selfdrivingcars/safety-and-ai-self-driving-cars-world-safety-summit-on-autonomous-tech/

For the second step of emitting the car controls commands, there is a possibility that the commands might be garbled by how they have been formatted or during their conveyance to the ECU. This sets up a rather dangerous situation. If the commands perchance are not intelligible when reaching the ECU, there’s a good chance that the ECU will realize something has gone awry, but if the commands are perchance intelligible, the ECU is likely going to try and act on them, though they aren’t what was emitted. In essence, the receiving of a wrong set of commands is bound to be worse than commands that are so unintelligible that they are obviously incorrect and improper.

Step three is an effort to checkpoint that the car control commands have indeed been received and an attempt to verify they are what was actually intended. This is a last-moment layer of defense against executing car controls that weren’t what was emitted to be undertaken. Note though that this checkpoint is not second-guessing the first two steps, since even if those steps have provided commands that might get the car into an untoward traffic situation, that’s not what this third step is trying to ascertain.

In the fourth step, the car controls commands are translated into the myriad of other electronic messages that must be sent along the CAN to the subsystems of the self-driving car. This is when the physical operational activities are being established based on the car controls commands that were provided by step one and step two, and were verified in step three. There are lots of opportunities for things to go south at this juncture. Imagine sending boats along a river with lots of tributaries, and any of those boats might go astray.

During step five, the operational activities are now being carried out, such as the brakes being applied to the tires and the car beginning to slow down or the accelerated applied and the car starting to speed-up. Assuming that the car controls commands actually reached the subsystems in step four, this step five is the actual enactment of those commands. Things can go wrong. Suppose the brakes aren’t working right? Suppose the engine is not responsive?

At step six, there is a need to ascertain that the car controls commands were carried out. As a human driver, when you wrench the steering wheel to a hard right, you can feel as the car makes the right lurching motion. This is your way of ascertaining that your command, the steering wheel movement, got translated into the actual operational and physical outcome. The AI system has to do the same kind of sensing to realize whether the car controls commands were executed, which requires using the sensors such as the cameras, radar, LIDAR, and internally focused ones like the IMU.

For more about the IMU, see my article: https://www.aitrends.com/selfdrivingcars/proprioceptive-inertial-measurement-units-imu-self-driving-cars/

For more about the cognition timing of AI self-driving cars, see my article: https://www.aitrends.com/selfdrivingcars/cognitive-timing-for-ai-self-driving-cars/

Time Is A Crucial Factor

Each of the six major steps takes time to undertake.

The AI system during the action planning portion has to gauge how long each of those steps might take and use that estimation to determine what is feasible to do. If the amount of time to apply the brakes, let’s say, would exceed what the AI wants to do in terms of the necessity of trying to slow down or halt the self-driving car, the AI would need to ascertain what another alternative might be pursued instead. For example, if the braking cannot be done in time, would it be possible to turn the steering wheel in time, and avoid whatever collision is about to occur?

As might be rather evident, the AI system cannot just emit car controls commands and walk away from the effort. The AI has to ascertain whether the car controls commands were properly and appropriately disseminated and performed. This includes not just that the commands themselves were abided by, but it also includes that the timing of the effort went as planned too. The AI has to be ready to deal with contingencies in case the commands are improperly conveyed, or improperly executed, or executed on a delayed basis, etc.

As a human driver, you likely sometimes change your mind while driving the car and suddenly do something contrary to what you had just done, such as my earlier example of radically going from speeding up to suddenly slowing down. Can the AI “change its mind” in terms of opting to do something different from what it has already started to undertake?

The answer is yes, the AI can opt to try to change what it was trying to do.

This can be problematic to execute.

If the AI can catch the emitted commands before the ECU starts to push them along to the self-driving car subsystems, those commands can be potentially suppressed. That’s a kind of undo. If the commands are already in-flight of being performed by the physical elements of the self-driving car, there’s not much chance of an undo, and instead the AI would likely need to emit a new set of car controls commands, seeking to get those executed right away (such as doing a braking on top of having just done a speeding up action).

Suppose the AI emits commands to turn the steering wheel so radically that it would cause the self-driving car to topple over and roll onto its roof. What step should catch that aspect? Even if it is caught, does the act itself mean that it should never be executed? Perhaps the AI has ascertained that making such a radical turn is worth the risk, namely that it is “better” to turn and roll over the self-driving car versus say ramming into a truck that’s filled with petrol and would explode upon impact.

There are also car control commands that could be emitted that are not possible for the physical capabilities of the car. Remember that I earlier mentioned the notion of going from 0 to 60 miles per hour in 3 seconds. If the AI system emits commands that are based on that capability, and yet if the self-driving car cannot achieve that kind of pace, this is a dangerous disconnect or confusion. One place to have a final double-check is at step 3 of the six major steps.

It would be better though that such commands never get into the stream and the AI should not be relying on a slim chance hope that infeasible or impossible commands are going to get detected and rejected downstream.

Most of these steps are complex and complicated when you get into their respective details. They involve doing a variety of real-time programming and are not particularly AI-based in terms of how you would develop these portions of the overall self-driving car systems. They are more akin to traditional automotive engineering and systems development.

You might be wondering whether AI could help out in those seemingly non-AI related portions.

Yes, there are some ways that we can apply AI to those elements.

Use of Machine Learning Or Deep Learning

One aspect involves using Machine Learning or Deep Learning to ferret out patterns of car controls commands actions.

If there are patterns that can be found, it could make things easier for the AI system and the controlling of the self-driving car.

We’ve taken large datasets of emitting car controls commands and fed them into large-scale multi-layer artificial neural networks. This is a means to seek out patterns. There are other ways to do so too, including using other popular techniques. You often would be wise to use an ensemble of Deep Learning techniques. Doing so allows for a wide array of hopefully identifying useful patterns.

To explain why patterns of car controls commands might be handy, consider what you do as a human driver of a car.

Let’s say that each morning you back your car out of your garage, doing so in reverse, going slowly, and enter into the street while backing out. Once you get far enough into the street, you turn the wheel toward the end of the street and begin to accelerate. You accelerate somewhat toward the end of the block, and then usually make a right turn. All of this is a series of maneuvers that you do each morning, almost like clockwork.

In fact, the odds are that you do this driving sequence somewhat mindlessly. You are perhaps thinking about work and other matters, rather than concentrating on the driving task. You’ve done the sequence so much that it is rote or muscle memory, nearly acting without any awareness you are doing so. I admit that I do the same. There are some mornings that I reach the end of the block and suddenly am startled, amazed that I did that while backing out and driving forward sequence, yet my mind was drifting to other topics.

Let’s put AI into the driver’s seat. Under the usual circumstances, the AI would need to calculate each of those particular actions and figure out what to do. From a raw and unprocessed blank slate, the AI would determine that the self-driving car needs to back out of the garage. Those car control commands would be baked fresh to do so. The AI would determine the self-driving car needs to make its way down the street. Those car control commands would be baked fresh. And so on.

Suppose though that the AI self-driving car had used Machine Learning or Deep Learning to examine the voluminous amount of driving actions of the self-driving car over time. This pattern of each morning doing the same driving routine has a chance of being spotted by the Machine Learning or Deep Learning. If those kinds of driving patterns are identifiable, the AI could incorporate those sets or subsets into a collection or library of known driving patterns. These patterns could then be invoked when needed.

Does the invoking of these driving patterns imply that the AI can somehow fall asleep and merely allow those sets to be carried out? Nope. In the same manner that a human driver cannot (should not) blindly perform the same driving pattern, nor can the AI do so.

When I mentioned that I sometimes reach the corner of my street and realize that I was in a kind of mental fog, don’t misinterpret that suggestion to mean that I wasn’t paying attention to the driving task at all. One day, I started to back out of my garage, and my neighbor’s cat was sauntering in the street where I was backing out the car. Had I done my usual backing out, mindlessly, and if the cat had not realized the car was coming (perhaps it had not yet had its morning coffee!), it could have been a murky morning for that cat.

An astute human driver will be paying attention to the driving task even if the driving task is a repetitive one that has become part of their habits of driving. I’m not saying all humans will be quite so diligent and that’s part of the problem with human drivers. In any case, the AI would need to still stay on top of its game, doing so while performing a set of car controls commands. At any moment, the sequence might need to be interrupted or altered, depending upon the driving situation at-hand.

For ensemble Machine Learning aspects, see my article: https://www.aitrends.com/selfdrivingcars/ensemble-machine-learning-for-ai-self-driving-cars/

For my article about Support Vector Machines (SVM), see: https://www.aitrends.com/selfdrivingcars/support-vector-machines-svm-ai-self-driving-cars/

For my article about Deep Learning and Machine Learning benchmarks, see: https://www.aitrends.com/selfdrivingcars/machine-learning-benchmarks-and-ai-self-driving-cars/

For Deep Reinforcement Learning, see: https://www.aitrends.com/ai-insider/human-aided-training-deep-reinforcement-learning-ai-self-driving-cars/

Conclusion

Car controls commands are essential to the operation of a self-driving car. They don’t get the kind of media attention that you see going toward the sensor’s aspects of self-driving cars. Nonetheless, if the AI system and the car controls commands portion aren’t properly aligned, it can be a dangerous and very untoward situation.

Though the car control commands aspects are primarily automotive engineering based, there are opportunities to add AI into the mix. One approach involves examining large datasets of car controls commands emissions and trying to find useful patterns. Caution needs to be exercised in doing so. There could be patterns that are not viable for reuse or that are only reusable in quite narrow circumstances.

As a human driver, you are continually issuing car controls commands. They are coming from your brain, going to your limbs, and then involve using the pedals and the steering wheel. The use of the pedals and the steering wheel are then translated into the use of the car subsystems. Once those car subsystems undertake their efforts, the car physically attempts to perform those driving efforts.

It’s all a dance of the human driver and the car.

The same kind of dance has to happen with the AI and the self-driving car.

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]