Should Tesla disable your Autopilot if you're not diligent? - and a survey of robocar validation
Submitted by brad on Tue, 2016-07-05 10:24Executive Summary: A rundown of different approaches for validation of self-driving and driver assist systems, and a recommendation to Tesla and others to have countermeasures to detect drivers not watching the road, and permanently disable their Autopilot if they show a pattern of inattention.
The recent fatality for a man who was allowing his car to be driven by the Tesla "autopilot" system has ignited debate on whether it was appropriate for Tesla to allow their system to be used as it was.
Tesla's autopilot is a driver assist system, and Tesla tells customers it must always be supervised by an alert driver ready to take the controls at any time. The autopilot is not a working self-driving car system, and it's not rated for all sorts of driving conditions, and there are huge numbers of situations that it is not designed to handle and can't handle. Tesla knows that, but the public, press and Tesla customers forget that, and there are many Tesla users who are treating the autopilot like a real self-driving car system, and who are not paying attention to the road -- and Tesla is aware of that as well. Press made this mistake as well, regularly writing fanciful stories about how Tesla was ahead of Google and other teams.
Brown, the driver killed in the crash, was very likely one of those people, and if so, he paid for it with his life. In spite of all the warnings Tesla may give about the system, some users do get a sense of false security. There is debate if that means driver assist systems are a bad idea.
There have been partial self-driving systems that require supervision since the arrival of the cruise control. Adaptive cruise control is even better, and other car companies have released autopilot like systems which combine adaptive cruise control with lane-keeping and forward collision avoidance, which hits the brakes if you're about to rear end another car. Mercedes has sold a "traffic jam assist" like the Telsa autopilot since 2014 that only runs at low speeds in the USA. You can even go back to a Honda demo in 2005 of an autopilot like system.
With cruise control, you might relax a bit but you know you have to pay attention. You're steering and for a long time even the adaptive cruise controls did not slow down for stopped cars. The problem with Tesla's autopilot is that it was more comprehensive and better performing than earlier systems, and even though it had tons of things it could not handle, people started to trust it with their lives.
Tesla's plan can be viewed in several ways. One view is that Tesla was using customers as "beta testers," as guinea pigs for a primitive self-drive system which is not production ready, and that this is too much of a risk. Another is that Tesla built (and tested) a superior driver assist system with known and warned limitations, and customers should have listened to those warnings.
Neither is quite right. While Tesla has been clear about the latter stance, with the knowledge that people will over-trust it, we must face the fact that it is not only the daring drivers who are putting themselves at risk, it's also others on the road who are put at risk by the over-trusting drivers -- or perhaps by Tesla. What if the errant car had not gone under a truck, but instead hit another car, or even plowed into a pedestrian when it careened off the road after the crash?
At the same time, Tesla's early deployment approach is a powerful tool for the development and quality assurance of self-drive systems. I have written before about how testing is the big unsolved problem in self-driving cars. Companies like Google have spent many millions to use a staff of paid drivers to test their cars for 1.6 million miles. This is massively expensive and time consuming, and even Google's money can't easily generate the billions of miles of testing that some feel might be needed. Human drivers will have about 12 fatalities in a billion miles, and we want our self-driving cars to do much better. Just how we'll get enough verification and testing done to bring this technology to the world is not a solved problem.