The cell phone ride hail apps like Uber and Lyft are now reporting great success with actual ride-sharing, under the names UberPool, LyftLines and Lyft Carpool. In addition, a whole new raft of apps to enable semi-planned and planned carpooling are out making changes.
It's not surprising there is huge debate about the fatal Tesla autopilot crash revealed to us last week. The big surprise to me is actually that Tesla and MobilEye stock seem entirely unaffected. For many years, one of the most common refrains I would hear in discussions about robocars was, "This is all great, but the first fatality and it's all over." I never believed it would all be over, but I didn't think there would barely be a blip.
There's been lots of blips in the press and online, of course, but most of it has had some pretty wrong assumptions. Tesla's autopilot is a distant cousin of a real robocar, and that would explain why the fatality is no big deal for the field, but the press shows that people don't know that.
Tesla's autopilot is really a fancy cruise control. It combines several key features from the ADAS (Advance Driver Assist) world, such as adaptive cruise control, lane-keeping and forward collision avoidance, among others. All these features have been in cars for years, and they are also combined in similar products in other cars, both commercial offerings and demonstrated prototypes. In fact, Honda promoted such a function over 10 years ago!
Tesla's autopilot primarily uses the MobilEye EyeQ3 camera, combined with radars and some ultrasonic sensors. It doesn't have a lidar (the gold standard in robocar sensors) and it doesn't use a map to help it understand the road and environment.
Most importantly, it is far from complete. There is tons of stuff it's not able to handle. Some of those things it can't do are known, some are unknown. Because of this, it is designed to only work under constant supervision by a driver. Tesla drivers get this explained in detail in their manual and when they turn on the autopilot.
This is nothing new -- lots of cars have lots of features to help drive (including the components used like cruise controls, each available on their own) which are not good enough to drive the car, and only are supposed to augment an alert driver, not replace one. Because car companies have been selling things like this for years, when the first robocar laws were drafted, they made sure there was a carve-out in the laws so that their systems would not be subject to the robocar regulations companies like Google wanted.
The Florida law, similar to other laws, says:
The term [Autonomous Vehicle] excludes a motor vehicle enabled with active safety systems or driver assistance systems, including, without limitation, a system to provide electronic blind spot assistance, crash avoidance, emergency braking, parking assistance, adaptive cruise control, lane keep assistance, lane departure warning, or traffic jam and queuing assistant, unless any such system alone or in combination with other systems enables the vehicle on which the technology is installed to drive without the active control or monitoring by a human operator.
There's been a lot of writing (and I did some of it) about the particulars of the failure of Tesla's technology, and what might be done to fix it. That's an interesting topic, but it misses a very key point. Tesla's system did not fail. It operated within its design parameters, and according to the way Tesla describes it in its manuals and warnings. The Tesla system, not being a robocar system, has tons of stuff it does not properly detect. A truck crossing the road is just one of those things. It's also poor on stopped vehicles and many other situations.
Tesla could (and in time, will) fix the system's problem with cross traffic. (MobilEye itself has that planned for its EyeQ4 chip coming out in 2018, and freely admits that the EyeQ3 Tesla uses does not detect cross traffic well.) But fixing that problem would not change what the system is, and not change the need for constant monitoring that Tesla has always declared it to have.
Today at Starship, we announced our first pilot projects for robotic delivery which will begin operating this summer. We'll be working with a London food delivery startup Pronto as well as German parcel company Hermes and the Metro Group of retailers, plus Just Eat restaurant food delivery to trial on-your-schedule delivery of packages, groceries and meals to people's homes.
Executive Summary: A rundown of different approaches for validation of self-driving and driver assist systems, and a recommendation to Tesla and others to have countermeasures to detect drivers not watching the road, and permanently disable their Autopilot if they show a pattern of inattention.
The recent fatality for a man who was allowing his car to be driven by the Tesla "autopilot" system has ignited debate on whether it was appropriate for Tesla to allow their system to be used as it was.
Tesla's autopilot is a driver assist system, and Tesla tells customers it must always be supervised by an alert driver ready to take the controls at any time. The autopilot is not a working self-driving car system, and it's not rated for all sorts of driving conditions, and there are huge numbers of situations that it is not designed to handle and can't handle. Tesla knows that, but the public, press and Tesla customers forget that, and there are many Tesla users who are treating the autopilot like a real self-driving car system, and who are not paying attention to the road -- and Tesla is aware of that as well. Press made this mistake as well, regularly writing fanciful stories about how Tesla was ahead of Google and other teams.
Brown, the driver killed in the crash, was very likely one of those people, and if so, he paid for it with his life. In spite of all the warnings Tesla may give about the system, some users do get a sense of false security. There is debate if that means driver assist systems are a bad idea.
There have been partial self-driving systems that require supervision since the arrival of the cruise control. Adaptive cruise control is even better, and other car companies have released autopilot like systems which combine adaptive cruise control with lane-keeping and forward collision avoidance, which hits the brakes if you're about to rear end another car. Mercedes has sold a "traffic jam assist" like the Telsa autopilot since 2014 that only runs at low speeds in the USA. You can even go back to a Honda demo in 2005 of an autopilot like system.
With cruise control, you might relax a bit but you know you have to pay attention. You're steering and for a long time even the adaptive cruise controls did not slow down for stopped cars. The problem with Tesla's autopilot is that it was more comprehensive and better performing than earlier systems, and even though it had tons of things it could not handle, people started to trust it with their lives.
Tesla's plan can be viewed in several ways. One view is that Tesla was using customers as "beta testers," as guinea pigs for a primitive self-drive system which is not production ready, and that this is too much of a risk. Another is that Tesla built (and tested) a superior driver assist system with known and warned limitations, and customers should have listened to those warnings.
Neither is quite right. While Tesla has been clear about the latter stance, with the knowledge that people will over-trust it, we must face the fact that it is not only the daring drivers who are putting themselves at risk, it's also others on the road who are put at risk by the over-trusting drivers -- or perhaps by Tesla. What if the errant car had not gone under a truck, but instead hit another car, or even plowed into a pedestrian when it careened off the road after the crash?
At the same time, Tesla's early deployment approach is a powerful tool for the development and quality assurance of self-drive systems. I have written before about how testing is the big unsolved problem in self-driving cars. Companies like Google have spent many millions to use a staff of paid drivers to test their cars for 1.6 million miles. This is massively expensive and time consuming, and even Google's money can't easily generate the billions of miles of testing that some feel might be needed. Human drivers will have about 12 fatalities in a billion miles, and we want our self-driving cars to do much better. Just how we'll get enough verification and testing done to bring this technology to the world is not a solved problem.
A Tesla blog post describes the first fatality involving a self drive system. A Tesla was driving on autopilot down a divided highway. A truck made a left turn and crossed the Tesla's lanes. A white truck body against a bright sky is not something the MobilEye camera system in the Tesla perceives well, and it is not designed for cross traffic.
With Mobility on Demand, you don't buy a car, you buy rides. That's certainly Uber's plan, and is a plan that makes sense for Google, Apple and other no-car companies. But even Daimler, with Car2Go/Car2Come, BMW with DriveNow and GM with Lyft plan to sell you a ride rather than a car, because it's the more lucrative thing to do.
So what does that car of the future look like? There is no one answer, because in this world, the car that is sent to pick you up is tailored to your trip. The more people traveling, the bigger the car is. If your trip does not involve a highway, it may not be a car capable of the highway. If your trip is up to a mountain cabin, it's more like an SUV, but you never use an SUV to go get a bottle of milk the way we do today. If it's for a cruise to the beach on a sunny day, the roof may have been removed at the depot. If it's for an overnight trip to a country home, it may be just beds.
I outlined many of these changes in this article on design changes in cars but today I will focus on the incredibly cheap and simple design of what should become the most common vehicle made, namely the car designed for a short urban trip by one person. That's 80% of trips and around 45% of miles, so this should be a large fraction of the fleet. I predict a lot of these cars will be made every year -- more than all the cars made today, even though they are used as taxis and shared among many passengers.
What does it look like?
A car for 1-2 people will be small. It will probably be around 1.5m wide, narrow enough that you can fit two in a lane, and have it park very efficiently when it has to wait. If it's for just one person, it won't be very long either. For two people, there will be a "face to face" configuration which is longer and an "tandem" configuration which is a bit shorter. The 2 person vehicles aren't a lot bigger or heavier than the one person, so they might be the most common cars, since you can serve a solo rider fairly efficiently with one, even if not perfectly efficient.
A car that is so narrow can't corner very fast. A wide stance is much more stable. There are a few solutions to that, including combinations of these:
The car already has a big win when it comes to aerodynamic drag by only being half-width. The non-highway version probably gives back a bit of that because you don't need to worry as much about that if you are not going fast. Energy lost to drag goes up with the square of velocity. So a 30mph car has 1/4 the drag of a 60mph car, and 1/8th the drag of a similar car of full width. The highway car needs to be shaped as close to a "teardrop" as you can, but the city car can get away with being a bit taller for more comfortable seating and entry/exit.
Political debate is going overboard these days. I travel overseas all the time and if I reveal I live in the USA, you can't stop people from asking about Trump. It's getting frustrating and boring. But to avoid contentious topics, let's talk about guns!
Elon Musk likes to say pretty controversial things off the cuff, and so do I, but he inspired a number of threads by saying at Re:Code that there's a billion to one chance we're living in base reality. To use his word, this world is almost surely a "simulation."
When I give talks on robocars, the most common question, asked almost all the time, is the one known as the "trolley problem" question, "What will the car do if it has to choose between killing one person or another" or other related dilemmas. I have written frequently about how this is a very low priority question in reality, much more interesting to philosophy classes than it is important. It is a super-rare event and there are much more important everyday ethical questions that self-driving car developers have to solve long before they will tackle this one.
In spite of this, the question persists in the public mind. We are fascinated and afraid of the idea of machines making life or death decisions. The tiny number of humans faced with such dilemmas don't have a detailed ethical debate in their minds; they can only go with their "gut" or very simple and quick reasoning. We are troubled because machines don't have a difference between instant and carefully pondered reactions. The one time in billions of miles(*) that a machine faces such a question it would presumably make a calculated decision based on its programming. That's foreign to our nature, and indeed not a task desired by programmers or vendors of robocars.
There have been calls to come up with "ethical calculus" algorithms and put them in the cars. As a programmer, I could imagine coding such an algorithm, but I certainly would not want to, nor would I want to be held accountable for what it does, because by definition, it's going to do something bad. The programmer's job is to make driving safer. On their own, I think most builders of robocars would try to punt the decision elsewhere if possible. The simplest way to punt the decision is to program the car to follow the law, which generally means to stay in its right-of-way. Yes, that means running over 3 toddlers who ran into the road instead of veering onto the sidewalk to run over Hitler. Staying in our lane is what the law says to do, and you are not punished for doing it. The law strongly forbids going onto the sidewalk or another lane to deliberately hit something, no matter who you might be saving.
We might not like the law, but we do have the ability to change it.
Thus I propose the following: Driving regulators should create a special panel which can rule on driving ethics questions. If a robocar developer sees a question which requires some sort of ethical calculation whose answer is unclear, they can submit that question to the panel. The panel can deliberate and provide an answer. If the developer conforms to the ruling, they are absolved of responsibility. They did the right thing.
The panel would of course have people with technical skill on it, to make sure rulings are reasonable and can be implemented. Petitioners could also appeal rulings that would impede development, though they would probably suggest answers and describe their difficulty to the panel in any petition.
The panel would not simply be presented with questions like, "How do you choose between hitting 2 adults or one child?" It might make more sense to propose formulae for evaluating multiple different situations. In the end, it would need to be reduced to something you can do with code.
Very important to the rulings would be an understanding of how certain requirements could slow down robocar development or raise costs. For example, a ruling that car must make a decision based on the number of pedestrians it might hit demands it be able to count pedestrians. Today's robocars may often be unsure whether a blob is 2 or 3 pedestrians, and nobody cares because generally the result is the same -- you don't want to hit any number of pedestrians. Likeways, requirements to know the age of people on the road demands a great deal more of the car's perception system than anybody would normally develop, particularly if you imagine you will ask it to tell a dwarf adult from a child.
Reports from Tesla suggest they are gathering huge amounts of driving data from logs in their cars -- 780 million miles of driving, and as much as 100 million miles in autopilot mode. This contrasts with the 1.6 million miles of test operations at Google. Huge numbers, but what do they mean now, and in the future?