GM/Cruise leaks show them way, way behind Waymo. It's time for better metrics from everybody
GM's "Cruise" robocar unit is often cited as #2 behind Waymo. Some recent leaks of their internal metrics for progress paint a dim picture; that they aren't nearly as far along as they hoped, which does not bode well for the planned 2019 launch. In fact, they show as an order of magnitude behind where Google/Waymo was back in 2015.
The numbers that get published due to legal requirements tell us almost nothing. The methodology used by Waymo and cruse, the distance between simulated contacts, is a good start, and more teams should use it. And we should also dream up better metrics. (The simulated contact metric means every time the human safety drivers have to intervene, you build the situation in simulator, and you figure out if the car would have hit anything or not.)
In the new article linked below, I outline what this means for Cruise and what better metrics we might use. Measuring the safety of Robocars is now the biggest problem they face. We aren't that good at it yet, and even once you've done it, you have to prove that you've done it.
Comments
Craig Jenkins
Mon, 2019-06-10 14:50
Permalink
GMs lack of support
GM pulled it's best people off this effort once Softbank made its investment last year. I used to be working on the project and now I'm not along with many others.
Cruise thought they could do a better job running this but apparently not. I work at GM but own Google stock, go figure
FKA
Tue, 2019-06-11 19:57
Permalink
Metrics
These metrics strike me as such a naive way to judge progress in creating a self-driving vehicle. Like judging two five-year-olds on how far along they are in becoming a Jeopardy champion by seeing how many Jeopardy questions they get right.
And each of them is tested on a different set of questions, each from only a single category, with 99% of the questions being the exact same questions repeated over and over again.
Doggydogworld
Tue, 2019-06-11 20:00
Permalink
A set of standard metrics
A set of standard metrics would be awesome, and you'd think Waymo would want to define the standard instead of letting someone else do it. I can see Cruise being so enthusiastic, if they are really as far behind as Amir Efrati says. Then again, he has plenty bad to say about Waymo, too, and they're also way behind schedule.
Add new comment