NHTSA Regulations part 3: Data Sharing, Privacy, Safety, Security and HMI
Submitted by brad on Fri, 2016-09-30 12:49After my initial reactions and Overall Analysis here is a point by point consideration of the elements from NHTSA's 15 point certification list for robocars. See also the second half and the whole series
Let's dig in:
Data Recording and Sharing
These regulations require a plan about how the vehicle keep logs around any incident (while following privacy rules.) This is something everybody already does -- in fact they keep logs of everything for now -- since they want to debug any problems they encounter. NHTSA wants the logs to be available to NHTSA for crash investigation.
NHTSA also wants recordings of positive events (the system avoided a problem.)
Most interesting is a requirement for a data sharing plan. NHTSA wants companies to share their logs with their competitors in the event of incidents and important non-incidents, like near misses or detection of difficult objects.
This is perhaps the most interesting element of the plan, but it has seen some resistance from vendors. And it is indeed something that might not happen at scale without regulation. Many teams will consider their set of test data to be part of their crown jewels. Such test data is only gathered by spending many millions of dollars to send drivers out on the roads, or by convincing customers or others to voluntarily supervise while their cars gather test data, as Tesla has done. A large part of the head-start that leaders have in this field is the amount of different road situations they have been able to expose their vehicles to.
Recordings of mundane driving activity are less exciting and will be easier to gather. Real world incidents are rare and gold for testing. The sharing is not as golden, because each vehicle will have different sensors, located in different places, so it will not be easy to adapt logs from one vehicle directly to another. While a vehicle system can play its own raw logs back directly to see how it performs in the same situation, other vehicles won't readily do that.
Instead this offers the ability to build something that all vendors want and need, and the world needs, which is a high quality simulator where cars can be tested against real world recordings and entirely synthetic events. The data sharing requirement will allow the input of all these situations into the simulator, so every car can test how it would have performed. This simulation will mostly be at the "post perception level" where the car has (roughly) identified all the things on the road and is figuring out what to do with them, but some simulation could be done at lower levels.
These data logs and simulator scenarios will create what is known as a regression test suite. You test your car in all the situations, and every time you modify the software, you test that your modifications didn't break something that used to work. It's an essential tool.
In the history of software, there have been shared public test suites (often sourced from academia) and private ones that are closely guarded. For some time, I have proposed that it might be very useful if there were a a public and open source simulator environment which all teams could contribute scenarios to, but I always expected most contributions would come from academics and the open source community. Without this rule, the teams with the most test miles under their belts might be less willing to contribute.
Such a simulator would help all teams and level the playing field. It would allow small innovators to even build and test prototype ideas entirely in simulator, with very low cost and zero risk compared to building it in physical hardware.
This is a great example of where NHTSA could use its money rather than its regulatory power to improve safety, by funding the development of such test tools. In fact, if done open source, the agencies and academic institutions of the world could fund a global one. (This would face opposition from companies hoping to sell test tools, but there will still be openings for proprietary test tools.)
Privacy
This section demands a privacy policy. I'm not against that, though of course the history of privacy policies is not a great one. They mostly involve people clicking "I agree" to things they don't read. More important is the requirement that vendors be thinking about privacy.
The requirement for user choice is an interesting one, and it conflicts with the logging requirements. People are wary of technology that will betray them in court. Of course, as long as the car is not a hybrid car that mixes human driving with self-driving, and the passenger is not liable in an accident, there should be minimal risk to the passenger from accidents being recorded.
The rules require that personal information be scrubbed from any published data. This is a good idea but history shows it is remarkably hard to do properly.