Self-driving cars operated by Waymo are logging significantly more miles on California public roads than their competitors, and the Google project’s cars are traveling significantly more miles without experiencing disengagements of their autonomous technology, according to reports released by the California Department of Motor Vehicles. The reports, released on February 1, show that Waymo’s vehicles accumulated 635,868 miles of driving on the state’s public roads in 2016, more than the 10 other companies reporting figures combined.
More important, Waymo’s cars traveled farther before experiencing disengagements of their technology for software malfunctions or because the human drivers intervened. Waymo, the company spun off from Google last year, experienced an average of one disengagement for every 5127.9 miles traveled, a rate far better than its competitors reported.
“By any measure, we’re happy with our progress, but anxious to work even harder,” wrote Dmitri Dolgov, head of self-driving technology for Waymo. Reports Measure Progress—and Technical Struggles The public gets an annual snapshot at these autonomous driving reports once a year, thanks to California regulations that mandate disclosure of the number of miles traveled and the number of disengagements experienced. The reports are helpful in gauging general competence and progress, and in some cases, companies offered details on particular technical struggles. But overall, the reports are just a snapshot, not a comprehensive view of a company’s entire autonomous program. While it’s easy to tell that Waymo had an average of 0.2 disengagements per 1000 miles traveled while Mercedes-Benz had an average of 498.9 per 1000 miles traveled, for example, it’s harder to discern what specific aspects of the self-driving technology were being tested, whether the tests occurred in complex environments or on easier-to-navigate highways.
Further, the reports don’t track how the companies are using simulated driving miles to teach their self-driving systems, nor do they encompass how they’re testing in jurisdictions that don’t require reporting. There’s also no specific standard on how or when test drivers should decide to intervene and disengage the technology. For one example, Tesla’s report would seem to pale in comparison to Waymo’s exploits. Tesla reported 182 disengagements during 550 total miles of testing, and almost all of those took place in wet-road environments. It’d be tempting to conclude its vehicles have trouble operating in the rain. But it’s important to understand the broader context: Tesla accumulated 96 percent of those miles during a three-day span in October when it was rainy, and it tested primarily as a means to film footage to be used in advertisements for its Autopilot feature. In some ways, the reports say almost as much about how the companies value testing on public roads in California as they do about the mettle of the technology.
How Much Testing? Caveats aside, the reports are useful for measuring progress. Waymo’s rate of 0.2 disengagements per 1000 miles was a fourfold improvement over the 0.8 rate that Google reported for 2015. That’s a significant technical improvement for a company that, on the business end, said last month it had devised a way to reduce the cost of crucial sensors such as radar and lidar by more than 90 percent
With the 11 companies reporting 2648 disengagements in 2016, not everyone was ready to celebrate any improvements in the overall state of autonomous technology. John Simpson, privacy director of Consumer Watchdog, a nonprofit that advocates for stringent oversight of automated vehicles, said the statistics showed that self-driving cars aren’t ready for the road. “While there has been an improvement, the reports show the robots simply aren’t ready to be released to roam our roads without human drivers,” he said, later praising the California DMV for mandating that such reports be made public. “It’s the only way the public can find out what’s happening when companies use public roads as their laboratories.” Twenty-one companies currently hold permits to test in California. By statute, companies need to report their numbers each November 30.
The reports released Wednesday unearthed insights that go beyond disengagement rates. Ford disclosed that it kept its two vehicles in the state far from the Silicon Valley epicenter of autonomous technology, conducting its testing along Interstate 10 between Los Angeles and the Arizona border. Ford experienced three disengagements during 590 total miles of testing. Two came when their safety drivers intervened, aborting system-initiated lane changes because another vehicle was coming up from behind in the passing lane at high speeds. BMW reported that it only tested in March and April, and its lone disengagement in 638 miles of driving came in the northbound lanes of Highway 101 because the lane markings were not clear enough for its system to detect the lane, a potential infrastructure problem that has some industry leaders concerned. In testing its two autonomous Audi SQ5 vehicles, automotive supplier Delphi Automotive provided details on its 178 disengagements, noting that 19 occurred in construction zones and 28 came when its system had trouble detecting traffic-light signals due to “poor sun conditions.”
Mercedes-Benz reported that it conducted all of its tests on urban streets and logged zero miles of highway driving. Volkswagen and Honda, which both logged miles in 2015, both reported they had logged zero miles in their 2016 reports.
Cruise Automation, a company owned by General Motors and based in San Francisco, revealed that it is testing 20 autonomous 2017 Chevrolet Bolts and five autonomous Nissan Leafs, two from the 2012 model year and three from the 2016 model year. Perhaps to help keep them straight, the company has given its 25 autonomous vehicles individual names that include Platypus, Wombat, Pronghorn, and Beluga.
