Benchmarks for Retrospective Automated Driving System Crash Rate Analysis Using Police-Reported Crash Data
Study uses police-reported data to benchmark ADS crash rates, addressing data selection and reporting bias issues.
Key Findings
Methodology
The study uses police-reported crash data to generate benchmark crash rates, addressing challenges like data selection, reporting bias, and identifying driver populations. It employs Blincoe and Blanco adjustments to enhance comparability with ADS data.
Key Results
- San Francisco's adjusted crash rate is 5.82 IPMM, 330% higher than the national average.
- Maricopa County's crash rate is nearly identical to the national average at 1.80 IPMM.
- Los Angeles County's crash rate is 28% higher than the national average at 2.26 IPMM.
Significance
This study provides accurate benchmark crash rates, aiding researchers and regulators in evaluating ADS safety, filling a gap in existing literature.
Technical Contribution
The study introduces a novel benchmarking method combining multiple data sources and adjustment techniques, enhancing comparability with ADS data.
Novelty
This is the first systematic use of police-reported data to generate ADS crash rate benchmarks, addressing data selection and reporting bias issues.
Limitations
- Data selection and reporting bias remain challenges in generating accurate benchmarks.
- Property damage-only crashes are underreported in some regions.
Future Work
Future research could further optimize data correction methods and explore the potential of other data sources.
AI Executive Summary
As fully automated driving systems expand in the US, the need to assess their safety impact grows. This study uses police-reported crash data to generate human crash rate benchmarks across multiple regions for comparison with ADS. By employing Blincoe and Blanco adjustments, the study corrects reporting biases, enhancing data comparability. Results show significant regional differences, with San Francisco having the highest crash rate. The study provides critical references for ADS safety evaluation but highlights ongoing challenges with data selection and reporting biases. Future work could further optimize correction methods and explore additional data sources.
Deep Analysis
Background
The rapid development of automated driving technology makes assessing its safety crucial. Traditional safety assessments often rely on human driving data, which can suffer from reporting biases and selection issues.
Core Problem
Generating crash rate benchmarks comparable to ADS faces challenges like data selection, reporting bias, and identifying driver populations. These issues affect the accuracy and usability of benchmarks.
Innovation
The study innovatively uses police-reported data and employs Blincoe and Blanco adjustments to address reporting biases, enhancing data comparability.
Methodology
- �� Collect police-reported crash data
- �� Apply Blincoe and Blanco adjustments to correct data
- �� Generate crash rate benchmarks for different regions
- �� Analyze the impact of geographic region, road type, and vehicle type
Experiments
The experiment uses 2022 police-reported data covering Maricopa County, San Francisco, and Los Angeles. By applying adjustment methods, it generates benchmarks for various crash severity levels.
Results
The study finds that San Francisco's crash rate significantly exceeds the national average, while Maricopa County's is nearly identical. Los Angeles has a slightly higher rate than the national average.
Applications
The study's results can be used to evaluate ADS safety, providing a basis for policy-making and technological improvements.
Limitations & Outlook
Data selection and reporting biases remain challenges, and future work should further optimize adjustment methods.
Plain Language Accessible to non-experts
Imagine a busy kitchen where each chef represents a car and the kitchen is the road. The automated driving system is like a smart assistant helping chefs avoid collisions and accidents. The study is like observing these chefs to see if the smart assistant really reduces accidents. By analyzing the chefs' work reports, we can better understand the assistant's effectiveness.
ELI14 Explained like you're 14
Imagine you're playing a racing game with an auto-drive mode. The study is like analyzing the auto-drive mode's performance to see if it's safer than driving manually. By looking at the game's crash reports, researchers can better understand the pros and cons of auto-drive mode.
Glossary
Automated Driving System (ADS)
A system capable of fully controlling a vehicle under specific conditions without human intervention.
Used to evaluate the safety of automated driving systems.
Blincoe Adjustment
A method to correct crash report biases based on telephone surveys and insurance records.
Used to correct biases in police-reported data.
Blanco Adjustment
A correction method based on naturalistic driving studies to improve crash report accuracy.
Used to generate more accurate crash rate benchmarks.
Crash Rate Benchmark
A standardized crash rate used to compare the safety of different driving systems.
The core goal of the study is to generate accurate benchmarks.
Police-Reported Data
Traffic crash data recorded by police, including crash details.
The primary data source for generating benchmarks.
Open Questions Unanswered questions from this research
- 1 How can data selection and reporting biases be further reduced in benchmarks?
- 2 Are there other data sources that can improve benchmark accuracy?
Applications
Immediate Applications
ADS Safety Evaluation
The study's results can be used to evaluate ADS safety, aiding regulators in policy-making.
Long-term Vision
Intelligent Traffic Management
Improving crash rate benchmark accuracy can drive the development of intelligent traffic systems.
Abstract
With fully automated driving systems (ADS; SAE level 4) ride-hailing services expanding in the US, we are now approaching an inflection point, where the process of retrospectively evaluating ADS safety impact can start to yield statistically credible conclusions. An ADS safety impact measurement requires a comparison to a "benchmark" crash rate. This study aims to address, update, and extend the existing literature by leveraging police-reported crashes to generate human crash rates for multiple geographic areas with current ADS deployments. All of the data leveraged is publicly accessible, and the benchmark determination methodology is intended to be repeatable and transparent. Generating a benchmark that is comparable to ADS crash data is associated with certain challenges, including data selection, handling underreporting and reporting thresholds, identifying the population of drivers and vehicles to compare against, choosing an appropriate severity level to assess, and matching crash and mileage exposure data. Consequently, we identify essential steps when generating benchmarks, and present our analyses amongst a backdrop of existing ADS benchmark literature. One analysis presented is the usage of established underreporting correction methodology to publicly available human driver police-reported data to improve comparability to publicly available ADS crash data. We also identify important dependencies in controlling for geographic region, road type, and vehicle type, and show how failing to control for these features can bias results. This body of work aims to contribute to the ability of the community - researchers, regulators, industry, and experts - to reach consensus on how to estimate accurate benchmarks.