Closed Track Testing To Assess Prototype Level-3 Autonomous Vehicle Readiness for Public Road Deployment

Abstract

Most of the Automated Driving Systems (ADS) technology development is targeting urban areas; there is still much to learn about how ADS will impact rural transportation. The DriveOhio team deployed level-3 ADS-equipped prototype vehicles in rural Ohio with the goal of discovering technical challenges for ADS deployment in such environments. However, before the deployment on public roads, it was essential to test the ADS-equipped vehicle for their safety limitations. At Transportation Research Center Inc. (TRC Inc.) proving grounds, we tested one such prototype system on a closed test track with soft targets and robotic platforms as surrogates for other road users. This paper presents an approach to safely conduct testing for ADS prototype and assess its readiness for public road deployment. The main goal of this testing was to identify a safe Operational Design Domain (ODD) of this system by gaining better understanding of the limitations of the system. The prototype system uses Apollo-based platform that provided perception, localization, routing, planning, and control modules. In this paper, we present a test matrix specifically designed for testing level-3 ADS-equipped vehicles on a closed test track. The test matrix includes a minimum set of test scenarios and procedures for each functionality of the ADS, for example, localization, routing, planning, object detection and collision avoidance, negotiating traffic at different types of intersections, car-following, etc. Various test parameters are chosen considering the intended public road deployment requirements and limitations. Further, we define safety metrics for various scenarios to objectively assess the limitations of the ADS. We also present sample data analysis results obtained from the testing. The ODD definitions were later used as ADS engagement/disengagement guidelines for the ADS operator to follow during public road deployment. The findings and lessons learned from this study will be useful in safely assessing limitations of future level-3 ADS-equipped prototypes.