Two is better than one : digital siblings to improve autonomous driving testing
Additional information
Authors
Biagiola M.,
Stocco A.,
Riccio V.,
Tonella P.
Type
Journal Article
Year
2024
Language
English
Abstract
Simulation-based testing represents an important step to ensure the reliability of autonomous driving software. In practice, when companies rely on third-party general-purpose simulators, either for in-house or outsourced testing, the generalizability of testing results to real autonomous vehicles is at stake. In this paper, we enhance simulation-based testing by introducing the notion of digital siblings—a multi-simulator approach that tests a given autonomous vehicle on multiple general-purpose simulators built with different technologies, that operate collectively as an ensemble in the testing process. We exemplify our approach on a case study focused on testing the lane-keeping component of an autonomous vehicle. We use two open-source simulators as digital siblings, and we empirically compare such a multi-simulator approach against a digital twin of a physical scaled autonomous vehicle on a large set of test cases. Our approach requires generating and running test cases for each individual simulator, in the form of sequences of road points. Then, test cases are migrated between simulators, using feature maps to characterize the exercised driving conditions. Finally, the joint predicted failure probability is computed, and a failure is reported only in cases of agreement among the siblings. Our empirical evaluation shows that the ensemble failure predictor by the digital siblings is superior to each individual simulator at predicting the failures of the digital twin. We discuss the findings of our case study and detail how our approach can help researchers interested in automated testing of autonomous driving software.
Keywords
AI testing, Self-driving cars, Simulation-based testing, Digital twins, Deep neural networks, Autonomous vehicles
Journal
Empirical Software Engineering
Volume
29
Number ( Month )
4
Pages (or article number)
72
ISSN
1382-3256, 1573-7616
Diffusion
License
CC BY
Visibility
Public
Status open access
Hybrid
