When uncertainty leads to unsafety: Empirical insights into the role of uncertainty in unmanned aerial vehicle safety
Additional information
Authors
MazraehKhatiri S.,
Mohammadi Amin F.,
Panichella S.,
Tonella P.
Type
Journal Article
Year
2025
Language
English
Abstract
Despite the recent developments in obstacle avoidance and other safety features, autonomous Unmanned Aerial Vehicles (UAVs) continue to face safety challenges. No previous work investigated the relationship between the behavioral uncertainty of a UAV, characterized in this work by inconsistent or erratic control signal patterns, and the unsafety of its flight. By quantifying uncertainty, it is possible to develop a predictor for unsafety, which acts as a flight supervisor. We conducted a large-scale empirical investigation of safety violations using PX4-Autopilot, an open-source UAV software platform. Our dataset of over 5,000 simulated flights, created to challenge obstacle avoidance, allowed us to explore the relation between uncertain UAV decisions and safety violations: up to 89% of unsafe UAV states exhibit significant decision uncertainty, and up to 74% of uncertain decisions lead to unsafe states. Based on these findings, we implemented Superialist (Supervising Autonomous Aerial Vehicles), a runtime uncertainty detector based on autoencoders, the state-of-the-art technology for anomaly detection. Superialist achieved high performance in detecting uncertain behaviors with up to 96% precision and 93% recall. Despite the observed performance degradation when using the same approach for predicting unsafety (up to 74% precision and 87% recall), Superialist enabled early prediction of unsafe states up to 50 seconds in advance.
Keywords
Autonomous systems, UAV safety & uncertainty, Real-time monitoring, Simulation
Journal
Empirical Software Engineering
Volume
30
Number ( Month )
6
Pages (or article number)
166
ISSN
1382-3256, 1573-7616
Diffusion
License
CC BY
Visibility
Public
Status open access
Hybrid
