Testing machine learning based systems
a systematic mapping
Informazioni aggiuntive
Autori
Riccio V.,
Jahangirova G.,
Stocco A.,
Humbatova N.,
Weiss M.,
Tonella P.
Tipo
Articolo pubblicato in rivista scientifica
Anno
2020
Lingua
Inglese
Sommario
Context: A Machine Learning based System (MLS) is a software system including one or more components that learn how to perform a task from a given data set. The increasing adoption of MLSs in safety critical domains such as autonomous driving, healthcare, and finance has fostered much attention towards the quality assurance of such systems. Despite the advances in software testing, MLSs bring novel and unprecedented challenges, since their behaviour is defined jointly by the code that implements them and the data used for training them. Objective: To identify the existing solutions for functional testing of MLSs, and classify them from three different perspectives: (1) the context of the problem they address, (2) their features, and (3) their empirical evaluation. To report demographic information about the ongoing research. To identify open challenges for future research. Method: We conducted a systematic mapping study about testing techniques for MLSs driven by 33 research questions. We followed existing guidelines when defining our research protocol so as to increase the repeatability and reliability of our results. Results: We identified 70 relevant primary studies, mostly published in the last years. We identified 11 problems addressed in the literature. We investigated multiple aspects of the testing approaches, such as the used/proposed adequacy criteria, the algorithms for test input generation, and the test oracles. Conclusions: The most active research areas in MLS testing address automated scenario/ input generation and test oracle creation. MLS testing is a rapidly growing and developing research area, with many open challenges, such as the generation of realistic inputs and the definition of reliable evaluation metrics and benchmarks.
Parole chiave
Systematic mapping, Systematic review, Software testing, Machine learning
Periodico
Empirical Software Engineering
Volume
25
Numero ( Mese )
6
Pagine (o numero dell’articolo)
5193–5254
Diffusione
Licenza
CC BY
Visibilità
Pubblico
Status open access
Hybrid
