A Kolmogorov-Smirnov Test to Detect Changes in Stationarity in Big Data
Informazioni aggiuntive
Autori
Zhao D.,
Bu L.,
Alippi C.,
Wei Q.
Tipo
Articolo pubblicato in rivista scientifica
Anno
2017
Lingua
Inglese
Abstract
The paper proposes an effective change detection test for online monitoring data streams by inspecting the least squares density difference (LSDD) features extracted from two non-overlapped windows. The first window contains samples associated with the pre-change probability distribution function (pdf) and the second one with the post-change one (that differs from the former if a change in stationarity occurs). This method can detect changes by also controlling the false positive rate. However, since the window sizes is fixed after the test has been configured (it has to be small to reduce the execution time), the method may fail to detect changes with small magnitude which need more samples to reach the requested level of confidence. In this paper, we extend our work to the Big Data framework by applying the Kolmogorov-Smirnov test (KS test) to infer changes. Experiments show that the proposed method is effective in detecting changes.
Rivista
IFAC-PapersOnLine
Volume
50
Pagina inizio
14260
Pagina fine
14265
ISSN
2405-8963
Parole chiave
change detection test, KS test, LSDD
