No free lunch but a cheaper supper: A general framework for streaming anomaly detection

• Published in: Expert systems with applications Vol. 155; p. 113453
• Main Authors: Calikus, Ece, Nowaczyk, Sławomir, Sant’Anna, Anita, Dikmen, Onur
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 01.10.2020; Elsevier BV
• Subjects: Algorithms; Anomalies; Anomaly detection; Data mining; Datasets; Online learning; Reservoir sampling; Stream mining; Stream mining; Online learning; Anomaly detection; Reservoir sampling
• ISSN: 0957-4174; 1873-6793; 1873-6793
• DOI: 10.1016/j.eswa.2020.113453

•We postulate the problem of unifying streaming anomaly detection.•We propose a meta-framework for a flexible and adaptable anomaly detection procedure.•Our framework helps to overcome the limitations of one-size-fitsall solutions.•We propose a novel anomaly-aware reservoir sampling scheme.•We conduct an extensive comparison study on 20 detectors using various datasets. In recent years, research interest in detecting anomalies in temporal streaming data has increased significantly. A variety of algorithms are being developed in the data mining community. They can be broadly divided into two categories, namely general-purpose and ad hoc ones. In most cases, general approaches assume a one-size-fits-all solution model, and strive to design a single “optimal” anomaly detector which can detect all anomalies in any domain. To date, there exists no universal method that has been shown to outperform the others across different anomaly types, use cases and datasets. In this paper, we propose SAFARI, a framework created by abstracting and unifying the fundamental tasks within the streaming anomaly detection. SAFARI provides a flexible and extensible anomaly detection procedure to overcome the limitations of one-size-fits-all solutions. Such abstraction helps to facilitate more elaborate algorithm comparisons by allowing us to isolate the effects of shared and unique characteristics of diverse algorithms on the performance. Using the framework, we have identified a research gap that motivated us to propose a novel learning strategy. We implemented twenty different anomaly detectors and conducted an extensive evaluation study, comparing their performances using real-world benchmark datasets with different properties. The results indicate that there is no single superior detector which works perfectly for every case, proving our hypothesis that “there is no free lunch” in the streaming anomaly detection world. Finally, we discuss the benefits and drawbacks of each method in-depth, drawing a set of conclusions and guidelines to guide future users of SAFARI.