A Comparative Analysis of Concept Drift Detection Methods with a Systematic and Innovative Approach of Method Selection

Abstract

One of the most significant challenges in data-driven modeling of complex systems is dealing with concept drift, i.e., the unpredictable changes in the underlying data distribution over time. In this work, nine concept drift detection (CDD) methods are evaluated with respect to different types of concept drift, including abrupt, gradual, incremental, and real concept drift, for both supervised and unsupervised application scenarios. For the supervised case, the methods EDDM, FHDDMSadd, MDDM-E, and EFDT are compared against a classification without change detection using Naïve Bayes as the base classifier. In the unsupervised application scenario, CluStream, ClusTree, DenStream, StreamKM++, and D-Stream are evaluated. The experiments are conducted using the Massive Online Analysis (MOA) evaluation platform, and the performance of each method is measured in terms of classification accuracy, memory consumption, and computation time. This empirical research shows that classification accuracy can be improved by 20% by implementing a CDD method, highlighting the importance of CDD in SHM data streams. However, there is no single method that proves to be superior in all scenarios, and the choice depends on the characteristics of the considered data stream and application requirements. Selecting the appropriate CDD method from the approximately 340 different methods found in the literature is not a trivial task and can lead to suboptimal selection. To tackle this issue, an innovative approach is proposed to assist researchers and practitioners find the appropriate CCD method for their application.