RealDriftGenerator: A Novel Approach to Generate Concept Drift in Real World Scenario

Borong Lin, Chao Huang, Xiaohui Zhu, Nanlin Jin*

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

Abstract

Concept drift refers to the probability distribution of data generation changes over time in a data stream environment. In recent years, there has been an increasing interest in drift detection models. However, due to the lack of labeled concept drift datasets, most researchers tend to using synthetic drift data generators for model training. These generators only have relatively simple feature distributions, which fail to capture the complexity found in real-world scenarios. This paper introduces a real scenario concept drift label generator (RealDriftGenerator). This generator aims to preserve the complexity and temporal correlation of real-world scenario while generating concept drifts with user defined drift positions and drift widths. The validation result show that the temporal correlation coefficients of RealDriftGenerator is significantly higher than benchmark drift generators. Additionally, the ability of RealDriftGenerator to capture the complexity in real-world scenarios is 20% higher than benchmark drift generators(measured by model performance). The source code of RealDriftGenerator has been published on https://github.com/sniperrifle71/realDriftGenerator.

Original languageEnglish
Title of host publication2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1124-1129
Number of pages6
ISBN (Electronic)9781665410205
DOIs
Publication statusPublished - 2024
Event2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024 - Kuching, Malaysia
Duration: 6 Oct 202410 Oct 2024

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X

Conference

Conference2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024
Country/TerritoryMalaysia
CityKuching
Period6/10/2410/10/24

Cite this