Data Mining on Imbalanced Data Sets

• Published in: 2008 International Conference on Advanced Computer Theory and Engineering pp. 1020 - 1024
• Main Authors: Qiong Gu, Zhihua Cai, Li Zhu, Bo Huang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2008
• Subjects: Algorithm design and analysis; Cancer; Classification algorithms; Cost-Sensitive Learning; Costs; Data engineering; Data mining; Diseases; Imbalanced data sets; Machine learning; Machine learning algorithms; over-sampling; Radar detection; under-sampling
• ISBN: 9780769534893; 0769534899
• ISSN: 2154-7491
• DOI: 10.1109/ICACTE.2008.26

The majority of machine learning algorithms previously designed usually assume that their training sets are well-balanced, and implicitly assume that all misclassification errors cost equally. But data in real-world is usually imbalanced. The class imbalance problem is pervasive and ubiquitous, causing trouble to a large segment of the data mining community. The tradition machine learning algorithms have bad performance when they learn from imbalanced data sets. Thus, machine learning on imbalanced data sets becomes an urgent problem. The importance of imbalanced data sets and their broad application domains in data mining are introduced, and then methods to deal with the class imbalance problem are discussed and their effectiveness are compared. Last but not least, the existing evaluation measures of class imbalance problem are systematically analyzed.