An efficient instance selection algorithm for k nearest neighbor regression

• Published in: Neurocomputing (Amsterdam) Vol. 251; pp. 26 - 34
• Main Authors: Song, Yunsheng, Liang, Jiye, Lu, Jing, Zhao, Xingwang
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 16.08.2017
• Subjects: Data reduction; Instance selection; Nearest neighbor; Regression; Significant difference; Nearest neighbor; Regression; Instance selection; Data reduction; Significant difference
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2017.04.018

The k-Nearest Neighbor algorithm(kNN) is an algorithm that is very simple to understand for classification or regression. It is also a lazy algorithm that does not use the training data points to do any generalization, in other words, it keeps all the training data during the testing phase. Thus, the population size becomes a major concern for kNN, since large population size may result in slow execution speed and large memory requirements. To solve this problem, many efforts have been devoted, but mainly focused on kNN classification. And now we propose an algorithm to decrease the size of the training set for kNN regression(DISKR). In this algorithm, we firstly remove the outlier instances that impact the performance of regressor, and then sorts the left instances by the difference on output among instances and their nearest neighbors. Finally, the left instances with little contribution measured by the training error are successively deleted following the rule. The proposed algorithm is compared with five state-of-the-art algorithms on 19 datasets, and experiment results show it could get the similar prediction ability but have the lowest instance storage ratio.