Localized generalization error and its application to RBFNN training

• Published in: 2005 International Conference on Machine Learning and Cybernetics Vol. 8; pp. 4667 - 4673 Vol. 8
• Main Authors: Ng, Yeung, De-Feng Wang, Tsang, Xi-Zhao Wang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2005
• Subjects: Analytical models; Computer errors; Computer science; EMP radiation effects; Generalization Error; Machine learning; Mathematics; Model Selection; Network Architecture; Neural networks; Pattern classification; Radial Basis Function NN; Support vector machine classification; Support vector machines
• ISBN: 0780390911; 9780780390911
• ISSN: 2160-133X
• DOI: 10.1109/ICMLC.2005.1527762

The generalization error bounds for the entire input space found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. But classifiers such as SVM, RBFNN and MLPNN, are really local learning machines used for many application problems, which consider unseen samples close to the training samples more important. In this paper, we propose a localized generalization error model which bounds above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure (expectation of the squared output perturbations). It is then used to develop a model selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments by using eight real world datasets show that, in comparison with cross-validation, sequential learning, and two other ad-hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.