A New Ensemble Strategy Based on Surprisingly Popular Algorithm and Classifier Prediction Confidence

• Published in: Applied sciences Vol. 15; no. 6; p. 3003
• Main Authors: Shi, Haochen, Yuan, Zirui, Zhang, Yankai, Zhang, Haoran, Wang, Xiujuan
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.03.2025
• Subjects: Accuracy; Algorithms; Bayesian Truth Serum; Classification; confidence; Datasets; Decision making; ensemble learning; Hypotheses; Machine learning; Methods; supervised classification; Voting
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app15063003

Traditional ensemble methods rely on majority voting, which may fail to recognize correct answers held by a minority in scenarios requiring specialized knowledge. Therefore, this paper proposes two novel ensemble methods for supervised classification, named Confidence Truth Serum (CTS) and Confidence Truth Serum with Single Regression (CTS-SR). The former is based on the principles of Bayesian Truth Serum (BTS) and introduces classification confidence to calculate the prior and posterior probabilities of events, enabling the recovery of correct judgments provided by a confident minority beyond majority voting. CTS-SR further simplifies the algorithm by constructing a single regression model to reduce computational overhead, making it suitable for large-scale applications. Experiments are conducted on multiple binary classification datasets to evaluate CTS and CTS-SR. Experimental results demonstrate that, compared with existing ensemble methods, both of the proposed methods significantly outperform baseline algorithms in terms of accuracy and F1 scores. Specifically, there is an average improvement of 2–6% in accuracy and an average increase of 2–4% in F1 score. Notably, on the Musk and Hilly datasets, our method achieves a 5% improvement compared to the traditional majority voting approach. Particularly on the Hilly dataset, which generally exhibits the poorest classification performance and poses the greatest prediction challenges, our method demonstrates the best discriminative performance. validating the importance of confidence as a feature in ensemble learning.