Alleviating conditional independence assumption of naive Bayes

• Published in: Statistical papers (Berlin, Germany) Vol. 65; no. 5; pp. 2835 - 2863
• Main Authors: Liu, Xu-Qing, Wang, Xiao-Cai, Tao, Li, An, Feng-Xian, Jiang, Gui-Ren
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.07.2024; Springer Nature B.V
• Subjects: Algorithms; Classification; Economic Theory/Quantitative Economics/Mathematical Methods; Economics; Equivalence; Finance; Independent variables; Insurance; Management; Mathematics and Statistics; Operations Research/Decision Theory; Principal components analysis; Probability Theory and Stochastic Processes; Regular Article; Statistics; Statistics for Business; Variables; Variance analysis; Naive Bayes (NB); Class-weighting supervised principal component analysis (CWSPCA); Conditional Independence Assumption (CIA); Multiple decremental association rearrangement (MDAR)
• ISSN: 0932-5026; 1613-9798
• DOI: 10.1007/s00362-023-01474-5

In this paper, we consider the problem of how to alleviate the conditional independence assumption of naive Bayes. We try to find an equivalent set of variables for the attributes of the class such that these variables are nearly conditionally independent. For the case that all attributes are continuous variables, we put forward the theory of class-weighting supervised principal component analysis (CWSPCA) to improve naive Bayes. For the categorical case, we construct the equivalent variables by rearranging the values of the attributes, and propose the decremental association rearrangement (DAR) algorithm and its multiple version (MDAR). Finally, we make a benchmarking study to show the performance of our methods. The experimental results reveal that naive Bayes can be greatly improved by means of properly transforming the original attributes.