Propositionalized attribute taxonomies from data for data-driven construction of concise classifiers

• Published in: Expert systems with applications Vol. 38; no. 10; pp. 12739 - 12746
• Main Authors: Kang, Dae-Ki, Kim, Myoung-Jong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.09.2011
• Subjects: Algorithms; Bayesian analysis; Classifiers; Criteria; Machine learning; Naive Bayes learner; Optimization; Propositionalization; Searching; Taxonomy; Propositionalization; Naive Bayes learner; Taxonomy
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2011.04.062

► We introduce a machine learning algorithm that utilizes taxonomy of propositionalized attributes. ►We extend classical naive Bayes learner to traverse over a propositionalized taxonomy to search for an optimal cut. ► Our experimental results indicate that our algorithm generates compact and accurate naive Bayes classifiers. In this paper, we consider the problem of generating concise but accurate naive Bayes classifiers using taxonomy of propositionalized attributes. For the problem, we introduce propositionalized attribute taxonomy guided naive Bayes Learner (PAT-NBL), a machine learning algorithm that effectively utilizes taxonomy to generate compact classifiers. We extend classical naive Bayes learner to the PAT-NBL algorithm that traverses over a propositionalized taxonomy to search for a locally optimal cut. PAT-NBL uses bottom-up search to find the locally optimal cut on a given taxonomy. For the evaluation of candidate cuts, we apply conditional log-likelihood, conditional minimum description length, and conditional Akaike information criterion. The detected cut enables PAT-NBL to construct an instance space which corresponds to the taxonomy and the data. That is, after PAT-NBL determines a cut according to its information-theoretic criteria, the algorithm generates a concise naive Bayes classifier based on the cut. Our experimental results on UCI Machine Learning benchmark data sets indicate that the proposed algorithm can generate naive Bayes classifiers that are more compact and often comparably accurate to those produced by standard naive Bayes learners.