Forest PA: Constructing a decision forest by penalizing attributes used in previous trees

• Published in: Expert systems with applications Vol. 89; pp. 389 - 403
• Main Authors: Adnan, Md Nasim, Islam, Md Zahidul
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 15.12.2017; Elsevier BV
• Subjects: Algorithms; Classification; Decision Forest; Decision Tree; Decision trees; Ensemble Accuracy; Experiments; Expert systems; Intelligent systems; Machine learning; Random Forest; System effectiveness; Ensemble Accuracy; Decision Tree; Classification; Decision Forest; Random Forest
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2017.08.002

•Forest PA assigns weights only on the attributes appearing in the latest tree.•Weights are obtained randomly from dynamically determined weight ranges.•Weights are incremented if the attributes do not appear in the subsequent tree(s).•Forest PA is applied on 20 well known data sets.•The experimental results indicate the effectiveness of Forest PA. In this paper, we propose a new decision forest algorithm that builds a set of highly accurate decision trees by exploiting the strength of all non-class attributes available in a data set, unlike some existing algorithms that use a subset of the non-class attributes. At the same time to promote strong diversity, the proposed algorithm imposes penalties (disadvantageous weights) to those attributes that participated in the latest tree in order to generate the subsequent trees. Besides, some other weight-related concerns are taken into account so that the trees generated by the proposed algorithm remain individually accurate and retain strong diversity. In order to show the worthiness of the proposed algorithm, we carry out experiments on 20 well known data sets that are publicly available from the UCI Machine Learning Repository. The experimental results indicate that the proposed algorithm is effective in generating highly accurate and more balanced decision forests compared to other prominent decision forest algorithms. Accordingly, the proposed algorithm is expected to be very effective in the domain of expert and intelligent systems.