Cost-Sensitive Boosting

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 33; no. 2; pp. 294 - 309
• Main Authors: Masnadi-Shirazi, H, Vasconcelos, N
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.02.2011; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: AdaBoost; Algorithm design and analysis; Algorithms; Applied sciences; Artificial intelligence; asymmetric boosting; Boosting; Boundaries; Calibration; Computer science; control theory; systems; Computer vision; Cost engineering; Cost function; cost-sensitive learning; Data processing. List processing. Character string processing; Derivation; Exact sciences and technology; Face detection; Laboratories; Learning; Medical diagnosis; Memory organisation. Data processing; Object detection; Operations research; Optimization; Pattern analysis; Pattern recognition. Digital image processing. Computational geometry; Proposals; Software; Standard data; Studies; Computer vision; Data analysis; Motor car; Probabilistic approach; Face recognition; cost-sensitive learning; Minimization; Boosting; Calibration; asymmetric boosting; Aggregate model; Decision rule; Gradient descent; Optimal decision; AdaBoost; Asymmetry; Supervised learning; Facies; Vector support machine
• ISSN: 0162-8828; 1939-3539; 1939-3539
• DOI: 10.1109/TPAMI.2010.71

A novel framework is proposed for the design of cost-sensitive boosting algorithms. The framework is based on the identification of two necessary conditions for optimal cost-sensitive learning that 1) expected losses must be minimized by optimal cost-sensitive decision rules and 2) empirical loss minimization must emphasize the neighborhood of the target cost-sensitive boundary. It is shown that these conditions enable the derivation of cost-sensitive losses that can be minimized by gradient descent, in the functional space of convex combinations of weak learners, to produce novel boosting algorithms. The proposed framework is applied to the derivation of cost-sensitive extensions of AdaBoost, RealBoost, and LogitBoost. Experimental evidence, with a synthetic problem, standard data sets, and the computer vision problems of face and car detection, is presented in support of the cost-sensitive optimality of the new algorithms. Their performance is also compared to those of various previous cost-sensitive boosting proposals, as well as the popular combination of large-margin classifiers and probability calibration. Cost-sensitive boosting is shown to consistently outperform all other methods.