Area under Precision-Recall Curves for Weighted and Unweighted Data

• Published in: PloS one Vol. 9; no. 3; p. e92209
• Main Authors: Keilwagen, Jens, Grosse, Ivo, Grau, Jan
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 20.03.2014; Public Library of Science (PLoS)
• Subjects: Algorithms; Analysis; Area Under Curve; Artificial intelligence; Bioinformatics; Biology and Life Sciences; Classification; Classifiers; Computer and Information Sciences; Computer science; Data points; Databases as Topic; Engineering and Technology; Gene expression; Interpolation; Knowledge discovery; Labels; Linguistics; Natural language processing; Normal distribution; Physical Sciences; Recall; ROC Curve; Statistics as Topic; Germany
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0092209

Precision-recall curves are highly informative about the performance of binary classifiers, and the area under these curves is a popular scalar performance measure for comparing different classifiers. However, for many applications class labels are not provided with absolute certainty, but with some degree of confidence, often reflected by weights or soft labels assigned to data points. Computing the area under the precision-recall curve requires interpolating between adjacent supporting points, but previous interpolation schemes are not directly applicable to weighted data. Hence, even in cases where weights were available, they had to be neglected for assessing classifiers using precision-recall curves. Here, we propose an interpolation for precision-recall curves that can also be used for weighted data, and we derive conditions for classification scores yielding the maximum and minimum area under the precision-recall curve. We investigate accordances and differences of the proposed interpolation and previous ones, and we demonstrate that taking into account existing weights of test data is important for the comparison of classifiers.