The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

• Published in: BMC genomics Vol. 21; no. 1; pp. 6 - 13
• Main Authors: Chicco, Davide, Jurman, Giuseppe
• Format: Journal Article
• Language: English
• Published: London BioMed Central 02.01.2020; BioMed Central Ltd; Springer Nature B.V; BMC
• Subjects: Accuracy; Algorithms; Analysis; Animal Genetics and Genomics; Binary classification; Biomedical and Life Sciences; Biostatistics; Business metrics; Classification; Comparative and evolutionary genomics; Computational Biology - statistics & numerical data; Confusion; Confusion matrices; Correlation coefficient; Correlation coefficients; Correlation of Data; Data Interpretation, Statistical; Datasets; Evaluation; F1 score; Gene expression; Genomics; Learning algorithms; Life Sciences; Machine learning; Machine Learning - statistics & numerical data; Mathematical analysis; Matrix methods; Matthews correlation coefficient; Microarrays; Microbial Genetics and Genomics; Plant Genetics and Genomics; Proteomics; Research Article; Researchers; Statistics; score; Biostatistics; Accuracy; Dataset imbalance; Confusion matrices; F; Genomics; Matthews correlation coefficient; Machine learning; Binary classification; F1 score
• ISSN: 1471-2164; 1471-2164
• DOI: 10.1186/s12864-019-6413-7

Background To evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F 1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets. Results The Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset. Conclusions In this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F 1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F 1 score in evaluating binary classification tasks by all scientific communities.