Receiver operating characteristic analysis and confidence-accuracy characteristic analysis in investigations of system variables and estimator variables that affect eyewitness memory

• Published in: Journal of applied research in memory and cognition Vol. 4; no. 2; pp. 93 - 102
• Main Author: Mickes, Laura
• Format: Journal Article
• Language: English
• Published: Washigton Elsevier Science 01.06.2015; Elsevier Inc; Society for Applied Research in Memory and Cognition
• Subjects: Calibration analysis; Confidence and accuracy; Decision Making; Errors; Eyewitness memory; Human; Legal Testimony; Male; Memory; Recollection and familiarity; ROC analysis; System variables and estimator variables; Witnesses; Confidence and accuracy; Recollection and familiarity; Calibration analysis; System variables and estimator variables; ROC analysis; Eyewitness memory
• ISSN: 2211-3681; 2211-369X
• DOI: 10.1016/j.jarmac.2015.01.003

Two graphical techniques, receiver operating characteristic (ROC) analysis and what might be termed "confidence-accuracy characteristic" (CAC) analysis, are important tools for investigating variables that affect the accuracy of eyewitness identifications (e.g., type of lineup, exposure duration, same-race vs. other-race identifications, etc.). CAC analysis (a close relative of calibration analysis) consists of simply plotting suspect identification accuracy for each level of confidence. Two parties interested in the results of such investigations include (1) legal policymakers (e.g., state legislators and police chiefs) and (2) triers of guilt and innocence (e.g., judges and jurors). Which type of analysis is the most relevant to which party? The answer is largely a matter of whether the variable in question is a system variable or an estimator variable. ROC analysis, which measures discriminability, is critical for understanding system variables that affect eyewitness accuracy (e.g., the best lineup procedures). Thus, policymakers should be particularly attuned to the results of ROC analysis when making decisions about those variables. CAC analysis, which directly measures the confidence-accuracy relationship for suspect IDs, is critical for understanding the effect of estimator variables on eyewitness accuracy (e.g., exposure duration). Thus, triers of guilt and innocence should be particularly attuned to the results of CAC analysis. The utility of both analyses to system and estimator variables is illustrated by examining both types of analyses on previously published experiments and new experiments. Highlights * ROC analysis is critical for understanding system variables. * Policymakers should consider results of ROC analysis. * Confidence-accuracy characteristic analysis is critical for understanding estimator variables. * Arbitrators of truth should consider results of confidence-accuracy characteristic analysis.