A Robust Method for Detecting Item Misfit in Large-Scale Assessments

• Published in: Educational and psychological measurement Vol. 83; no. 4; pp. 740 - 765
• Main Authors: von Davier, Matthias, Bezirhan, Ummugul
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.08.2023; SAGE PUBLICATIONS, INC
• Subjects: Classical test theory; Educational Assessment; Educational evaluation; Educational tests & measurements; Goodness of Fit; Item Analysis; Item Response Theory; Robustness (Statistics); Statistical Distributions; Statistics; Test Items; Test Theory; Testing Problems; DIF; item fit; mixture distribution model; outlier detection; Tukey’s contaminated distributions; robust statistics
• ISSN: 0013-1644; 1552-3888; 1552-3888
• DOI: 10.1177/00131644221105819

Viable methods for the identification of item misfit or Differential Item Functioning (DIF) are central to scale construction and sound measurement. Many approaches rely on the derivation of a limiting distribution under the assumption that a certain model fits the data perfectly. Typical DIF assumptions such as the monotonicity and population independence of item functions are present even in classical test theory but are more explicitly stated when using item response theory or other latent variable models for the assessment of item fit. The work presented here provides a robust approach for DIF detection that does not assume perfect model data fit, but rather uses Tukey’s concept of contaminated distributions. The approach uses robust outlier detection to flag items for which adequate model data fit cannot be established.