Reducing Attenuation Bias in Regression Analyses Involving Rating Scale Data via Psychometric Modeling

• Published in: Psychometrika Vol. 89; no. 1; pp. 42 - 63
• Main Authors: Glas, Cees A. W., Jorgensen, Terrence D., Hove, Debby ten
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.03.2024; Springer Nature B.V
• Subjects: Assessment; Behavioral Science and Psychology; Bias; Educational Measurement - methods; Error of Measurement; Generalizability Theory; Humanities; Humans; Integrated approach; Item response theory; Law; Linear Models; Models, Statistical; Observational studies; Psychology; Psychometrics; Psychometrics - methods; Quantitative psychology; Rating Scales; Regression (Statistics); Regression Analysis; Resistance (Psychology); Statistical Theory and Methods; Statistics for Social Sciences; Teachers; Testing and Evaluation; Theory & Methods; Variables; generalizability coefficients; generalized partial credit model; item response theory; disattenuation; hierarchical linear models; generalizability theory
• ISSN: 0033-3123; 1860-0980; 1860-0980
• DOI: 10.1007/s11336-024-09967-4

Many studies in fields such as psychology and educational sciences obtain information about attributes of subjects through observational studies, in which raters score subjects using multiple-item rating scales. Error variance due to measurement effects, such as items and raters, attenuate the regression coefficients and lower the power of (hierarchical) linear models. A modeling procedure is discussed to reduce the attenuation. The procedure consists of (1) an item response theory (IRT) model to map the discrete item responses to a continuous latent scale and (2) a generalizability theory (GT) model to separate the variance in the latent measurement into variance components of interest and nuisance variance components. It will be shown how measurements obtained from this mixture of IRT and GT models can be embedded in (hierarchical) linear models, both as predictor or criterion variables, such that error variance due to nuisance effects are partialled out. Using examples from the field of educational measurement, it is shown how general-purpose software can be used to implement the modeling procedure.