Position Bias in Multiple-Choice Questions

• Published in: Journal of marketing research Vol. 21; no. 2; pp. 216 - 220
• Main Author: Blunch, Niels J.
• Format: Journal Article
• Language: English
• Published: Chicago American Marketing Association 01.05.1984; SAGE PUBLICATIONS, INC
• Subjects: Bias; Estimation bias; Estimators; Marketing; Modeling; Parametric models; Proportions; Questionnaires; Research biases; Research Notes and Communications; Sampling bias; Sampling errors; Scaling; Statistical analysis; Statistical models
• ISSN: 0022-2437; 1547-7193
• DOI: 10.1177/002224378402100210

The author argues that rotation of answer alternatives in multiple-choice questions does not eliminate position bias-i.e., bias caused by respondents' tendency to opt for alternatives in certain positions (e.g., the first few mentioned) regardless of their content. Researchers seem to have overlooked this conclusion, possibly because they have approached the problem as a purely manifest phenomenon without reference to a statistical model of the data-generating process. The study of such models and their application to empirical data also reveals that position bias (1) is not necessarily disclosed by rotation, (2) is often much more important than sampling error, and (3) precludes usual estimation of the parameters of the data-generating process. These findings also have implications for nonverbal stimuli and raise questions about efforts to handle position-biased experimental data when the response variable is only nominally scaled.