Exploration of Item Selection in Dual-Purpose Cognitive Diagnostic Computerized Adaptive Testing Based on the RRUM

• Published in: Applied psychological measurement Vol. 40; no. 8; pp. 625 - 640
• Main Authors: Dai, Buyun, Zhang, Minqiang, Li, Guangming
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.11.2016
• Subjects: item selection method; “dapperness with information” index; knowledge state; cognitive diagnostic computerized adaptive testing
• ISSN: 0146-6216; 1552-3497; 1552-3497
• DOI: 10.1177/0146621616666008

Cognitive diagnostic computerized adaptive testing (CD-CAT) can be divided into two broad categories: (a) single-purpose tests, which are based on the subject’s knowledge state (KS) alone, and (b) dual-purpose tests, which are based on both the subject’s KS and traditional ability level ( θ ). This article seeks to identify the most efficient item selection method for the latter type of CD-CAT corresponding to various conditions and various evaluation criteria, respectively, based on the reduced reparameterized unified model (RRUM) and the two-parameter logistic model of item response theory (IRT-2PLM). The Shannon entropy (SHE) and Fisher information methods were combined to produce a new synthetic item selection index, that is, the “dapperness with information (DWI)” index, which concurrently considers both KS and θ within one step. The new method was compared with four other methods. The results showed that, in most conditions, the new method exhibited the best performance in terms of KS estimation and the second-best performance in terms of θ estimation. Item utilization uniformity and computing time are also considered for all the competing methods.