Research on the selection of cognitive diagnosis model from the perspective of experts

• Published in: Current psychology (New Brunswick, N.J.) Vol. 43; no. 15; pp. 13802 - 13810
• Main Authors: Wu, Xiaopeng, Sun, Siyu, Xu, Tianshu, Wang, Axi
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.04.2024; Springer Nature B.V
• Subjects: Behavioral Science and Psychology; Cognitive models; Mathematics education; Psychology; Social Sciences; Students; Teaching; Education in mathematics; Data fitness; Fractional subtraction; Model selection; Cognitive diagnosis
• ISSN: 1046-1310; 1936-4733
• DOI: 10.1007/s12144-023-05438-8

As a new generation of assessment theory, Cognitive Diagnostic Assessment (CDA) has unique advantages in diagnosing students' personalized information. Cognitive diagnostic models (CDMs) are the core of CDA, so the selection of models becomes the key link of CDA. Generally, the selection of the models is based on data driven methods, such as comparing Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and other indicators. Few studies pay attention to the voice of subject experts. This study selected 10% of Tatsuoka fraction subtraction data, which were analyzed by 5 mathematics education experts according to the criteria of master (1), not master (0), and part master (0.5) for 8 attributes. We further analyzed the Pearson correlation coefficient of expert results and common model analysis results, and concluded that the DINA (the Deterministic Input, Noisy ‘‘And’’ Gate) model diagnosis results had the highest correlation with expert results, with the coefficient reaching 0.8624. The results showed that, from the perspective of mathematical experts, DINA model was most suitable for the diagnosis of fractional subtraction, which provided evidence for the rationality of DINA model diagnosis of fractional subtraction.