Using process data to understand adults’ problem-solving behaviour in the Programme for the International Assessment of Adult Competencies (PIAAC): Identifying generalised patterns across multiple tas

• Published in: IDEAS Working Paper Series from RePEc
• Main Authors: He, Qiwei, Borgonovi, Francesca, Paccagnella, Marco
• Format: Paper
• Language: English
• Published: St. Louis Federal Reserve Bank of St. Louis 01.01.2019
• Subjects: Problem solving
• DOI: 10.1787/650918f2-en

The Survey of Adult Skills, a product of the OECD Programme for the International Assessment of Adult Competencies (PIAAC), used computers as the main assessment deliver platform. This enabled the Programme to collect data not only on whether respondents were able to solve specific tasks, but also on how they approached the problems at hand and how much time they spent on them. This paper draws on this information to characterise individuals’ problem-solving strategies using the longest common subsequence (LCS) method, a sequence-mining technique commonly used in natural language processing and biostatistics. The LCS is used to compare the action sequences followed by PIAAC respondents to a set of “optimal” predefined sequences identified by test developers and subject matter experts. This approach allows studying problem-solving behaviours across multiple assessment items.