AC-PLT: An algorithm for computer-assisted coding of semantic property listing data

• Published in: Behavior research methods Vol. 56; no. 4; pp. 3366 - 3379
• Main Authors: Ramos, Diego, Moreno, Sebastián, Canessa, Enrique, Chaigneau, Sergio E., Marchant, Nicolás
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2024; Springer Nature B.V
• Subjects: Algorithms; Behavioral Science and Psychology; Cognitive Psychology; Humans; Machine Learning; Natural Language Processing; Original Manuscript; Psychology; Reproducibility of Results; Semantics; Coding reliability; Assisted codification; Machine learning framework; Property listing task
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-023-02260-9

In this paper, we present a novel algorithm that uses machine learning and natural language processing techniques to facilitate the coding of feature listing data. Feature listing is a method in which participants are asked to provide a list of features that are typically true of a given concept or word. This method is commonly used in research studies to gain insights into people's understanding of various concepts. The standard procedure for extracting meaning from feature listings is to manually code the data, which can be time-consuming and prone to errors, leading to reliability concerns. Our algorithm aims at addressing these challenges by automatically assigning human-created codes to feature listing data that achieve a quantitatively good agreement with human coders. Our preliminary results suggest that our algorithm has the potential to improve the efficiency and accuracy of content analysis of feature listing data. Additionally, this tool is an important step toward developing a fully automated coding algorithm, which we are currently preliminarily devising.