Korean Syntactic Complexity Analyzer (KOSCA): An NLP application for the analysis of syntactic complexity in second language production

• Published in: Language testing Vol. 41; no. 3; pp. 506 - 529
• Main Authors: Hwang, Haerim, Kim, Hyunwoo
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.07.2024; Sage Publications Ltd
• Subjects: Korean language; Language assessment; Language proficiency; Multiple Regression Analysis; Natural language processing; Regression analysis; Second language learning; Second Languages; Statistical analysis; Syntactic analysis; Syntactic complexity; Validity; Korean; natural language processing; written production; syntactic complexity; spoken production
• ISSN: 0265-5322; 1477-0946
• DOI: 10.1177/02655322231222596

Given the lack of computational tools available for assessing second language (L2) production in Korean, this study introduces a novel automated tool called the Korean Syntactic Complexity Analyzer (KOSCA) for measuring syntactic complexity in L2 Korean production. As an open-source graphic user interface (GUI) developed in Python, KOSCA provides seven indices of syntactic complexity, including traditional and Korean-specific ones. Its validity was tested by investigating whether the syntactic complexity indices measured by it in L2 written and spoken production could explain the variability of L2 Korean learners’ proficiency. The results of mixed-effects regression analyses showed that all seven indices significantly accounted for learner proficiency in Korean. Subsequent stepwise multiple regression analyses revealed that the syntactic complexity indices explained 56.0% of the total variance in proficiency for the written data and 54.4% for the spoken data. These findings underscore the validity of the syntactic complexity indices measured by KOSCA as reliable indicators of L2 Korean proficiency, which can serve as a valuable resource for researchers and educators in the field of L2 Korean learning and assessment.