Syntactic complexity revisited: sensitivity of China’s AES-generated scores to syntactic measures, effects of discourse-mode and topic

• Published in: Reading & writing Vol. 34; no. 3; pp. 681 - 704
• Main Authors: Qian, Leyi, Yang, Yong, Zhao, Yali
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.03.2021; Springer; Springer Nature B.V
• Subjects: Automation; Chinese languages; College Freshmen; Computerized corpora; Difficulty Level; Discourse Modes; Education; English as a second language; Foreign Countries; Holistic Evaluation; Language and Literature; Linguistics; Literacy; Memory; Neurology; Psycholinguistics; Scores; Scoring; Skills; Social Sciences; Syntactic complexity; Syntax; Topics; Writers; Writing; Writing (Composition); Writing Evaluation; China; Quality writing; Syntactic complexity; Holistic scores
• ISSN: 0922-4777; 1573-0905
• DOI: 10.1007/s11145-020-10087-5

This study aims to explore to what extent syntactic complexity predicts holistic scores generated by China’s two prominent automated writing evaluation systems— pigai and iWrite , the results of which will have some implications for the validity of these two systems. In the meanwhile, this study targets how syntactic complexity differs across writing tasks—between- and within- discourse mode as well as the effects of topic on the use of syntactic measures. To these ends, we examined a corpus of 445 writing samples with 4 prompts on two discourse modes (argumentative and expository) written by Chinese freshmen. These samples were analyzed for the sensitivity of China’s AES-generated scores to syntactic complexity, effects of topic and discourse-mode. Results indicated that there were no relationships between the variations of syntactic measures and the computer-generated scores. Strong topic effects were found within the same discourse mode, but no significant changes were observed in the use of syntactic measures across discourse modes.