Syntactic complexity revisited: sensitivity of China’s AES-generated scores to syntactic measures, effects of discourse-mode and topic - Qian, Leyi; Yang, Yong; Zhao, Yali (Reading & writing, 2021)
This study aims to explore to what extent syntactic complexity predicts holistic scores generated by China’s two prominent automated writing evaluation systems— pigai and iWrite , the results of which will have some implications for the validity of these two systems. In the meanwhile, this study targets how syntactic complexity differs across writing tasks—between- and within- discourse mode as well as the effects of topic on the use of syntactic measures. To these ends, we examined a corpus of 445 writing samples with 4 prompts on two discourse modes (argumentative and expository) written by Chinese freshmen. These samples were analyzed for the sensitivity of China’s AES-generated scores to syntactic complexity, effects of topic and discourse-mode. Results indicated that there were no relationships between the variations of syntactic measures and the computer-generated scores. Strong topic effects were found within the same discourse mode, but no significant changes were observed in the use of syntactic measures across discourse modes.