On the benefits of multimodal annotations for vocabulary uptake from reading

• Published in: Computer assisted language learning Vol. 30; no. 7; pp. 709 - 725
• Main Authors: Boers, Frank, Warren, Paul, Grimshaw, Gina, Siyanova-Chanturia, Anna
• Format: Journal Article
• Language: English
• Published: Oxford Routledge 03.10.2017; Taylor & Francis Ltd
• Subjects: Adults; Annotations; Attention; Coding; Coding theory; Comparative Analysis; Computer Assisted Instruction; Computer assisted language learning; Documentation; English (Second Language); Eye Movements; Foreign Countries; Foreign Students; glosses; Memory; Multimedia Instruction; Multimodality; pictures; reading; Reading Comprehension; Recall (Psychology); Retention (Psychology); Second Language Instruction; Second Language Learning; Second language reading; Second language vocabulary learning; Semantics; Vocabulary; New Zealand
• ISSN: 0958-8221; 1744-3210
• DOI: 10.1080/09588221.2017.1356335

Several research articles published in the realm of Computer Assisted Language Learning (CALL) have reported evidence of the benefits of multimodal annotations, i.e. the provision of pictorial as well as verbal clarifications, for vocabulary uptake from reading. Almost invariably, these publications account for the observed benefits with reference to Paivio's Dual Coding Theory, suggesting it is the visual illustration of word meaning that enhances the quality of processing and hence makes new words more memorable. In this discussion article, we explore the possibility that it is not necessarily the multimodality per se that accounts for the reported benefits. Instead, we argue that the provision of multimodal annotations is one of several possible means of inviting more and/or longer attention to the annotations - with amounts of attention given to words being a significant predictor of their retention in memory. After reviewing the available research on the subject and questioning whether invoking Paivio's Dual Coding Theory is an optimal account for reported findings, we report an eye-tracking study the results of which are consistent with the alternative thesis that the advantage of multimodal glosses for word learning lies with the greater quantity of attention these glosses attract in comparison with single-mode glosses. We conclude with a call for further research on combinations and sequences of annotation types, regardless of multimodality, as ways of promoting vocabulary uptake from reading.