Analysing Multimodal Texts in Science—a Social Semiotic Perspective

• Published in: Research in science education (Australasian Science Education Research Association) Vol. 52; no. 3; pp. 891 - 907
• Main Authors: Wanselin, Hanna, Danielsson, Kristina, Wikman, Susanne
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.06.2022; Springer
• Subjects: Biology education; Classroom Communication; Didactics; Didaktik; Education; Formative Evaluation; Instructional Materials; Language Education; Linguistics; Multimedia Materials; Multimodal analysis; Multimodality; Reader Text Relationship; Science Education; Semiotics; SFL; Social semiotics; språkdidaktik; Student-generated texts; SFL; Multimodal analysis; Biology education; Multimodality; Social semiotics; Student-generated texts; student-generated texts; multimodality; social semiotics; multimodal analysis; biology education
• ISSN: 0157-244X; 1573-1898; 1573-1898
• DOI: 10.1007/s11165-021-10027-5

Teaching and learning in science disciplines are dependent on multimodal communication. Earlier research implies that students may be challenged when trying to interpret and use different semiotic resources. There have been calls for extensive frameworks that enable analysis of multimodal texts in science education. In this study, we combine analytical tools deriving from social semiotics, including systemic functional linguistics (SFL), where the ideational, interpersonal, and textual metafunctions are central. In regard to other modes than writing — and to analyse how textual resources are combined — we build on aspects highlighted in research on multimodality. The aim of this study is to uncover how such a framework can provide researchers and teachers with insights into the ways in which various aspects of the content in multimodal texts are communicated through different semiotic resources. Furthermore, we aim to explore how different text resources interact and, finally, how the students, or authors of teaching resources, position themselves in relation to the subject. Data consist of one student text and one teaching resource text, both comprising drawn and written elements in combination with symbols. Our analyses of the student text suggest that the proposed framework can provide insights into students’ content knowledge and, hence, how construction of multimodal texts may be a useful tool for formative assessment. When it comes to teaching resources, the framework may be a useful tool for teachers when choosing resources, particularly in relation to students’ possibilities of meaning making when engaging with such texts, but also, as a basis for classroom discussions.