Text analytics for uncovering untapped ideas at the intersection of learning design and learning analytics: Critical interpretative synthesis

• Published in: Journal of computer assisted learning Vol. 39; no. 3; pp. 899 - 920
• Main Authors: Stoyanov, Slavi, Kirschner, Paul A.
• Format: Journal Article
• Language: English
• Published: Chichester, UK John Wiley & Sons, Inc 01.06.2023; Wiley; Wiley Subscription Services, Inc
• Subjects: Content analysis; critical interpretative synthesis; Data analysis; Data Interpretation; Design analysis; Educational Environment; Evidence Based Practice; Instructional Design; Learning Analytics; learning design; Literature Reviews; Machine learning; Mathematical analysis; Qualitative analysis; Research Methodology; School environment; Synthesis; Teachers; text analytics
• ISSN: 0266-4909; 1365-2729; 1365-2729
• DOI: 10.1111/jcal.12775

Background Learning analytics (LA) collects, analyses, and reports data from the learning environment, to provide evidence of the effects of a particular learning design. Learning design (LD) outlines the conceptual framework for a meaningful interpretation of Learning analytics data. Objectives The study aims to identify the most relevant concepts at the intersection of learning design and learning analytics, how these concepts are integrated into more general thematic areas, and the implications for the research and practice of the learning design and learning analytics synergy. Methods To this end, the study employs a critical interpretive synthesis (CIS) of the selected papers, and complements it with elements of systematic literature review as well as qualitative content analysis and text analytics that employ machine learning and language technology. Results and conclusions The most important themes identified are ‘analytics’, ‘learning’, ‘data’, ‘tools’, ‘research’, ‘framework’, ‘informed’, ‘model‐driven’, ‘participatory’, ‘technique’, ‘impact’, insight’, and ‘findings'. The text analytics detected two topic rarely explicitly discussed in the literature: ‘evidence‐informed instructional design approaches' and ‘design‐based research’. Future research should attempt a holistic perspective towards the LD and LA synergy considering evidence‐informed instructional design approaches as part of a design‐based research methodology that implements evidence‐based teachers' practice and research‐based findings. Lay Description What is currently known about the subject matter? The number of studies exploring different aspects of the relationship between learning design (LD) and learning analytics (LA) has significantly increased. LD provides the context of LA and outlines the conceptual framework for a meaningful interpretation of LA data. LA collect, analyse and report data from the learning environment to improve learning design. LD and instructional design (ID) are closely related concepts and practices, although misinterpretation remains of their nature and relationship. What this paper adds to this? Critical interpretative synthesis (CIS) approach for literature review is introduced. Text analytics to analyse the literature linking LD and LA is applied. The text analytics shows the most relevant concepts, their relationships, and how are they grouped into broader themes. The text analytics identifies ‘design‐based research’(DBR) and ‘evidence‐informed instructional design approaches’ as important concepts that could be used in the future for improving research investigating the relationship between LD and LA. Implications of study findings for practitioners Integrate the principles and guidelines of evidence‐informed instructional design approaches into a design‐based research process to promote effective, efficient and enjoyable learning implementing process and content types LA indicators. DBR is a natural way of integrating teachers' evidence‐based practice (LD) and research‐based findings (ID).