Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT

• Published in: Proceedings (IEEE International Conference on Advanced Learning Technologies) pp. 323 - 325
• Main Authors: Dai, Wei, Lin, Jionghao, Jin, Hua, Li, Tongguang, Tsai, Yi-Shan, Gasevic, Dragan, Chen, Guanliang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.07.2023
• Subjects: Automated Feedback; Chatbots; Feedback Effectiveness; Feedback Generation; Large Language Model; Task analysis
• ISSN: 2161-377X
• DOI: 10.1109/ICALT58122.2023.00100

Educational feedback has been widely acknowledged as an effective approach to improving student learning. However, scaling effective practices can be laborious and costly, which motivated researchers to work on automated feedback systems (AFS). Inspired by the recent advancements in the pre-trained language models (e.g., ChatGPT), we posit that such models might advance the existing knowledge of textual feedback generation in AFS because of their capability to offer natural-sounding and detailed responses. Therefore, we aimed to investigate the feasibility of using ChatGPT to provide students with feedback to help them learn better. Our results show that i) ChatGPT is capable of generating more detailed feedback that fluently and coherently summarizes students' performance than human instructors; ii) ChatGPT achieved high agreement with the instructor when assessing the topic of students' assignments; and iii) ChatGPT could provide feedback on the process of students completing the task, which might benefit students developing learning skills.