Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students’ text revision, motivation, and positive emotions - Meyer, Jennifer; Jansen, Thorben; Schiller, Ronja; Liebenow, Lucas W.; Steinbach, Marlene; Horbach, Andrea; Fleckenstein, Johanna (Computers and Education: Artificial Intelligence, 2024)
Writing proficiency is an essential skill for upper secondary students that can be enhanced through effective feedback. Creating feedback on writing tasks, however, is time-intensive and presents a challenge for educators, often resulting in students receiving insufficient or no feedback. The advent of text-generating large language models (LLMs) offers a promising solution, namely, automated evidence-based feedback generation. Yet, empirical evidence from randomized controlled studies about the effectiveness of LLM-generated feedback is missing. To address this issue, the current study compared the effectiveness of LLM-generated feedback to no feedback. A sample of N = 459 upper secondary students of English as a foreign language wrote an argumentative essay. Students in the experimental group were asked to revise their text according to feedback that was generated using the LLM GPT-3.5-turbo. The control group revised their essays without receiving feedback. We assessed improvement in the revision using automated essay scoring. The results showed that LLM-generated feedback increased revision performance (d = .19) and task motivation (d = 0.36). Moreover, it increased positive emotions (d = 0.34) compared to revising without feedback. The findings highlight that using LLMs allows to create timely feedback that can positively relate to students’ cognitive and affective-motivational outcomes. Future perspectives and the implications for research and practice of using LLM-generated feedback in intelligent tutoring systems are discussed.
Keywords: AI: ChatGPT; Educational Area: Assess & Review; Educational Area: Positive Learning Environment; Educational Area: Student Learning; Evidence: Gaps in Current Research; Evidence: Improve a process; Evidence: Measured Student Behaviour; Evidence: Student Learning Outcomes; Location: Europe; Research Design: Randomised Control Trial; Research Method: Quantitative; Source: Open Source; artificial intelligence