LLM-enhanced Feedback

Nowadays, many learning environments are starting to produce a deluge of data. Diverse, complex data including digital logs, student portfolios, videos, sensor data, surveys, and information from registrars are often produced at overwhelming speeds and quantities. New technologies like artificial intelligence (AI) can not only help automate the process of separating information from data, but it can also power tools and scaffolds that support students and instructors. Currently, the most common way information is delivered is through analytic dashboards, where different visualizations summarize student states to support decision making. However, the effectiveness of dashboards has so far been mixed (Teasley, 2017), with some pointing out that dashboards can fail to engage users because of the time and data literacy skills needed to make sense of complex information (Verbert et al., 2020).

In this project, we instead explore how AI technologies can augment periodic feedback, a standard practice in many learning environments. Specifically, we explore how a cutting-edge natural language model, GPT-3 (Dalle, 2021), can generate personalized messages for students by considering data from periodic pulse surveys and student reflections. Our context is one that is less explored but ripe for automated data-based feedback systems: a makerspace, an example of a physical project-based learning environment. In constructionist learning environments like makerspaces, students are encouraged to construct their own learning at different speeds and directions, which can make monitoring and support a challenging task. Our approach thus asks, could AI be used to assist writing higher quality feedback for students going through different activities, states, and phases in their learning? 

References

  • Sung, G., Guillain, L., & Schneider, B. (2023). Can AI Help Teachers Write Higher Quality Feedback? Lessons Learned from Using the GPT-3 Engine in a Makerspace Course. In Proceedings of the 17th International Conference of the Learning Sciences (pp. 2094-2095). International Society of the Learning Sciences.