A typical CS1 class involves students working on solving programming problems. Before the pandemic, this occurred in a computer laboratory with a teacher who could quickly assist students having difficulty with their work.

Sometimes, there is a need for this intervention even without the student asking for help. An experienced teacher can sense the growing frustration of a student through their overall demeanor. A teacher can also watch how a student codes to provide quick hints to address potential problems.

This kind of intervention is challenging to do in an online learning setting. A typical online meeting software provides a small and limited view of a student, often crowded with all the other students. As such, the visual cues of frustration can be easily lost in the noise. Not being able to see the student’s code easily is also a problem.

The system we are developing aims to create an online IDE that automatically leverages gaze tracking and code logging to identify these struggling students. In the first phase of the research, a learning model will be trained on students’ gaze and code logs in line with their overall class performance. The second phase of the research will then use this model to predict the frustration level of student users.

Collaboration and gamification strategies will be explored in the final stage of the research that would assist interventions of not just teachers but also classmates who are willing to help.