Jointly Predicting Courses and Grades Using a Trans...

Jointly Predicting Courses and Grades Using a Trans... | AI Research

Key Takeaways

Jointly Predicting Courses and Grades Using a Transformer-Based Model introduces TRACE (TRansformer for Academic Course-grade Estimation), a model designed t...
Existing predictive models in learning analytics often treat student academic history as a simple sequence, overlooking the concurrent nature of courses taken within a semester.
This simplification can lead to inaccurate performance predictions, particularly for students with heavy or challenging course loads.
Our approach encodes courses on a per-semester basis to capture the effects of course concurrency and utilizes a novel loss function combining course-set prediction with grade prediction.
We demonstrate that predicting courses taken in addition to the grades in those courses leads to significant improvements in prediction quality.

Paper AbstractExpand

Existing predictive models in learning analytics often treat student academic history as a simple sequence, overlooking the concurrent nature of courses taken within a semester. This simplification can lead to inaccurate performance predictions, particularly for students with heavy or challenging course loads. This paper introduces a TRansformer for Academic Course-grade Estimation (TRACE) that addresses this limitation by jointly predicting both the set of courses a student will take and their corresponding grades for an upcoming semester. Our approach encodes courses on a per-semester basis to capture the effects of course concurrency and utilizes a novel loss function combining course-set prediction with grade prediction. We demonstrate that predicting courses taken in addition to the grades in those courses leads to significant improvements in prediction quality. Trained on ten years of institutional data, our joint prediction model reduces mean absolute error by nearly 50% compared to an identical architecture that predicts grades alone. The model also outperforms traditional LSTM-based sequential models, as well as graph neural network-based approaches, and offers natural ways to incorporate student attribute data. This work demonstrates the utility of modern neural architectures for creating interpretable models that can be adapted to new institutions via retraining and recalibration, as well as the importance of key techniques, such as predicting courses taken during training. We discuss how this model could be incorporated into early detection systems at institutions of higher education.

Jointly Predicting Courses and Grades Using a Transformer-Based Model introduces TRACE (TRansformer for Academic Course-grade Estimation), a model designed to predict both the specific courses a student will take and the grades they will earn in an upcoming semester. By treating course selection and academic performance as a joint task, the model accounts for the concurrent nature of a student's course load, which traditional sequential models often overlook.

Addressing Course Concurrency

Existing predictive models in learning analytics frequently treat academic history as a simple, linear sequence. This approach fails to capture the reality that students take multiple courses simultaneously within a single semester, where the combination of courses can influence performance. TRACE addresses this by using semester-level positional encodings. By assigning the same temporal vector to all courses taken within the same term, the model enforces a permutation-invariant representation that reflects the unordered nature of concurrent enrollments while maintaining the temporal order of semesters.

Joint Prediction Architecture

The model uses a Transformer architecture to perform multi-task learning, predicting course sets and grades simultaneously. The author, Paul Savala, notes that this joint approach forces the model to learn more meaningful course representations, similar to how auxiliary tasks improve performance in natural language processing models like BERT. The model utilizes a custom loss function that combines Kullback-Leibler (KL) divergence for course-set prediction and Mean Squared Error (MSE) for grade regression. This design avoids the order artifacts that can occur when using standard token-level cross-entropy for set-valued predictions.

Performance and Results

Trained on ten years of institutional data from over 5,000 students, the TRACE model demonstrated significant improvements in predictive accuracy. The research reports that the joint prediction model reduced mean absolute error by nearly 50% compared to an identical architecture that predicted grades alone. The model also outperformed traditional LSTM-based sequential models and graph neural network-based approaches. These results suggest that incorporating course-set prediction as an auxiliary task is a critical factor in improving the quality of grade predictions.

Practical Considerations

The model is designed to be interpretable and adaptable, allowing institutions to retrain and recalibrate it for new environments. By using label-encoded course names and majors, the model learns latent representations without requiring manual feature engineering, such as identifying prerequisites or course difficulty levels. While the model shows promise for early detection systems in higher education, the author notes that the primary goal of the joint prediction task is to improve the model's internal representation of student academic history rather than to serve as a course recommendation tool for students.