The necessity of multimodal feedback for learning effective pedagogical policies with reinforcement learning

Abstract

Traditional teaching evaluation methods are often retrospective and coarse-grained, lacking the continuous, moment-to-moment feedback required for real-time pedagogical adaptation in language training. Existing intelligent tutoring systems frequently fail to address this, underutilizing multimodal behavioral signals and leaving a critical gap in understanding whether such signals are essential for learning effective policies. This paper introduces a Reinforcement Learning (RL) framework to solve this problem. We propose a hybrid cognitive-linguistic model using a Proximal Policy Optimization (PPO) actor-critic agent, which operates on a novel 516-dimensional state vector that fuses a 512-dimensional semantic embedding from a pre-trained T5 model with a 4-dimensional vector of simulated cognitive-behavioral signals (correctness, response time, attention, hint request). Tested in a simulated learner environment built on the Tatoeba corpus, our agent autonomously discovers a highly effective policy, achieving a mean episodic reward of 6.563, on par with the optimal heuristic baseline. The identified policy, albeit optimal in the simulation, embodies a counterintuitive method that significantly prioritized task repetition by the learner. Our hypothesis is confirmed by a critical ablation experiment: an agent that is deprived of the cognitive-behavioral signals does not learn, and they are as good as a random baseline (mean reward 5.213). The present work gives conclusive evidence that multimodal cognitive-behavioral cues are not only supplementary but are an inevitable part of learning by adaptive pedagogical agents. We mainly provide validation of a hybrid state representation that allows an RL agent to learn effective teaching strategies, making way for more useful and customized educational technologies.