A Model-Free Affective Reinforcement Learning Approach to Personalization of an Autonomous Social Robot Companion for Early Literacy Education
Open accessAbstract
Personalized education technologies capable of delivering adaptive interventions could play an important role in addressing the needs of diverse young learners at a critical time of school readiness. We present an innovative personalized social robot learning companion system that utilizes children’s verbal and nonverbal affective cues to modulate their engagement and maximize their long-term learning gains. We propose an affective reinforcement learning approach to train a personalized policy for each student during an educational activity where a child and a robot tell stories to each other. Using the personalized policy, the robot selects stories that are optimized for each child’s engagement and linguistic skill progression. We recruited 67 bilingual and English language learners between the ages of 4–6 years old to participate in a between-subjects study to evaluate our system. Over a three-month deployment in schools, a unique storytelling policy was trained to deliver a personalized story curriculum for each child in the Personalized group. We compared their engagement and learning outcomes to a Non-personalized group with a fixed curriculum robot, and a baseline group that had no robot intervention. In the Personalization condition, our results show that the affective policy successfully personalized to each child to boost their engagement and outcomes with respect to learning and retaining more target words as well as using more target syntax structures as compared to children in the other groups.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This presents a great opportunity for AI's applications in education for further enhancing students' learning outcomes, reducing teachers' workloads and making learning personalized, interactive and fun for teachers and students.
“This presents a great opportunity for AI’s applications in education for further enhancing students’ learning outcomes, reducing teachers’ workloads [30] and making learning personalized [64], interactive and fun [50, 53] for teachers and students.”
From Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021 (2022) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 3× · see citing works
“We used OpenPose (Cao et al. 2017), an open-source tool for 2D body-pose estimation, to analyze the dynamics of children leaning either toward or away from the robot.”
MethodMeaningful Differences in the Everyday Experience of Young American Children
Panagos, R. J. (1998) · Remedial and Special Education · doi:10.1177/074193259801900609 · Cited by 4,183 · see citing works
Meaningful Differences in the Everyday Experience of Young American Children
(1996) · Choice Reviews Online · doi:10.5860/choice.33-3435 · Cited by 4,103 · see citing works
National Center for Educational Statistics
Culpepper, J. C. (2007) · Reference Reviews · doi:10.1108/09504120710838778 · Cited by 2,039 · see citing works
Social robots for education: a review.
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018) · Science Robotics · doi:10.1126/scirobotics.aat5954 · Cited by 1,558 · see citing works
The American Heritage Word Frequency Book
Warren, R. E., Carroll, J. B., Davies, P. S., & Richman, B. (1973) · The American Journal of Psychology · doi:10.2307/1421864 · Cited by 1,379 · see citing works
Building Intelligent Interactive Tutors : student-centered strategies for revolutionizing e-learning
Woolf, B. P. (2009) · HAL · Cited by 716 · see citing works
(1995) · Medical Entomology and Zoology · Cited by 706 · see citing works
Reading in the Brain: The New Science of How We Read
Dehaene, S. (2010) · Cited by 645 · see citing works
Scarborough, H. S. (1990) · Applied Psycholinguistics · doi:10.1017/s0142716400008262 · Cited by 517 · see citing works
Individualized Bayesian Knowledge Tracing Models
Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_18 · Cited by 502 · see citing works
PPVT: Peabody Picture Vocabulary Test
(2021) · Springer eBooks · doi:10.1007/978-3-319-91280-6_301234 · Cited by 495 · see citing works
Mapping the Early Language Environment Using All-Day Recordings and Automated Analysis
Gilkerson, J., Richards, J. A., Warren, S. F., Montgomery, J. K., Greenwood, C. R., Kimbrough Oller, D., Hansen, J. H. L., & Paul, T. D. (2017) · American Journal of Speech-Language Pathology · doi:10.1044/2016_ajslp-15-0169 · Cited by 472 · see citing works
A review of recent advances in learner and skill modeling in intelligent learning environments
Desmarais, M. C., & Baker, R. S. (2011) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9106-8 · Cited by 458 · see citing works
The Theoretical Basis For Relationships Between Language and Literacy in Development
Snow, C. E. (1991) · Journal of Research in Childhood Education · doi:10.1080/02568549109594817 · Cited by 424 · see citing works
Affective Personalization of a Social Robot Tutor for Children’s Second Language Skills
Gordon, G., Spaulding, S., Kory-Westlund, J. M., Lee, J. J., Plummer, L., Martinez, M., Das, M., & Breazeal, C. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9914 · Cited by 322 · see citing works
Self-citationMcDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & el Kaliouby, R. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2890247 · Cited by 283 · see citing works
Language Exposure Relates to Structural Neural Connectivity in Childhood
Romeo, R. R., Segaran, J., Leonard, J., Robinson, S. T., West, M. R., Mackey, A. P., Yendiki, A., Rowe, M. L., & Gabrieli, J. D. E. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0484-18.2018 · Cited by 272 · see citing works
The Condition of Education 2017. NCES 2017-144.
McFarland, J., Hussar, B., de Brey, C., Snyder, T., Wang, X., Wilkinson-Flicker, S., Gebrekristos, S., Zhang, J., Rathbun, A. H., Barmer, A., et al. (2017) · National Center for Education Statistics · Cited by 239 · see citing works
ELL preschoolers’ English vocabulary acquisition from storybook reading
Collins, M. F. (2009) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2009.07.009 · Cited by 229 · see citing works
Personalizing robot tutors to individuals' learning differences
Leyzberg, D., Spaulding, S., & Scassellati, B. (2014) · Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction · doi:10.1145/2559636.2559671 · Cited by 223 · see citing works
Self-citationIs Literacy Enough?: Pathways to Academic Success for Adolescents
Snow, C. E., Porche, M. V. M., Tabors, P. O., & Harris, S. R. (2007) · Cited by 218 · see citing works
Robot education peers in a situated primary school study: Personalisation promotes child learning
Baxter, P. E., Ashurst, E. J., Read, R., Kennedy, J., & Belpaeme, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178126 · Cited by 207 · see citing works
Using electrodermal activity to recognize ease of engagement in children during social interactions
Hernández, J., Riobo, I., Rozga, A., Abowd, G., & Picard, R. W. (2014) · Ubiquitous Computing · doi:10.1145/2632048.2636065 · Cited by 155 · see citing works
Growing Growth Mindset with a Social Robot Peer
Park, H. W., Rosenberg-Kima, R., Rosenberg, M., Gordon, G., & Breazeal, C. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020213 · Cited by 140 · see citing works
Self-citationKory-Westlund, J. M., Jeong, S., Park, H. W., Ronfard, S., Adhikari, A., Harris, P. L., DeSteno, D. A., & Breazeal, C. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00295 · Cited by 137 · see citing works
Self-citationReading socially: Transforming the in-home reading experience with a learning-companion robot
Michaelis, J. E., & Mutlu, B. (2018) · Science Robotics · doi:10.1126/scirobotics.aat5999 · Cited by 108 · see citing works
Park, H. W., Gelsomini, M., Lee, J. J., & Breazeal, C. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2909824.3020245 · Cited by 103 · see citing works
Self-citationFish, M., & Pinkerman, B. (2003) · Journal of Applied Developmental Psychology · doi:10.1016/s0193-3973(02)00141-7 · Cited by 92 · see citing works
Bayesian Active Learning-Based Robot Tutor for Children's Word-Reading Skills
Gordon, G., & Breazeal, C. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9376 · Cited by 75 · see citing works
Self-citationThe pace of vocabulary growth during preschool predicts cortical structure at school age
Asaridou, S. S., Demir‐Lira, Ö. E., Susan Goldin-meadow, A., & Small, S. L. (2016) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2016.05.018 · Cited by 70 · see citing works
Giving our children a fighting chance: Poverty, literacy, and the development of information capital
Sensenig, V. J. (2013) · International Review of Education · doi:10.1007/s11159-013-9383-z · Cited by 68 · see citing works
National Institute for Early Education Research
(2009) · Choice Reviews Online · doi:10.5860/choice.47-0418 · Cited by 54 · see citing works
Backchannel opportunity prediction for social robot listeners
Park, H. W., Gelsomini, M., Lee, J. J., Zhu, T., & Breazeal, C. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989266 · Cited by 23 · see citing works
Self-citationStorytelling with robots : effects of robot language level on children's language learning
Kory, J. (2014) · Cited by 19 · see citing works
Retrieving experience: Interactive instance-based learning methods for building robot companions
Park, H. W., & Howard, A. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7140061 · Cited by 13 · see citing works
Self-citationUsing a shared tablet workspace for interactive demonstrations during human-robot learning scenarios
Park, H. W., Coogle, R. A., & Howard, A. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907248 · Cited by 13 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 7
- Browne, C., et al. 2013. The new general service list: Cel- ebrating 60 years of vocabulary learning. The Language Teacher 37(4):13-16.
- Corbett, A. T., and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4(4):253-278.
- McDuff, D.; Mahmoud, A.; Mavadati, M.; Amr, M.; Turcot, J.; and Kaliouby, R. e. 2016. Affdex sdk: A cross-platform real-time multi-face expression recognition toolkit. In Pro- ceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 3723-3726. ACM.
- Páez, M. M.; Tabors, P. O.; and López, L. M. 2007. Dual language and literacy development of spanish-speaking preschool children. Journal of applied developmental psy- chology 28(2):85-102.
- Watkins, C. J., and Dayan, P. 1992. Q-learning. Machine learning 8(3-4):279-292.
- Wolf, M., and Gottwald, S. 2016. Tales of Literacy in the 21st Century. Literary Agenda Series. Oxford University Press.
- Morgan Kaufmann.
Cited by
Sorted by citations
Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021
Chaudhry, M. A., & Kazim, E. (2022) · AI and Ethics · doi:10.1007/s43681-021-00074-z · Cited by 425 · see citing works
Halkiopoulos, C., & Gkintoni, E. (2024) · Electronics · doi:10.3390/electronics13183762 · Cited by 328 · see citing works
“Affective Reinforcement Learning uses emotional and engagement cues to personalize educational content, which may enhance learning outcomes [182].”
InfluentialLudic-HRI: Designing Playful Experiences with Robots
Lee, W.-Y., & Jung, M. (2020) · Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3371382.3377429 · Cited by 23 · see citing works
Can Children Emulate a Robotic Non-Player Character's Figural Creativity?
Ali, S., Park, H. W., & Breazeal, C. (2020) · Proceedings of the Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3410404.3414251 · Cited by 20 · see citing works
Self-citationPatricc: A Platform for Triadic Interaction with Changeable Characters
Gvirsman, O., Koren, Y., Norman, T., & Gordon, G. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374792 · Cited by 18 · see citing works
Multimodal User Feedback During Adaptive Robot-Human Presentations
Axelsson, A., & Skantze, G. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.741148 · Cited by 14 · see citing works
- Authors
- Park, H. W., Grover, I., Spaulding, S., Gomez, L., & Breazeal, C.
- Affiliations
- Massachusetts Institute of Technology (US)
- Wichita State University (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 687-694
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2905180035
- Cited by
- 146
- References
- 37
- Citations by year
- 2026: 6 · 2025: 23 · 2024: 21 · 2023: 23 · 2022: 23 · 2021: 26 · 2020: 16 · 2019: 8
- FWCI
- 30.13
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- Funders
- National Science Foundation (1523118)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- social robots, personalized learning, early literacy, school readiness, bilingual children, English language learners, storytelling, child engagement, emotional cues, vocabulary acquisition, syntax learning, adaptive interventions, preschool children
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
