Automatic Deictic Gestures for Animated Pedagogical Agents
The DOI matches its Crossref record; 83 of the 88 references it cites were found in a registry or an index.
Abstract
We present a system that automatically generates deictic gestures for animated pedagogical agents (APAs). The system takes audio and text as input, which define what the APA has to say, and generates animated gestures based on a set of rules. The automatically generated gestures point to the exact locations of elements on a whiteboard nearby the APA, which are calculated by searching for keywords mentioned in the speech. We conducted a study with 100 subjects, in which we compared lecture videos containing gestures automatically-scripted by the system to videos of the same lecture containing manually-scripted gestures. The study results show that the manually-scripted and automatically-scripted lectures had comparable number of gestures, and that the gestures were timed equally well.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“All of these systems are based on rules and/or procedural generation, which limits the degree of naturalness, fluidity, and variability that can be achieved in the resulting animations, and makes it difficult to match the gesture quality produced by contemporary generative models for co-speech gestures as described above.”
From Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents (2024) · cites this paper as [14]
Citations
Cites
Embodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Moreno, R., & Mayer, R. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9047-2 · Cited by 1,666
Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments
Johnson, W. L., Rickel, J. W., & Lester, J. C. (2000) · Cited by 820
AutoTutor: an intelligent tutoring system with mixed-initiative dialogue.
Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005) · IEEE Transactions on Education · doi:10.1109/te.2005.856149 · Cited by 750 · cited 1×
“A majority of prior APA systems used talking heads rather than fully embodied agents [27], [29], [59]–[62].”
BackgroundBEAT: the Behavior Expression Animation Toolkit
Cassell, J., Vilhjálmsson, H. H., & Bickmore, T. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383315 · Cited by 739 · cited 1×
“One of the first systems that used this approach was the BEAT toolkit [76], which generated nonverbal behavior and synthesized speech, taking”
MethodEmbodiment in Mathematics Teaching and Learning: Evidence From Learners' and Teachers' Gestures
Alibali, M. W., & Nathan, M. J. (2012) · Journal of the Learning Sciences · doi:10.1080/10508406.2011.611446 · Cited by 691 · cited 1×
“Primary rules: Research on instructor gesture has shown that a large number of important instructor gestures are deictic gestures [85], [86], which connect the graphical representation of lesson elements, appearing on a whiteboard, on paper, or in a textbook, to their utterance within the…”
BackgroundWhen do gestures communicate? A meta-analysis.
Hostetter, A. B. (2011) · Psychological Bulletin · doi:10.1037/a0022128 · Cited by 558 · cited 1×
“Furthermore, research in instructor gesture has shown that a large percentage of instructor gestures are deictic gestures [8], and that instructor deictic gestures are essential for learning [9].”
BackgroundGestures: Their Role in Teaching and Learning
Roth, W.-M. (2001) · Review of Educational Research · doi:10.3102/00346543071003365 · Cited by 496 · cited 1×
“Primary Rules: Research on instructor gesture has shown that a large number of important instructor gestures are deictic gestures [85], [86], which connect the graphical representation of lesson elements, appearing on a whiteboard, on paper, or in a textbook, to their utterance within the instructor speech.”
BackgroundResearch-Based Design of Pedagogical Agent Roles: a Review, Progress, and Recommendations
Kim, Y., & Baylor, A. L. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0055-y · Cited by 422
Brand, M. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311537 · Cited by 411 · cited 1×
“Some prior research on generating agent animations from speech has focused on lip sync, facial expressions and head and eye movements [67], [68], [69], [70], [71].”
BackgroundTeachers’ gestures facilitate students’ learning: A lesson in symmetry
Valenzeno, L., Alibali, M. W., & Klatzky, R. (2003) · Contemporary Educational Psychology · doi:10.1016/s0361-476x(02)00007-3 · Cited by 366 · cited 1×
“Using humans as teachers in live experiment settings [53], [54] or using video recordings of humans as lectures [55], [56] can add confounding variables.”
BackgroundChildren Learn When Their Teacher's Gestures and Speech Differ
Singer, M. A., & Goldin-Meadow, S. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.00786.x · Cited by 333 · cited 2×
“Research on generating gestures automatically requires producing two sets of lectures as stimuli, one set having gestures and one set not having gestures [53]–[56].”
BackgroundAn embodiment effect in computer-based learning with animated pedagogical agents.
Mayer, R. E., & DaPra, C. S. (2012) · Journal of Experimental Psychology: Applied · doi:10.1037/a0028616 · Cited by 323 · cited 2×
“A few systems provided full-bodied human-like APAs that used gaze and gestures to direct user’s attention [26], [63]–[65].”
BackgroundD’Mello, S. K., & Graesser, A. (2010) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9074-4 · Cited by 323 · cited 1×
“A majority of prior APA systems used talking heads rather than fully embodied agents [27], [29], [59]–[62].”
BackgroundThe Role of Gesture in Learning: Do Children Use Their Hands to Change Their Minds?
Cook, S. W., & Goldin-Meadow, S. (2006) · Journal of Cognition and Development · doi:10.1207/s15327647jcd0702_4 · Cited by 320 · cited 1×
“Using humans as teachers in live experiment settings [53], [54] or using video recordings of humans as lectures [55], [56] can add confounding variables.”
BackgroundMayer, R. E. (2014) · The Cambridge Handbook of Multimedia Learning · doi:10.1017/cbo9781139547369.017 · Cited by 309
How Effective are Pedagogical Agents for Learning? A Meta-Analytic Review
Schroeder, N. L., Adesope, O. O., & Gilbert, R. B. (2013) · Journal of Educational Computing Research · doi:10.2190/ec.49.1.a · Cited by 299 · cited 2×
“ANIMATED Pedagogical Agents (APAs) are on-screen instructor avatars embedded into e-learning environments to facilitate instruction [1], [2].”
BackgroundDo pedagogical agents make a difference to student motivation and learning?
Heidig, S., & Clarebout, G. (2011) · Educational Research Review · doi:10.1016/j.edurev.2010.07.004 · Cited by 295 · cited 1×
“While some studies on APAs have resulted in new models to conduct APA research [57] and guidelines to design APAs [27], not many studies focused on developing actual systems to make the process of conducting APA research easier across multiple disciplines.”
BackgroundThe politeness effect: Pedagogical agents and learning outcomes
Wang, N., Johnson, W. L., Mayer, R. E., Rizzo, P., Shaw, E., & Collins, H. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.09.003 · Cited by 294 · cited 1×
“Other studies suggest that agent’s features such as voice and appearance [31], [32], visual presence [33], nonverbal communication [34] and communication style [35] could impact learning and motivation.”
BackgroundGenerating facial expressions for speech
Pelachaud, C. (1996) · Cognitive Science · doi:10.1016/s0364-0213(99)80001-9 · Cited by 285 · cited 1×
“Some prior research on generating agent animations from speech has focused on lip sync, facial expressions and head and eye movements [67]–[71].”
BackgroundEmbodied conversational agents: representation and intelligence in user interfaces
Cassell, J. (2001) · AI Magazine · doi:10.1609/aimag.v22i4.1593 · Cited by 269 · cited 1×
“One of the first researchers who studied the use of animated agents in learning and communication, developed the Embodied Conversational Agent, an interactive virtual agent that can speak and exhibit nonverbal behaviors [19], [20].”
BackgroundNonverbal Behavior Generator for Embodied Conversational Agents
Lee, J., & Marsella, S. (2006) · Lecture Notes in Computer Science · doi:10.1007/11821830_20 · Cited by 238 · cited 1×
“Other automatic gesture generation methods use a combination of prosody and text to produce the agent nonverbal behavior [79], [80].”
MethodLester, J. C., Stone, B. A., & Stelling, G. D. (1999) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1008374607830 · Cited by 202 · cited 1×
“Early examples of APAs are Cosmo [10], Herman [11], STEVE [12], PETA [13] and the “Thinking Head” [14].”
BackgroundAshby Plant, E., Baylor, A. L., Doerr, C. E., & Rosenberg-Kima, R. B. (2009) · Computers & Education · doi:10.1016/j.compedu.2009.01.013 · Cited by 194 · cited 1×
“High school students preferred to work with an agent with the same ethnicity more than with a different one [37], [38], [39].”
BackgroundDesigning nonverbal communication for pedagogical agents: When less is more
Baylor, A. L., & Kim, S. (2009) · Computers in Human Behavior · doi:10.1016/j.chb.2008.10.008 · Cited by 188 · cited 1×
“Other studies suggest that agent’s features such as voice and appearance [31], [32], visual presence [33], nonverbal communication [34] and communication style [35] could impact learning and motivation.”
BackgroundRigid Head Motion in Expressive Speech Animation: Analysis and Synthesis
Busso, C., Deng, Z., Grimm, M., Neumann, U., & Narayanan, S. (2007) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tasl.2006.885910 · Cited by 180 · cited 1×
“Some prior research on generating agent animations from speech has focused on lip sync, facial expressions and head and eye movements [67], [68], [69], [70], [71].”
BackgroundMoreno, R., & Flowerday, T. (2006) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2005.05.002 · Cited by 178 · cited 2×
“College students of color felt more comfortable interacting with a similar agent than with a dissimilar one [37].”
BackgroundVirtual character performance from speech
Marsella, S., Xu, Y., Lhommet, M., Feng, A., Scherer, S., & Shapiro, A. (2013) · Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation · doi:10.1145/2485895.2485900 · Cited by 168 · cited 1×
“Other automatic gesture generation methods use a combination of prosody and text to produce the agent nonverbal behavior [79], [80].”
MethodDesigning agents to support learning by explaining
Holmes, J. (2005) · Computers & Education · doi:10.1016/j.compedu.2005.02.007 · Cited by 166 · cited 1×
“Many studies confirm the positive learning effects of systems using these agents [15]–[18].”
BackgroundDeictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents
Lester, J. C., Voerman, J. L., Towns, S. G., & Callaway, C. B. (1999) · Applied Artificial Intelligence · doi:10.1080/088395199117324 · Cited by 161 · cited 1×
“Some researchers argue that the gestures of an APA provide deictic believability [47] to the agent, making it more lifelike.”
BackgroundLusk, M. M., & Atkinson, R. K. (2007) · Applied Cognitive Psychology · doi:10.1002/acp.1347 · Cited by 161 · cited 1×
“Many studies confirm the positive learning effects of systems using these agents [15], [16], [17], [18].”
BackgroundRosenberg-Kima, R. B., Baylor, A. L., Plant, E. A., & Doerr, C. E. (2008) · Computers in Human Behavior · doi:10.1016/j.chb.2008.03.017 · Cited by 161 · cited 1×
“Other studies suggest that agent’s features such as voice and appearance [31], [32], visual presence [33], nonverbal communication [34] and communication style [35] could impact learning and motivation.”
BackgroundHow Teachers Link Ideas in Mathematics Instruction Using Speech and Gesture: A Corpus Analysis
Alibali, M. W., Nathan, M. J., Wolfgram, M. S., Church, R. B., Jacobs, S. A., Johnson Martinez, C., & Knuth, E. J. (2014) · Cognition and Instruction · doi:10.1080/07370008.2013.858161 · Cited by 157 · cited 1×
“For example, research on mathematical learning suggests that certain gestures such as point, sweep and balance promote conceptual understanding [44].”
BackgroundDomagk, S. (2010) · Journal of Media Psychology · doi:10.1027/1864-1105/a000011 · Cited by 147
Evaluation of Speech-to-Gesture Generation Using Bi-Directional LSTM Network
Hasegawa, D., Kaneko, N., Shirakawa, S., Sakuta, H., & Sumi, K. (2018) · Proceedings of the 18th International Conference on Intelligent Virtual Agents · doi:10.1145/3267851.3267878 · Cited by 131 · cited 1×
“One recent approach [78] uses a deep neural network for 3D gesture motion generation and a Bi-Directional Recurrent Neural Networks (Bi-Directional RNN) with Bi-Directional Long Short-Term Memory (Bi-Directional LSTM) for natural language processing.”
MethodLevine, S., Krähenbühl, P., Thrun, S., & Koltun, V. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778861 · Cited by 129
Integrating Pedagogical Agents into Virtual Environments
Johnson, W. L., Rickel, J., Stiles, R., & Munro, A. (1998) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474698565929 · Cited by 128 · cited 2×
“Early examples of APAs are Cosmo [10], Herman [11], STEVE [12], PETA [13] and the “Thinking Head” [14].”
BackgroundThe impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis
Davis, R. O. (2018) · Educational Research Review · doi:10.1016/j.edurev.2018.05.002 · Cited by 128 · cited 2×
“A more recent meta-analysis of 20 experiments revealed that gesturing pedagogical agents in multimedia environments had a small-to-medium impact on near transfer of knowledge and retention of learning [40].”
BackgroundJohnson, A. M., Ozogul, G., & Reisslein, M. (2015) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12078 · Cited by 125 · cited 1×
“A few studies suggest that APAs have a positive impact on students with low prior knowledge of the subject, and have no impact and sometimes, a negative impact on students with high prior knowledge [22], [23].”
BackgroundReal-time prosody-driven synthesis of body language
Levine, S., Theobalt, C., & Koltun, V. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618518 · Cited by 120
The impact of learner attributes and learner choice in an agent-based environment
Kim, Y. A., & Wei, Q. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.09.016 · Cited by 114 · cited 1×
“High school students preferred to work with an agent with the same ethnicity more than with a different one [37]–[39].”
BackgroundCongdon, E. L., Novack, M. A., Brooks, N., Hemani-Lopez, N., O'Keefe, L., & Goldin-Meadow, S. (2017) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.03.005 · Cited by 109 · cited 1×
“A beneficial effect of gesture on learning has been demonstrated in multiple domains, including mathematics, science, and foreign language education [3], [43].”
BackgroundJohnson, A. M., Ozogul, G., Moreno, R., & Reisslein, M. (2013) · Journal of Engineering Education · doi:10.1002/jee.20009 · Cited by 103 · cited 1×
“A few studies suggest that APAs have a positive impact on students with low prior knowledge of the subject, and have no impact and sometimes, a negative impact on students with high prior knowledge [22], [23].”
BackgroundDesigning multimedia learning environments using animated pedagogical agents: factors and issues
Woo, H. L. (2009) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2008.00299.x · Cited by 100 · cited 1×
“A few studies suggest that gestures, in addition with facial expressions and body movements define the persona of an agent [48].”
BackgroundPredicting Co-verbal Gestures: A Deep and Temporal Modeling Approach
Chiu, C., Morency, L., & Marsella, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-21996-7_17 · Cited by 91 · cited 1×
“Another fairly recent system [83] uses deep learning in order to map gestures from text and prosody.”
BackgroundA Teachable Agent Game Engaging Primary School Children to Learn Arithmetic Concepts and Reasoning
Pareto, L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0018-8 · Cited by 88 · cited 1×
“An experiment showed that teachable agents in educational games could help achieve deeper levels of mathematics learning for elementary and middle school children [30].”
BackgroundMeasuring pedagogical agent persona and the influence of agent persona on learning
Schroeder, N. L., Romine, W. L., & Craig, S. D. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.02.015 · Cited by 87 · cited 2×
“Findings showed that perceptions measured by the Agent Persona Instrument (API) had no significant effect on learning [36].”
BackgroundAffect in Embodied Pedagogical Agents: Meta-Analytic Review
Guo, Y. R., & Goh, D. H.-L. (2015) · Journal of Educational Computing Research · doi:10.1177/0735633115588774 · Cited by 83
Using Virtual Peers to Guide Visual Attention During Learning: A Test of the Persona Hypothesis
Moreno, R., Reislein, M., & Ozogul, G. (2010) · Journal of Media Psychology · doi:10.1027/1864-1105/a000008 · Cited by 82 · cited 1×
“In an experiment [49] conducted with 159 middle school students, it was found that students who learned with the help of an animated arrow and students who learned with the help of deictic movements of an APA outperformed students who learned without any visual attention-guiding method (control…”
MethodBuilding a Case for Agent-Assisted Learning as a Catalyst for Curriculum Reform in Medical Education
Shaw, E., Ganeshan, R., & Millar, D. B. (2000) · Cited by 76
Gesture enhances learning of a complex statistical concept
Rueckert, L. M., Church, R. B., Avila, A., & Trejo, T. (2017) · Cognitive Research Principles and Implications · doi:10.1186/s41235-016-0036-1 · Cited by 75 · cited 2×
“Research on generating gestures automatically requires producing two sets of lectures as stimuli, one set having gestures and one set not having gestures [53]–[56].”
BackgroundCosmo: A Life-Like Animated Pedagogical Agent With Deictic Believability
Towns, S. (1997) · International Joint Conference on Artificial Intelligence · Cited by 74
Simulating Instructional Roles through Pedagogical Agents
Baylor, A. L., Kim, Y., & Ph., D. (2005) · Cited by 72
Creating Presence and Community in a Synchronous Virtual Learning Environment Using Avatars
Annetta, L. A. (2006) · International journal of instructional technology and distance learning · Cited by 71
Research-based principles for designing multimedia instruction.
Mayer, R. E. (2014) · Cited by 70
Badler, N. I. (2024) · Synthesis lectures on computer science · doi:10.1007/978-3-031-63945-6_23 · Cited by 64
Hand Gesture and Mathematics Learning: Lessons From an Avatar
Cook, S. W., Friedman, H. S., Duggan, K. A., Cui, J., & Popescu, V. (2017) · Cognitive Science · doi:10.1111/cogs.12344 · Cited by 63 · cited 4×
“A beneficial effect of gesture on learning has been demonstrated in multiple domains, including mathematics, science, and foreign language education [3], [43].”
BackgroundMethodInfluentialSelf-citationPedagogical Agents: Back to the Future
Johnson, W. L., & Lester, J. C. (2018) · AI Magazine · doi:10.1609/aimag.v39i2.2793 · Cited by 62 · cited 1×
“Researchers point out that one factor that will drive future demand for pedagogical agents is the need for scalable online learning that engages and retains students [7].”
A Look at the Roles of Look & Roles in Embodied Pedagogical Agents – A User Preference Perspective
Haake, M., & Gulz, A. (2009) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2009-19(1)03 · Cited by 59 · cited 1×
“Amajority of prior APA systems used talking heads rather than fully embodied agents [27], [59], [60], [61], [62], [29].”
BackgroundAutomatic Generation of Non-Verbal Facial Expressions from Speech
Albrecht, I., Haber, J., & Seidel, H. (2002) · Springer eBooks · doi:10.1007/978-1-4471-0103-1_18 · Cited by 54 · cited 1×
“Some prior research on generating agent animations from speech has focused on lip sync, facial expressions and head and eye movements [67], [68], [69], [70], [71].”
BackgroundThe Deep Versus the Shallow: Effects of Co‐Speech Gestures in Learning From Discourse
Cutica, I., & Bucciarelli, M. (2008) · Cognitive Science · doi:10.1080/03640210802222039 · Cited by 51 · cited 1×
“In a wide variety of studies, speech accompanied by gesture has been shown to be more likely to be subsequently remembered than speech that is not accompanied by gesture [45], [46].”
BackgroundExplanatory lifelike avatars: performing user-centered tasks in 3D learning environments
Lester, J. C., Zettlemoyer, L., Grégoire, J. P., & Bares, W. H. (1999) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/301136.301154 · Cited by 48 · cited 1×
“A few early systems created APAs that could navigate through the virtual environment and generate deictic gestures [10], [12], [66].”
BackgroundDesirable Characteristics of Learning Companions
Kim, Y. (2007) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2007-17(4)03 · Cited by 47 · cited 1×
“Amajority of prior APA systems used talking heads rather than fully embodied agents [27], [59], [60], [61], [62], [29].”
BackgroundGesture in Instruction: Evidence from Live and Video Lessons
Koumoutsakis, T., Church, R. B., Alibali, M. W., Singer, M. A., & Ayman‐Nolley, S. (2016) · Journal of Nonverbal Behavior · doi:10.1007/s10919-016-0234-z · Cited by 45 · cited 1×
“Furthermore, research in instructor gesture has shown that a large percentage of instructor gestures are deictic gestures [8], and that instructor deictic gestures are essential for learning [9].”
BackgroundPedagogical Agents as Social Models to Influence Learner Attitudes
Kim, Y. A., & Baylor, A. L. (2007) · Digital Commons - USU (Utah State University) · Cited by 44
Combined Gesture-Speech Analysis and Speech Driven Gesture Synthesis
Sargin, M. E., Aran, O., Karpov, A. A., Ofli, F., Yasinnik, Y., Wilson, S., Erzin, E., Yemez, Y., & Tekalp, A. M. (2006) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2006.262663 · Cited by 41 · cited 1×
“One other system synthesized deictic gestures (describing directions) from speech [77].”
BackgroundSocial and visual style in virtual pedagogical agents
Gulz, A., & Haake, M. (2005) · Lund University Publications (Lund University) · Cited by 39
Interlanguage pragmatics with a pedagogical agent: the request game
Yang, H.-C., & Zapata-Rivera, D. (2010) · Computer Assisted Language Learning · doi:10.1080/09588221.2010.520274 · Cited by 37 · cited 2×
“Compared to an e-learning activity that relies only on text and images, an activity presented by an APA can lead to more learning, especially for younger learners [3], for learners not yet proficient in English [4], and for learners with special needs [5].”
BackgroundKim, Y. A., & Lim, J. H. (2013) · Journal of Educational Psychology · doi:10.1037/a0031027 · Cited by 35
Head and facial gestures synthesis using PAD model for an expressive talking avatar
Jia, J., Wu, Z., Zhang, S., Meng, H. M. L., & Cai, L. (2013) · Multimedia Tools and Applications · doi:10.1007/s11042-013-1604-8 · Cited by 34 · cited 1×
“Some prior research on generating agent animations from speech has focused on lip sync, facial expressions and head and eye movements [67]–[71].”
BackgroundTwo Novel Technologies for Accessible Math and Science Education
Adamo‐Villani, N., & Wilbur, R. Β. (2008) · IEEE Multimedia · doi:10.1109/mmul.2008.97 · Cited by 31 · cited 2×
“Compared to an e-learning activity that relies only on text and images, an activity presented by an APA can lead to more learning, especially for younger learners [3], for learners not yet proficient in English [4], and for learners with special needs [5].”
BackgroundSelf-citationLhommet, M., & Marsella, S. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40415-3_27 · Cited by 31 · cited 1×
“Another approach generates gestures (locations and type) based on the text of the speech [75].”
BackgroundAutomating the Production of Communicative Gestures in Embodied Characters
Ravenet, B., Pélachaud, C., Clavel, C., & Marsella, S. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01144 · Cited by 29 · cited 1×
“One recently proposed approach generates the agents metaphoric gestures from spoken text using Image Schemas [81], [82].”
Alseid, M. (2010) · The Open Virtual Reality Journal · doi:10.2174/1875323x01002010008 · Cited by 15 · cited 1×
“Agents interacting using multiple modalities appear to lead to greater learning than agents that interact only in a single channel [24], [25].”
BackgroundThe role of speech-gesture congruency and delay in remembering action events
Galati, A., & Samuel, A. G. (2010) · Language and Cognitive Processes · doi:10.1080/01690965.2010.494846 · Cited by 14 · cited 1×
“In a wide variety of studies, speech accompanied by gesture has been shown to be more likely to be subsequently remembered than speech that is not accompanied by gesture [45], [46].”
BackgroundHand Gesture Synthesis for Conversational Characters
Neff, M. (2016) · Springer eBooks · doi:10.1007/978-3-319-30808-1_5-1 · Cited by 14 · cited 1×
“The main problem with prosody-based approaches is that they do not capture semantics and therefore the generated gestures do not augment the meaning being conveyed by the speech with information not present in the spoken message [74].”
BackgroundDavis, R. O., & Antonenko, P. (2017) · The Journal of Interactive Learning Research · Cited by 14
PETA: a pedagogical embodied teaching agent
Powers, D. M. W., Leibbrandt, R. E., Luerssen, M. H., Lewis, T. W., & Lawson, M. (2008) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/1389586.1389658 · Cited by 13
Digital learning activities delivered by eloquent instructor avatars: scaling with problem instance
Anasingaraju, S., Wu, M.-L., Adamo‐Villani, N., Popescu, V. S., Cook, S. W., Nathan, M. J., & Alibali, M. W. (2016) · SIGGRAPH Asia Symposium on Education · doi:10.1145/2993352.2993355 · Cited by 10 · cited 1×
“Recent work has demonstrated such e-learning content creation scalability with the problem instance [6].”
BackgroundSelf-citationAutomatic Nonverbal Behavior Generation from Image Schemas
Ravenet, B., Clavel, C., & Pélachaud, C. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/nhje8666 · Cited by 10 · cited 2×
“The system extracts Image Schemas from spoken texts, aligns them with text using a set of rules based on lexical and prosodic models, translates the image schemas into gesture invariants, combines the invariants into full gestures, and then syncs the gestures with speech.”
Making a thinking-talking head
Davis, C., Kim, J., Kuratate, T., Chen, J., Stelarc, & Burnham, D. K. (2007) · Cited by 9
Animation Stimuli System for Research on Instructor Gestures in Education
Cui, J., Popescu, V. S., Adamo‐Villani, N., Cook, S. W., Duggan, K. A., & Friedman, H. S. (2017) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2017.3271471 · Cited by 8 · cited 4×
“The animation commands in the script file are converted into APA animations using inverse kinematics algorithms and predefined animations [52].”
BackgroundMethodInfluentialSelf-citationScripted Animation Towards Scalable Content Creation for eLearning—A Quality Analysis
Adamo‐Villani, N., Cui, J., & Popescu, V. S. (2014) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-13293-8_1 · Cited by 4 · cited 4×
“In the current version of the system, the user has to specify manually the content that should be present on the board at different times using a script [51], [52].”
BackgroundMethodInfluentialSelf-citation
Research works printed but not linked: 5
- baylor, 0, Pedagogical agents as social models for engineering: The influence of agent appearance on female choice, Proc Conf Artif Intell Educ Supporting Learn Intell Socially Informed Technol, 125, 65
- baylor, 0, Designing effective nonverbal communication for pedagogical agents, Proc Conf Artif Intell Educ Supporting Learn Intell Socially Informed Technol, 744
- 10.1207/s15516709cog2001_1 · doi:10.1207/s15516709cog2001_1 · Paper page
- 10.1002/acp.1347 · DOI · Paper page
- rosenfelder, 2011, Fave (forced alignment and vowel extraction) program suite
Cited by
Dai, C.-P., & Ke, F. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100087 · Cited by 271
Shute, V., Rahimi, S., Smith, G., Ke, F., Almond, R., Dai, C., Kuba, R., Liu, Z., Yang, X., & Sun, C. (2021) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12473 · Cited by 132
How pedagogical agents communicate with students: A two-phase systematic review
Sikström, P., Valentini, C., Sivunen, A., & Kärkkäinen, T. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104564 · Cited by 103
Pérez‐Sanagustín, M., Sapunar‐Opazo, D., Álvarez, R. A. P., Hilliger, I., Bey, A., Maldonado‐Mahauad, J., & Baier, J. A. (2020) · Computer Applications in Engineering Education · doi:10.1002/cae.22337 · Cited by 89
Twenty-five Years of Learning with Pedagogical Agents: History, Barriers, and Opportunities
Siegle, R. F., Schroeder, N. L., Lane, H. C., & Craig, S. D. (2023) · TechTrends · doi:10.1007/s11528-023-00869-3 · Cited by 36
“In addition, gesturing systems have been developed to allow PAs to point at things within the virtual world, not from being programed to point, but to naturally point when discussing an object (Kappagantula et al., 2019).”
Background
Metadata
- Authors
- Kappagantula, S. R. K., Adamo-Villani, N., Wu, M.-L., & Popescu, V.
- Affiliations
- Purdue University West Lafayette (US)
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2020-01-01
- Volume
- 13
- Issue
- 1
- Pages
- 1-13
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W2952862469
- Cited by
- 13
- References
- 88
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 1 · 2022: 3 · 2021: 2 · 2020: 2
- FWCI
- 0.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Learning Technologies (publishedVersion)
- Funders
- Institute of Education Sciences
- U.S. Department of Education (R305A130016)
- National Science Foundation (1217215)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- pointing gestures, animated pedagogical agents, video lectures, gesture timing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.