CoChat
Sign Up
Cited by 161Intelligent Tutoring Systems and Adaptive LearningSocial Robot Interaction and HRISpeech and dialogue systems

Deictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents

Open access
Lester, James C., Voerman, Jennifer L., Towns, Stuart G., Callaway, Charles B.·1999·Applied Artificial Intelligence
DOI 10.1080/088395199117324
Open this paper in CoChat — free accountOpen PDF

Abstract

Lifelike animated agents for knowledge - based learning environments can provide timely , customized advice to support students' problem solving . Because of their strong visual presence , they hold significant promise for substantially increasing students' enjoyment of their learning experiences . A key problemposed by lifelike agents that inhabit artificial worlds is deictic believability. In the same manner that humans refer to objects in their environment through judicious combinations of speech , locomotion , and gesture , animated agents should be able to move through their environment and point to and refer to objects appropriately as they provide problem - solving advice . In this paper we describe a framework for achieving deictic believability in animated agents . A deictic behavior planner exploits a world model and the evolving explanation plan as it selects and coordinates locomotive , gestural , and speech behaviors . The resulting behaviors and utterances are believable , and the references exhibit a lack of ambiguity . This approach to spatial deixis has been implemented in a lifelike animated agent , COSMO , who inhabits a learning environment for the domain of Internet packet routing . COSMO provides real - time advice to students as they escort packets through a virtual world of interconnected routers . Results of an informal focus group study with the COSMOagent suggest that the spatial deixis framework produces clear explanatory animated behaviors .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/088395199117324

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Situated gesture generation systems for virtual agents and human-robot communication are based on rules and/or procedural generation, which limits naturalness, fluidity and variability and makes it difficult to match contemporary generative co-speech gesture models.

    “All of these systems are based on rules and/or procedural generation, which limits the degree of naturalness, fluidity, and variability that can be achieved in the resulting animations, and makes it difficult to match the gesture quality produced by contemporary generative models for co-speech gestures as described above.”

    From Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents (2024) · cites this paper as [18]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 161

Research works printed but not linked: 50

  1. Andre  , E., and T. Rist. 1996. Coping with temporal constraints in multimedia presentation planning. In Proc. T hirteenth National Conference on Arti cial Intelligence, 142 147.
  2. M. T. Maybury, chap. 3. New York : AAAI Press.
  3. Andre  , E., T. Rist, and J. Mu È ller. 1998. Integrating reactive and scripted behaviors in a lifelike presen- tation agent. In Proc. Second International Conference on Autonomous Agents, 261 268.
  4. Bares, W. H., and J. C. Lester. 1997. Real-time generation of customized 3-D animated explanations for knowledge-based learning environments. In AAAI-97: Proc. Fourteenth National Conference on Arti cial Intelligence, 347 354.
  5. Bates, J. 1994. The role of emotion in believableagents. Commun. ACM 37(7):122 125.
  6. Blumberg, B., and T. Galyean. 1995. Multi-level direction of autonomous creatures for real-time virtual environments. In Computer graphics proceedings, 47 54.
  7. Butz, A. 1997. Anymation with CATHI. In Proc. Ninth Innovative Applications of Arti cial Intelligence Conference, 957 962.
  8. Carr, B., and I. P. Goldstein. 1977. Overlays: A theory of modelling for computer aided instruction. Technical Report AI Memo 406, Arti cial Intelligence laboratory, Massachusetts Institute of Tech- nology, Cambridge, Mass.
  9. Cassell, J., C. Pelachaud, N. Badler, M. Steedman, B. Achorn, T. Becket, B. Douville, S. Prevost, and M. Stone. 1994. Modeling the interaction between speech and gesture. In SIGGRAPH '94.
  10. Cawsey, A. 1992. Explanation and interaction: T he computer generation of explanatory dialogues. Cam- bridge, Mass.: MIT Press.
  11. Claassen, W. 1992. Generating referring expressions in a multimodal environment. In Aspects of automa- ted natural language generation, eds. R. Dale, E. Hovy, D. Rosner, and O. Stock, 247 262. Berlin: Springer-Verlag.
  12. Conati, C., A. Gertner, K. VanLehn, and M. Druzdzel. 1997. On-line student modeling for coached problem solving using Bayesian networks. In Proc. Sixth International Conference on User Model- ing, 231 242.
  13. Dale, R. 1992. Generating referring expressions. Cambridge, Mass.: MIT Press.
  14. Elliott, C. 1992. The aV ective reasoner: A process model of emotions in a multi-agent system. Ph.D. thesis Northwestern University, Evanston, Ill.
  15. Feiner, S. K., and K. R. McKeown. 1990. Coordinating text and graphics in explanation generation. In Proc. Eighth National Conference on Arti cial Intelligence, 442 449.
  16. Freedman, R. K. 1996. Interaction of discourse planning instructional planning and dialogue manage- ment in an interactive tutoring system. Ph.D. thesis, Northwestern University, Evanston, Ill.
  17. Granieri, J. P., W. Becket, B. D. Reich, J. Crabtree, and N. I. Badler. 1995. Behavioral control for real-time simulated human agents. In Proc. 1995 Symposium on Interactive 3D Graphics, 173 180.
  18. Guinn, C. I. 1995. Meta-dialogue behaviors: Improving the efficiency of human-machine dialogue-A computational model of variable initiative and negotiation in collaborative problem-solving, Ph.D. thesis, Duke University, Durham, N.C.
  19. Jarvella, R. J., and W. Klein (eds.) 1982. Speech, place, and action: Studies in deixes and related topics. New York : John Wiley.
  20. Karp, P., and S. Feiner. 1993. Automated presentation planning of animation using task decomposition with heuristic reasoning. In Proc. Graphics Interface '93, 118 127.
  21. Kurlander, D., and D. T. Ling. 1995. Planning-based control of interface animation. In Proc. CHI '95, 472 479.
  22. Lester, J. C., S. A. Converse, S. E. Kahler, S. T. Barlow, B. A. Stone, and R. Bhogal. 1997a. The persona eVect : AVective impact of animated pedagogical agents. In Proc. CHI'97 (Human Factors in Com- puting Systems), 359 366.
  23. Lester, J. C., S. A. Converse, B. A. Stone, S. E. Kahler, and S. T. Barlow. 1997b. Animated pedagogical agents and problem-solving eVectiveness: A large-scale empirical evaluation. In Proc. Eighth World Conference on Arti cial Intelligence in Education, 23 30.
  24. Lester, J. C., P. J. FitzGerald, and B. A. Stone. 1997c. The pedagogical design studio: Exploiting artifact- based task models for constructivist learning. In Proc. of the T hird International Conference on Intelligent User Interfaces, 155 162.
  25. Lester, J. C., and B. W. Porter. 1997. Developing and empirically evaluating robust explanation gener- ators: The KNIGHT experiments. Comput. Linguistics 23(1):65 101.
  26. Lester, J. C., and B. A. Stone. 1997. Increasing believability in animated pedagogical agents. In Proc. First International Conference on Autonomous Agents, 16 21.
  27. Lester, J. C., B. A. Stone, M. A. O'Leary, and R. B. Stevenson. 1996. Focusing problem solving in design-centered learning environments. In Proc. Third International Conference on Intelligent Tutor- ing Systems, 475 483.
  28. Lester, J., B. Stone and G. Stelling. In press. Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments. In User modeling and user-adapted interaction.
  29. Maes, P., T. Darrell, B. Blumberg and A. Pentland. 1995. The ALIVE system: Full-body interaction with autonomous agents. In Proc. Computer Animation '95 Conference.
  30. Maybury, M. T. 1991. Planning multimedia explanations using communicative acts. In Proc. Ninth National Conference on Arti cial Intelligence, 61 66.
  31. Mittal, V. O. 1993. Generating natural language descriptions with integrated text and examples. Ph.D. thesis, University of Southern California, Los Angeles, Calif.
  32. Mittal, V., S. Roth, J. D. Moore, J. Mattis, and G. Carenini. 1995. Generating explanatory captions for information graphics. In Proc. International J oint Conference on Arti cial Intelligence, 1276 1283.
  33. Moore, J. D. 1995. Participating in explanatory dialogues. Cambridge, Mass.: MIT Press.
  34. Nass, C., J. Steuer, L. Henriksen, and H. Reeder. 1993. Anthropomorphism, agency and ethopoeia: Com- puters as social actors. In Proc. International CHI Conference.
  35. Neal, J. G., and S. C. Shapiro. 1991. Intelligent multi-media interface technology. In Intelligent user interfaces, eds. J. W. Sullivan and S. W. Tyler, 11 43. Reading, Mass.: Addison-Wesley.
  36. Rickel, J., and L. Johnson. 1997a. Integrating pedagogical capabilities in a virtual environment agent. In Proc. First International Conference on Autonomous Agents, 30 38.
  37. Rickel, J., and L. Johnson. 1997b. Intelligent tutoring in virtual reality: A preliminary report. In Proc. Eighth W orld Conference on AI in Education, 294 301.
  38. Roberts, L. D. 1993. How reference works : explanatory models for indexicals, descriptions, and opacity. State University of New York Press.
  39. Roth, S. F., J. Mattis, and X. Mesnard. 1991. Graphics and natural language as components of automatic explanation. In Intelligent user interfaces, eds. J. W. Sullivan and S. W. Tyler, 207 239. Reading, Mass.: Addison-Wesley.
  40. Sibun, P. 1992. Generating text without trees. Comput. Intelligence 8(1):102 122.
  41. Smith, R. W. and D. R. Hipp. 1994. Spoken natural language dialog systems. New York : Oxford Uni- versity Press.
  42. Stone, B. A., and J. C. Lester. 1996. Dynamically sequencing an animated pedagogical agent. In Proc. T hirteenth National Conference on Arti cial Intelligence, 424 431.
  43. Suthers, D. D. 1991. A task-appropriate hybrid architecture for explanation. Comput. Intelligence 7(4):315 333.
  44. Towns, S. G., P. J. FitzGerald, and J. C. Lester. 1998. Visual emotive communication in lifelike peda- gogical agents. In Proc. Fourth International Conference on Intelligent Tutoring Systems.
  45. Traum D. 1994. A computational theory of grounding in natural language generation. Ph.D. thesis, University of Rochester, Rochester, N.Y.
  46. Velasquez, J. D. 1997. Modeling emotions and other motivations in synthetic agents. In AAAI-97 : Proc. Fourteenth National Conference on Arti cial Intelligence, 10 15.
  47. Walker, M. 1993. Informational redundancy and resource bounds in dialogue. Ph.D. thesis, University of Pennsylvania.
  48. Walker, M. A., J. E. Cahn, and S. J. Whittaker. 1997. Improvising linguistic style: Social and aV ective bases of agent personality. In Proc. First International Conference on Autonomous Agents.
  49. Westcourt, K. T., M. Beard, and A. Barr. 1981. Curriculum information networks for CAI: Research on testing and evaluation by simulation. In University-level computer-assisted instruction at Stanford: 1968 1980, ed. P. Suppes, 817 839. Stanford, Calif.: Stanford University Press.
  50. Zhou, M. X., and S. K. Feiner. 1997. Top-down hierarchical planning of coherent visual discourse. In Proc. T hird International Conference on Intelligent User Interfaces.

Authors
Lester, J. C., Voerman, J. L., Towns, S. G., & Callaway, C. B.
Affiliations
North Carolina State University (US)
ORCID
Venue
Applied Artificial Intelligence
Publisher
Informa UK Limited
Published
1999-05
Volume
13
Issue
4-5
Pages
383-414
ISSN
  • 0883-9514 (ISSN-L)
  • 1087-6545
DOI
10.1080/088395199117324
OpenAlex
W2056052471
Cited by
161
References
0
Citations by year
2026: 1 · 2025: 2 · 2024: 3 · 2023: 2 · 2022: 2 · 2021: 3 · 2019: 3 · 2018: 6 · 2017: 2 · 2016: 4 · 2015: 2 · 2013: 9 · 2012: 6
FWCI
9.11
Type
journal article
Language
en
Open access
Locations
Applied Artificial Intelligence (publishedVersion)
Funders
  • National Science Foundation (CAREER, 9701503, 9720395)
  • North Carolina State University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
spatial deixis, educational agents, locomotion, behavior planning, world models, virtual learning environments, student problem solving

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review