CoChat
Sign Up
Cited by 378Intelligent Tutoring Systems and Adaptive LearningVisual and Cognitive Learning ProcessesInnovative Teaching and Learning Methods

When Are Tutorial Dialogues More Effective Than Reading?

Open access
VanLehn, Kurt, Graesser, Arthur C., Jackson, G. Tanner, Jordan, Pamela, Olney, Andrew, Rosé, Carolyn P.·2007·Cognitive Science
DOI 10.1080/03640210709336984
Open this paper in CoChat — free accountOpen PDF

Abstract

It is often assumed that engaging in a one-on-one dialogue with a tutor is more effective than listening to a lecture or reading a text. Although earlier experiments have not always supported this hypothesis, this may be due in part to allowing the tutors to cover different content than the noninteractive instruction. In 7 experiments, we tested the interaction hypothesis under the constraint that (a) all students covered the same content during instruction, (b) the task domain was qualitative physics, (c) the instruction was in natural language as opposed to mathematical or other formal languages, and (d) the instruction conformed with a widely observed pattern in human tutoring: Graesser, Person, and Magliano's 5-step frame. In the experiments, we compared 2 kinds of human tutoring (spoken and computer mediated) with 2 kinds of natural-language-based computer tutoring (Why2-Atlas and Why2-AutoTutor) and 3 control conditions that involved studying texts. The results depended on whether the students' preparation matched the content of the instruction. When novices (students who had not taken college physics) studied content that was written for intermediates (students who had taken college physics), then tutorial dialogue was reliably more beneficial than less interactive instruction, with large effect sizes. When novices studied material written for novices or intermediates studied material written for intermediates, then tutorial dialogue was not reliably more effective than the text-based control conditions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/03640210709336984

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 77 · Cited by 378

Research works printed but not linked: 46

  1. Aleven, V., Ogden, A., Popescu, O., Torrey, C., & Koedinger, K. (2004). Evaluating the effectiveness of a tutorial di- alogue system for self-explanation. In J. Lester, R. M. Vicari, & F. Paraguaca (Eds.), Intelligent Tutoring Sys- tems: Seventh International Conference, ITS 2004 (pp. 443-454). Berlin, Germany: Springer.
  2. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience and school. Washington, DC: National Academy Press.
  3. Charney, D. H., Reder, L. M., & Wells, G. W. (1988). Studies of elaboration in instructional texts. In S. Doheny-Farina (Ed.), Effective documentation: What we have learned from research (pp. 47-72). Cambridge, MA: MIT Press.
  4. Cohen, P. A., Kulik, J. A., & Kulik, C.-L. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19, 237-248.
  5. Core, M. G., Moore, J. D., & Zinn, C. (2003). The role of initiative in tutorial dialogue. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL) (pp. 67-74). Morristown, NJ: Association of Computation Linguistics.
  6. Craig, S. D., Driscoll, D. M., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning and pedagogical agents. Journal of Educational Multimedia and Hypermedia, 13, 163-183.
  7. Craig, S. D., Sullins, J., Witherspoon, A., & Gholson, B. (in press). The deep-level reasoning questions effect: The role of dialog and deep-level reasoning questions during vicarious learning. Cognition and Instruction.
  8. de Kleer, J. (1977). Multiple representations of knowledge in a mechanics problem-solver. In International joint conference on Artificial Intelligence (pp. 299-304). Cambridge, MA: MIT Press.
  9. Freedman, R., Rosé, C. P., Ringenberg, M. A., & VanLehn, K. (2000). ITS Tools for natural language dialogue: A domain-independent parser and planner. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent Tutoring Systems: 5th International Conference, ITS 2000 (pp. 433-442). Berlin, Germany: Springer.
  10. Graesser, A. C., Moreno, K., Marineau, J., Adcock, A., Olney, A., & Person, N. (2003). AutoTutor improves deep learning of computer literacy: Is it the dialog or the talking head? In U. Hoppe, F. Verdejo, & J. Kay (Eds.), Pro- ceedings of Artificial Intelligence in Education (pp. 47-54). Amsterdam: IOS.
  11. Graesser, A. C., & Olde, B. A. (2003). How does one know whether a person understands a device? The quality of questions the person asks when the device breaks down. Journal of Educational Psychology, 95, 524-536.
  12. Graesser, A. C., & Person, N. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137.
  13. Graesser, A. C., Person, N., Harter, D., & Tutoring Research Group. (2001). Teaching tactics and dialog in AutoTutor. International Journal of Artificial Intelligence and Education, 12, 257-279.
  14. Graesser, A. C., Wiemer-Hastings, K., Wiemer-Hastings, P., & Kreuz, R. (1999). AutoTutor: A simulation of a hu- man tutor. Journal of Cognitive Systems Research, 1, 15-25.
  15. Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Halter, D., Person, N., & Tutoring Research Group. (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor. Interactive Learning Environments, 8(2), 129-147.
  16. Hewitt, P. G. (1987). Conceptual physics. Menlo Park, CA: Addison-Wesley.
  17. Hu, X., Cai, Z., Louwerse, M. M., Olney, A., Penumatsa, P., Graesser, A. C., et al. (2003). A revised algorithm for latent semantic analysis. In G. Gottlob & T. Walsh (Eds.), Proceedings of the 2003 International Joint Confer- ence on Artificial Intelligence (pp. 1489-1491). Menlo Park, CA: AAAI Press.
  18. Jordan, P., Makatchev, M., Pappuswamy, U., VanLehn, K., & Albacete, P. (2006). A natural language tutorial dia- logue system for physics. In Proceedings of FLAIRS 2006. Menlo Park, CA: AAAI Press.
  19. Jordan, P., Makatchev, M., & VanLehn, K. (2004). Combining competing language understanding approaches in an intelligent tutoring system. In J. C. Lester, R. M. Vicari, & F. Praguacu (Eds.), Intelligent Tutoring Systems: 7th International Conference, ITS 2004 (pp. 346-357). Berlin, Germany: Springer-Verlag.
  20. Jordan, P., Rosé, C. P., & VanLehn, K. (2001). Tools for authoring tutorial dialogue knowledge. In J. D. Moore, C. Redfield, & W. L. Johnson (Eds.), Artificial intelligence in education: AI-Ed in the wired and wireless future (pp. 222-233). Washington, DC: IOS.
  21. Jordan, P., & VanLehn, K. (2002). Discourse processing for explanatory essyas in tutorial applications. In Proceed- ings of the third SIGdial Workshop on Discourse and Dialogue (pp. 74-83). Philadelphia, PA: Association for Computational Linguistics.
  22. Katz, S., Connelly, J., & Allbritton, D. (2003). Going beyond the problem given: How human tutors use post-solution discussions to support transfer. International Journal of Artificial Intelligence in Education, 13, 79-116.
  23. Lane, H. C., & VanLehn, K. (2005). Teaching the tacit knowledge of programming to novices with natural language tutoring. Computer Science Education, 15(3), 183-201.
  24. Litman, D., Rosé, C., Forbes-Riley, K., VanLehn, K., Bhembe, D., & Silliman, S. (2006). Spoken versus typed hu- man and computer dialogue. International Journal of Artificial Intelligence and Education, 16, 145-170.
  25. Makatchev, M., Hall, B. S., Jordan, P., Papuswamy, U., & VanLehn, K. (2005, August). Mixed language processing in the Why2-Atlas tutoring system. Paper presented at the Proceedings of the Workshop on Mixed Language Ex- planations in Learning Environments, AIED2005, Amsterdam, the Netherlands.
  26. Makatchev, M., Jordan, P., & VanLehn, K. (2004). Abductive theorem proving for analyzing student explanations and guiding feedback in intelligent tutoring systems. Journal of Automated Reasoning, 32, 187-226.
  27. McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts: Effects of text sequence and prior knowledge. Canadian Journal of Experimental Psychology, 55, 51-62.
  28. Mean, H. (1979). Learning lessons: Social organization in the classroom. Cambridge, MA: Harvard University Press.
  29. Olde, B. A., Franceschetti, D., Karnavat, A., Graesser, A. C., & Tutoring Research Group. (2002). The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In W. D. Gray & C. D. Schunn (Eds.), Proceedings of the 24th Annual Conference of the Cognitive Science Society (pp. 708-713). Mahwah, NJ: Law- rence Erlbaum Associates, Inc.
  30. Olney, A., Louwerse, M. M., Mathews, E. C., Marineau, J., Hite-Mitchell, H., & Graesser, A. C. (2003). Utterance classification in AutoTutor. In Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applica- tions Using Natural Language Processing (pp. 1-8). Philadelphia, PA: Association for Computational Linguis- tics.
  31. Person, N., Graesser, A. C., Bautista, L., Mathews, E. C., & Tutoring Research Group. (2001). Evaluating student learning gains in two version of AutoTutor. In J. D. Moore, C. Redfield, & W. L. Johnson (Eds.), Artificial intel- ligence in education: AI-ED in the wired and wireless future (pp. 268-293). Amsterdam: IOS.
  32. Person, N., Graesser, A. C., Kreuz, R. J., Pomeroy, V., & Tutoring Research Group. (2001). Simulating human tutor dialog moves in AutoTutor. International Journal of Artificial Intelligence in Education, 12, 23-39.
  33. Person, N., Graesser, A. C., & Tutoring Research Group. (2002). Human or computer?: AutoTutor in a bystander Turing test. In S. A. Cerri, G. Gouarderes, & F. Paraguaca (Eds.), Intelligent Tutoring Systems, ITS 2002 (pp. 821-830). Berlin, Germany: Springer.
  34. Ranney, M., & Thagard, P. (1988). Explanatory coherence and belief revision in naive physics. In V. Patel & G. J. Groen (Eds.), The Tenth Annual Conference of the Cognitive Science Society (pp. 426-432). Mahwah, NJ: Law- rence Erlbaum Associates, Inc.
  35. Reif, F., & Scott, L. A. (1999). Teaching scientific thinking skills: Students and computers coaching each other. American Journal of Physics, 67, 819-831.
  36. Rosé, C. P., Bhembe, D., Siler, S., Srivastava, R., & VanLehn, K. (2003). Exploring the effectiveness of knowledge construction dialogues. In U. Hoppe, F. Verdejo, & J. Kay (Eds.), Artificial intelligence in education: Shaping the future of learning through intelligent technologies (pp. 497-499). Amsterdam, the Netherlands: IOS Press.
  37. Rosé, C. P., Jordan, P. W., Ringenberg, M., Siler, S., VanLehn, K., & Weinstein, A. (2001). Interactive conceptual tu- toring in Atlas-Andes. In J. D. Moore, C. Redfield, & W. L. Johnson (Eds.), Artificial intelligence in education: AI-Ed in the wired and wireless future (pp. 256-266). Washington, DC: IOS.
  38. Rosé, C. P., Moore, J. D., VanLehn, K., & Allbritton, D. (2001). A comparative evaluation of Socratic versus didac- tic tutoring. In J. D. Moore & K. Stenning (Eds.), Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society (pp. 897-902).
  39. Rosé, C. P., Roque, A., Bhembe, D., & VanLehn, K. (2002). A hybrid language understanding approach for robust selection of tutoring goals. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.), Intelligent Tutoring Systems, 2002: 6th International Conference (pp. 552-561). Berlin, Germany: Springer.
  40. Sinclair, J. M., & Coulthard, R. M. (1975). Towards an analysis of discourse: The English used by teachers and pu- pils. London: Oxford University Press.
  41. Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592-604.
  42. VanLehn, K. (in press). When is tutoring more effective than less interactive instruction? In M. T. H. Chi (Ed.), Festshrift for Lauren Resnick.
  43. VanLehn, K., Jordan, P., Rosé, C. P., Bhembe, D., Bottner, M., Gaydos, A., et al. (2002). The architecture of Why2-Atlas: A coach for qualitative physics essay writing. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.), Intelligent Tutoring Systems, 2002, 6th International Conference (pp. 158-167). Berlin, Germany: Springer.
  44. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Har- vard University Pres.
  45. Wood, D. J., & Middleton, D. (1975). A study of assisted problem-solving. British Journal of Psychology, 66, 181-191.
  46. Wood, D. J., Wood, H., & Middleton, D. (1978). An experimental evaluation of four face-to-face teaching strate- gies. International Journal of Behavioral Development, 1, 131-147.

Authors
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P.
Affiliations
  • University of Pittsburgh (US)
  • University of Memphis (US)
  • Carnegie Mellon University (US)
Corresponding author
VanLehn, Kurt
ORCID
Venue
Cognitive Science
Publisher
Wiley
Published
2007-02
Volume
31
Issue
1
Pages
3-62
ISSN
  • 0364-0213 (ISSN-L)
  • 1551-6709
DOI
10.1080/03640210709336984
OpenAlex
W2165079349
Cited by
378
References
77
Citations by year
2026: 3 · 2025: 4 · 2024: 10 · 2023: 13 · 2022: 11 · 2021: 9 · 2020: 18 · 2019: 11 · 2018: 13 · 2017: 19 · 2016: 22 · 2015: 30 · 2014: 30 · 2013: 28 · 2012: 37
FWCI
33.54
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Cognitive Science (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Science Foundation (0106965, 0354420, N00014-00-1-0600, 0325428, 9720314, 0325054, 9720359)
  • University of Pittsburgh
  • University of Memphis
  • Multidisciplinary University Research Initiative
  • Office of Naval Research (N00014-00-1-0600, N00014)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
Keywords
tutorial dialogue, human tutoring, natural language instructions, qualitative physics, interaction hypothesis, text-based teaching, prior knowledge level, AutoTutor, learning effectiveness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review