AI in Informal Science Education: Bringing Turing Back to Life to Perform the Turing Test
Open accessThe DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Abstract
This paper describes an interactive museum exhibit featuring an avatar of Alan Turing that informs museum visitors about artificial intelligence and Turing’s seminal Turing Test for machine intelligence. The objective of the exhibit is to engage and motivate visiting children in the hope of sparking an interest in them about computer science and artificial intelligence, and cause them to consider pursuing future studies and/or careers in these fields. The exhibit interacts with the visitors, allowing them to participate in a simplified version of Turing’s test that is brief and informal to suit the limitations of a five-minute exhibit. In this exhibit, the visitor (targeted towards middle school age children) invokes an avatar of his/her own choice, and acts to endow it with human-like qualities (voice, brain, eyesight and hearing). Then, the visitor engages the avatar in a (brief) question-and-answer session to determine whether the visitor thinks that he/she is interacting with a real human on a video conference or with an avatar. We consider this interaction to be an extension of the original Turing Test because, unlike Turing’s original that used text via a teletype, this version features a graphical embodiment of an agent with which one can converse in spoken natural language. This extension serves to make passing the Turing Test more difficult, as now the avatar must not only communicate like a human, but also look, sound and act the part. It also makes the exhibit visual, dynamic and interesting to the visitors. Evaluations were performed with museum visitors, both in backrooms with prototypes as well as on the museum floor with the final version. The formative and summative evaluations performed indicated overall success in engaging the museum visitors and increasing their interest in computer science. More specifically, the formative testing, mostly done in quiet back rooms with selected test subjects, indicated that on the important questions about enjoyment of exhibit and increased interest in computer science by the test subjects, their self-reported Likert scale responses (1 being negative and 5 being positive) increased from 3.16 in the first evaluation to 4.38 in the third one for increased interest in CS. Likewise for the question about exhibit enjoyment (from 3.92 to 4.56). The summative evaluation, done through unobtrusive observation of exhibit use on museum floor, indicated that almost 74% of the parties that initiated the exhibit were either highly or moderately engaged by the exhibit. However, there was one major negative finding, namely the overly long duration of the exhibit, which may have caused premature abandonment of the exhibit in several cases during the summative evaluation. These tests and their results are presented and discussed in detail in this paper. The exhibit has been on permanent display at the Orlando (FL) Science Center since June 2014 and has received a strongly positive response from visitors since that time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These activities included conducting discussions after watching AI-related movies (Tims et al., 2012), carrying out computer-based simulations of human-like behaviors (Ho et al., 2019), experimenting as active users of social robots (Gonzalez et al., 2017) and programming AI-based conversational agents (Van Brummelen et al., 2021b).”
From AI literacy in K-12: a systematic literature review (2023) · cites this paper as Gonzalez et al., 2017
Citations
Cites
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
Computing Machinery and Intelligence (1950)
Turing, A. (2004) · The Essential Turing · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,118
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Rising Above The Gathering Storm: Energizing and Employing America for a Brighter Economic Future
Augustine, N. R. (2006) · Cited by 2,064
Learning science in informal environments : people, places, and pursuits
Bell, P., Lewenstein, B. V., Shouse, A. W., & Feder, M. A. (2009) · Cited by 1,702
Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments
Johnson, W. L., Rickel, J. W., & Lester, J. C. (2000) · Cited by 820
Designing robots for long-term social interaction
Gockley, R., Bruce, A., Forlizzi, J., Michalowski, M., Mundell, A., Rosenthal, S., Sellner, B., Simmons, R., Snipes, K., Schultz, A. C., et al. (2005) · 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2005.1545303 · Cited by 407
A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application
Kopp, S., Gesellensetter, L., Krämer, N. C., & Wachsmuth, I. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550617_28 · Cited by 393
Human Conversation as a System Framework: Designing Embodied Conversational Agents
Cassell, J., Bickmore, T., Campbell, L., Vilhjálmsson, H., & Yan, H. (2000) · Embodied Conversational Agents · doi:10.7551/mitpress/2697.003.0004 · Cited by 342
An Oz-Centric Review of Interactive Drama and Believable Agents
Mateas, M. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-48317-9_12 · Cited by 245 · cited 1×
“Mateas [Mateas 1997] provides a comparative overview of advances in chatbots and related technologies in the 1990s.”
BackgroundEmbodied agents for multi-party dialogue in immersive virtual worlds
Traum, D., & Rickel, J. (2002) · Proceedings of the first international joint conference on Autonomous agents and multiagent systems part 2 - AAMAS '02 · doi:10.1145/544862.544922 · Cited by 229 · cited 1×
“Further research into chatbots saw a shift towards enhanced immersive reality for dialog systems, emphasizing face-to-face avatar presentations and dialog evaluation improvements [Traum and Rickel 2002; Duch et al. 2006], including moving from textbased to speech-based interaction.”
BackgroundInteractive humanoid robots for a science museum
Shiomi, M., Kanda, T., Ishiguro, H., & Hagita, N. (2006) · Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction · doi:10.1145/1121241.1121293 · Cited by 221
Ada and Grace: Toward Realistic and Engaging Virtual Museum Guides
Swartout, W., Traum, D., Artstein, R., Noren, D., Debevec, P., Bronnenkant, K., Williams, J., Leuski, A., Narayanan, S., Piepol, D., et al. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15892-6_30 · Cited by 165
Simulating the Emotion Dynamics of a Multimodal Conversational Agent
Becker, C., Kopp, S., & Wachsmuth, I. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24842-2_15 · Cited by 151 · cited 8×
“Becker and Wachsmuth [2004] explored the representation and actuation of coherent emotional states in a virtual conversational agent. Kopp et al. [2005] extends this research by presenting a model for sustainable conversation in a real-world application. Virtual humans have been used in museums before. Kopp et al. [20…”
InfluentialBuilding Interactive Virtual Humans for Training Environments
Kenny, P. G., Hartholt, A., Gratch, J. S., Swartout, W., Traum, D. R., Marsella, S., & Piepol, D. (2007) · Cited by 125
Bickmore, T. W., & Picard, R. W. (2004) · CHI '04 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/985921.986097 · Cited by 122 · cited 1×
“Bickmore and Picard [2004] presented their studies with Laura, a personal trainer agent.”
BackgroundWhat’s An Agent, Anyway? A Sociological Case Study
Foner, L. N. (1999) · Cited by 101
Relational Agents Improve Engagement and Learning in Science Museum Visitors
Bickmore, T., Pfeifer, L. M., & Schulman, D. J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23974-8_7 · Cited by 96
Hayes‐Roth, B., & Doyle, P. (1998) · Autonomous Agents and Multi-Agent Systems · doi:10.1023/a:1010019818773 · Cited by 87
Representation of procedures and practices in contextual graphs
Brézillon, P. (2003) · The Knowledge Engineering Review · doi:10.1017/s0269888903000675 · Cited by 73 · cited 8×
“At its most fundamental level, the Exhibit Framework uses a modified version of Brézillon’s Contextual Graph (CxG) paradigm [Brézillon 2003] to hold the knowledge required to run the exhibit.”
BackgroundMethodInfluentialAda and Grace: Direct Interaction with Museum Visitors
Traum, D. R., Aggarwal, P., Artstein, R., Foutz, S., Gerten, J., Katsamanis, A., Leuski, A., Noren, D., & Swartout, W. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33197-8_25 · Cited by 69
Sorge, C. (2007) · Science educator · Cited by 64
Lane, H. C., Cahill, C., Foutz, S., Auerbach, D., Noren, D., Lussenhop, C. J., & Swartout, W. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_32 · Cited by 55
Embodied agents for multi-party dialogue in immersive virtual worlds
Traum, D. R., & Rickel, J. W. (2002) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/544920.544922 · Cited by 30
Performance Indices for Parallel Marker-Propagation.
DeMara, R. F., & MOLDOVAN, D. I. (1991) · Proceedings of the International Conference on Parallel Processing · Cited by 20
Self-citationTowards Avatars with Artificial Minds: Role of Semantic MemoryPreprint
Szymański, J., Sarnatowicz, T., & Duch, W. (2007) · CogPrints (University of Southampton) · Cited by 20
Context-Centric Speech-Based Human-Computer Interaction
Hung, V. C., & González, A. J. (2013) · International Journal of Intelligent Systems · doi:10.1002/int.21614 · Cited by 12 · cited 2×
“Our dialog management system is a variation of Hung's CONtext-centric Corpus-based Utterance Robustness (CONCUR) system [Hung and Gonzalez 2013].”
MethodGonzález, A. J., Leigh, J., DeMara, R. F., Johnson, A. E., Jones, S. R., Lee, S., Hung, V. C., Renambot, L., Leon-Barth, C., Brown, M. D., et al. (2013) · Journal of Intelligent Systems · doi:10.1515/jisys-2013-0016 · Cited by 4 · cited 1×
“Our dialog management system is a variation of Hung's CONtext-centric Corpus-based Utterance Robustness (CONCUR) system [Hung and Gonzalez 2013].”
MethodSelf-citationAn Extended Turing Test: A Context Based Approach Designed to Educate Youth in Computing
Hollister, J. R., Parker, S. T., González, A. J., & DeMara, R. F. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40972-1_16 · Cited by 3
Self-citation
Research works printed but not linked: 1
- Lee, C., Sidner, C., Kidd, C. Engagement during dialogues with robots, 2005. AAAI Spring Symposia.
Cited by
AI technologies for education: Recent research & future directions
Zhang, K., & Aslan, A. B. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100025 · Cited by 923
AI literacy in K-12: a systematic literature review
Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023) · International Journal of STEM Education · doi:10.1186/s40594-023-00418-7 · Cited by 724
Almasri, F. (2024) · Research in Science Education · doi:10.1007/s11165-024-10176-3 · Cited by 361
“Par - ticularly, the application of AI in science teaching and learning is becoming more popular, even as interest in AI’s effects on general education is growing (Chiu et al., 2023; Gonzalez et al., 2017).”
Affordances and challenges of artificial intelligence in K-12 education: a systematic review
Crompton, H., Jones, M. V., & Burke, D. (2024) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2121344 · Cited by 213
“Animated Pedagogical Agents (APAs; Bowman, 2012 ) and Intelligent Avatars (Gonzalez et al., 2017 ) were used to focus on AI as tutors with a visual avatar appearance that accompanied the tutoring.”
MethodThe use of Artificial intelligence in school science: a systematic literature review
Heeg, D. M., & Avraamidou, L. (2023) · Educational Media International · doi:10.1080/09523987.2023.2264990 · Cited by 85
Metadata
- Authors
- Gonzalez, A. J., Hollister, J. R., DeMara, R. F., Leigh, J., Lanman, B., Lee, S.-Y., Parker, S., Walls, C., Parker, J., Wong, J., Barham, C., & Wilder, B.
- Affiliations
- University of Central Florida (US)
- University of Illinois Chicago (US)
- Orlando Science Center (US)
- Venue
- International Journal of Artificial Intelligence in Education
- Publisher
- Elsevier BV
- Published
- 2017-06
- Volume
- 27
- Issue
- 2
- Pages
- 353-384
- ISSN
- 1560-4292 (ISSN-L)
- OpenAlex
- W2596069950
- Cited by
- 31
- References
- 31
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 6 · 2023: 5 · 2022: 4 · 2021: 2 · 2019: 2 · 2018: 2 · 2017: 1
- FWCI
- 23.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor, 3253 d)
- Locations
- International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
- Journal of International Crisis and Risk Communication Research (submittedVersion)
- Funders
- Directorate for Education and Human Resources (DRL1138325)
- Topics
- Social Sciences › Arts and Humanities › Museology › Museums and Cultural Heritage
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
- Keywords
- Turing test, Alan Turing, interactive museum exhibits, artificial intelligence in education, informal science education, embodied avatars, chatbots, natural language interaction, junior high school students, STEM outreach, exhibition evaluation, human-robot interaction, museum visitor engagement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.