BEETLE II: Deep Natural Language Understanding and Automatic Feedback Generation for Intelligent Tutoring in Basic Electricity and Electronics
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Within STEM domains, physics is considered to be one of the most difficult topics to master, in part because many of the underlying principles are counter-intuitive. Effective teaching methods rely on engaging the student in active experimentation and encouraging deep reasoning, often through the use of self-explanation. Supporting such instructional approaches poses a challenge for developers of Intelligent Tutoring Systems. We describe a system that addresses this challenge by teaching conceptual knowledge about basic electronics and electricity through guided experimentation with a circuit simulator and reflective dialogue to encourage effective self-explanation. The Basic Electricity and Electronics Tutorial Learning Environment (BEETLE II) advances the state of the art in dynamic adaptive feedback generation and natural language processing (NLP) by extending symbolic NLP techniques to support unrestricted student natural language input in the context of a dynamically changing simulation environment in a moderately complex domain. This allows contextually-appropriate feedback to be generated “on the fly” without requiring curriculum designers to anticipate possible student answers and manually author multiple feedback messages. We present the results of a system evaluation. Our curriculum is highly effective, achieving effect sizes of 1.72 when comparing pre- to post-test learning gains from our system to those of a no-training control group. However, we are unable to demonstrate that dynamically generated feedback is superior to a non-NLP feedback condition. Evaluation of interpretation quality demonstrates its link with instructional effectiveness, and provides directions for future research and development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Scientific discovery learning with computer simulations of conceptual domains
De Jong, T., & Van Joolingen, W. R. (1998) · Review of Educational Research · doi:10.3102/00346543068002179 · Cited by 1,593
Chinn, C. A., & Brewer, W. F. (1993) · Review of Educational Research · doi:10.3102/00346543063001001 · Cited by 1,393
Conceptual change: A powerful framework for improving science teaching and learning
Duit, R., & Treagust, D. F. (2003) · International Journal of Science Education · doi:10.1080/09500690305016 · Cited by 1,293
Eliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
When Are Tutorial Dialogues More Effective Than Reading?
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007) · Cognitive Science · doi:10.1080/03640210709336984 · Cited by 378 · cited 3×
“The statistical methods employ techniques from textual entailment and paraphrase identification to compare the text of the student answer against sets of possible correct and incorrect answers anticipated by system developers (Jordan et al. 2007; McCarthy et al. 2008; VanLehn et al. 2007).”
BackgroundMethodMore accurate tests for the statistical significance of result differences
Yeh, A. (2000) · Proceedings of the 18th conference on Computational linguistics - · doi:10.3115/992730.992783 · Cited by 378
Evaluating spoken dialogue systems according to de-facto standards: a case study
Möller, S., Smeele, P., Boland, H., & Krebber, J. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2005.11.003 · Cited by 69 · cited 2×
“Similar studies have previously been carried out in tutorial dialogue (Aleven et al. 2004) and task-oriented spoken dialogue systems (Mller et al. 2007).”
Result
Cited by
Evolution and Revolution in Artificial Intelligence in Education
Roll, I., & Wylie, R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0110-3 · Cited by 1,229
“Also most of the five empirical papers from 2014 that were classified as “complex problems” present problems that can be easily evaluated, such as electronics (Dzikovska et al., 2014) and programming (Weragama & Reye, 2014).”
BackgroundMousavinasab, E., Zarifsanaiey, N., R. Niakan Kalhori, S., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021) · Interactive Learning Environments · doi:10.1080/10494820.2018.1558257 · Cited by 605
Automatic feedback in online learning environments: A systematic literature review
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100027 · Cited by 440
Mining Big Data in Education: Affordances and Challenges
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020) · Review of Research in Education · doi:10.3102/0091732x20903304 · Cited by 412
Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.100284 · Cited by 339
Metadata
- Authors
- Dzikovska, M., Steinhauser, N., Farrow, E., Moore, J., & Campbell, G.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2014-06-10
- Volume
- 24
- Issue
- 3
- Pages
- 284-332
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2123159101
- Cited by
- 122
- References
- 8
- FWCI
- 8.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.