The DOI matches its Crossref record; 70 of the 74 references it cites were found in a registry or an index.
Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science
Open accessAbstract
This article describes several approaches to assessing student understanding using written explanations that students generate as part of a multiple-document inquiry activity on a scientific topic (global warming). The current work attempts to capture the causal structure of student explanations as a way to detect the quality of the students’ mental models and understanding of the topic by combining approaches from Cognitive Science and Artificial Intelligence, and applying them to Education. First, several attributes of the explanations are explored by hand coding and leveraging existing technologies (LSA and Coh-Metrix). Then, we describe an approach for inferring the quality of the explanations using a novel, two-phase machine-learning approach for detecting causal relations and the causal chains that are present within student essays. The results demonstrate the benefits of using a machine-learning approach for detecting content, but also highlight the promise of hybrid methods that combine ML, LSA and Coh-Metrix approaches for detecting student understanding. Opportunities to use automated approaches as part of Intelligent Tutoring Systems that provide feedback toward improving student explanations and understanding are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The main use of AI for assessment is focused mainly on higher education contexts, although there are some examples of its use at secondary education level [43,47].”
From Artificial Intelligence for Student Assessment: A Systematic Review (2021) · cites this paper as [43]
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3×
“…relations between events using a wide range of linguistic resources and techniques, including part-of-speech tagging, syntactic parsing, WordNet (Miller 1995), VerbOcean for semantic links between verbs (Chklovski and Pantel 2004), dependency parsing, word sense disambiguation (Mihalcea and…”
BackgroundAn introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · cited 1×
“In this work, a common assessment method has been to assess the similarity of each student response to a set of idealized target responses using Latent Semantic Analysis (LSA, Landauer et al. 1998).”
MethodA meta-analysis of writing instruction for adolescent students
Graham, S., & Perin, D. (2007) · Journal of Educational Psychology · doi:10.1037/0022-0663.99.3.445 · Cited by 1,609 · cited 1×
“For example, a metaanalysis on the effectiveness of feedback on quality of student compositions has shown moderate effect sizes (e.g., 0.77, Graham and Perin 2007).”
BackgroundInfluentialHandbook of Latent Semantic Analysis
(2007) · Psychology Press eBooks · doi:10.4324/9780203936399 · Cited by 1,554
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025 · cited 1×
“Similarly, revision is essential to improving the quality of written compositions (Flower and Hayes 1981).”
BackgroundAutomated Evaluation of Text and Discourse with Coh-Metrix
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511894664 · Cited by 892
Putting stress into words: Health, linguistic, and therapeutic implications
Pennebaker, J. W. (1993) · Behaviour Research and Therapy · doi:10.1016/0005-7967(93)90105-4 · Cited by 672 · cited 1×
“Lexical diversity has been suggested to reflect the coherence of a writer's mental representation about an event or topic (Pennebaker 1993; Wade-Stein and Kintsch 2004).”
BackgroundImproving Students' Ability to Identify and Use Source Information
Britt, M. A., & Aglinskas, C. (2002) · Cognition and Instruction · doi:10.1207/s1532690xci2004_2 · Cited by 502 · cited 6×
“Although the significant relation between causality (CAUSAL) and learning (IVT) replicates prior work using hand coding for causal expressions (Britt and Aglinskas 2002; Voss and Table 7 Holistic explanation quality scores as predicted by similarity to idealized essay and source documents Variable…”
BackgroundMethodInfluentialSelf-citationReading Comprehension Strategies
(2007) · Psychology Press eBooks · doi:10.4324/9780203810033 · Cited by 491
Wiley, J., & Voss, J. F. (1999) · Journal of Educational Psychology · doi:10.1037//0022-0663.91.2.301 · Cited by 482 · cited 20×
“Negative correlations can be seen when essays include misconceptions (Hemmerich and Wiley 2002).”
BackgroundResultInfluentialSelf-citation
Research works printed but not linked: 3
- Shermis, M. D., & Hamner, B. (2012). Contrasting state-of-the-art automated scoring of essays: Analysis. In Annual national council on measurement in education meeting (pp. 14–16).
- Wiley, 1996, The effects of “playing” historian on learning in history, Applied Cognitive Psychology, 10, 63, 10.1002/(SICI)1099-0720(199611)10:7<63::AID-ACP438>3.0.CO;2-5 · doi:10.1002/(sici)1099-0720(199611)10:7<63::aid-acp438>3.0.co;2-5
- Wiley, 2011, Clarifying readers’ goals for learning from expository science texts, 353
Cited by
Artificial Intelligence for Student Assessment: A Systematic Review
González-Calatayud, V., Prendes-Espinosa, P., & Roig-Vila, R. (2021) · Applied Sciences · doi:10.3390/app11125467 · Cited by 503
“Three of the papers that do not use formative assessment focus on the use of AI in English language teaching [27,44,50], another two used AI in secondary education [43,47] and the other two in mathematics, but at university level [48,49].”
BackgroundApplying machine learning in science assessment: A systematic review
Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020) · Studies in Science Education · doi:10.1080/03057267.2020.1735757 · Cited by 257
Ferreira‐Mello, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1332 · Cited by 193
From substitution to redefinition: A framework of machine learning‐based science assessment
Zhaı, X., Haudek, K. C., Shi, L., Nehm, R. H., & Urban‐Lurain, M. (2020) · Journal of Research in Science Teaching · doi:10.1002/tea.21658 · Cited by 158
Zhaı, X., Shi, L., & Nehm, R. H. (2020) · Journal of Science Education and Technology · doi:10.1007/s10956-020-09875-z · Cited by 97
Metadata
- Authors
- Wiley, J., Hastings, P., Blaum, D., Jaeger, A. J., Hughes, S., Wallace, P., Griffin, T. D., & Britt, M. A.
- Affiliations
- University of Illinois Chicago (US)
- DePaul University (US)
- Northern Illinois University (US)
- Corresponding author
- Wiley, Jennifer
- ORCID
- Venue
- International Journal of Artificial Intelligence in Education
- Publisher
- Elsevier BV
- Published
- 2017-12
- Volume
- 27
- Issue
- 4
- Pages
- 758-790
- ISSN
- 1560-4292 (ISSN-L)
- OpenAlex
- W2586120904
- Cited by
- 49
- References
- 74
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 8 · 2023: 9 · 2022: 5 · 2021: 5 · 2020: 8 · 2019: 4 · 2018: 6
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor, 3071 d)
- Locations
- International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
- Figshare (submittedVersion, cc-by)
- Funders
- Institute of Education Sciences (Grant R305F100007, R305B070460)
- National Science Foundation (1535299)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Science Education and Pedagogy
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- student explanations, global warming, causal chains, student mental models, latent semantic analysis, Coh-Metrix, automated essay scoring, intelligent tutoring systems, science education, student understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science. https://app.cochat.ai/papers/10.1007/s40593-017-0138-z
- The paper
- Wiley, J., Hastings, P., Blaum, D., Jaeger, A. J., Hughes, S., Wallace, P., Griffin, T. D., & Britt, M. A. (2017). Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-017-0138-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.