Comparing comprehension measured by multiple-choice and open-ended questions.
The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Abstract
This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Indeed, verifying and falsifying suggestions is a different cognitive task than independently gathering findings and deducing a diagnosis15,16.”
From Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays (2023) · cites this paper as [15]
Citations
Cites
(1998) · Cited by 4,875
The role of knowledge in discourse comprehension: A construction-integration model.
Kintsch, W. (1988) · Psychological Review · doi:10.1037/0033-295x.95.2.163 · Cited by 4,363
The Nature of Recollection and Familiarity: A Review of 30 Years of Research
Yonelinas, A. P. (2002) · Journal of Memory and Language · doi:10.1006/jmla.2002.2864 · Cited by 3,840 · cited 1×
“This is because the target information or its variant is present as a part of the answer options (see Yonelinas, 2002 for discussions on recollection and familiarity).”
BackgroundTheoretical Models and Processes of Reading
Barnett, M. A., Singer, H., & Ruddell, R. B. (1987) · The Modern Language Journal · doi:10.2307/326762 · Cited by 3,086
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867
Levels of processing versus transfer appropriate processing
Morris, C. D., Bransford, J. D., & Franks, J. J. (1977) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(77)80016-9 · Cited by 2,296 · cited 1×
“Further, note that the transfer appropriate processing perspective (Morris et al., 1977) states that memory retrieval is best when the nature of the encoding process corresponds to the nature of the retrieval process.”
BackgroundMcNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996) · Cognition and Instruction · doi:10.1207/s1532690xci1401_1 · Cited by 1,341
Self-citationEliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
Learning from texts: Effects of prior knowledge and text coherence
McNamara, D. S., & Kintsch, W. (1996) · Discourse Processes · doi:10.1080/01638539609544975 · Cited by 745 · cited 3×
“We presented the prior knowledge questions at the end of the experiment to avoid exposing participants to topic-related questions that could influence participants’ reading behaviour (see McNamara & Kintsch, 1996).”
BackgroundSelf-citationToward an Interactive Model of Reading
Rumelhart, D. (2013) · Theoretical Models and Processes of Reading · doi:10.1598/0710.29 · Cited by 628
SERT: Self-Explanation Reading Training
McNamara, D. S. (2004) · Discourse Processes · doi:10.1207/s15326950dp3801_1 · Cited by 519
Self-citationTest format and corrective feedback modify the effect of testing on long-term retention
Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440601056620 · Cited by 513
Role of the Reader's Schema in Comprehension, Learning, and Memory
Anderson, R. C. (2018) · Theoretical Models and Processes of Literacy · doi:10.4324/9781315110592-9 · Cited by 425
Amnesia and recognition memory: A re-analysis of psychometric data
Aggleton, J. P., & Shaw, C. (1996) · Neuropsychologia · doi:10.1016/0028-3932(95)00150-6 · Cited by 322 · cited 1×
“…researchers argue that answering forced-choice questions (e.g., multiplechoice) relies on familiarity more so than does answering yes–no questions (Aggleton & Shaw, 1996), others argue for similar contributions of familiarity across forced-choice and yes–no recognitions (Khoe, Kroll, Yonellinas,…”
BackgroundHow to Construct Achievement Tests to Assess Comprehension
Anderson, R. C. (1972) · Review of Educational Research · doi:10.2307/1170014 · Cited by 294
A schema-theoretic view of reading
Adams, M. J., & Collins, A. (1977) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 216
Improving Adolescent Students' Reading Comprehension with Istart
McNamara, D. S., O'Reilly, T. P., Best, R. M., & Ozuru, Y. (2006) · Journal of Educational Computing Research · doi:10.2190/1ru5-hdtj-a5c8-jvwe · Cited by 205
Self-citationBastin, C., & Van der Linden, M. (2003) · Neuropsychology · doi:10.1037/0894-4105.17.1.14 · Cited by 205
Learning to Read in American Schools: Basal Readers and Content Texts
Anderson, R. C., Osborn, J., & Tierney, R. J. (1984) · Medical Entomology and Zoology · Cited by 195
Assessing Reading Skill With a Think-Aloud Procedure and Latent Semantic Analysis
Magliano, J. P., & Millis, K. K. (2003) · Cognition and Instruction · doi:10.1207/s1532690xci2103_02 · Cited by 174 · cited 4×
“However, there are many differences between the Magliano and Millis (2003) study and the current study.”
BackgroundMethodResultInfluentialOzuru, Y., Best, R. M., Bell, C. M., Witherspoon, A. M., & McNamara, D. S. (2007) · Cognition and Instruction · doi:10.1080/07370000701632371 · Cited by 170
Self-citationBastin, C., & Van der Linden, M. (2003) · Neuropsychology · doi:10.1037//0894-4105.17.1.14 · Cited by 158
Transfer-appropriate processing (TAP)
Franks, J. J., Bilbrey, C., Lien, K. G., & McNamara, T. P. (2000) · Memory & Cognition · doi:10.3758/bf03211815 · Cited by 146
Information accretion and reduction in text processing: Inferences
Kintsch, W. (1993) · Discourse Processes · doi:10.1080/01638539309544837 · Cited by 143
Dissociating Automatic and Consciously Controlled Effects of Study/Test Compatibility
Jacoby, L. L. (1996) · Journal of Memory and Language · doi:10.1006/jmla.1996.0002 · Cited by 138
What is readily available during reading? A memory‐based view of text processing
Cook, A. E., Halleran, J. G., & O'Brien, E. J. (1998) · Discourse Processes · doi:10.1080/01638539809545041 · Cited by 129
Processing discourse roles in scripted narratives: The influences of context and world knowledge
Cook, A. E., & Myers, J. L. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2003.11.003 · Cited by 120 · cited 1×
“Some evidence suggests that readers engage in extensive semantic processing by default (e.g., Cook & Myers, 2004), and others suggest that readers do not tend to engage in active processing when reading difficult texts (Noordman, Vonk, & Kempff, 1992).”
BackgroundReaders' knowledge and the control of inferences in reading
Noordman, L. G. M., & Vonk, W. (1992) · Language and Cognitive Processes · doi:10.1080/01690969208409392 · Cited by 94 · cited 1×
“It is known that readers tend to process a text relatively passively without actively generating inferences when they read an unfamiliar text without any specific reading purpose (Noordman et al., 1992).”
BackgroundLeahey, T. H., & Harris, R. J. (1993) · Cited by 93
Khoe, W., Kroll, N. E. A., Yonelinas, A. P., Dobbins, I. G., & Knight, R. T. (2000) · Neuropsychologia · doi:10.1016/s0028-3932(00)00055-5 · Cited by 92
Graesser, A. C., Ozuru, Y., & Sullins, J. (2010) · Cited by 64
Self-citationWriting is Applied Metacognition
Hacker, D. J., Keener, M. C., & Kircher, J. C. (2009) · Routledge eBooks · doi:10.4324/9780203876428-13 · Cited by 64 · cited 1×
“McNamara and Magliano (2009) suggest that verbal protocols reflect relatively unaltered processing, whereas self-explanation changes comprehension during reading, which suggests that think-alouds and self-explanations induce and tap into different processes.”
BackgroundAnswering questions from oceanography texts: Learner, task, and text characteristics
Goldman, S. R., & Durán, R. P. (1988) · Discourse Processes · doi:10.1080/01638538809544710 · Cited by 58
Self-Explanation Reading Training: Effects for Low-Knowledge Readers
O’Reilly, T., Best, R. M., & McNamara, D. S. (2004) · Proceedings of the Annual Meeting of the Cognitive Science Society · Cited by 53
Self-citationThe effect of metacomprehension judgment task on comprehension monitoring and metacognitive accuracy
Ozuru, Y., Kurby, C. A., & McNamara, D. S. (2012) · Metacognition and Learning · doi:10.1007/s11409-012-9087-y · Cited by 43
Self-citationRetrieving text inferences: Controlled and automatic influences
Singer, M., & Remillard, G. (2004) · Memory & Cognition · doi:10.3758/bf03206314 · Cited by 29 · cited 1×
“…open-ended and multiple-choice versions of the same questions relies on different aspects of text comprehension because the cognitive processes involved in answering these questions reinstate different types of cognitive processes engaged at the time of reading the text (Singer & Remillard, 2004).”
Background
Cited by
Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2021) · Journal of Computers in Education · doi:10.1007/s40692-020-00169-2 · Cited by 1,081
Elgort, I., & Warren, P. (2014) · Language Learning · doi:10.1111/lang.12052 · Cited by 240
Connor Desai, S., & Reimers, S. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1066-z · Cited by 127
“This finding is particularly interesting in light of the fact that responses to open and closed questions might be supported by different underlying retrieval processes (Fisher, Brewer, & Mitchell, 2009; Ozuru et al., 2013; Shapiro, 2006).”
BackgroundStuijfzand, S., Creswell, C., Field, A. P., Pearcey, S., & Dodd, H. (2018) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12822 · Cited by 151
“These response formats require active and passive information generation, respectively, which may influence responses and subsequent conclusions regarding bias (Ozuru, Best, Bell, Witherspoon, & McNamara, 2007; Ozuru, Briner, Kurby, & McNamara, 2013).”
A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects
Dai, L., Jung, M. M., Postma, M., & Louwerse, M. M. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104607 · Cited by 128
Metadata
- Authors
- Ozuru, Y., Briner, S., Kurby, C. A., & McNamara, D. S.
- Affiliations
- University of Illinois Chicago (US)
- Grand Valley State University (US)
- Arizona State University (US)
- Venue
- Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale
- Published
- 2013-09
- Volume
- 67
- Issue
- 3
- Pages
- 215-227
- ISSN
- 1196-1961 (ISSN-L)
- 1878-7290
- OpenAlex
- W2202297923
- Cited by
- 134
- References
- 37
- Citations by year
- 2026: 14 · 2025: 7 · 2024: 12 · 2023: 12 · 2022: 17 · 2021: 18 · 2020: 10 · 2019: 13 · 2018: 10 · 2017: 7 · 2016: 5 · 2015: 6 · 2014: 3
- FWCI
- 9.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- text comprehension, multiple-choice questions, open-ended questions, self-explanation, prior knowledge, dual-process theory, reading comprehension
- MeSH terms
- Choice Behavior, Female, Humans, Language, Male, Surveys and Questionnaires, Reading, Mental Recall, Semantics, Statistics as Topic, Students, Universities, Vocabulary, Knowledge, Comprehension, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.