Civic Online Reasoning Across the Curriculum: Developing and Testing the Efficacy of Digital Literacy Lessons
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Given the current threat posed by toxic digital content, preparing students to evaluate online sources cannot be relegated to a single subject area—this instruction should happen across the curriculum. This article focuses on materials designed to teach students to evaluate online information across subject areas. ninth-grade biology and world geography teachers taught a series of curriculum-embedded lessons based on the following design principles: (a) Focus on a core question and strategy; (b) engage students in evaluating real online content; (c) feature cognitive apprenticeship and formative assessment; and (d) support teacher learning. We examine whether these lessons helped students become more skilled evaluators of online content. Pretest/posttest data (N =574) showed statistically significant growth in students’ ability to evaluate the credibility of online content. We analyze the role played by the curriculum design principles in this interdisciplinary intervention and explore implications for future initiatives.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(1978) · Medical Entomology and Zoology · Cited by 53,488
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152 · cited 1×
“Early in the year, they helped us tailor lessons to students’ strengths and needs (Black & Wiliam, 1998; Popham, 2006; Shepard, 2008).”
BackgroundHandbook of research on educational communications and technology
(2004) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2004.00409_15.x · Cited by 1,840
Designing Educative Curriculum Materials to Promote Teacher Learning
Davis, E. A., & Krajcik, J. S. (2005) · Educational Researcher · doi:10.3102/0013189x034003003 · Cited by 1,140 · cited 1×
“We designed the lesson plans to be educational for teachers because we recognized that they may need support to learn civic online reasoning themselves (Davis & Krajcik, 2005; McGrew, 2021).”
BackgroundModern robust statistical methods: An easy way to maximize the accuracy and power of your research.
Erceg-Hurn, D. M., & Mirosevich, V. M. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.591 · Cited by 959
How do users evaluate the credibility of Web sites?: a study with over 2,500 participants
Fogg, B. J., Soohoo, C., Danielson, D. R., Marable, L., Stanford, J., & Tauber, E. R. (2003) · Proceedings of the 2003 conference on Designing for user experiences · doi:10.1145/997078.997097 · Cited by 565 · cited 1×
“Many adults, including teachers, need help learning to effectively evaluate online information themselves (Fogg et al., 2003; Hargittai & Dobransky, 2017; McGrew, 2021).”
BackgroundLindle, J. C. (2006) · The SAGE Handbook for Research in Education · doi:10.4135/9781412976039.n13 · Cited by 551
Can Students Evaluate Online Sources? Learning From Assessments of Civic Online Reasoning
McGrew, S., Breakstone, J., Ortega, T., Smith, M., & Wineburg, S. (2018) · Theory & Research in Social Education · doi:10.1080/00933104.2017.1416320 · Cited by 533 · cited 3×
“Members of the research team scored the constructed-response items by using rubrics developed as part of prior research projects (McGrew et al., 2018).”
BackgroundMethodSelf-citationWineburg, S., & McGrew, S. (2019) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146811912101102 · Cited by 394 · cited 4×
“They quickly left the page and opened new browser tabs to investigate the source by reading what other reliable sources said about it (Wineburg & McGrew, 2019).”
BackgroundInfluentialSelf-citationOld Dogs, New Clicks: Digital Inequality in Skills and Uses among Older Adults
Hargittai, E., & Dobransky, K. (2017) · Canadian Journal of Communication · doi:10.22230/cjc.2017v42n2a3176 · Cited by 351 · cited 1×
“Many adults, including teachers, need help learning to effectively evaluate online information themselves (Fogg et al., 2003; Hargittai & Dobransky, 2017; McGrew, 2021).”
Students’ Civic Online Reasoning: A National Portrait
Breakstone, J., Smith, M., Wineburg, S., Rapaport, A., Carle, J., Garland, M., & Saavedra, A. (2021) · Educational Researcher · doi:10.3102/0013189x211017495 · Cited by 267
Self-citationEpistemic Thinking in Action: Evaluating and Integrating Online Sources
Barzilai, S., & Zohar, A. (2012) · Cognition and Instruction · doi:10.1080/07370008.2011.636495 · Cited by 239 · cited 3×
“(e.g., Barzilai & Zohar, 2012; Breakstone, Smith, Wineburg, et al., 2021; Coiro et al., 2015; McGrew et al., 2018).”
BackgroundChucking the Checklist: A Contextual Approach to Teaching Undergraduates Web-Site Evaluation
Meola, M. (2004) · portal: Libraries and the Academy · doi:10.1353/pla.2004.0055 · Cited by 200 · cited 1×
“Questions like these treat evaluation as a straightforward process of checking for specific features and adding up these credibility cues (Breakstone et al., 2018; Meola, 2004).”
BackgroundLateral reading: Reading less and learning more when evaluating digital information
Wineburg, S., & McGrew, S. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3048994 · Cited by 198
Self-citationLearning to evaluate: An intervention in civic online reasoning
McGrew, S. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103711 · Cited by 179 · cited 3×
“Although the length of the interventions varied, from a single 90-minute lesson (Kohnen et al., 2020) to a series of eight lessons (McGrew, 2020), student performance left substantial room for improvement.”
BackgroundSelf-citationLateral reading on the open Internet: A district-wide field study in high school government classes.
Wineburg, S., Breakstone, J., McGrew, S., Smith, M. D., & Ortega, T. (2022) · Journal of Educational Psychology · doi:10.1037/edu0000740 · Cited by 165 · cited 2×
“Wineburg et al. (2022) reported that after high school students completed six civic online reasoning lessons, their scores improved from an average of 2.9 points on pretest to 5.1 points on a posttest; however, the score was out of a possible 14 points.”
BackgroundInfluentialSelf-citationWhy we need a new approach to teaching digital literacy
Breakstone, J., McGrew, S., Smith, M., Ortega, T., & Wineburg, S. (2018) · Phi Delta Kappan · doi:10.1177/0031721718762419 · Cited by 134 · cited 4×
“Members of the research team scored the constructed-response items by using rubrics developed as part of prior research projects (McGrew et al., 2018).”
BackgroundMethodInfluentialSelf-citationDe La Paz, S., Monte‐Sano, C., Felton, M., Croninger, R., Jackson, C., & Piantedosi, K. W. (2017) · Reading Research Quarterly · doi:10.1002/rrq.147 · Cited by 132 · cited 1×
“Lessons that introduce evaluation strategies (e.g., lateral reading) incorporate cognitive modeling, which allows teachers to make expert cognitive processes visible to novice learners (Collins et al., 1991; De La Paz et al., 2016).”
BackgroundImproving university students’ web savvy: An intervention study
McGrew, S., Smith, M., Breakstone, J., Ortega, T., & Wineburg, S. (2019) · British Journal of Educational Psychology · doi:10.1111/bjep.12279 · Cited by 131 · cited 1×
“…2020; Breakstone, Smith, Connors, et al., 2021; Brodsky, Brooks, Scimeca, Galati, et al., 2021; Brodsky, Brooks, Scimeca, Todorova, et al., 2021; McGrew et al., 2019) have shown that college students improve in their approaches to evaluating online information, including reading laterally more…”
BackgroundSelf-citationInvestigating Criteria That Seventh Graders Use to Evaluate the Quality of Online Information
Coiro, J., Coscarelli, C., Maykel, C., & Forzani, E. (2015) · Journal of Adolescent & Adult Literacy · doi:10.1002/jaal.448 · Cited by 126 · cited 2×
“(e.g., Barzilai & Zohar, 2012; Breakstone, Smith, Wineburg, et al., 2021; Coiro et al., 2015; McGrew et al., 2018).”
BackgroundPérez, A., Potocki, A., Stadtler, M., Macedo-Rouet, M., Paul, J., Salmerón, L., & Rouet, J.-F. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.04.006 · Cited by 110 · cited 1×
“Instead, researchers have shown students printouts of websites (Walraven et al., 2013), adapted paper sources to make them appear web-like (Macedo-Rouet et al., 2019), and prompted students to “search for information” on a hard-copy version of a website (Pérez et al., 2018, p. 57).”
BackgroundBrodsky, J. E., Brooks, P. J., Scimeca, D., Todorova, R., Galati, P., Batson, M., Grosso, R., Matthews, M., Miller, V., & Caulfield, M. (2021) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-021-00291-4 · Cited by 93 · cited 4×
“Finally, lateral reading relies on students’ reading comprehension: To search for, read, and interpret clues about a site’s reliability, students need to be able to read and process information accurately (e.g., Brodsky, Brooks, Scimeca, Galati, et al., 2021; Leu et al., 2018).”
BackgroundInfluentialLateral reading: College students learn to critically evaluate internet sources in an online course.
Breakstone, J., Smith, M., Connors, P., Ortega, T., Kerr, D., & Wineburg, S. (2021) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-56 · Cited by 91 · cited 4×
“A series of studies (Addy, 2020; Breakstone, Smith, Connors, et al., 2021; Brodsky, Brooks, Scimeca, Galati, et al., 2021; Brodsky, Brooks, Scimeca, Todorova, et al., 2021; McGrew et al., 2019) have shown that college students improve in their approaches to evaluating online information, including…”
BackgroundInfluentialSelf-citationMacedo‐Rouet, M., Potocki, A., Scharrer, L., Ros, C., Stadtler, M., Salmerón, L., & Rouet, J. (2019) · Reading Research Quarterly · doi:10.1002/rrq.241 · Cited by 87
Fostering students’ evaluation behaviour while searching the internet
Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2013) · Instructional Science · doi:10.1007/s11251-012-9221-x · Cited by 82 · cited 1×
“Instead, researchers have shown students printouts of websites (Walraven et al., 2013), adapted paper sources to make them appear web-like (Macedo-Rouet et al., 2019), and prompted students to “search for information” on a hard-copy version of a website (Pérez et al., 2018, p. 57).”
BackgroundKohnen, A. M., Mertens, G. E., & Boehm, S. M. (2020) · Journal of Media Literacy Education · doi:10.23860/jmle-2020-12-2-6 · Cited by 64 · cited 5×
“Although the length of the interventions varied, from a single 90-minute lesson (Kohnen et al., 2020) to a series of eight lessons (McGrew, 2020), student performance left substantial room for improvement.”
BackgroundInfluentialBrodsky, J. E., Brooks, P. J., Scimeca, D., Galati, P., Todorova, R., & Caulfield, M. (2021) · AERA Open · doi:10.1177/23328584211038937 · Cited by 59
Breakstone, J., Smith, M. D., Ziv, N., & Wineburg, S. (2022) · The Journal of Higher Education · doi:10.1080/00221546.2022.2082783 · Cited by 56 · cited 2×
“They give undue weight to websites’ top-level domain, often believing that a .org or . edu site is preferable to a .com ( Breakstone, Smith, Wineburg, et al., 2021; Breakstone et al., 2022; Kohnen et al., 2020).”
BackgroundSelf-citationThe art of the real: fact checking as information literacy instruction
Addy, J. (2020) · Reference Services Review · doi:10.1108/rsr-09-2019-0067 · Cited by 50 · cited 1×
“A series of studies (Addy, 2020; Breakstone, Smith, Connors, et al., 2021; Brodsky, Brooks, Scimeca, Galati, et al., 2021; Brodsky, Brooks, Scimeca, Todorova, et al., 2021; McGrew et al., 2019) have shown that college students improve in their approaches to evaluating online information, including…”
BackgroundLeu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2018) · Routledge eBooks · doi:10.4324/9781315110592-19 · Cited by 42
Challenging approaches: Sharing and responding to weak digital heuristics in class discussions
McGrew, S. (2021) · Teaching and Teacher Education · doi:10.1016/j.tate.2021.103512 · Cited by 37 · cited 2×
“We designed the lesson plans to be educational for teachers because we recognized that they may need support to learn civic online reasoning themselves (Davis & Krajcik, 2005; McGrew, 2021).”
BackgroundSelf-citationInternet or Archive? Expertise in Searching for Digital Sources on a Contentious Historical Question
McGrew, S. (2021) · Cognition and Instruction · doi:10.1080/07370008.2021.1908288 · Cited by 21 · cited 3×
“We designed the lesson plans to be educational for teachers because we recognized that they may need support to learn civic online reasoning themselves (Davis & Krajcik, 2005; McGrew, 2021).”
BackgroundSelf-citation
Research works printed but not linked: 4
- Collins A., 1991, American Educator, 15, 6
- Collins A., 1989, Knowing, learning, and instruction: Essays in honor of Robert Glaser, 453
- Pavlounis D., 2021, The digital media literacy gap: How to build widespread resilience to false and misleading information using evidence-based classroom tools
- Shepard L. A., 2008, The future of assessment: Shaping teaching and learning, 279
Cited by
Teaching lateral reading: Interventions to help people read like fact checkers.
McGrew, S. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2023.101737 · Cited by 41
Self-citationUpdating Digital Citizenship Education for a Postdigital Society
Webster, J. (2023) · New Zealand Journal of Educational Studies · doi:10.1007/s40841-023-00305-3 · Cited by 20
“Accordingly, DCE is not confined to one single subject area and happens across the curriculum (McGrew & Breakstone, 2023).”
Understanding the impact of misinformation on adolescents
Ma, I., Sultan, M., Kozyreva, A., & van den Bos, W. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02338-8 · Cited by 24
“Their effectiveness has been demonstrated primarily as part of the Civic Online Reasoning curriculum with high school and college students in the USA 169–172 .”
McGrew, S., Reynolds, E. C., & Glass, A. C. (2024) · Cognition and Instruction · doi:10.1080/07370008.2024.2340981 · Cited by 19
“This requires skills like effective keyword searching and knowledge of sources that tend to be credible for lateral reading (McGrew & Breakstone, 2023; Walsh-Moorman & Hovick, 2021).”
BackgroundSelf-citationMarten, P. L., & Stadtler, M. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108548 · Cited by 22
Metadata
- Authors
- McGrew, S., & Breakstone, J.
- Affiliations
- University of Maryland, College Park (US)
- Stanford University (US)
- Corresponding author
- McGrew, Sarah
- Venue
- AERA Open
- Publisher
- SAGE Publications
- Published
- 2023-01
- Volume
- 9
- ISSN
- 2332-8584 (ISSN-L)
- OpenAlex
- W4379647578
- Cited by
- 49
- References
- 32
- Citations by year
- 2026: 14 · 2025: 24 · 2024: 8 · 2023: 3
- FWCI
- 23.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- AERA Open (publishedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Stanford Digital Repository
- Funders
- Brinson Foundation
- McCormick Foundation
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
- Social Sciences › Education › Literacy, Media, and Education
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Keywords
- digital literacy, evaluating online information, source credibility, interdisciplinary teaching, ninth grade, biology education, world geography education, cognitive apprenticeship, formative assessment, teacher learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.