Practice makes pretty good: Assessment of primary literature reading abilities across multiple large-enrollment biology laboratory courses
Open accessAbstract
Primary literature is essential for scientific communication and is commonly utilized in undergraduate biology education. Despite this, there is often little time spent training our students how to critically analyze a paper. To address this, we introduced a primary literature module in multiple upper-division laboratory courses. In this module, instructors conduct classroom discussions that dissect a paper as researchers do. While previous work has identified classroom interventions that improve primary literature comprehension within a single course, our goal was to determine whether including a scientific paper module in our classes could produce long-term benefits. On the basis of performance in an assessment exam, we found that our module resulted in longitudinal gains, including increased comprehension and critical-thinking abilities in subsequent lab courses. These learning gains were specific to courses utilizing our module, as no longitudinal gains were seen in students who had taken other upper-division labs that lacked extensive primary literature discussion. In addition, we assessed whether performance on our assessment correlated with a variety of factors, including grade point average, course performance, research background, and self-reported confidence in understanding of the article. Furthermore, all of the study conclusions are independent of biology disciplines, as we observe similar trends within each course.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many strategy papers rely on anecdotal evidence that undergraduates adopt superficial reading strategies and lack analytical skills, and make a tacit assumption that more experienced researchers read the literature correctly.
“Many of these ‘strategy’ papers rely on anecdotal evidence that undergraduates adopt superficial reading strategies and lack analytical skills [17], and make a tacit assumption that more experienced researchers read the literature ‘correctly’.”
From Perceptions of scientific research literature and strategies for reading papers depend on academic career stage (2017) · cites this paper as [17]
The Introduction (page unavailable) says that, based on personal communication and classroom discussions, the average student lacks the skill of critically reading scientific papers. It does not say that many strategy papers rely on anecdotal evidence, that students use superficial reading strategies, or that those papers assume experienced researchers read correctly. (LLM, conf=0.83)
Experimental — accuracy not yet measuredBenefits of reading PSL can persist in the long term, with students demonstrating improved scientific process skills and academic advantages in subsequent courses.
“These benefits can persist in the long term, with students demonstrating improved scientific process skills and academic advantages in subsequent courses (Kozeracki et al., 2006; Sato et al., 2014).”
From Reading Primary Scientific Literature: Approaches for Teaching Students in the Undergraduate STEM Classroom (2023) · cites this paper as Sato et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A revision of bloom's taxonomy: an overview
Krathwohl, D. R. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4104_2 · Cited by 8,132 · see citing works
Benefits of Undergraduate Research Experiences
Russell, S. H., Hancock, M. P., & McCullough, J. (2007) · Science · doi:10.1126/science.1140384 · Cited by 1,380 · see citing works
Seymour, E., Hunter, A., Laursen, S. L., & DeAntoni, T. (2004) · Science Education · doi:10.1002/sce.10131 · Cited by 1,332 · see citing works
Undergraduate research experiences support science career decisions and active learning
Lopatto, D. (2007) · CBE—Life Sciences Education · doi:10.1187/cbe.07-06-0039 · Cited by 1,009 · see citing works
Survey of undergraduate research experiences (SURE): First findings
Lopatto, D. (2004) · Cell Biology Education · doi:10.1187/cbe.04-07-0045 · Cited by 969 · see citing works
Biology in Bloom: Implementing Bloom's Taxonomy to Enhance Student Learning in Biology
Crowe, A., Dirks, C., & Wenderoth, M. P. (2008) · CBE—Life Sciences Education · doi:10.1187/cbe.08-05-0024 · Cited by 629 · see citing works
Academic Self-Efficacy as a Predictor of College Outcomes: Two Incremental Validity Studies
Gore, P. A. (2005) · Journal of Career Assessment · doi:10.1177/1069072705281367 · Cited by 465 · see citing works
Comparison between medical students' experience, confidence and competence
Morgan, P. J., & Cleave‐Hogg, D. M. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01228.x · Cited by 295 · see citing works
Just the Facts? Introductory Undergraduate Biology Courses Focus on Low-Level Cognitive Skills
Momsen, J. L., Long, T. M., Wyse, S. A., & Ebert‐May, D. (2010) · CBE—Life Sciences Education · doi:10.1187/cbe.10-01-0001 · Cited by 290 · see citing works
Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes
Stanger-Hall, K. F. (2012) · CBE—Life Sciences Education · doi:10.1187/cbe.11-11-0100 · Cited by 243 · see citing works
Selective Use of the Primary Literature Transforms the Classroom Into a Virtual Laboratory
Hoskins, S. G., Stevens, L. M., & Nehm, R. H. (2007) · Genetics · doi:10.1534/genetics.107.071183 · Cited by 220 · see citing works
Critical Thinking in the Management Classroom: Bloom's Taxonomy as a Learning Tool
Athanassiou, N. A., McNett, J. M., & Harvey, C. (2003) · Organizational Behavior Teaching Review · doi:10.1177/1052562903252515 · Cited by 215 · see citing works
Hoskins, S. G., Lopatto, D., & Stevens, L. M. (2011) · CBE—Life Sciences Education · doi:10.1187/cbe.11-03-0027 · Cited by 207 · see citing works
A New Method for Assessing Critical Thinking in the Classroom
Bissell, A. N., & Lemons, P. P. (2006) · BioScience · doi:10.1641/0006-3568(2006)056[0066:anmfac]2.0.co;2 · Cited by 191 · see citing works
Kozeracki, C. A., Carey, M. F., Colicelli, J., & Levis-Fitzgerald, M. (2006) · CBE—Life Sciences Education · doi:10.1187/cbe.06-02-0144 · Cited by 124 · see citing works
Gottesman, A. J., & Hoskins, S. G. (2013) · CBE—Life Sciences Education · doi:10.1187/cbe.12-11-0201 · Cited by 123 · cited 1× · see citing works
“…one of the most well-documented techniques to improve student primary literature reading abilities in biology education, with data showing that it is beneficial for students in various courses and at different educational levels (Hoskins et al. , 2007; Hoskins, 2008; Gottesman and Hoskins, 2013).”
DebBurman, S. K. (2002) · Cell Biology Education · doi:10.1187/cbe.02-07-0024 · Cited by 117 · see citing works
Luckie, D. B., Maleszewski, J. J., Loznak, S. D., & Krha, M. (2004) · AJP Advances in Physiology Education · doi:10.1152/advan.00025.2004 · Cited by 115 · see citing works
Round, J. E., & Campbell, A. M. (2013) · CBE—Life Sciences Education · doi:10.1187/cbe.11-07-0057 · Cited by 110 · cited 1× · see citing works
“CBE—Life Sciences Education 2013; Round and Campbell, 2013).”
Self-esteem and achievement motivation as determinants of students’ approaches to studying
Abouserie, R. (1995) · Studies in Higher Education · doi:10.1080/03075079512331381770 · Cited by 90 · see citing works
Critical thinking in undergraduate athletic training education.
Fuller, D. A. (1997) · PubMed · Cited by 50 · see citing works
Livengood, J. M. (1992) · Research in Higher Education · doi:10.1007/bf00973581 · Cited by 45 · see citing works
Clark, I. E., Romero‐Calderón, R., Olson, J. M., Jaworski, L., Lopatto, D. E., & Banerjee, U. K. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000264 · Cited by 44 · see citing works
(2008) · Europe PMC (PubMed Central) · Cited by 30 · see citing works
A Comparison of Pharmacy Students' Confidence and Test Performance
Valdez, C. A., Thompson, D., Ulrich, H., Bi, H., & Paulsen, S. M. (2006) · American Journal of Pharmaceutical Education · doi:10.5688/aj700476 · Cited by 29 · see citing works
Summer Workshop in Metagenomics: One Week Plus Eight Students Equals Gigabases of Cloned DNA
Rı́os-Velázquez, C., Williamson, L. L., Cloud-Hansen, K. A., Allen, H. K., McMahon, M. D., Sabree, Z. L., Donato, J. J., & Handelsman, J. (2011) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v12i2.177 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hubbard, K. E., & Dunbar, S. D. (2017) · PLoS ONE · doi:10.1371/journal.pone.0189753 · Cited by 108 · see citing works
La formación en investigación educativa de los futuros profesores
Perinés, H. (2020) · Formación universitaria · doi:10.4067/s0718-50062020000400139 · Cited by 57 · see citing works
Nelms, A. A., & Segura-Totten, M. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.18-05-0077 · Cited by 50 · see citing works
Annotated primary scientific literature: A pedagogical tool for undergraduate courses
Kararo, M. J., & McCartney, M. L. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000103 · Cited by 49 · see citing works
Lie, R., Abdullah, C., He, W., & Tour, E. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.15-09-0198 · Cited by 29 · see citing works
“…science literacy skills (Krontiris-Litowitz, 2013), ability to critically read research articles (Van Lacum et al., 2014), and science process/critical -thinking skills (Hoskins et al., 2007; Gottesman and Hoskins, 2013; Segura-Totten and Dalman, 2013; Sato et al., 2014; Abdullah et al., 2015).”
BackgroundSelf-citationGoudsouzian, L. K., & Hsu, J. L. (2023) · CBE - Life Sciences Education · doi:10.1187/cbe.22-10-0211 · Cited by 26 · see citing works
“The importance of reading PSL stems from the many cognitive and affective benefits that students gain when reading and engaging with it, including increased knowledge of scientific process skills and improved self-confidence in science (Brownell et al., 2013; Krontiris-Litowitz, 2013; Sato et al., 2014).”
BackgroundAnalyzing the Language of an Adapted Primary Literature Article
Ariely, M., Livnat, Z., & Yarden, A. (2019) · Science & Education · doi:10.1007/s11191-019-00033-5 · Cited by 24 · see citing works
Insights on biology student motivations and challenges when reading and analyzing primary literature
Howard, K. N., Stapleton, E. K., Nelms, A. A., Ryan, K. C., & Segura-Totten, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251275 · Cited by 23 · see citing works
The Grass Isn’t Always Greener: Perceptions of and Performance on Open-Note Exams
Sato, B. K., He, W., Warschauer, M., & Kadandale, P. (2015) · CBE—Life Sciences Education · doi:10.1187/cbe.14-08-0121 · Cited by 23 · see citing works
Self-citationUse of primary literature in the undergraduate analytical class
Kovarik, M. L. (2016) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-016-9467-2 · Cited by 23 · see citing works
Barnard-Brak, L. M., Stevens, T., & Ritter, W. A. (2017) · Learning and Individual Differences · doi:10.1016/j.lindif.2017.07.001 · Cited by 20 · see citing works
Undergraduate Science Students’ Attitudes Toward and Approaches to Scientific Reading and Writing
Verkade, H. M., & Lim, S. H. (2016) · The journal of college science teaching · doi:10.2505/4/jcst16_045_04_83 · Cited by 19 · see citing works
Lo, S. M., Luu, T. B., & Tran, J. (2020) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v21i1.1881 · Cited by 15 · see citing works
Brewing for Students: An Inquiry-Based Microbiology Lab
Sato, B. K., Alam, U., Dacanay, S. J., Lee, A. K., & Shaffer, J. F. (2015) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v16i2.914 · Cited by 15 · see citing works
Self-citationImplementing a Synthesis Tutorial to Improve Student Literature Reviews
Darowski, E. S., Patson, N. D., & Helder, E. (2016) · Behavioral & Social Sciences Librarian · doi:10.1080/01639269.2016.1243437 · Cited by 14 · see citing works
Vision & Change: Why It Matters
McLaughlin, J., & Metz, A. M. (2016) · The American Biology Teacher · doi:10.1525/abt.2016.78.6.456 · Cited by 13 · see citing works
Mini-review: CREATE-ive use of primary literature in the science classroom
Pugh-Bernard, A. E., & Kenyon, K. L. (2020) · Neuroscience Letters · doi:10.1016/j.neulet.2020.135532 · Cited by 13 · see citing works
Helix, M. R., Coté, L. E., Stachl, C. N., Linn, M. C., Stone, E. M., & Baranger, A. M. (2021) · Chemistry Education Research and Practice · doi:10.1039/d1rp00104c · Cited by 11 · see citing works
Kulkarni, A., & Vartak, R. (2019) · Biochemistry and Molecular Biology Education · doi:10.1002/bmb.21281 · Cited by 11 · see citing works
Chatzikyriakidou, K., & McCartney, M. (2022) · International Journal of Science Education · doi:10.1080/09500693.2022.2073482 · Cited by 10 · see citing works
Exploring Primary Scientific Literature through the Lens of the 5 Core Concepts of Biology
Chatzikyriakidou, K., Manrique, C., Tacloban, M. J., & McCartney, M. L. (2021) · CourseSource · doi:10.24918/cs.2021.5 · Cited by 10 · see citing works
Primary Literature in Undergraduate Science Courses: What are the Outcomes?
Sloane, J. D. (2021) · Journal of College Science Teaching · doi:10.1080/0047231x.2021.12290508 · Cited by 9 · see citing works
Research Competencies of Medical University Teachers: Evaluation, Perception, and Perspective
Koshmaganbetova, G. K., Kurmangaliyeva, S. S., Kashkinbayeva, A. R., Kurmangaliyev, K. B., & Alekenova, N. U. (2020) · Open Access Macedonian Journal of Medical Sciences · doi:10.3889/oamjms.2020.3954 · Cited by 9 · see citing works
Stengel, A., Stanke, K. M., McPherson, M. R., & Drijber, R. A. (2021) · Natural sciences education · doi:10.1002/nse2.20051 · Cited by 9 · see citing works
Just Figures: A Method to Introduce Students to Data Analysis One Figure at a Time
Massimelli, J., Denaro, K., Sato, B., Kadandale, P., & Boury, N. (2019) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe..v20i2.1690 · Cited by 8 · see citing works
Self-citationBeck, C. W. (2019) · CourseSource · doi:10.24918/cs.2019.25 · Cited by 8 · see citing works
Lee, S., Zhong, M., Foster, C., Segura-Totten, M., & McCartney, M. (2022) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe.00126-22 · Cited by 7 · see citing works
Chatzikyriakidou, K., Tacloban, M. J., Concepcion, K., & McCartney, M. L. (2022) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00059-22 · Cited by 7 · see citing works
O’Keeffe, G. W., & McCarthy, M. M. (2017) · PubMed Central · Cited by 7 · see citing works
The Literature Discussion: A Signature Pedagogy for Chemistry
Reisner, B. A., & Stewart, J. L. (2020) · ACS symposium series · doi:10.1021/bk-2020-1370.ch002 · Cited by 7 · see citing works
Wise, M. J. (2021) · Teaching and Teacher Education · doi:10.1016/j.tate.2021.103422 · Cited by 7 · see citing works
Teaching from the primary inorganic literature: lessons from Richard Andersen
Stewart, J. L., Bentley, A. K., Johnson, A. R., Nataro, C., Reisner, B. A., & Watson, L. A. (2018) · Dalton Transactions · doi:10.1039/c8dt02669f · Cited by 6 · see citing works
Spix, T., & Brasier, D. J. (2018) · Europe PMC (PubMed Central) · Cited by 6 · see citing works
Ferenc, J., Červenák, F., Birčák, E., Juríková, K., Goffová, I., Gorilák, P., Huraiová, B., Plavá, J., Demecsová, L., Ďuríková, N., et al. (2017) · Biochemistry and Molecular Biology Education · doi:10.1002/bmb.21084 · Cited by 6 · see citing works
Bjorn, G. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1257747 · Cited by 5 · see citing works
Using “Research Boxes” to Enhance Understanding of Primary Literature and the Process of Science
Carmichael, J. S., & Allison, L. A. (2019) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v20i2.1743 · Cited by 5 · see citing works
Shorbagi, S. H., & Ashok, A. (2016) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v17i3.1110 · Cited by 4 · see citing works
Jablonski, G., & Grinath, A. S. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.22-11-0218 · Cited by 3 · see citing works
Palavalli‐Nettimi, R., Fieldsend, T. W., Acosta, A., Catenazzi, A., & McCartney, M. L. (2022) · The American Biology Teacher · doi:10.1525/abt.2022.84.7.428 · Cited by 3 · see citing works
Using Primary Research Literature to Teach Critical Thinking in Pre-Service Teacher Education
Wei, W., Hudson, M., & Cook, A. E. (2018) · Advances in higher education and professional development book series · doi:10.4018/978-1-5225-7823-9.ch022 · Cited by 3 · see citing works
A Model for a Data Analysis-and Literature-Intensive Undergraduate Course
Resendes, K. K. (2022) · The journal of college science teaching · doi:10.1080/0047231x.2022.12290580 · Cited by 2 · see citing works
Méndez, Y. C. M., Listyg, B. S., Mardones-Segovia, C., Bowers, B., Cotto, K. M. C., Hilton, L., Koscik, I., Outlaw, B., Portner, S., Barekzi, N. A., et al. (2025) · CBE—Life Sciences Education · doi:10.1187/cbe.24-11-0271 · Cited by 2 · see citing works
CBE—Life Sciences Education: the story of a “great journal scientists might be caught reading”
Dolan, E. L. (2018) · Molecular Biology of the Cell · doi:10.1091/mbc.e18-06-0410 · Cited by 2 · see citing works
Explorations: a New Approach to Self-Directed Learning in a Competency-Based Curriculum
Keator, C. S., Dickinson, B. L., Lackey, W., Morris, A., Quesnelle, K. M., Riddle, D. R., Sheakley, M. L., Heuvel, G. B. V., & Vandré, D. D. (2016) · Medical Science Educator · doi:10.1007/s40670-016-0293-0 · Cited by 2 · see citing works
Introducing Immunology Research Literature to Understand B-cell Receptor Gene Expression
Morgan, D. E. (2022) · CourseSource · doi:10.24918/cs.2022.36 · Cited by 2 · see citing works
How Do Students Interact With the Primary Scientific Literature in an Undergraduate Science Program?
Yi, H., Verkade, H. M., & DRIEL, J. H. V. (2023) · The journal of college science teaching · doi:10.1080/0047231x.2023.12290701 · Cited by 1 · see citing works
Hippee, C. E., Klingelhutz, A. J., & Wu, L. J. (2024) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00184-24 · Cited by 1 · see citing works
Worthington, D., & Kelp, N. (2025) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00078-25 · Cited by 1 · see citing works
Integrating Research Across Biology to Strengthen Senior Capstone Research
Resendes, K. K., Balczon, J. M., Corrette-Bennett, J. C., Cornell, K. L., & Robertson, K. (2016) · Council on Undergraduate Research Quarterly · doi:10.18833/curq/37/1/6 · Cited by 1 · see citing works
Introducing a molecular basis to physiology in undergraduate medical education
Mukhopadhyay, A., Savitha, D., & Kurpad, A. V. (2018) · The National Medical Journal of India · doi:10.4103/0970-258x.258225 · Cited by 1 · see citing works
Romaní, F., & Gutiérrez, C. A. (2023) · Investigación en Educación Médica · doi:10.22201/fm.20075057e.2023.45.22455 · Cited by 1 · see citing works
Writing with Research: Understanding How Students Perceive Sources in the Sciences
Klucevsek, K. M. (2022) · Across the Disciplines · doi:10.37514/atd-j.2022.19.1-2.06
Peer-reviewed presentation exchange in an undergraduate classroom
Kiselyov, K. I., & Schunn, C. D. (2024) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00067-23
Nathan, D., & Moryossef, I. G. (2025) · Science Journal of Education · doi:10.11648/j.sjedu.20251303.12
Strategies for navigating primary scientific literature in undergraduate immunology courses
Bergerson, R. J., Basta, H. A., Coleman, S. T., & Pandey, S. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1606578
Korko, M. (2025) · Edukacja Zawodowa i Ustawiczna · doi:10.71358/ezu.2237
Developing critical thinking skills in engineering students through a primary literature exercise
Kays, J. C. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--58780
- Authors
- Sato, B. K., Kadandale, P., He, W., Murata, P. M. N., Latif, Y., & Warschauer, M.
- Affiliations
- University of California, Irvine (US)
- Corresponding author
- Sato, Brian K.
- Venue
- CBE—Life Sciences Education
- Published
- 2014-12
- Volume
- 13
- Issue
- 4
- Pages
- 677-686
- ISSN
- 1931-7913 (ISSN-L)
- PMC
- PMC4255354
- OpenAlex
- W2096215774
- Cited by
- 58
- References
- 26
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 3 · 2023: 4 · 2022: 7 · 2021: 6 · 2020: 6 · 2019: 7 · 2018: 5 · 2017: 4 · 2016: 8 · 2015: 2
- FWCI
- 23.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- CBE—Life Sciences Education (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- University of California, Irvine
- Topics
- Social Sciences › Education › Science Education and Pedagogy
- Social Sciences › Education › Innovative Teaching Methods
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Keywords
- critical thinking, classroom discussion, student self-confidence, research background, grade point average
- MeSH terms
- Biology, Communication, Curriculum, Educational Measurement, Periodicals as Topic, Reading, Regression Analysis, Universities, Models, Educational
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review