Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments
Open accessAbstract
Life sciences faculty agree that developing scientific literacy is an integral part of undergraduate education and report that they teach these skills. However, few measures of scientific literacy are available to assess students’ proficiency in using scientific literacy skills to solve scenarios in and beyond the undergraduate biology classroom. In this paper, we describe the development, validation, and testing of the Test of Scientific Literacy Skills (TOSLS) in five general education biology classes at three undergraduate institutions. The test measures skills related to major aspects of scientific literacy: recognizing and analyzing the use of methods of inquiry that lead to scientific knowledge and the ability to organize, analyze, and interpret quantitative data and scientific information. Measures of validity included correspondence between items and scientific literacy goals of the National Research Council and Project 2061, findings from a survey of biology faculty, expert biology educator reviews, student interviews, and statistical analyses. Classroom testing contexts varied both in terms of student demographics and pedagogical approaches. We propose that biology instructors can use the TOSLS to evaluate their students’ proficiencies in using scientific literacy skills and to document the impacts of curricular reform on students’ scientific literacy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Test of Scientific Literacy Skills measures students' scientific literacy.
“the Test of Scientific Literacy Skills measures students’ scientific literacy (Gormally et al., 2012)”
From Reading Primary Scientific Literature: Approaches for Teaching Students in the Undergraduate STEM Classroom (2023) · cites this paper as Gormally et al., 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Willis, G. (2005) · doi:10.4135/9781412983655 · Cited by 1,778 · see citing works
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693 · see citing works
What We Say Is Not What We Do: Effective Evaluation of Faculty Professional Development Programs
Ebert-May, D., Derting, T. L., Hodder, J., Momsen, J. L., Long, T. M., & Jardeleza, S. E. (2011) · BioScience · doi:10.1525/bio.2011.61.7.9 · Cited by 420 · cited 2× · see citing works
“This dissonance between what many faculty say they do and classroom reality has been documented by others and may be indicative of such concerns as the need to cover content and lack of time or expertise to develop and incorporate opportunities for skill development (Coil et al., 2010; Andrews et al., 2011; Ebert-May…”
BackgroundTeaching the process of science: faculty perceptions and an effective methodology
Coil, D., Wenderoth, M. P., Cunningham, M., & Dirks, C. (2010) · CBE—Life Sciences Education · doi:10.1187/cbe.10-01-0005 · Cited by 282 · see citing works
Kuiper, E., Volman, M., & Terwel, J. (2005) · Review of Educational Research · doi:10.3102/00346543075003285 · Cited by 273 · see citing works
Hoskins, S. G. (2010) · The American Biology Teacher · doi:10.1525/abt.2010.72.7.5 · Cited by 21 · see citing works
Flagged first, then by citations
Research works printed but not linked: 23
- American Association for the Advancement of Science (AAAS) Science for All Americans 1990 New York Oxford University Press
- AAAS Benchmarks for Science Literacy 1993 New York Oxford University Press
- Andrews TM Leonard MJ Colgrove CA Kalinowski ST Active learning NOT associated with student learning in a random sample of college biology courses CBE Life Sci Educ 2011 10 394 405 22135373 PMC3228657
- Bialek W Botstein D Introductory science and mathematics education for 21st-century biologists Science 2004 303 788 790 14764865
- Bray Speth E Momsen JL Moyerbrailean GA Ebert-May D Long TM Wyse S Linton D 1, 2, 3, 4: infusing quantitative literacy into introductory biology CBE Life Sci Educ 2010 9 323 332 20810965 PMC2931680
- Bybee RW Reforming Science Education: Social Perspectives and Personal Reflections 1993 New York Teachers College Press
- Colon-Berlingeri M Borrowes PA Teaching biology through statistics: application of statistical methods in genetics and zoology courses CBE Life Sci Educ 2011 10 259 267 21885822 PMC3164565
- Ebel R Measuring Educational Achievement 1965 Englewood Cliffs, NJ Prentice Hall
- Facione PA Using the California Critical Thinking Skills Test in Research, Evaluation, and Assessment 1991 Millbrae, CA California Academic Press
- Garvin-Doxas K Klymkowsky MW Understanding randomness and its impact on student learning: lessons learned from building the Biology Concept Inventory (BCI) CBE Life Sci Educ 2008 7 227 233 18519614 PMC2424310
- Gross LJ Interdisciplinarity and the undergraduate biology curriculum: finding a balance Cell Biol Educ 2004 3 85 87 15257334 PMC437648
- Labov JB The challenges and opportunities for improving undergraduate science education through introductory courses Cell Biol Educ 2004 3 212 214 15592592 PMC533118
- Marsteller P de Pillis L Findley A Joplin K Delesko J Nelson K Thompson K Toward integration: from quantitative biology to mathbio-biomath? CBE Life Sci Educ 2010 9 165 171 20810946 PMC2931661
- Meinwald J Hildebrand JG Science and the Educated American: A Core Component of Liberal Education 2010 Cambridge, MA American Academy of Arts and Sciences
- National Research Council (NRC) National Science Education Standards 1996 Washington, DC National Academies Press
- NRC BIO2010: Transforming Undergraduate Education for Future Research Biologists 2003 Washington, DC National Academies Press 20669482
- Osborne J Arguing to learn in science: the role of collaborative, critical discourse Science 2010 328 463 466 20413492
- Osterlind SJ Modern Measurement: Theory, Principles, and Applications of Mental Appraisal 2010 2nd ed. Upper Saddle River, NJ Pearson
- Quitadamo IJ Faiola CL Johnson JE Kurtz MJ Community-based inquiry improves critical thinking in general education biology CBE Life Sci Edu 2008 7 327 337 PMC2527977 18765755
- Semsar K Knight JK Birol G Smith MK The Colorado Learning Attitudes about Science Survey (CLASS) for use in biology CBE Life Sci Educ 2011 10 268 278 21885823 PMC3164566
- Shi J Wood WB Martin JM Guild NA Vicens Q Knight JK A diagnostic assessment for introductory molecular and cell biology CBE Life Sci Educ 2010 9 453 461 21123692 PMC2995763
- Smith MK Wood WB Knight JK The Genetics Concept Assessment: a new concept inventory for gauging student understanding of genetics CBE Life Sci Educ 2008 7 422 430 19047428 PMC2592048
- Tanner K Allen D Approaches to biology teaching and learning: understanding the wrong answers—teaching toward conceptual change Cell Biol Educ 2005 4 112 117 15917868 PMC1103711
Cited by
Sorted by citations
Nelms, A. A., & Segura-Totten, M. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.18-05-0077 · Cited by 50 · see citing works
Kenyon, K. L., Onorato, M. E., Gottesman, A. J., Hoque, J., & Hoskins, S. G. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.15-07-0146 · Cited by 32 · see citing works
Goudsouzian, L. K., & Hsu, J. L. (2023) · CBE - Life Sciences Education · doi:10.1187/cbe.22-10-0211 · Cited by 26 · see citing works
“For instance, the Test of Scientific Literacy Skills measures students’ scientific literacy (Gormally et al., 2012), while there are separate instruments focusing specifically on STEM students’ critical thinking (e.”
BackgroundInsights on biology student motivations and challenges when reading and analyzing primary literature
Howard, K. N., Stapleton, E. K., Nelms, A. A., Ryan, K. C., & Segura-Totten, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251275 · Cited by 23 · see citing works
- Authors
- Gormally, C., Brickman, P., & Lutz, M.
- Affiliations
- Georgia Institute of Technology (US)
- University of Georgia (US)
- Corresponding author
- Gormally, Cara
- Brickman, Peggy
- ORCID
- Brickman, Peggy
- Venue
- CBE—Life Sciences Education
- Published
- 2012-12
- Volume
- 11
- Issue
- 4
- Pages
- 364-377
- ISSN
- 1931-7913 (ISSN-L)
- PMC
- PMC3516792
- OpenAlex
- W2148272413
- Cited by
- 437
- References
- 7
- Citations by year
- 2026: 15 · 2025: 32 · 2024: 52 · 2023: 75 · 2022: 45 · 2021: 59 · 2020: 37 · 2019: 31 · 2018: 26 · 2017: 16 · 2016: 19 · 2015: 12 · 2014: 14 · 2013: 3
- FWCI
- 45.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- CBE—Life Sciences Education (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- University of Georgia
- Topics
- Social Sciences › Education › Science Education and Pedagogy
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Evolution and Science Education
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Keywords
- scientific literacy, scale development, undergraduate biology education, methods of inquiry, quantitative data interpretation, Project 2061, curriculum reform, student assessment
- MeSH terms
- Data Collection, Demography, Educational Measurement, Faculty, Female, Humans, Learning, Male, Science, Students, Reproducibility of Results, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
