Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol
Open accessAbstract
Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Three-Dimensional Learning Assessment Protocol is a published framework for evaluating assessments.
“Instructors may also wish to rely on published frameworks for evaluating assessments, such as the Three-Dimensional Learning Assessment Protocol (Laverty et al., 2016), when searching for and deciding on appropriate assessments to use.”
From Reading Primary Scientific Literature: Approaches for Teaching Students in the Undergraduate STEM Classroom (2023) · cites this paper as Laverty et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1999) · Cited by 28,356 · see citing works
(2001) · Cited by 13,930 · see citing works
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · see citing works
National Science Education Standards
Shea, J. H. (1995) · Journal of Geological Education · doi:10.5408/0022-1368-43.2.102 · Cited by 7,888 · see citing works
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,975 · see citing works
(2013) · Cited by 6,708 · see citing works
Hake, R. R. (1998) · American Journal of Physics · doi:10.1119/1.18809 · Cited by 6,072 · see citing works
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248 · see citing works
Hestenes, D., Wells, M., & Swackhamer, G. (1992) · The Physics Teacher · doi:10.1119/1.2343497 · Cited by 3,235 · see citing works
The National Science Education Standards
McComas, W. F. (2013) · SensePublishers eBooks · doi:10.1007/978-94-6209-497-0_58 · Cited by 2,885 · see citing works
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,858 · see citing works
(2000) · Cited by 2,779 · see citing works
Taking science to school: learning and teaching science in grades K-8
Duschl, R. A., & National Research Council (USA) (2008) · Choice Reviews Online · doi:10.5860/choice.45-3327 · Cited by 2,480 · see citing works
Knowing What Students Know: The Science and Design of Educational Assessment
Pellegrino, J. W., Chudowsky, N., & Glaser, R. A. (2013) · Cited by 2,416 · see citing works
Douglas, O. B., & Holland, B. F. (2015) · MacMillan Co eBooks · doi:10.1037/14910-020 · Cited by 2,283 · see citing works
A Nation at Risk: The Imperative for Educational Reform
Levin, H. M. (1984) · Journal of Policy Analysis and Management · doi:10.2307/3323945 · Cited by 1,326 · see citing works
A Framework for K-12 Science Education
National Research Council (2012) · National Academies Press eBooks · doi:10.17226/13165 · Cited by 1,230 · see citing works
Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L. O., Acher, A., Fortus, D., Shwartz, Y., Hug, B., & Krajcik, J. (2009) · Journal of Research in Science Teaching · doi:10.1002/tea.20311 · Cited by 1,159 · see citing works
Henderson, C., Beach, A., & Finkelstein, N. (2011) · Journal of Research in Science Teaching · doi:10.1002/tea.20439 · Cited by 1,055 · cited 1× · see citing works
“Furthermore, development of the 3D-LAP was part of a larger initiative to transform the introductory science courses in each discipline [53].”
MethodWindschitl, M., Thompson, J. J., & Braaten, M. (2008) · Science Education · doi:10.1002/sce.20259 · Cited by 929 · see citing works
Development and use of diagnostic tests to evaluate students’ misconceptions in science
Treagust, D. F. (1988) · International Journal of Science Education · doi:10.1080/0950069880100204 · Cited by 844 · see citing works
Novick, L. R. (2012) · PsycEXTRA Dataset · doi:10.1037/e502412013-211 · Cited by 691 · see citing works
Biology in Bloom: Implementing Bloom's Taxonomy to Enhance Student Learning in Biology
Crowe, A., Dirks, C., & Wenderoth, M. P. (2008) · CBE—Life Sciences Education · doi:10.1187/cbe.08-05-0024 · Cited by 629 · see citing works
Toward a modeling theory of physics instruction
Hestenes, D. (1987) · American Journal of Physics · doi:10.1119/1.15129 · Cited by 583 · see citing works
A BRIEF INTRODUCTION TO EVIDENCE‐CENTERED DESIGN
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003) · ETS Research Report Series · doi:10.1002/j.2333-8504.2003.tb01908.x · Cited by 537 · see citing works
Scientific argument and explanation: A necessary distinction?
Osborne, J. F., & Williams, A. P. (2011) · Science Education · doi:10.1002/sce.20438 · Cited by 381 · see citing works
Developing Assessments for the Next Generation Science Standards
National Research Council (2014) · National Academies Press eBooks · doi:10.17226/18409 · Cited by 353 · see citing works
Rubio, A. M. D. (2017) · Colombian Applied Linguistics Journal · doi:10.14483/calj.v19n1.11490 · Cited by 351 · see citing works
An Inventory for Alternate Conceptions among First-Semester General Chemistry Students
Mulford, D. R., & Robinson, W. R. (2002) · Journal of Chemical Education · doi:10.1021/ed079p739 · Cited by 349 · see citing works
Smith, M. K., Wood, W. B., & Knight, J. (2008) · CBE—Life Sciences Education · doi:10.1187/cbe.08-08-0045 · Cited by 290 · see citing works
Kober, N. (2015) · Cited by 282 · see citing works
Innovations in Teaching Undergraduate Biology and Why We Need Them
Wood, W. B. (2009) · Annual Review of Cell and Developmental Biology · doi:10.1146/annurev.cellbio.24.110707.175306 · Cited by 273 · see citing works
Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes
Stanger-Hall, K. F. (2012) · CBE—Life Sciences Education · doi:10.1187/cbe.11-11-0100 · Cited by 243 · see citing works
Formative assessment at the crossroads: Conformative, deformative and transformative assessment
Torrance, H. (2012) · Oxford Review of Education · doi:10.1080/03054985.2012.689693 · Cited by 217 · see citing works
A Brief Introduction to Evidence-Centered Design
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2004) · PsycEXTRA Dataset · doi:10.1037/e646212011-001 · Cited by 164 · see citing works
Challenge faculty to transform STEM learning
Cooper, M. M., Caballero, M. D., Ebert‐May, D., et al. (2015) · Science · doi:10.1126/science.aab0933 · Cited by 160 · see citing works
Self-citationAnderson, L. W., & Krathwohl, D. R. (2001) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 156 · see citing works
Application of Bloom's Taxonomy Debunks the "MCAT Myth"
Zheng, A. Y., Lawhorn, J. K., Lumley, T., & Freeman, S. (2008) · Science · doi:10.1126/science.1147852 · Cited by 146 · see citing works
What sort of science education do we really need?
Duggan, S., & Gott, R. (2002) · International Journal of Science Education · doi:10.1080/09500690110110133 · Cited by 131 · see citing works
Using Assessments to Investigate and Compare the Nature of Learning in Undergraduate Science Courses
Momsen, J. L., Offerdahl, E. G., Kryjevskaia, M., Montplaisir, L. M., Anderson, E., & Grosz, N. (2013) · CBE—Life Sciences Education · doi:10.1187/cbe.12-08-0130 · Cited by 116 · see citing works
Guide To Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments
Towns, M. H. (2014) · Journal of Chemical Education · doi:10.1021/ed500076x · Cited by 113 · see citing works
For whom is argument and explanation a necessary distinction? A response to Osborne and Patterson
Berland, L. K., & McNeill, K. L. (2012) · Science Education · doi:10.1002/sce.21000 · Cited by 111 · see citing works
McComas, W. F. (2013) · SensePublishers eBooks · doi:10.1007/978-94-6209-497-0_8 · Cited by 110 · see citing works
Questions for Assessing Higher-Order Cognitive Skills: It's Not Just Bloom’s
Lemons, P. P., & Lemons, J. D. (2013) · CBE—Life Sciences Education · doi:10.1187/cbe.12-03-0024 · Cited by 96 · see citing works
Plake, B. S., & Wise, L. L. (2014) · Educational Measurement Issues and Practice · doi:10.1111/emip.12045 · Cited by 93 · see citing works
Knight, J., Wise, S. B., Rentsch, J. D., & Furtak, E. M. (2015) · CBE—Life Sciences Education · doi:10.1187/cbe.15-04-0093 · Cited by 90 · see citing works
Science-based occupations and the science curriculum: Concepts of evidence
Aikenhead, G. S. (2004) · Science Education · doi:10.1002/sce.20046 · Cited by 75 · see citing works
Variables affecting physics achievement
Lawrenz, F. P., Wood, N. B., Kirchhoff, A., et al. (2009) · Journal of Research in Science Teaching · doi:10.1002/tea.20292 · Cited by 47 · see citing works
Osborne, J., & Patterson, A. D. (2012) · Science Education · doi:10.1002/sce.21034 · Cited by 27 · see citing works
Designing Assessment Tools To Measure Students’ Conceptual Knowledge of Chemistry
Bretz, S. L. (2014) · ACS symposium series · doi:10.1021/bk-2014-1166.ch009 · Cited by 19 · see citing works
Examining the Effects of Students' Classroom Expectations on Undergraduate Biology Course Reform
Hall, K. L. (2013) · University Libraries (University of Maryland) · Cited by 14 · see citing works
Point of View: The Next Generation Science Standards: Impact on College Science Teaching
McDonald, J. (2015) · The journal of college science teaching · doi:10.2505/4/jcst15_045_01_13 · Cited by 7 · see citing works
Engaging Physics Faculty in Course TransformationPreprint
<ns0:vcard>BEGIN:vCardVERSION:3.0FN:James T. LavertyTITLE:ORG:Kansas State UniversityADR;TYPE=WORK, P., Tessmer, S., <ns0:vcard>BEGIN:vCardVERSION:3.0FN:Melanie M. CooperTITLE:ORG:Michigan State UniversityADR;TYPE=WORK, P., et al. (2016) · doi:10.1119/perc.2014.pr.033 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [2]Committee on Prospering in the Global Economy of the 21st Century An Agenda for American Science Technology Rising Above the Gathering Storm. National Academies Press; 2007. (Energizing and Employing America for a Brighter Economic Future).
- [3]Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5 Washington, D.C.: National Academies Press; 2010. 25506949
- [18]McDonald J. The Next Generation Science Standards: Impact on College Science Teaching. J Coll Sci Teach. 2015 10 9; 45(1): 13– 4.
Cited by
Sorted by citations
Probing Internal Assumptions of the Revised Bloom’s Taxonomy
Larsen, T. M., Endo, B. H., Yee, A. T., Do, T., & Lo, S. M. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.20-08-0170 · Cited by 54 · see citing works
Goudsouzian, L. K., & Hsu, J. L. (2023) · CBE - Life Sciences Education · doi:10.1187/cbe.22-10-0211 · Cited by 26 · see citing works
“Instructors may also wish to rely on published frameworks for evaluating assessments, such as the Three-Dimensional Learning Assessment Protocol (Laverty et al., 2016), when searching for and deciding on appropriate assessments to use.”
Method
- Authors
- Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., Fata-Hartley, C. L., Ebert-May, D., Jardeleza, S. E., & Cooper, M. M.
- Affiliations
- Michigan State University (US)
- Corresponding author
- Laverty, James T.
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-09-08
- Volume
- 11
- Issue
- 9
- Pages
- e0162333
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5015839
- OpenAlex
- W2412226538
- Cited by
- 262
- References
- 54
- Citations by year
- 2026: 25 · 2025: 21 · 2024: 28 · 2023: 21 · 2022: 26 · 2021: 38 · 2020: 34 · 2019: 34 · 2018: 24 · 2017: 9 · 2016: 2
- FWCI
- 102.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- K-State Research Exchange (Kansas State University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- American Association of Universities
- Leona M. and Harry B. Helmsley Charitable Trust (US)
- Topics
- Social Sciences › Education › Science Education and Pedagogy
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- three-dimensional learning, science education reform, scientific and engineering practices, crosscutting concepts, disciplinary core ideas, assessment validity and reliability
- MeSH terms
- Biology, Chemistry, Curriculum, Educational Measurement, Engineering, Learning, Physics, Science, Universities, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
