The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Consequences Validity Evidence: Evaluating the Impact of Educational Assessments
Abstract
Because tests that do not alter management (i.e., influence decisions and actions) should not be performed, data on the consequences of assessment constitute a critical source of validity evidence. Consequences validity evidence is challenging for many educators to understand, perhaps because it has no counterpart in the older framework of content, criterion, and construct validity. The authors’ purpose is to explain consequences validity evidence and propose a framework for organizing its collection and interpretation. Both clinical and educational assessments can be viewed as interventions. The act of administering or taking a test, the interpretation of scores, and the ensuing decisions and actions influence those being assessed (e.g., patients or students) and other people and systems (e.g., physicians, teachers, hospitals, schools). Consequences validity evidence examines such impacts of assessments. Despite its importance, consequences evidence is reported infrequently in health professions education (range 5%–20% of studies in recent systematic reviews) and is typically limited in scope and rigor. Consequences validity evidence can derive from evaluations of the impact on examinees, educators, schools, or the end target of practice (e.g., patients or health care systems); and the downstream impact of classifications (e.g., different score cut points and labels). Impact can result from the uses of scores or from the assessment activity itself, and can be intended or unintended and beneficial or harmful. Both quantitative and qualitative research methods are useful. The type, quantity, and rigor of consequences evidence required will vary depending on the assessment and the claims for its use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Motivational processes affecting learning.
Dweck, C. S. (1986) · American Psychologist · doi:10.1037/0003-066x.41.10.1040 · Cited by 6,971
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application
Cook, D. A., & Beckman, T. J. (2006) · The American Journal of Medicine · doi:10.1016/j.amjmed.2005.10.036 · Cited by 1,745
Self-citationScreening for Breast Cancer: U.S. Preventive Services Task Force Recommendation Statement
U.S. Preventive Services Task Force* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-10-200911170-00008 · Cited by 1,686
Validity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434
Screening for Breast Cancer: An Update for the U.S. Preventive Services Task Force
Nelson, H. D., Tyne, K., Naik, A., Bougatsos, C., Chan, B. K., & Humphrey, L. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-10-200911170-00009 · Cited by 1,248
Colorectal-Cancer Incidence and Mortality with Screening Flexible Sigmoidoscopy
Schoen, R. E., Pinsky, P. F., Weissfeld, J. L., Yokochi, L. A., Church, T., Laiyemo, A. O., Bresalier, R., Andriole, G. L., Buys, S. S., Crawford, E. D., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1114635 · Cited by 1,021
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 598
A contemporary approach to validity arguments: a practical guide to Kane's framework
Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. (2015) · Medical Education · doi:10.1111/medu.12678 · Cited by 554
Self-citationHubbard, R. A., Kerlikowske, K., Flowers, C. I., Yankaskas, B. C., Zhu, W., & Miglioretti, D. L. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00004 · Cited by 414
Cited by
Validation of educational assessments: a primer for simulation and beyond
Cook, D. A., & Hatala, R. (2016) · Advances in Simulation · doi:10.1186/s41077-016-0033-y · Cited by 304
Self-citationLörwald, A. C., Lahner, F.-M., Nouns, Z. M., Berendonk, C., Norcini, J., Greif, R., & Huwendiek, S. (2018) · PLOS ONE · doi:10.1371/journal.pone.0198009 · Cited by 117
The impact of large scale licensing examinations in highly developed countries: a systematic review
Archer, J., Lynn, N., Coombes, L., Roberts, M., Gale, T., Price, T., & Regan de Bere, S. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0729-7 · Cited by 42
Song, X., Cleaves, E., Gluzman, E., Kotlyar, B., Russo, R. A., Schilling, D. C., Tsao, C. P., & West, J. C. (2025) · Academic Psychiatry · doi:10.1007/s40596-025-02136-4 · Cited by 6
Grace Chen, C. C., Bar-El, L., Long, J. B., Frost, A., King, C. R., Katz, A., Moawad, N. S., Levy, B., Wu, H., & Lerner, V. (2026) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2026.02.040
Metadata
- Authors
- Cook, D. A., & Lineberry, M.
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-06
- Volume
- 91
- Issue
- 6
- Pages
- 785-795
- ISSN
- 1040-2446
- Cited by
- 129
- References
- 50
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Consequences Validity Evidence: Evaluating the Impact of Educational Assessments. https://app.cochat.ai/papers/10.1097/acm.0000000000001114
- The paper
- Cook, D. A., & Lineberry, M. (2016). Consequences Validity Evidence: Evaluating the Impact of Educational Assessments. Academic Medicine. https://doi.org/10.1097/acm.0000000000001114
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
