The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study
Abstract
BACKGROUND: Worldwide efforts are under way to improve the quality of clinical practice. Most quality measurements, however, are poorly validated, expensive, and difficult to compare among sites. OBJECTIVE: To validate whether vignettes accurately measure the quality of clinical practice by using a comparison with standardized patients (the gold standard method), and to determine whether vignettes are a more or less accurate method than medical record abstraction. DESIGN: Prospective, multisite study. SETTING: Outpatient primary care clinics in 2 Veterans Affairs medical centers and 2 large, private medical centers. PARTICIPANTS: 144 of 163 eligible physicians agreed to participate, and, of these, 116 were randomly selected to see standardized patients, to complete vignettes, or both. MEASUREMENTS: Scores, expressed as the percentage of explicit quality criteria correctly completed, were obtained by using 3 methods. RESULTS: Among all physicians, the quality of clinical practice as measured by the standardized patients was 73% correct (95% CI, 72.1% to 73.4%). By using exactly the same criteria, physicians scored 68% (CI, 67.9% to 68.9%) when measured by the vignettes but only 63% (CI, 62.7% to 64.0%) when assessed by medical record abstraction. These findings were consistent across all diseases and were independent of case complexity or physician training level. Vignettes also accurately measured unnecessary care. Finally, vignettes seem to capture the range in the quality of clinical practice among physicians within a site. LIMITATIONS: Despite finding variation in the quality of clinical practice, we did not determine whether poorer quality translated into worse health status for patients. In addition, the quality scores are based on measurements from 1 patient-provider interaction. As with all other scoring criteria, vignette criteria must be regularly updated. CONCLUSIONS: Vignettes are a valid tool for measuring the quality of clinical practice. They can be used for diverse clinical settings, diseases, physician types, and situations in which case-mix variation is a concern. They are inexpensive and easy to use. Vignettes are particularly useful for comparing quality among and within sites and may be useful for longitudinal evaluations of interventions intended to change clinical practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,736
The quality of health care delivered to adults in the United States
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa022615 · Cited by 5,079
Peabody, J. W., Luck, J., Glassman, P., Dresselhaus, T. R., & Lee, M. (2000) · JAMA · doi:10.1001/jama.283.13.1715 · Cited by 1,241
Self-citationThe application of vignettes in social and nursing research
Hughes, R., & Huby, M. (2002) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2002.02100.x · Cited by 701
Quality of hospital care for seriously ill children in less-developed countries
Nolan, T., Angos, P., Cunha, A. J., Muhe, L., Qazi, S., Simoes, E. A., Tamburlini, G., Weber, M., & Pierce, N. F. (2001) · The Lancet · doi:10.1016/s0140-6736(00)03542-x · Cited by 397
Luck, J., Peabody, J. W., Dresselhaus, T. R., Lee, M., & Glassman, P. (2000) · The American Journal of Medicine · doi:10.1016/s0002-9343(00)00363-6 · Cited by 335
Self-citationUsing vignettes to collect data for nursing research studies: how valid are the findings?
GOULD, D. (1996) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.1996.tb00253.x · Cited by 290
Physician and Coding Errors in Patient Records
Lloyd, S. S. (1985) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1985.03360100080018 · Cited by 270
Assessing the Accuracy of Administrative Data in Health Information Systems
Peabody, J. W., Luck, J., Jain, S., Bertenthal, D., & Glassman, P. (2004) · Medical Care · doi:10.1097/00005650-200411000-00005 · Cited by 254
Self-citationRethans, J. J., Sturmans, F., Drop, R., van der Vleuten, C., & Hobus, P. (1991) · BMJ · doi:10.1136/bmj.303.6814.1377 · Cited by 201
Cited by
Cognitive biases associated with medical decisions: a systematic review
Saposnik, G., Redelmeier, D., Ruff, C. C., & Tobler, P. N. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0377-1 · Cited by 1,001
Evans, S. C., Roberts, M. C., Keeley, J. W., Blossom, J. B., Amaro, C. M., Garcia, A. M., Stough, C. O., Canter, K. S., Robles, R., & Reed, G. M. (2015) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2014.12.001 · Cited by 757
Improving the accuracy of medical diagnosis with causal machine learning
Richens, J. G., Lee, C. M., & Johri, S. (2020) · Nature Communications · doi:10.1038/s41467-020-17419-7 · Cited by 598
Sabin, J. A., & Greenwald, A. G. (2012) · American Journal of Public Health · doi:10.2105/ajph.2011.300621 · Cited by 525
Clinical Vignette-Based Surveys: A Tool for Assessing Physician Practice Variation
Veloski, J., Tai, S., Evans, A. S., & Nash, D. B. (2005) · American Journal of Medical Quality · doi:10.1177/1062860605274520 · Cited by 381
Metadata
- Authors
- Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., & Lee, M.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2004-11-16
- Volume
- 141
- Issue
- 10
- Pages
- 771-780
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2165974787
- Cited by
- 714
- References
- 44
- FWCI
- 32.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study. https://app.cochat.ai/papers/10.7326/0003-4819-141-10-200411160-00008
- The paper
- Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., & Lee, M. (2004). Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-141-10-200411160-00008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.