The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Vignette methodologies for studying clinicians’ decision-making: Validity, utility, and application in ICD-11 field studies
Open accessAbstract
Vignette-based methodologies are frequently used to examine judgments and decision-making processes, including clinical judgments made by health professionals. Concerns are sometimes raised that vignettes do not accurately reflect “real world” phenomena, and that this affects the validity of results and conclusions of these studies. This article provides an overview of the defining features, design variations, strengths, and weaknesses of vignette studies as a way of examining how health professionals form clinical judgments (e.g., assigning diagnoses, selecting treatments). As a “hybrid” of traditional survey and experimental methods, vignette studies can offer aspects of both the high internal validity of experiments and the high external validity of survey research in order to disentangle multiple predictors of clinician behavior. When vignette studies are well designed to test specific questions about judgments and decision-making, they can be highly generalizable to “real life” behavior, while overcoming the ethical, practical, and scientific limitations associated with alternative methods (e.g., observation, self-report, standardized patients, archival analysis). We conclude with methodological recommendations and a description of how vignette methodologies are being used to investigate clinicians’ diagnostic decisions in case-controlled field studies for the ICD-11 classification of mental and behavioural disorders, and how these studies illustrate the preceding concepts and recommendations Las metodologías basadas en viñetas se utilizan frecuentemente para examinar los procesos de toma de decisiones, incluyendo los de profesionales sanitarios. No obstante, existen dudas sobre si las viñetas reflejan adecuadamente los fenómenos del “mundo real” permitiendo resultados y conclusiones válidas. Ofrecemos una visión de las características, variaciones de diseño, fortalezas y debilidades de estos estudios para examinar cómo los profesionales forman juicios clínicos (como el diagnóstico y tratamiento). Siendo “híbridos” de las encuestas tradicionales y los métodos experimentales, estos estudios pueden ofrecer la alta validez interna de los experimentos y la alta validez externa de las encuestas, para aislar múltiples factores predictivos del comportamiento de los clínicos. Un diseño adecuado para poner a prueba preguntas específicas acerca de los juicios y la toma de decisiones permite resultados altamente generalizables a la “vida real”, sin las limitaciones éticas, prácticas y científicas de los métodos alternativos (observación, auto-informe, pacientes estandarizados, análisis de archivos clínicos). Concluimos con recomendaciones metodológicas que se ilustran tras una descripción del uso de las metodologías de viñetas para investigar las decisiones diagnósticas de los clínicos en los estudios de campo de casos y controles para la clasificación de los trastornos mentales y del comportamiento en la CIE-11
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evans et al. (2015) reviewed the limitations of using vignettes in research and methods for facilitating external validity of research findings.
“They offered a thorough review of the limitations for the use of vignettes in research, as well as methods for facilitating external validity of the research findings.”
From Examining Counselor Trainees’ Clinical Impressions of Pornography Use: Implications for Couple and Family Counselors (2026) · cites this paper as Evans et al. (2015)
When properly addressed, such limitations can be minimized.
“However, when properly addressed, such limitations can be minimized (Evans et al., 2015).”
From Examining Counselor Trainees’ Clinical Impressions of Pornography Use: Implications for Couple and Family Counselors (2026) · cites this paper as Evans et al., 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnosis and classification of disorders specifically associated with stress: proposals for ICD-11
Maercker, A., Brewin, C. R., Bryant, R. A., Cloitre, M., van Ommeren, M., Jones, L. M., Humayan, A., Kagee, A., Llosa, A. E., Rousseau, C., et al. (2013) · World Psychiatry · doi:10.1002/wps.20057 · Cited by 762
Self-citationBarter, C., & Renold, E. (2000) · International Journal of Social Research Methodology · doi:10.1080/13645570050178594 · Cited by 543
International Advisory Group for the Revision of ICD‐10 Mental and Behavioural Disorders (2011) · World Psychiatry · doi:10.1002/j.2051-5545.2011.tb00022.x · Cited by 300
Keeley, J. W., Reed, G. M., Roberts, M. C., Evans, S. C., Medina-Mora, M. E., Robles, R., Rebello, T., Sharan, P., Gureje, O., First, M. B., et al. (2016) · American Psychologist · doi:10.1037/a0039972 · Cited by 173
Self-citation
Flagged first, then by citations
Research works printed but not linked: 56
- Alexander, 1978, The use of vignettes in survey research, Public Opinion Quarterly, 42, 93, 10.1086/268432 · DOI · Paper page
- Atzmüller, 2010, Experimental vignette studies in survey research, Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 6, 128, 10.1027/1614-2241/a000014 · DOI
- Bachmann, 2008, Vignette studies of medical choice and judgment to study caregivers’ medical decision behavior: systematic review, BMC Research Methodology, 8
- Baudson, 2013, Teachers’ implicit personality theories about the gifted: An experimental approach, School Psychology Quarterly, 28, 37, 10.1037/spq0000011 · DOI
- Cullen, 2010, Survey-driven romanticism, Review of Philosophy and Psychology, 1, 275, 10.1007/s13164-009-0016-1 · DOI · Paper page
- Evans, 2013, Psychologists’ perspectives on the diagnostic classification of mental disorders: Results from the WHO-IUPsyS Global Survey, International Journal of Psychology, 48, 177, 10.1080/00207594.2013.804189 · DOI
- Faia, 1980, The vagaries of the vignette world: A comment on Alves and Rossi, American Journal of Sociology, 85, 951, 10.1086/227096 · DOI
- Flaskerud, 1979, Use of vignettes to elicit responses toward broad concepts, Nursing Research, 28, 210, 10.1097/00006199-197907000-00004 · DOI
- Ganong, 2006, Multiple segment factorial vignette designs, Journal of Marriage and Family, 68, 455, 10.1111/j.1741-3737.2006.00264.x · DOI
- Gould, 1996, Using vignettes to collect data for nursing research studies: How valid are the findings?, Journal of Clinical Nursing, 5, 207, 10.1111/j.1365-2702.1996.tb00253.x · DOI
- Grøndal, 2009, A comparative case vignette study of decision making in forensic psychiatric cases, International Journal of Forensic Mental Health, 8, 263, 10.1080/14999011003635639 · DOI
- Grønhøj, 2010, Using vignettes to study family consumption processes, Psychology & Marketing, 27, 445, 10.1002/mar.20338 · DOI
- Herskovitz, 1950, The hypothetical situation: A technique of field research, Southwestern Journal of Anthropology, 6, 32, 10.1086/soutjanth.6.1.3628688 · DOI
- Howie, 2012, Children's event reports: Factors affecting responses to repeated questions in vignette scenarios and event recall interviews, British Journal of Developmental Psychology, 30, 550, 10.1111/j.2044-835X.2011.02064.x · DOI
- Hughes, 1998, Considering the vignette technique and its application to a study of drug injecting and HIV risk and safer behaviour, Sociology of Health & Illness, 20, 381, 10.1111/1467-9566.00107 · DOI · Paper page
- Hughes, 2001, The application of vignettes in social and nursing research, Methodological Issues in Nursing Research, 37, 382
- Jenkins, 2010, Putting it in context: The use of vignettes in qualitative interviewing, Qualitative Research, 10, 175, 10.1177/1468794109356737 · DOI
- Jones, 1973, Attribution of fault to a rape victim as a function of respectability of the victim, Journal of Personality and Social Psychology, 26, 415, 10.1037/h0034463 · DOI
- Kim, 2012, Scenarios in information seeking and information retrieval research: A methodological application and discussion, Library & Information Science Research, 34, 300, 10.1016/j.lisr.2012.04.002 · DOI · Paper page
- Kirwan, 1983, Clinical judgment in rheumatoid arthritis. I. Rheumatologists’ opinions and the development of ‘paper patients’, Annals of the Rheumatic Diseases, 42, 644, 10.1136/ard.42.6.644 · DOI
- Langley, 1991, Use of written cases to study factors associated with regional variations in referral rates, Journal of Clinical Epidemiology, 44, 391, 10.1016/0895-4356(91)90077-M · DOI
- Lauder, 2002, Factorial survey methods: A valuable but under-utilised research method in nursing research?, Nursing Times Research, 7, 35, 10.1177/136140960200700106 · DOI
- Levy Cole, 1996, Children's perceptions of ill peers: Effects of disease, grade, and impact variables, Children's Health Care, 25, 107, 10.1207/s15326888chc2502_3 · DOI
- Ludwick, 2004, An improved methodology for advancing nursing research: Factorial surveys, Advances in Nursing Science, 27, 224, 10.1097/00012272-200407000-00007 · DOI
- Ludwick, 2001, The factorial survey: An experimental method to replicate real world problems, Nursing Research, 50, 129, 10.1097/00006199-200103000-00009 · DOI
- Lunza, 1990, A methodological approach to enhance external validity in simulation based research, Issues in Mental Health Nursing, 11, 407, 10.3109/01612849009006526 · DOI
- Macpherson, 2010, Medical student attitudes about bioethics, Cambridge Quarterly of Healthcare Ethics, 19, 488, 10.1017/S0963180110000381 · DOI
- Maieron, 1996, Children's perceptions of peers with AIDS: Assessing the impact of contagion information, perceived similarity, and illness conceptualization, Journal of Pediatric Psychology, 21, 321, 10.1093/jpepsy/21.3.321 · DOI
- McCarron, 2011, A Canadian perspective on using vignettes to teach ethics in psychology, Canadian Psychology, 52, 185, 10.1037/a0024504 · DOI
- McGlynn, 1976, Sex and race as factors affecting the attribution of insanity in a murder trial, The Journal of Psychology, 93, 93, 10.1080/00223980.1976.9921378 · DOI
- Murphy, 1986, Evaluating the performance of paper people, Journal of Applied Psychology, 71, 654, 10.1037/0021-9010.71.4.654 · DOI
- Peabody, 2007, A cross-national comparison of the quality of clinical care using vignettes, Health Policy and Planning, 22, 294, 10.1093/heapol/czm020 · DOI
- Peabody, 2000, Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of 3 methods for measuring quality, Journal of the American Medical Association, 283, 1715, 10.1001/jama.283.13.1715 · DOI
- Peabody, 2004, Measuring the quality of physician practice by using clinical vignettes: A prospective validation study, Annals of Internal Medicine, 141, 771, 10.7326/0003-4819-141-10-200411160-00008 · DOI
- Reed, 2010, Toward ICD-11: Improving the clinical utility of WHO's international classification of mental disorders, Professional Psychology: Research and Practice, 41, 457, 10.1037/a0021701 · DOI
- Reed, 2011, The WPA-WHO global survey of psychiatrists’ attitudes toward mental disorders classification, World Psychiatry, 10, 118, 10.1002/j.2051-5545.2011.tb00034.x · DOI
- Reed, 2013, Mental health professionals’ natural taxonomies of mental disorders: Implications for the clinical utility of the ICD-11 and the DSM-5, Journal of Clinical Psychology, 69, 1191, 10.1002/jclp.22031 · DOI
- Roberts, 1984, The role of socioeconomic status on children's perceptions of medical and psychological disorders, Journal of Clinical Child Psychology, 13, 243, 10.1080/15374418409533197 · DOI
- Roberts, 2012, A global clinicians’ map of mental disorders to improve ICD-11: Analysing meta-structure to enhance clinical utility, International Review of Psychiatry, 24, 578, 10.3109/09540261.2012.736368 · DOI
- Ross, 1999, Sex and attitude: A randomized vignette study of the management of depression by general practitioners, British Journal of General Practice, 49, 17
- Rossi, 1974, Measuring household social standing, Social Science Research, 3, 169, 10.1016/0049-089X(74)90011-8 · DOI
- Schoenberg, 2000, Using vignettes in awareness and attitudinal research, International Journal of Social Research Methodology, 3, 63, 10.1080/136455700294932 · DOI
- Shah, 2007, Measuring clinical practice, Ophthalmic and Physiological Optics, 27, 113, 10.1111/j.1475-1313.2006.00481.x · DOI
- Sniderman, 1996, Innovations in experimental design in attitude surveys, Annual Review of Sociology, 22, 377, 10.1146/annurev.soc.22.1.377 · DOI
- Spalding, 2007, Exploring the use of vignettes: From validity to trustworthiness, Qualitative Health Research, 17, 954, 10.1177/1049732307306187 · DOI
- Spreng, C. P., Ojo, I. P., Burger, N. E., Sood, N., Peabody, J., W., & Demaria, L. M. (2014). Does stewardship make a difference in the quality of care? Evidence from clinics and pharmacies in Kenya and Ghana. International Journal for Quality of Health Care. Advance online publication. http://dx.doi.org/10.1093/intqhc/mzu054. · DOI
- Stokes, 2011, Race, poverty and child protection decision making, British Journal of Social Work, 41, 1105, 10.1093/bjsw/bcr009 · DOI
- Stolte, 1994, The context of satisficing in vignette research, The Journal of Social Psychology, 134, 727, 10.1080/00224545.1994.9923007 · DOI
- Taylor, 2006, Factorial surveys: Using vignettes to study professional judgment, British Journal of Social Work, 36, 1187, 10.1093/bjsw/bch345 · DOI
- Veloski, 2005, Clinical vignette-based surveys: A tool for assessing physician practice variation, American Journal of Medical Quality, 20, 151, 10.1177/1062860605274520 · DOI
- Wainwright, 2010, The use of vignettes within a Delphi exercise: A useful approach in empirical ethics, Journal of Medical Ethics, 36, 656, 10.1136/jme.2010.036616 · DOI
- Wallander, 2009, 25 years of factorial surveys in sociology: A review, Social Science Research, 38, 505, 10.1016/j.ssresearch.2009.03.004 · DOI
- Wallander, 2012, Measuring social workers’ judgments: Why and how to use the factorial survey approach in the study of professional judgments, Journal of Social Work, 12, 364, 10.1177/1468017310387463 · DOI
- Walster, 1966, Assignment of responsibility for an automobile accident, Journal of Personality and Social Psychology, 3, 73, 10.1037/h0022733 · DOI
- Wason, 2002, Designing vignette studies in marketing, Australasian Marketing Journal, 10, 41, 10.1016/S1441-3582(02)70157-2 · DOI
- Williamson, 1965, Assessing clinical judgment, Journal of Medical Evaluation, 40, 180
Cited by
Sorted by citations
Clark, L. A., Cuthbert, B., Lewis-Fernández, R., Narrow, W. E., & Reed, G. M. (2017) · Psychological Science in the Public Interest · doi:10.1177/1529100617727266 · Cited by 688
Self-citationAelterman, N., Vansteenkiste, M., Haerens, L., Soenens, B., Fontaine, J. R. J., & Reeve, J. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000293 · Cited by 614
The classification of obsessive-compulsive and related disorders in the ICD-11
Stein, D. J., Kogan, C. S., Atmaca, M., Fineberg, N. A., Fontenelle, L. F., Grant, J. E., Matsunaga, H., Reddy, Y. C. J., Simpson, H. B., Thomsen, P. H., et al. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.10.061 · Cited by 204
Self-citationGilbert, S., Mehl, A., Baluch, A., Cawley, C., Challiner, J., Fraser, H., Millen, E., Montazeri, M., Multmeier, J., Pick, F., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040269 · Cited by 183
Keeley, J. W., Reed, G. M., Roberts, M. C., Evans, S. C., Medina-Mora, M. E., Robles, R., Rebello, T., Sharan, P., Gureje, O., First, M. B., et al. (2016) · American Psychologist · doi:10.1037/a0039972 · Cited by 173
Self-citationClaudino, A. M., Pike, K. M., Hay, P., Keeley, J. W., Evans, S. C., Rebello, T. J., Bryant-Waugh, R., Dai, Y., Zhao, M., Matsumoto, C., et al. (2019) · BMC Medicine · doi:10.1186/s12916-019-1327-4 · Cited by 151
Self-citationKeeley, J. W., Reed, G. M., Roberts, M. C., Evans, S. C., Robles, R., Matsumoto, C., Brewin, C. R., Cloitre, M., Perkonigg, A., Rousseau, C., et al. (2016) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2015.09.002 · Cited by 134
Self-citationReed, G. M., Sharan, P., Rebello, T. J., Keeley, J. W., Elena Medina‐Mora, M., Gureje, O., Luis Ayuso‐Mateos, J., Kanba, S., Khoury, B., Kogan, C. S., et al. (2018) · World Psychiatry · doi:10.1002/wps.20524 · Cited by 126
Self-citationChanges from ICD-10 to ICD-11 and future directionsin psychiatric classification
Gaebel, W., Stricker, J., & Kerst, A. (2020) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2020.22.1/wgaebel · Cited by 98
WHO's Global Clinical Practice Network for mental health
Reed, G. M., Rebello, T. J., Pike, K. M., Medina-Mora, M. E., Gureje, O., Zhao, M., Dai, Y., Roberts, M. C., Maruta, T., Matsumoto, C., et al. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00183-2 · Cited by 68
Self-citationThe Role of Religiosity in Ethical Decision-Making: A Study on Islam and the Malaysian Workplace
Sulaiman, R., Toulson, P., Brougham, D., Lempp, F., & Haar, J. (2022) · Journal of Business Ethics · doi:10.1007/s10551-021-04836-x · Cited by 62
Job, C., Adenipekun, B., Cleves, A., Gill, P., & Samuriwo, R. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-081723 · Cited by 44
“17 A comprehensive discussion about the utility or otherwise of different discrete or proxy measures is beyond the remit of this paper, but there are some constraints to the use of some discrete measures such as income as a proxy for SES.”
BackgroundJeon, Y. A. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.03.028 · Cited by 39
Mental health literacy in a sample of Canadian adults.
Gallagher, C. E., & Watt, M. C. (2019) · Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement · doi:10.1037/cbs0000129 · Cited by 9
Gaebel, W., Riesbeck, M., Zielasek, J., Kerst, A., Meisenzahl-Lechner, E., Köllner, V., Rose, M., Hofmann, T., Schäfer, I., Lotzin, A., et al. (2018) · Fortschritte der Neurologie · Psychiatrie · doi:10.1055/s-0044-100508 · Cited by 9
Self-citationReliability of suicide risk estimates: a vignette study
Kolochowski, F. D., Kreckeler, N., Forkmann, T., & Teismann, T. (2025) · Archives of Suicide Research · doi:10.1080/13811118.2024.2382709 · Cited by 8
Racial triangulation and shifting standards in mental health assessments
Gushue, G. V., Lee, T. R., & Kim, J. E. (2022) · Journal of Counseling & Development · doi:10.1002/jcad.12420 · Cited by 6
Flinchum, M. R., Bradley, L. J., Noble, N., & Jung, K. (2026) · The Family Journal · doi:10.1177/10664807261505019
Metadata
- Authors
- Evans, S. C., Roberts, M. C., Keeley, J. W., Blossom, J. B., Amaro, C. M., Garcia, A. M., Stough, C. O., Canter, K. S., Robles, R., & Reed, G. M.
- Affiliations
- University of Kansas (US)
- Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz (MX)
- World Health Organization (CH)
- Corresponding author
- Roberts, Michael C.
- Venue
- International Journal of Clinical and Health Psychology
- Publisher
- Elsevier BV
- Published
- 2015-05
- Volume
- 15
- Issue
- 2
- Pages
- 160-170
- ISSN
- 1697-2600 (ISSN-L)
- PMC
- PMC6224682
- OpenAlex
- W2050607941
- Cited by
- 757
- References
- 4
- Citations by year
- 2026: 102 · 2025: 79 · 2024: 86 · 2023: 74 · 2022: 69 · 2021: 102 · 2020: 62 · 2019: 56 · 2018: 54 · 2017: 32 · 2016: 31 · 2015: 9
- FWCI
- 45.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- International Journal of Clinical and Health Psychology (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- KU ScholarWorks (University of Kansas) (submittedVersion, cc-by-nc-nd)
- Redalyc (Universidad Autónoma del Estado de México) (submittedVersion)
- Funders
- World Health Organization
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- vignette study, clinician decision-making, clinical judgment, validity, generalizability, internal validity, diagnostic decision-making, ICD-11, treatment selection, methodological recommendations, standardized patients, archival analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
