The DOI matches its Crossref record; 64 of the 66 references it cites were found in a registry or an index.
Fostering and Evaluating Reflective Capacity in Medical Education: Developing the REFLECT Rubric for Assessing Reflective Writing
Abstract
PURPOSE: Reflective writing (RW) curriculum initiatives to promote reflective capacity are proliferating within medical education. The authors developed a new evaluative tool that can be effectively applied to assess students' reflective levels and assist with the process of providing individualized written feedback to guide reflective capacity promotion. METHOD: Following a comprehensive search and analysis of the literature, the authors developed an analytic rubric through repeated iterative cycles of development, including empiric testing and determination of interrater reliability, reevaluation and refinement, and redesign. Rubric iterations were applied in successive development phases to Warren Alpert Medical School of Brown University students' 2009 and 2010 RW narratives with determination of intraclass correlations (ICCs). RESULTS: The final rubric, the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT), consisted of four reflective capacity levels ranging from habitual action to critical reflection, with focused criteria for each level. The rubric also evaluated RW for transformative reflection and learning and confirmatory learning. ICC ranged from 0.376 to 0.748 for datasets and rater combinations and was 0.632 for the final REFLECT iteration analysis. CONCLUSIONS: The REFLECT is a rigorously developed, theory-informed analytic rubric, demonstrating adequate interrater reliability, face validity, feasibility, and acceptability. The REFLECT rubric is a reflective analysis innovation supporting development of a reflective clinician via formative assessment and enhanced crafting of faculty feedback to reflective narratives.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
The Reflective Practitioner: How Professionals Think in Action
Checkoway, B., & Schon, D. A. (1985) · Journal of Policy Analysis and Management · doi:10.2307/3324262 · Cited by 21,268
Transformative Dimensions of Adult Learning
Mezirow, J. (1991) · Cited by 7,430
Nicol, D. J., & Macfarlane‐Dick, D. (2006) · Studies in Higher Education · doi:10.1080/03075070600572090 · Cited by 5,846
(1985) · Medical Entomology and Zoology · Cited by 3,952
Goodman, L. A. (1961) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177705148 · Cited by 3,606
Defining and Assessing Professional Competence
Epstein, R. M. (2002) · JAMA · doi:10.1001/jama.287.2.226 · Cited by 3,107
Reflection and reflective practice in health professions education: a systematic review
Mann, K., Gordon, J., & MacLeod, A. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-007-9090-2 · Cited by 2,175
Narrative Medicine: Honoring the Stories of Illness
(2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-2-200701160-00021 · Cited by 1,613
The use of reflection in medical education: AMEE Guide No. 44
Sandars, J. (2009) · Medical Teacher · doi:10.1080/01421590903050374 · Cited by 1,085
Epstein, R. M. (1999) · JAMA · doi:10.1001/jama.282.9.833 · Cited by 1,036
Overconfidence as a Cause of Diagnostic Error in Medicine
Berner, E. S., & Graber, M. L. (2008) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.01.001 · Cited by 1,034
Using Rubrics To Promote Thinking and Learning.
Andrade, H. G. (2000) · Educational leadership · Cited by 553
How Doctors Think: Clinical Judgment and the Practice of Medicine
Montgomery, K. (2005) · Cited by 448
A four‐category scheme for coding and assessing the level of reflection in written work
Kember, D., McKay, J., Sinclair, K., & Wong, F. K. Y. (2008) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930701293355 · Cited by 444
Effects of reflective practice on the accuracy of medical diagnoses
Mamede, S., Schmidt, H. G., & Penaforte, J. C. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03030.x · Cited by 382
The Developing Physician — Becoming a Professional
Stern, D. T., & Papadakis, M. (2006) · New England Journal of Medicine · doi:10.1056/nejmra054783 · Cited by 368
A handbook of reflective and experiential learning
Moon, J. A. (2004) · Cited by 333
Twelve tips for teaching reflection at all levels of medical education
Aronson, L. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.507714 · Cited by 332
KEMBER, D. (1999) · International Journal of Lifelong Education · doi:10.1080/026013799293928 · Cited by 323
Reflection: a link between receiving and using assessment feedback
Sargeant, J. M., Mann, K. V., van der Vleuten, C. P., & Metsemakers, J. F. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-008-9124-4 · Cited by 296
Goodrich, S. M. (1995) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(95)90252-x · Cited by 254
A Conceptual Framework for the Use of Illness Narratives in Medical Education
Kumagai, A. K. (2008) · Academic Medicine · doi:10.1097/acm.0b013e3181782e17 · Cited by 248
The development of a scale to measure personal reflection in medical practice and education
Aukes, L. C., Geertsma, J., Cohen-Schotanus, J., Zwierstra, R. P., & Slaets, J. P. J. (2007) · Medical Teacher · doi:10.1080/01421590701299272 · Cited by 230
Beyond the Margins: Reflective Writing and Development of Reflective Capacity in Medical Education
Wald, H. S., & Reis, S. P. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1347-4 · Cited by 225
Self-citationWhy medical educators may be failing at feedback.
Bing-You, R. G. (2009) · JAMA · doi:10.1001/jama.2009.1393 · Cited by 224
A Call for Outcomes Research in Medical Education
Chen, F. M., Bauchner, H., & Burstin, H. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00010 · Cited by 205
Chirema, K. D. (2007) · Nurse Education Today · doi:10.1016/j.nedt.2006.04.007 · Cited by 185
Wald, H. S., Davis, S. W., Reis, S. P., Monroe, A. D., & Borkan, J. M. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181a8592f · Cited by 174
Self-citationThe self critical doctor: helping students become more reflective
Driessen, E. W., van Tartwijk, J., & Dornan, T. (2008) · BMJ · doi:10.1136/bmj.39503.608032.ad · Cited by 169
The Reflective Practitioner: Reaching for Excellence in Practice
Plack, M. M., & Greenberg, L. (2005) · Pediatrics · doi:10.1542/peds.2005-0209 · Cited by 165
Words and Wards: A Model of Reflective Writing and Its Uses in Medical Education
Shapiro, J., Kasman, D., & Shafer, A. (2006) · Journal of Medical Humanities · doi:10.1007/s10912-006-9020-y · Cited by 159
Brady, D. W., Corbie-Smith, G. M., & Branch, W. T. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-137-3-200208060-00025 · Cited by 151
Levine, R. B., Kern, D. E., & Wright, S. M. (2007) · Advances in Health Sciences Education · doi:10.1007/s10459-007-9079-x · Cited by 141
Evaluation and the Improvement of Academic Learning
Sadler, D. R. (1983) · The Journal of Higher Education · doi:10.2307/1981645 · Cited by 136
Huddle, T. S. (2005) · Academic Medicine · doi:10.1097/00001888-200510000-00002 · Cited by 134
A method for assessing reflective journal writing.
Plack, M. M., Driscoll, M., Blissett, S., McKenna, R., & Plack, T. P. (2005) · PubMed · Cited by 133
Chretien, K. C., Goldman, E. F., & Faselis, C. J. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0796-5 · Cited by 132
Appraising and assessing reflection in students' writing on a structured worksheet
Pee, B., Woodman, T., Fry, H., et al. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01227.x · Cited by 116
Assessing Reflective Writing on a Pediatric Clerkship by Using a Modified Bloom’s Taxonomy
Plack, M. M., Driscoll, M., Marquez, M., Cuppernull, L., Maring, J., & Greenberg, L. W. (2007) · Ambulatory Pediatrics · doi:10.1016/j.ambp.2007.04.006 · Cited by 105
Reflection, perception and the acquisition of wisdom
Epstein, R. M. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03181.x · Cited by 98
Developing and maintaining reflection in clinical journals
Paterson, B. L. (1995) · Nurse Education Today · doi:10.1016/s0260-6917(95)80108-1 · Cited by 90
Reis, S. P., Wald, H. S., Monroe, A. D. H., & Borkan, J. (2010) · Patient Education and Counseling · doi:10.1016/j.pec.2009.11.014 · Cited by 85
Self-citationBuilding Character: A Model for Reflective Practice
Bryan, C. S., & Babelay, A. M. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181b6a79c · Cited by 84
Assessing the Written Communication Skills of Medical School Graduates
Boulet, J. R., Rebbecchi, T. A., Denton, E. C., McKinley, D. W., & Whelan, G. P. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000012216.39378.15 · Cited by 82
A Categorization Scheme for Assessing Pharmacy Students' Levels of Reflection During Internships
Wallman, A., Lindblad, Å. K., Hall, S., Lundmark, A., & Ring, L. (2008) · American Journal of Pharmaceutical Education · doi:10.5688/aj720105 · Cited by 77
Meitar, D., Karnieli‐Miller, O., & Eidelman, S. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181bb2b94 · Cited by 69
Developing a Peer Assessment of Lecturing Instrument: Lessons Learned
Newman, L. R., Lown, B. A., Jones, R. N., Johansson, A. C., & Schwartzstein, R. M. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181ad18f9 · Cited by 67
Assessing the Reliability of Grading Reflective Journal Writing
Williams, R. M., Sundelin, G., Foster-Seargeant, E., et al. (2000) · Journal of Physical Therapy Education · doi:10.1097/00001416-200007000-00005 · Cited by 58
Literature and Medicine: A Problem of Assessment
Kuper, A. (2006) · Academic Medicine · doi:10.1097/00001888-200610001-00032 · Cited by 52
Guillemin, M., McDougall, R. J., & Gillam, L. H. (2009) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s096318010909032x · Cited by 52
First year specialist trainees’ engagement with reflective practice in the e-portfolio
McNeill, H., Brown, J., & Shaw, N. J. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9217-8 · Cited by 51
Reflective Ability Rubric and User Guide
O’Sullivan, P., Aronson, L., Chittenden, E., Niehaus, B., & Learman, L. A. (2010) · MedEdPORTAL · doi:10.15766/mep_2374-8265.8133 · Cited by 33
Reflection rubric development: evaluating medical students’ reflective writing
Wald, H. S., Reis, S. P., & Borkan, J. M. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03470.x · Cited by 27
Self-citationClerkship‐based reflective writing: a rubric for feedback
Devlin, M. J., Mutnick, A., Balmer, D. F., & RICHARDS, B. F. (2010) · Medical Education · doi:10.1111/j.1365-2923.2010.03815.x · Cited by 24
Aronson, L., Niehaus, B., DeVries, C. D., Siegel, J. R., & O’Sullivan, P. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181ed3aa7 · Cited by 22
Developing a Scoring Rubric for Resident Research Presentations: A Pilot Study
Musial, J. L., Rubinfeld, I. S., Parker, A., et al. (2007) · Journal of Surgical Research · doi:10.1016/j.jss.2007.03.060 · Cited by 21
Commentary: Forks in the Road: Disruption and Transformation in Professional Development
Kumagai, A. K. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181fa2a59 · Cited by 15
Hopkins, R. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31824485ac · Cited by 8
Bolton, G. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61223-x · Cited by 8
Wald, H. S. (2008) · PubMed · Cited by 7
Self-citationDr. Irmfried Eberl (1910-1948): mass murdering MD.
Strous, R. D. (2009) · PubMed · Cited by 7
Wald, H. S. (2008) · JAMA · doi:10.1001/jama.299.19.2255 · Cited by 5
Self-citationWald, H. S. (2009) · Academic Medicine · doi:10.1097/acm.0b013e31819fb9ab · Cited by 2
Self-citation
Research works printed but not linked: 5
- Avrahami, 2009, Isr Med Assoc J, 11, 216
- Wald, 2008, Fam Med, 40, 393
- Plack, 2005, J Allied Health, 34, 199
- Andrade, 2000, Educ Leadersh, 57, 1
- Aronson, 2011, Acad Med, 33, 220
Cited by
Wald, H. S., Anthony, D., Hutchinson, T. A., Liben, S., Smilovitch, M., & Donato, A. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000725 · Cited by 331
Self-citationTeaching and Learning in Medical Education: How Theory can Inform Practice
Kaufman, D. M., & Mann, K. V. (2010) · Understanding Medical Education · doi:10.1002/9781444320282.ch2 · Cited by 284
The quality and effectiveness of descriptive rubrics
Brookhart, S. M., & Chen, F. (2015) · Educational Review · doi:10.1080/00131911.2014.929565 · Cited by 266
Feedback for learners in medical education: What is known? A scoping review.
Bing-You, R., Hayes, V., Varaklis, K., Trowbridge, R., Kemp, H., & McKelvy, D. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001578 · Cited by 256
Appropriate criteria: key to effective rubrics
Brookhart, S. M. (2018) · Frontiers in Education · doi:10.3389/feduc.2018.00022 · Cited by 243
“Wald et al., 2012 Graduate Reflective writing Analytic rubric with 5 criteria (+1 optional) and descriptive plds at 4 levels 10–60 narratives over 5 trials Rubric content based on literature Wallace et al., 2011 Undergraduate Astronomy – Cosmology Task-specific, holistic rubrics for each test item,…”
BackgroundMethodInfluential
Metadata
- Authors
- Wald, H. S., Borkan, J. M., Taylor, J. S., Anthony, D., & Reis, S. P.
- Affiliations
- Brown University (US)
- Corresponding author
- Wald, Hedy S.
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-01
- Volume
- 87
- Issue
- 1
- Pages
- 41-50
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W2152338595
- Semantic Scholar
- 9b76d4ff3ea189020a42bf2289a484a963021a58
- Cited by
- 391
- References
- 66
- Citations by year
- 2026: 40 · 2025: 32 · 2024: 29 · 2023: 27 · 2022: 23 · 2021: 30 · 2020: 34 · 2019: 26 · 2018: 26 · 2017: 27 · 2016: 19 · 2015: 29 · 2014: 26 · 2013: 13 · 2012: 6
- FWCI
- 70.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Academic Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Reflective Practices in Education
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- reflective writing, reflective capacity, inter-rater reliability, formative assessment, instructor feedback, reflective practice, medical students, critical reflection
- MeSH terms
- Attitude of Health Personnel, Clinical Competence, Curriculum, Education, Medical, Undergraduate, Educational Measurement, Feedback, Humans, Rhode Island, Thinking, Writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.