Impact Studies in Continuing Education For Health Professionals: A Critique of the Research Syntheses
Abstract
Documenting impact has been a continual pursuit in continuing education (CE) for health professionals. Hundreds of primary impact studies have been published, along with 16 impact study syntheses that have sought to generalize about the effectiveness of CE and sometimes how and why the effects occur. This article describes and critiques the methodology of the 16 syntheses and summarizes their findings. A first wave of syntheses established a general causal connection between CE and impacts, but explained impact variability only in the dependent variable-knowledge, competence, performance, or outcome. A second wave added a search for causal explanation through analysis of variables that moderate impact. This wave has begun to identify the most appropriate types of programs for promoting performance changes. The article concludes by suggesting questions and methods for future primary studies and meta-analyses, including improved experimental and meta-analytical methods, along with case and naturalistic studies and action research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing evidence about the design and outcomes of CCPs is mixed.
“Although mandatory continuing professional development is an approach that is used widely by professional regulators globally, existing evidence about the design and outcomes of CCPs is mixed [18,247–251].”
From Nursing regulation in Canada: Insights from a scoping review (2025) · cites this paper as [251]
Cites
(1984) · Cited by 81,983 · see citing works
Case Study Research: Design and Methods
Chaiklin, H. (1991) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199102000-00025 · Cited by 39,631 · see citing works
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · Routledge eBooks · doi:10.4324/9780203710753-35 · Cited by 37,557 · see citing works
Educating the reflective practitioner
Schön, D. A. (1987) · Cited by 11,335 · see citing works
Hietala, M. (1983) · Scandinavian Journal of History · doi:10.1080/03468758308579015 · Cited by 6,232 · see citing works
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,896 · see citing works
(2020) · Journal of Medical Education · doi:10.5812/jme · Cited by 3,505 · see citing works
TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH
Baldwin, T. T., & Ford, J. K. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00632.x · Cited by 3,449 · see citing works
Hinkemeyer, M. T. (1971) · The Clearing House A Journal of Educational Strategies Issues and Ideas · doi:10.1080/00098655.1971.11478291 · Cited by 1,847 · see citing works
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486 · see citing works
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422 · see citing works
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,168 · see citing works
Explaining Organizational Behavior.
Pfeffer, J., & Mohr, L. B. (1983) · Administrative Science Quarterly · doi:10.2307/2392635 · Cited by 970 · see citing works
Health Education Planning: A diagnostic approach
(1986) · Journal of Nutrition Education · doi:10.1016/s0022-3182(86)80109-1 · Cited by 950 · see citing works
Discovering the future of the case study method in evaluation research
Yin, R. K. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90023-x · Cited by 745 · see citing works
Best-Evidence Synthesis: An Alternative to Meta-Analytic and Traditional Reviews
Slavin, R. E. (1986) · Educational Researcher · doi:10.3102/0013189x015009005 · Cited by 699 · see citing works
Lipsey, M. W. (1991) · Evaluation and Program Planning · doi:10.1016/0149-7189(91)90027-e · Cited by 598 · see citing works
Vote-counting methods in research synthesis.
Hedges, L., & Olkin, I. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.2.359 · Cited by 511 · see citing works
The effectiveness of continuing medical education
Stein, L. S. (1981) · Academic Medicine · doi:10.1097/00001888-198102000-00004 · Cited by 480 · see citing works
Improving Drug Prescribing in Primary Care: A Critical Analysis of the Experimental Literature
Soumerai, S. B., McLaughlin, T. J., & Avorn, J. (2005) · Milbank Quarterly · doi:10.1111/j.1468-0009.2005.00435.x · Cited by 327 · see citing works
Theory as method: small theories of treatments
Lipsey, M. W. (1993) · New Directions for Program Evaluation · doi:10.1002/ev.1637 · Cited by 322 · see citing works
Impact analysis for program evaluation
Mohr, L. B. (1988) · Cited by 299 · see citing works
A Critical Appraisal of the Efficacy of Continuing Medical Education
Haynes, R. B. (1984) · JAMA · doi:10.1001/jama.1984.03340250041018 · Cited by 294 · see citing works
Israel, B. A. (1985) · Health Education Quarterly · doi:10.1177/109019818501200106 · Cited by 289 · see citing works
Improving Drug Prescribing in Primary Care: A Critical Analysis of the Experimental Literature
Soumerai, S. B., McLaughlin, T., & Avorn, J. (1989) · Milbank Quarterly · doi:10.2307/3350142 · Cited by 267 · see citing works
Changing and Learning in the Lives of Physicians
Fox, R., Mazmanian, P. E., Putnam, R. W., & Knowles, M. S. (1989) · Medical Entomology and Zoology · Cited by 201 · see citing works
Changing Physician Behavior to Improve Disease Prevention
Cohen, S. J., Halvorson, H. W., & Gosselink, C. A. (1994) · Preventive Medicine · doi:10.1006/pmed.1994.1040 · Cited by 179 · see citing works
Changing and Learning in the Lives of Physicians
Manning, P. R. (1990) · JAMA · doi:10.1001/jama.1990.03450150117045 · Cited by 179 · see citing works
The "Too Few Cases/Too Many Variables" Problem in Implementation Research
Goggin, M. L. (1986) · The Western Political Quarterly · doi:10.1177/106591298603900210 · Cited by 173 · see citing works
Patton, M. Q. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90026-4 · Cited by 164 · see citing works
How physicians view the process of change in their practice behavior
Geertsma, R. H., Parker, R. C., & Whitbourne, S. K. (1982) · Academic Medicine · doi:10.1097/00001888-198210000-00003 · Cited by 137 · see citing works
Professional practice, learning, and continuing education: an integrated perspective‡
Cervero, R. (1992) · International Journal of Lifelong Education · doi:10.1080/0260137920110202 · Cited by 135 · see citing works
Self-citationThe Evaluation of Continuing Medical Education: A Literature Review
Bertram, D. A., & Brooks-Bertram, P. (1977) · Health Education Monographs · doi:10.1177/109019817700500404 · Cited by 131 · see citing works
Effectivness of Continuing Medical Education
Lloyd, J., & Abrahamson, S. (1979) · Evaluation & the Health Professions · doi:10.1177/016327877900200301 · Cited by 121 · see citing works
The Effects of Continuing Education on Nursing Practice: A Meta-Analysis
Waddell, D. L. (1992) · The Journal of Continuing Education in Nursing · doi:10.3928/0022-0124-19920701-08 · Cited by 118 · see citing works
Continuing Professional Education and Behavioral Change: A Model for Research and Evaluation
Cervero, R. M. (1985) · The Journal of Continuing Education in Nursing · doi:10.3928/0022-0124-19850501-05 · Cited by 116 · see citing works
Self-citationGottfredson, G. D. (1984) · American Psychologist · doi:10.1037/0003-066x.39.10.1101 · Cited by 108 · see citing works
Efficacy and Cost-Containment in Hospital Pharmacotherapy: State of the Art and Future Directions
Soumerai, S. B., & Avorn, J. (1984) · The Milbank Memorial Fund Quarterly Health and Society · doi:10.2307/3349860 · Cited by 103 · see citing works
Critical multiplism: A research strategy and its attendant tactics
Shadish, W. R. (1993) · New Directions for Program Evaluation · doi:10.1002/ev.1660 · Cited by 96 · see citing works
The Physician as Learner: Linking Research to Practice
Nelson, M. S. (1995) · JAMA · doi:10.1001/jama.1995.03530090111036 · Cited by 89 · see citing works
A quasi‐sampling theory of the generalization of causal relationships
Cook, T. D. (1993) · New Directions for Program Evaluation · doi:10.1002/ev.1638 · Cited by 79 · see citing works
The Effects of Continuing Education on Nursing Practice: A Meta-Analysis
Waddell, D. L. (1991) · The Journal of Continuing Education in Nursing · doi:10.3928/0022-0124-19910501-09 · Cited by 78 · see citing works
Program Implementation: The Organizational Context
Scheirer, M. A. (1981) · Medical Entomology and Zoology · Cited by 71 · see citing works
The "Too Few Cases/Too Many Variables" Problem in Implementation Research
Goggin, M. L. (1986) · The Western Political Quarterly · doi:10.2307/448302 · Cited by 71 · see citing works
rrez, G. G., Guiscafr, H. C., Bronfman, M., Walsh, J., nez, H. M. ́., & oz, O. M. (1994) · Medical Care · doi:10.1097/00005650-199405000-00003 · Cited by 64 · see citing works
The effectiveness of continuing medical education: A quantitative synthesis
Beaudry, J. S. (1989) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.4750090414 · Cited by 61 · see citing works
Determining Physicians' Performance
Sanazaro, P. J. (1983) · Evaluation & the Health Professions · doi:10.1177/016327878300600205 · Cited by 56 · see citing works
Davis, D., Haynes, R. B., Chambers, L. W., Neufield, V. R., McKibbon, A., & Tugwell, P. (1984) · Evaluation & the Health Professions · doi:10.1177/016327878400700301 · Cited by 54 · see citing works
Program implementation: The organizational context
T.W.W. (1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90028-3 · Cited by 50 · see citing works
Impact analysis for program evaluation
Wholey, J. S. (1990) · Evaluation Practice · doi:10.1016/0886-1633(90)90048-i · Cited by 44 · see citing works
Evaluation of Continuing Education in the Health Professions
Abrahamson, S. (1984) · Evaluation & the Health Professions · doi:10.1177/016327878400700101 · Cited by 37 · see citing works
CME and change in practice: An alternative perspective
Wergin, J. F., Mazmanian, P. E., Miller, W. W., et al. (1988) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.4750080302 · Cited by 34 · see citing works
Evaluating Educational Programs: An Integrative, Causal-modeling Approach
Wang, M. C., & Walberg, H. J. (1983) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737005003347 · Cited by 33 · see citing works
Quality health care : the role of continuing medical education
Egdahl, R. H., & Gertman, P. M. (1977) · Medical Entomology and Zoology · Cited by 26 · see citing works
Evaluation: Review of the past, preview of the future
Smith, M. F. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90015-9 · Cited by 25 · see citing works
Evaluation of continuing medical education programs: A selected literature, 1984-1988
McLaughlin, P. J., & Donaldson, J. F. (1991) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.4750110108 · Cited by 24 · see citing works
Six evaluation issues that synthesis can resolve better than single studies
Light, R. J. (1984) · New Directions for Program Evaluation · doi:10.1002/ev.1378 · Cited by 21 · see citing works
The Effectiveness of Continuing Education as Reflected in the Literature of the Health Professions
Nona, D. A., Kenny, W. R., & Johnson, D. K. (1988) · American Journal of Pharmaceutical Education · doi:10.1016/s0002-9459(24)02999-1 · Cited by 19 · see citing works
A review of evaluations of effectiveness in continuing dental education
Bader, J. D. (1987) · Möbius A Journal for Continuing Education Professionals in Health Sciences · doi:10.1002/chp.4760070309 · Cited by 16 · see citing works
Strengthening causal interpretations of nonexperimental data: The role of meta‐analysis
Cordray, D. S. (1993) · New Directions for Program Evaluation · doi:10.1002/ev.1661 · Cited by 12 · see citing works
How physicians integrate advances into clinical practices
Lockyer, J., & Chugh, U. (1985) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.4760050203 · Cited by 11 · see citing works
After the Meta-Analytic Revolution
Bausell, R. B. (1993) · Evaluation & the Health Professions · doi:10.1177/016327879301600101 · Cited by 8 · see citing works
Bayley, E. W. (1988) · Scholarly Commons (University of Pennsylvania) · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cervero, R. M., & Gaines, J. K. (2015) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.21290 · Cited by 515 · see citing works
Self-citationImpact studies in continuing education for health professions: update
Robertson, M. K., Umble, K. E., & Cervero, R. M. (2003) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340230305 · Cited by 145 · see citing works
Self-citationSamuel, A., Cervero, R. M., Durning, S. J., & Maggio, L. A. (2021) · Academic Medicine · doi:10.1097/acm.0000000000003899 · Cited by 147 · see citing works
Self-citationMain, P. A. E., & Anderson, S. (2023) · Human Resources for Health · doi:10.1186/s12960-023-00803-x · Cited by 74 · see citing works
Nursing regulation in Canada: Insights from a scoping review
Chiu, P., Thiessen, N. J., Idrees, S., Leslie, K., & Kung, J. Y. (2025) · PLOS ONE · doi:10.1371/journal.pone.0323716 · Cited by 12 · see citing works
“Although mandatory continuing professional development is an approach that is used widely by professional regulators globally, existing evidence about the design and outcomes of CCPs is mixed [18,247–251].”
- Authors
- Umble, K. E., & Cervero, R. M.
- Affiliations
- University of Georgia (US)
- Venue
- Evaluation & the Health Professions
- Publisher
- SAGE Publications
- Published
- 1996-06
- Volume
- 19
- Issue
- 2
- Pages
- 148-174
- ISSN
- 0163-2787 (ISSN-L)
- 1552-3918
- OpenAlex
- W2049363298
- Cited by
- 76
- References
- 63
- Citations by year
- 2025: 1 · 2024: 3 · 2023: 5 · 2022: 1 · 2021: 2 · 2020: 2 · 2019: 1 · 2018: 3 · 2017: 2 · 2016: 3 · 2015: 3 · 2014: 2 · 2013: 3
- FWCI
- 2.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation & the Health Professions (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Psychology › Applied Psychology › Human Resource Development and Performance Evaluation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- continuing education, healthcare workers, impact studies, causal explanation, knowledge change, competence, performance change, evaluation methodology
- MeSH terms
- Clinical Competence, Data Interpretation, Statistical, Education, Medical, Continuing, Educational Measurement, Humans, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review