The DOI matches its Crossref record; 41 of the 44 references it cites were found in a registry or an index.
A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Course evaluation using questionnaires and assessment of participant satisfaction are level 1 criteria of Kirkpatrick’s evaluation model of training programs (subjective evaluation).”
From Neurosurgery resident training using blended learning concepts: Course development and participant evaluation (2022) · cites this paper as [18]
In the citing paper
“it remains to be evaluated whether these skills will be transferred actively into clinical practice where they can have an impact on patient outcome (level 3).18”
From Neurosurgery resident training using blended learning concepts: Course development and participant evaluation (2022) · cites this paper as [18]
In the citing paper
“Analysis of objective, automated measures of the e-learning account for level 2.”
From Neurosurgery resident training using blended learning concepts: Course development and participant evaluation (2022) · cites this paper as [18]
Citations
Cites
Evaluating Training Programs : The Four Levels
Kirkpatrick, D. L. (2009) · Cited by 4,720
Principles of Biomedical Ethics.
(1984) · Annals of Internal Medicine · doi:10.7326/0003-4819-100-1-168_1 · Cited by 4,559
Utilization-Focused Evaluation
(2018) · Sage eBooks · doi:10.4135/9781506326139.n729 · Cited by 2,315
Kraiger, K., Ford, J. K., & Salas, E. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.311 · Cited by 1,831 · cited 1×
“Kraiger, Ford and Salas (1993) have forwarded a multidimensional model of learning outcomes from training and have described a process training evaluators could use to developing learning evaluation measures (Kraiger & Jung, 1997).”
BackgroundThe science of training: a decade of progress
Salas, E., & Cannon-Bowers, J. A. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.471 · Cited by 1,703
Techniques for Evaluating Training Programs
Kirkpatrick, D. L. (1975) · Training and development journal · Cited by 1,234
Training and Development in Work Organizations
Tannenbaum, S. I., & YUKL, G. A. (1992) · Annual Review of Psychology · doi:10.1146/annurev.ps.43.020192.002151 · Cited by 1,023
Knowles, M. S., Holton, E. F., & Swanson, R. A. (1973) · Cited by 970
The flawed four‐level evaluation model
Holton, E. F. (1996) · Human Resource Development Quarterly · doi:10.1002/hrdq.3920070103 · Cited by 965
Principles of Biomedical Ethics
Beauchamp, T. L., & Childless, J. F. (1991) · International Clinical Psychopharmacology · doi:10.1097/00004850-199100620-00013 · Cited by 964
A META‐ANALYSIS OF THE RELATIONS AMONG TRAINING CRITERIA
ALLIGER, G. M., TANNENBAUM, S. I., BENNETT, W., TRAVER, H., & SHOTLAND, A. (1997) · Personnel Psychology · doi:10.1111/j.1744-6570.1997.tb00911.x · Cited by 948
Reeves, M. (1993) · Cited by 906
Development of a generalized learning transfer system inventory
Holton, E. F., Bates, R. A., & Ruona, W. E. A. (2000) · Human Resource Development Quarterly · doi:10.1002/1532-1096(200024)11:4 · Cited by 889
Applying trained skills on the job: The importance of the work environment.
Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. (1995) · Journal of Applied Psychology · doi:10.1037/0021-9010.80.2.239 · Cited by 826
Shakhov, A. N., Collart, M. A., Vassalli, P., Nedospasov, S. A., & Jongeneel, C. V. (1990) · The Journal of Experimental Medicine · doi:10.1084/jem.171.1.35 · Cited by 819
The relationship between organizational transfer climate and positive transfer of training
Rouiller, J. Z., & Goldstein, I. L. (1993) · Human Resource Development Quarterly · doi:10.1002/hrdq.3920040408 · Cited by 771
Foundations of Human Resource Development
McGuire, D. (2014) · SAGE Publications Ltd eBooks · doi:10.4135/9781473906891.n1 · Cited by 725
KIRKPATRICK'S LEVELS OF TRAINING CRITERIA: THIRTY YEARS LATER
Alliger, G. M., & Janak, E. A. (1989) · Personnel Psychology · doi:10.1111/j.1744-6570.1989.tb00661.x · Cited by 722
MODELING JOB PERFORMANCE IN A POPULATION OF JOBS
Campbell, J. P., McHenry, J. J., & Wise, L. L. (1990) · Personnel Psychology · doi:10.1111/j.1744-6570.1990.tb01561.x · Cited by 614
Kitchener, K. S. (1984) · The Counseling Psychologist · doi:10.1177/0011000084123005 · Cited by 560
FACTORS AFFECTING THE OPPORTUNITY TO PERFORM TRAINED TASKS ON THE JOB
Ford, J. K., Quiñones, M. A., Sego, D. J., & Sorra, J. (1992) · Personnel Psychology · doi:10.1111/j.1744-6570.1992.tb00858.x · Cited by 498
Training and development handbook: A guide to human resource development
Craig, R. L. (1976) · Medical Entomology and Zoology · Cited by 452
Smith, M. C. (1991) · doi:10.1007/978-1-349-21542-3_4 · Cited by 400
Utilization-focused evaluation
Westbrook, L. (1998) · Library & Information Science Research · doi:10.1016/s0740-8188(98)90032-9 · Cited by 389
SIOP (2002) · PsycEXTRA Dataset · doi:10.1037/e553002012-001 · Cited by 280
Student Evaluations of Teachers
Rodin, M. J., & Rodin, B. (1972) · Science · doi:10.1126/science.177.4055.1164 · Cited by 251
Cannon‐Bowers, J. A., Eduardo Salas, A., Tannenbaum, S. I., & Mathieu, J. E. (1995) · Military Psychology · doi:10.1207/s15327876mp0703_1 · Cited by 206
Bates, R. A., Holton, E. F., Seyler, D. L., & Carvalho, M. A. (2000) · Human Resource Development International · doi:10.1080/136788600361920 · Cited by 142
Foundations of Human Resource Development
McGuire, D. (2011) · SAGE Publications Ltd eBooks · doi:10.4135/9781446251065.n1 · Cited by 126
Measuring Knowledge Organization as a Method for Assessing Learning during Training
Kraiger, K., Eduardo Salas, A., & Cannon‐Bowers, J. A. (1995) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095778995535 · Cited by 103
Ethical Issues in Graduate Education: Faculty and Student Responsibilities
Brown, R. D., & Krager, L. (1985) · The Journal of Higher Education · doi:10.2307/1981303 · Cited by 94
Applied ethics of program evaluation
Sieber, J. E. (1997) · Evaluation and Program Planning · doi:10.1016/s0149-7189(97)85673-x · Cited by 66
Michalski, G. V., & Cousins, J. B. (2000) · Evaluation and Program Planning · doi:10.1016/s0149-7189(00)00005-7 · Cited by 58
Who's Afraid of Level 4 Evaluation? A Practical Approach.
Shelton, S. K., & Alliger, G. M. (1993) · Cited by 50
A quantitative needs assessment technique for cross-cultural work adjustment training
Selmer, J. (2000) · Human Resource Development Quarterly · doi:10.1002/1532-1096(200023)11:3 · Cited by 50
Does Your Training Make a Difference? Prove It!.
Geber, B. (1995) · Cited by 35
Newstrom, J. W. (1995) · Human Resource Development Quarterly · doi:10.1002/hrdq.3920060310 · Cited by 34
Linking training objectives to evaluation criteria.
Kraiger, K., & Jung, K. M. (1997) · American Psychological Association eBooks · doi:10.1037/10260-006 · Cited by 32
Invited reaction: Linking learning with financial performance
Baldwin, T. T., & Danielson, C. (2002) · Human Resource Development Quarterly · doi:10.1002/hrdq.1011 · Cited by 26
Implementing an ROI Measurement Process at Dell Computer
Tesoro, F. (2008) · Performance Improvement Quarterly · doi:10.1111/j.1937-8327.1998.tb00110.x · Cited by 15
Valuing Learning: The Measurement Journey.
Wang, G. (2003) · Educational Technology archive · Cited by 12
Flagged first, then by citations
3 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Ford, 1995, The application of cognitive constructs and principles to the instructional systems design model of training: Implications for needs assessment, design, and transfer, International Review of Industrial and Organizational Psychology, 10, 1
- Grider, 1988, Evaluating valuations, Training and Development Journal, 42, 11
- Schwandt, T. A., (1998). How we think about evaluation practice. Paper presented at the American Evaluation Association Conference, Chicago, IL.
Cited by
Sorted by citations
Program evaluation models and related theories: AMEE Guide No. 67
FRYE, A. W., & Hemmer, P. A. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.668637 · Cited by 641
Thieme, A., Belgrave, D., & Doherty, G. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3398069 · Cited by 406
“The principle of beneficence however does not only entail encouraging human flourishing and wellbeing by doing the right thing, but also suggests to ‘do it well’ [13].”
BackgroundSurr, C. A., Gates, C., Irving, D., Oyebode, J., Smith, S. J., Parveen, S., Drury, M., & Dennison, A. (2017) · Review of Educational Research · doi:10.3102/0034654317723305 · Cited by 327
“Kirkpatrick’s (1979, 1984) “Return on Investment” model developed specifically for evaluation of training consists of four levels and is widely adopted in the evaluation of training and education provision (Bates, 2004).”
A Critique of Kirkpatrick's Evaluation Model
Reio, T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017) · New Horizons in Adult Education and Human Resource Development · doi:10.1002/nha3.20178 · Cited by 227
“, Bates, 2004; Holton, 1996; Swanson, 1994). Some researchers, however, have developed their own models based on Kirkpatrick’s. Bushnell (1990) and Hayes, Scott, Braczinskas, Scaccia, Stout, and Wandersman (2016) contend that Kirkpatrick’s model focuses only on what happens after the training, and therefore is not suf…”
BackgroundMethodInfluentialSanches, P., Janson, A., Karpashevich, P., Nadal, C., Qu, C., Roquet, C. D., Umair, M., Windlin, C., Doherty, G. J., Höök, K., et al. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300475 · Cited by 248
“Others argued for the value of increasing user’s feeling of control by augmenting tracking technologies with support for reflection [18], or increasing user’s engagement with technology through DIY approaches to design [148].”
BackgroundInfluential
Metadata
- Authors
- Bates, R.
- Affiliations
- Louisiana State University (US)
- Corresponding author
- Bates, Reid
- Venue
- Evaluation and Program Planning
- Publisher
- Elsevier BV
- Published
- 2004-08
- Volume
- 27
- Issue
- 3
- Pages
- 341-347
- ISSN
- 0149-7189 (ISSN-L)
- OpenAlex
- W2097592370
- Cited by
- 623
- References
- 44
- Citations by year
- 2026: 53 · 2025: 23 · 2024: 42 · 2023: 53 · 2022: 49 · 2021: 39 · 2020: 43 · 2019: 44 · 2018: 32 · 2017: 45 · 2016: 32 · 2015: 32 · 2014: 30 · 2013: 29 · 2012: 22
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation and Program Planning (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Human Resource Development and Performance Evaluation
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- Kirkpatrick model, training evaluation, beneficence, moral obligation, formative assessment, summative assessment, training effectiveness, human resource development, evaluation limitations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
