Cited by 623Human Resource Development and Performance EvaluationEvaluation and Performance AssessmentEvaluation of Teaching Practices

The DOI matches its Crossref record; 41 of the 44 references it cites were found in a registry or an index.

A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence

Bates, Reid·2004·Evaluation and Program Planning

DOI 10.1016/j.evalprogplan.2004.04.011

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1016/j.evalprogplan.2004.04.011

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Evaluating Training Programs : The Four Levels
Kirkpatrick, D. L. (2009) · Cited by 4,720
Principles of Biomedical Ethics.
(1984) · Annals of Internal Medicine · doi:10.7326/0003-4819-100-1-168_1 · Cited by 4,559
Utilization-Focused Evaluation
(2018) · Sage eBooks · doi:10.4135/9781506326139.n729 · Cited by 2,315
Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation.
Kraiger, K., Ford, J. K., & Salas, E. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.311 · Cited by 1,831 · cited 1×
“Kraiger, Ford and Salas (1993) have forwarded a multidimensional model of learning outcomes from training and have described a process training evaluators could use to developing learning evaluation measures (Kraiger & Jung, 1997).”
Background
The science of training: a decade of progress
Salas, E., & Cannon-Bowers, J. A. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.471 · Cited by 1,703
Techniques for Evaluating Training Programs
Kirkpatrick, D. L. (1975) · Training and development journal · Cited by 1,234
Training and Development in Work Organizations
Tannenbaum, S. I., & YUKL, G. A. (1992) · Annual Review of Psychology · doi:10.1146/annurev.ps.43.020192.002151 · Cited by 1,023
The adult learner
Knowles, M. S., Holton, E. F., & Swanson, R. A. (1973) · Cited by 970
The flawed four‐level evaluation model
Holton, E. F. (1996) · Human Resource Development Quarterly · doi:10.1002/hrdq.3920070103 · Cited by 965
Principles of Biomedical Ethics
Beauchamp, T. L., & Childless, J. F. (1991) · International Clinical Psychopharmacology · doi:10.1097/00004850-199100620-00013 · Cited by 964

Flagged first, then by citations

Show all 44 references

3 of the 44 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

Ford, 1995, The application of cognitive constructs and principles to the instructional systems design model of training: Implications for needs assessment, design, and transfer, International Review of Industrial and Organizational Psychology, 10, 1
Grider, 1988, Evaluating valuations, Training and Development Journal, 42, 11
Schwandt, T. A., (1998). How we think about evaluation practice. Paper presented at the American Evaluation Association Conference, Chicago, IL.

Sorted by citations

Items 201–300 of 623

Evaluating Interactive Fatigue Management Workshops for Occupational Health Professionals in the United Kingdom
Ali, S., Chalder, T., & Madan, I. (2014) · Safety and Health at Work · doi:10.1016/j.shaw.2014.07.002 · Cited by 11
“Also Kirkpatrick’s [12] model is beneficial in that it provides a simplified way of evaluating training [13], which can be easily replicated.”
Background
Delivering training strategies: the balanced scorecard at work
Baraldi, S., & Cifalinò, A. (2015) · International Journal of Training and Development · doi:10.1111/ijtd.12055 · Cited by 11
Nurse Executives’ Perceptions of the Executive Training for Research Application (EXTRA) Program
Lavoie‐Tremblay, M., Anderson, M. I., Bonneville‐Roussy, A., Drevniok, U., & Lavigne, G. L. (2011) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2011.00218.x · Cited by 11
“The Kirkpatrick four-level model (Reaction, Learning, Behavior, and Results) is the most well known and used model for assessing learning in organizations (Kirkpatrick 1994, 1998; Bates, 2004).”
Method
Sharpening the lens to evaluate interprofessional education and interprofessional collaboration by improving the conceptual framework: a critical discussion
Neubauer, F. B., Wagner, F. L., Lörwald, A. C., & Huwendiek, S. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05590-0 · Cited by 13
“This question is posed variously under well-known labels: Authors aware of said stagnancy either call for “formative evaluation” [52], “process evaluation” [14], or “realist evalua-tion” [42] in order to understand why interventions work as intended or not.”
BackgroundMethodInfluential
The training pathway for residents: ‘Robotic Curriculum for young Surgeons’ (RoCS) does not impair patient outcome during implementation into clinical routine
Stockheim, J., Andriof, S., Andric, M., Al-Madhi, S., Acciuffi, S., Franz, M., Lorenz, E. P. M., Peglow, S., Benedix, F., Perrakis, A., et al. (2024) · Journal of Robotic Surgery · doi:10.1007/s11701-024-02056-9 · Cited by 10
“However, it is not feasible to deduce recommendations for improving the training program specifically based on this evaluation [16].”
Background
The Influencing Factors on the Effectiveness of Foundation Training Programs: A Case Study of the Health Cadre Service in Bangladesh
Rahman, M. Z., Kumarasamy, M. M., & Bhuiyan, A. B. (2019) · International Journal of Business and Management Future · doi:10.46281/ijbmf.v3i2.367 · Cited by 10
“The items described has been found to be helpful to measure the turning effectiveness of training programs(Bates, 2004; Kirkpatrick & Kirkpatrick, 2015).”
Background
Trainees’ aversions of employee training programs
Andoh, R. P. K., Annan-Prah, E. C., Owusu, E. A., & Agyei, P. M. (2022) · European journal of training and development · doi:10.1108/ejtd-02-2022-0022 · Cited by 10
A Critical Review of Transition-to-Professional-Practice Programs: Applying a Standard Model of Evaluation
Mellor, P. D., Gregoric, C., Atkinson, L. M., & Greenhill, J. (2017) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(17)30095-9 · Cited by 10
“Also, concern exists regarding the subjective nature of levels 1 and 2 as well as claims that the model does not include contextual influences when assessing the transfer of behavior or skills to the workplace, such as the learning culture of the organization (Bates, 2004; Galloway, 2005).”
Background
Laparoscopic Liver Surgery Training Course on Thiel‐Embalmed Human Cadavers: Program Evaluation, Trainer's Long‐Term Feedback and Steps Forward
Rashidian, N., Willaert, W., Giglio, M. C., Scuderi, V., Tozzi, F., Vanlander, A., D'Herde, K. G. M. A., Alseidi, A. A., & TROISI, R. I. (2019) · World Journal of Surgery · doi:10.1007/s00268-019-05103-x · Cited by 10
A validated audio-visual educational module on examination skills in ophthalmology for undergraduate medical students in the COVID-19 season - An observational longitudinal study
Rose, J. S., Lalgudi, S., Joshua, R. A., Paul, J., Susanne, M. A., Phillips, A. C., Jeyaraj, C., Abraham, G., Joshua, R. A., Vinay, S., et al. (2020) · Indian Journal of Ophthalmology · doi:10.4103/ijo.ijo_2054_20 · Cited by 10
“The 1‐week outcome measures only evaluate the primary levels of Kirkpatrick’s model of learning.[16] A longer follow up would be required in assessing clinical competence in work place based practice over a period of time.”
Background
Assessment of Professional Training Programmes in International Agricultural Research Institutions: The Case of ICRAF
Wanjiku, J., Mairura, F., & Place, F. (2010) · The Journal of Agricultural Education and Extension · doi:10.1080/1389224x.2010.515064 · Cited by 13
“Despite the methodological challenges of assessing training impact (Sharples et al, 2003 ; Gardner, 2003 ; Bates, 2004 ), several other factors including, poor monitoring and evaluation structures, lack of resources, weak policy support, and poor documentation and information systems have contributed to ineffective or…”
Background
Building Evaluation Capability in the Public Health Workforce
Dickinson, P., & Adams, J. (2012) · Evaluation Journal of Australasia · doi:10.1177/1035719x1201200204 · Cited by 10
“While there has been some criticism of Kirkpatrick’s model for its oversimplification, including the assumptions of causality between levels and that information is increasingly important as the levels of outcomes are ascended (Bates 2004), it is still considered a useful framework.”
Asthma education for school staff
Kew, K. M., Carr, R., Donovan, T., & Gordon, M. A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012255 · Cited by 10
Safe surgical training: evaluation of a national functional endoscopic sinus surgery model simulation course using the Kirkpatrick evaluation model
Gillanders, S. L., McHugh, A., Lacy, P. D., & Thornton, M. (2023) · Irish Journal of Medical Science (1971 -) · doi:10.1007/s11845-023-03309-6 · Cited by 9
“Level three behaviour outcomes address how the knowledge is implemented, and finally, level four results provide a measure of the impact that training has had on broader organizational goals, timing or financial objectives [10].”
Background
Effectiveness of Virtual Laboratory Teacher Training Workshops: A Kirkpatrick Model Analysis
Achuthan, K., Kolil, V. K., & Jyothy, S. N. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i15.40037 · Cited by 9
Teaching Undergraduate Medical Students Non-Technical Skills: An Evaluation Study of a Simulated Ward Experience
Pollard, J., & Tombs, M. (2022) · Advances in Medical Education and Practice · doi:10.2147/amep.s344301 · Cited by 9
“A follow-up longitudinal study evaluating attitudes of newly qualified FY1ʹs who participated in SWE as undergraduates may help ascertain other aspects and possible benefits of the educational intervention.(32,33)”
Background
Evaluating training effectiveness in India: Exploring the relationship between training components, metacognition and learning outcomes
Bhat, Z. H. (2023) · International Journal of Training and Development · doi:10.1111/ijtd.12311 · Cited by 9
Does actors' engagement with capacity-building training programs enable delivery of SDG-aligned public services? The case of senior public officials
Azmat, F., Ferdous, A. S., Wali, F. I., Muttakin, M. B., & Haque, M. Z. (2024) · Journal of Service Theory and Practice · doi:10.1108/jstp-03-2023-0103 · Cited by 9
Rethinking training evaluation: omnidirectional evaluation, range of audience and intentional change
Langmann, S., & Thomas, S. R. (2017) · The International Journal of Human Resource Management · doi:10.1080/09585192.2017.1303782 · Cited by 9
“Limitations include current evaluation models being too simplistic (Bates, 2004), too complex or starting from the wrong place of inquiry (Alliger, Tannenbaum, Bennett, Traver, & Shotland,1997), having an incomplete understanding of evaluating stakeholders and differences in their expectations of…”
BackgroundMethodInfluential
Involving teachers in reducing children’s media risks
Stiller, A., Schwendemann, H., Bleckmann, P., Bitzer, E., & Mößle, T. (2017) · Health Education · doi:10.1108/he-11-2016-0061 · Cited by 9
Training for Tuberculosis Elimination in Indonesia: Achievements, Reflections, and Potential for Impact
Main, S., Lestari, T., Triasih, R., Chan, G., Davidson, L., Majumdar, S. S., Santoso, D., Phung, S., Laukkala, J., Graham, S. M., et al. (2019) · Tropical Medicine and Infectious Disease · doi:10.3390/tropicalmed4030107 · Cited by 9
“They should attempt to measure not just participant reactions, but also learning, behaviour change, and, although difficult, impact on TB programs or health systems [15].”
Background
Firm size, ownership, training duration and training evaluation practices
Asadullah, M. A., Peretti, J. M., Arain, G. A., & Bourgain, M. (2015) · European journal of training and development · doi:10.1108/ejtd-10-2014-0072 · Cited by 9
“Various researchers criticized TKMand different frameworks of evaluation emerged from this model (Bates, 2004). As training is an investment requiring training professionals to determine its value in dollar terms, therefore, Phillips (1997), like various authors, suggested to include a financial perspective ROI as a f…”
Background
Evidence for improvement in the quality of care given during emergencies in pregnancy, infancy and childhood following training in life-saving skills: a postal survey.
Haq, Z. U., Qureshi, F., Hafeez, A., Zafar, S., Mohamud, B. K., & Southal, D. P. (2009) · PubMed · Cited by 9
Evaluation Models for Evaluating Educational Programs
Brewer, E. W. (2011) · IGI Global eBooks · doi:10.4018/978-1-61520-745-9.ch007 · Cited by 9
Using the Kirkpatrick Model to Evaluate the Effect of a Primary Trauma Care Course on Health Care Workers’ Knowledge, Attitude, and Practice in Two Vietnamese Local Hospitals: Prospective Intervention Study
Nguyen, B. T., Nguyen, V. A. T., Blizzard, C. L., Palmer, A. J., Nguyen, H. T., Quyet, T. C., Tran, V., Skinner, M., Perndt, H., & Nelson, M. R. (2024) · JMIR Medical Education · doi:10.2196/47127 · Cited by 12
“The assessments were conducted using an Objective Structured Clinical Examination format [20,24] with 4 stations, each station lasting up to 10 minutes, with one observer who used a standardized checklist to rate the performance of the team.”
Method
Contorting identities: figuring literacy and identity in adolescent worlds
Quinlan, A., & Curtin, A. J. (2017) · Irish Educational Studies · doi:10.1080/03323315.2017.1362352 · Cited by 8
“According to Bates (2004), a significant principle of ethics to consider is that of beneficence which dictates that the good obtained must be balanced against the possible harm.”
Background
Evaluation of the use of a guided bur during preclinical teaching of tooth preparation: A pilot study
Yee, S., Richert, R., Viguié, G., Couraud, S., Dehurtevent, M., Fages, M., Corne, P., & Ducret, M. (2019) · Clinical and Experimental Dental Research · doi:10.1002/cre2.184 · Cited by 8
“Performance requires the use of scales and instruments to examine a range of variables during training (Bates, 2004).”
BackgroundInfluential
Evaluation of Three Blended Learning Courses to Strengthen Health Professionals' Capacity in Primary Health Care, Management of Sexual and Reproductive Health Services and Research Methods in Guinea
Millimouno, T. M., Delvaux, T., Kolié, J. M., Kourouma, K., Bastelaere, S. V., Tsunami, C. K., Béavogui, A. H., Garcia, M., Damme, W. V., & Délamou, A. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.911089 · Cited by 8
“Similarly, it leads to some limitations - it fails to grasp complexity of the context and related factors - it shows a certain rigidity - and the hierarchical order and importance of the different levels can be questioned (41, 42).”
Background
Service Learning in the Nursing Bachelor Thesis: A Mixed-Methods Study
Roca, J., Gros, S., Vélez, O. C., Ortega, J. T., Tort‐Nasarre, G., Postic, T., & Martínez-Alarcón, L. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191912387 · Cited by 8
“It also simplifies the assessment of training programs and services [30,31].”
Background
A Systematic Review of Educational Interventions to Equip Health and Social Care Professionals to Promote End-of-Life Supportive Care when a Parent with Dependent Children is Dying with Cancer
Sheehan, S., Hanna, J. R., Drury, A., McCance, T., Semple, C. J., & O’Neill, C. (2023) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2023.151474 · Cited by 8
Exploring Factors Affecting Learner's Perception of Learning Information and Communication Technology: A HLM Analysis of a National Farmers' Training Program in Taiwan
Yueh, H., Chen, T., Chiu, L.-A., & Lin, W.-C. M. (2013) · Educational Technology & Society · Cited by 8
The patient suicide attempt – An ethical dilemma case study
Jie, L. (2015) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2015.01.013 · Cited by 8
Visual hermeneutics as a tool to introduce empathy and core physician attributes in doctor-patient relationship for first-year medical undergraduate students
Prabhath, S., Kulkarni, U., Vani Lakshmi, R., Patra, B., Eshwari, K., Prabhu, D. A., & Nayak, K. R. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06742-6 · Cited by 7
“The session’s effectiveness was evaluated based on the first two levels of the Kirkpatrick evaluation model [15, 16].”
Differing associations between organizational training types and operational resilience
Ataburo, H., Essuman, D., Mensah, H. K., Aditchere, J., & Nkrumah, P. (2025) · Management Decision · doi:10.1108/md-05-2024-1194 · Cited by 8
Sustainance of Community Engagement – Is it a Feasible Feat?
Ramachandra, A., Mansor, N. N. A., Anvari, R., & Rahman, A. A. (2014) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2014.03.661 · Cited by 7
A Framework to Guide the Development of Health Care Professional Education and Training in Best Evidence Osteoarthritis Care
Kobayashi, S., Allen, K. D., Bennell, K. L., Bowden, J. L., Briggs, A. M., Burgess, A. W., Hinman, R. S., Holden, M. A., Østerås, N., Risberg, M. A., et al. (2022) · Clinics in Geriatric Medicine · doi:10.1016/j.cger.2021.11.008 · Cited by 7
Surgical education interventions in liver surgery: a systematic review
Rashidian, N., Vierstraete, M., Alseidi, A. A., TROISI, R. I., & Willaert, W. (2020) · Updates in Surgery · doi:10.1007/s13304-020-00766-x · Cited by 7
Impact of internet supported dental education: Initial outcomes in a study sample
Sarsar, F. (2016) · Journal of Human Sciences · doi:10.14687/jhs.v13i3.4209 · Cited by 7
“Discussion Students’ affective responses and instructors’ comments are generally utilized as the first step in evaluating a novel system prior to its incorporation into education (Bates, 2004; Willcockson & Phelps, 2010), as was performed in the present study.”
BackgroundMethodInfluential
“L'évocation de la liberté améliore-t-elle l'efficacité d'une formation ?”
Gillet, I., Gilibert, D., & Girandola, F. (2013) · Les cahiers internationaux de psychologie sociale · doi:10.3917/cips.098.0147 · Cited by 7
Healthcare decision-makers’ perspectives on evaluating conflict management training in paediatric healthcare: a utilisation-focused qualitative study
Phillipson, J., Barclay, S. F., Menson, E. N., & Lyons, O. (2024) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2024-003047 · Cited by 6
Enhancing nursing practice through simulation: Addressing barriers and advancing the integration of artificial intelligence in healthcare
Benfatah, M., Elazizi, I., Belhaj, H., & Lamiri, A. (2025) · Journal of Nursing Regulation · doi:10.1016/j.jnr.2025.08.004 · Cited by 8
Medical student education for injury prevention: closing the gap
Zosel, A. E., Kohlbeck, S. A., Davis, C. S., Meurer, L. N., & Hargarten, S. W. (2020) · Injury Prevention · doi:10.1136/injuryprev-2020-043759 · Cited by 6
Research on the Influence of Nonmorphological Elements’ Cognition on Architectural Design Education in Universities: Third Year Architecture Core Studio in Special Topics “Urban Village Renovation Design”
Qiu, Z., Su, W., Chen, X., Xiang, X., Chen, Q., & Kong, J. (2023) · Buildings · doi:10.3390/buildings13092255 · Cited by 6
Measuring the effectiveness of L&D: Time to move on from Kirkpatrick and Phillips?
Kirwan, C. (2024) · International Journal of Training and Development · doi:10.1111/ijtd.12342 · Cited by 6
Drop-in sessions as an effective format for teaching information literacy:
Funnell, P. (2015) · Journal of Information Literacy · doi:10.11645/9.2.1982 · Cited by 6
Teaching peroral endoscopic pyloromyotomy (POP) to practicing endoscopists: An “into-the-fire” approach to simulation
Wong, H. J., Su, B., Attaar, M., Kuchta, K. M., Linn, J. G., Denham, W., Haggerty, S. P., & Ujiki, M. B. (2020) · Surgery · doi:10.1016/j.surg.2020.08.023 · Cited by 6
M3: The military medicine module: A focussed competency-based program
Lall, M., Datta, K., Iyengar, M. A., Shakya, A., & Kanitkar, M. (2021) · Medical Journal Armed Forces India · doi:10.1016/j.mjafi.2021.01.004 · Cited by 6
“specifically done to avoid bias and it also ensured the conduct of independent feedback analysis and genuine modification based on the feedback and not affected by the perception of the involved faculty and administration.(33) We have described a focussed competency-based modular approach to integrate military medical…”
Method
E-learning for WASH systems strengthening: lessons from a capacity-building platform
Leal, A., Saleh, A., & Verhoeven, J. (2022) · H2Open Journal · doi:10.2166/h2oj.2022.066 · Cited by 6
“Although the Kirkpatrick model has been criticised (Bates 2004), it has been used for decades and remains one of the most widely used tools for the evaluation of workplace training due to its systematic approach (Paula et al.”
Method
Towards high reliability in national pathology education: Evaluating the United States and Canadian Academy of Pathology educational product
Harris, C. K., Chen, Y., Jensen, K. C., Hornick, J. L., Kilfoyle, C., Lamps, L. W., & Heher, Y. (2022) · Academic Pathology · doi:10.1016/j.acpath.2022.100048 · Cited by 6
Development of a microlearning intervention regarding nursing nutritional care for older adults: A multi-methods study
ten Cate, D., Dikken, J., Ettema, R. G. A., Schoonhoven, L., & Schuurmans, M. J. (2022) · Nurse Education Today · doi:10.1016/j.nedt.2022.105623 · Cited by 6
Assessment of improvement in knowledge and skills amongst trainees of workshop on "labour and partograph".
Nausheen, S., Jalil, S., Anwer, T., & Akhter, A. Z. (2010) · PubMed · Cited by 6
Integrating Learning Outcome Typologies for HRD: Review and Current Status
Lim, D. H., Yoon, S. W., & Park, S. (2013) · New Horizons in Adult Education and Human Resource Development · doi:10.1002/nha.20015 · Cited by 6
“…little assumption of causal linkages between the levels, and little attention to the importance of positive correlations between the levels (Bates, 2004), this model, for almost five decades since its inception, has been one of the most frequently used evaluation models among HRD…”
Promoting Transition Resilience through Personal Development Planning: An Evaluation of the Perspectives of Preparation for Transition of Final-Year Undergraduate Nursing Students
Leigh, J., & Darvill, A. (2019) · International Journal of Practice-based Learning in Health and Social Care · doi:10.18552/ijpblhsc.v7i1.513 · Cited by 6
“(Bates 2004).”
The use of actor-based immersive health and safety inductions: Lessons from the Thames Tideway Tunnel megaproject
Harvey, E. J., Pinder, J. A., Haslam, R. A., Dainty, A., & Gibb, A. (2019) · Applied Ergonomics · doi:10.1016/j.apergo.2019.102955 · Cited by 6
The Impact of Emerging Simulation-Based Technologies on the Management of Deteriorating Patients: Aiming for a Gold Standard Educational Evaluation
Cooper, S. J., Cant, R., & Chung, C. E. (2020) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2020.05.004 · Cited by 6
Evaluation of Educational Workshops for Family Medicine Residents Using the Kirkpatrick Framework
Almeneessier, A. S., Alyousefi, N. A., Alwatban, L., Alodhayani, A. A., Alzahrani, A. M., Alwalan, S. I., AlSaad, S. Z., & Alonezan, A. (2021) · Advances in Medical Education and Practice · doi:10.2147/amep.s283379 · Cited by 6
“Level four of the Kirkpatrick model may not provide information on improving the structure and process of training programs.(11) The majority of program evaluations that used the Kirkpatrick model focused on level 1 (72%) and level 2 (31%) and to a lesser extent level 3 (20%) and level 4 (12%).”
Method
How Far Can a Short Leadership and Management Programme Address the Challenges for First Line Social Work Managers? An Evaluation of One of the Skills for Care Leadership and Management Demonstration Sites
Ward, J., & Bailey, D. (2015) · Practice · doi:10.1080/09503153.2015.1087492 · Cited by 6
“However, Kirkpatrick fails to connect these levels with an evaluation of the context in which the training takes place and it has also been criticised as being too simplistic (Bates 2004; Laughrin 2005).”
Background
Learning, behaviour and reaction framework: a model for training raters to improve assessment quality
Chen, C., Chang, H., Hsu, W., & Sheen, G. (2016) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2016.1180663 · Cited by 7
Evaluation of a Capacity-Building Programme of District Health Managers in India: A Contextualised Theoretical FrameworkPreprint
Srinivas, P. N., Marchal, B., Macq, J., Kegels, G., & Criel, B. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2451132 · Cited by 6
Palliative care simulation for internal medicine trainees: development and pilot study
Dewhurst, F., Howorth, K., Billett, H. C., Brown, J., Charles, M., Fleming, E., Gouldthorpe, C., Huggin, A., Kavanagh, E., Kiltie, R., et al. (2021) · BMJ Supportive & Palliative Care · doi:10.1136/bmjspcare-2021-003272 · Cited by 5
The Effects Of An Intervention In Student Teachers’ Classroom Management Learning Processes
Adams, T., Köster, B., & Brok, P. J. D. (2024) · Pedagogische Studiën · doi:10.59302/ps.v101i1.18784 · Cited by 5
“These studies also indicate that there is no valuable information found on Kirkpatricks’ fourth level (Reio et al., 2017; Bates, 2004).”
Background
Measuring the impact of faculty development programmes on business schools’ performance in Pakistan
Maqsood, N., Prendeville, P., & Bhatti, S. H. (2024) · Journal of Education for Business · doi:10.1080/08832323.2024.2372498 · Cited by 5
“Kirkpatrick’s model can provide a heuristic to guide training and developmental evaluations (Bates, 2004).”
A reflection on a leadership training programme of the Ministry of Interior, Thailand
Siengthai, S. (2015) · Human Resource Development International · doi:10.1080/13678868.2015.1071992 · Cited by 5
“By far, the most popular approach used to evaluate training in organizations today is Kirkpatrick’s (1975) framework of four progressive ‘levels’ of criteria: 1) reaction, 2) learning, 3) behaviour, and 4) results (Bates 2004; Antle, Barbee, and Van Zyl 2008).”
Method
Implementation evaluation in paralympic sports achievement development
Fabiano, K., Hidayatullah, M. F., & Purnama, S. K. (2021) · International journal of social sciences · doi:10.31295/ijss.v4n1.410 · Cited by 5
“Evaluation activities are carried out periodically to determine the level of success in implementing the training program (Bates, 2004; Faith, 1992).”
Background
Effects of different kinds of trust and leadership on outputs of collaborative processes: the case of personal social services in Israel
Lahat, L., & Sabah, Y. (2021) · International Journal of Public Sector Management · doi:10.1108/ijpsm-03-2020-0064 · Cited by 5
Improving Noncognitive Constructs for Career Readiness and Success: A Theory of Change for Postsecondary, Workplace, and Research Applications
Williams, K. M., Martín‐Raugh, M. P., & Lentini, J. E. (2022) · ETS Research Report Series · doi:10.1002/ets2.12346 · Cited by 7
Protocol for an integrated evaluation framework to study training, curricular and contextual factors impacting the success of a measurement for improvement training programme for healthcare staff in Ireland
Khurshid, Z., Brún, A. D., & McAuliffe, É. (2022) · BMJ Open · doi:10.1136/bmjopen-2020-047639 · Cited by 5
The form and function of U.S.-based science communication fellowship programs: interviews with program directors
Bennett, N., Dudo, A. D., & Besley, J. C. (2022) · International Journal of Science Education Part B · doi:10.1080/21548455.2022.2155495 · Cited by 8
“Instead, program directors can pull from, adapt, and apply procedures from the extensive, multi-disciplinary literature focused on evaluation practices (Barel-Ben David & Baram-Tsabari, 2000; Bates, 2004; Kirkpatrick, 1967; Praslova, 2010; Rodgers et al., 2020).”
Background
Evaluation of the Effectiveness of E-Training: A Case Study on In-Service Teachers' Training
Zheng, L., Huang, R., & Yu, J. (2013) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2013.71 · Cited by 5
The Incompatibility of End-User Learning Styles and the Current ERP Training Approach
Chayakonvikom, M., Fuangvut, P., & Prinyapol, N. (2015) · International Journal of Information and Education Technology · doi:10.7763/ijiet.2016.v6.736 · Cited by 8
Perceptions of robotics emulation of human ethics in educational settings: a content analysis
Fedock, B., Paladino, A. E., Bailey, L. W., & Moses, B. (2018) · Journal of Research in Innovative Teaching & Learning · doi:10.1108/jrit-02-2018-0004 · Cited by 5
“Though the framework was created before the emulation of human ethics was used by robots in educational settings, educators tend to create ethical policies which are based on Kitchener’s framework (Bates, 2004).”
Method
Effectiveness of training: The conceptual framework and methodological strategy of a longitudinal study in the Portuguese armed forces
Loureiro, N. A. R. S., & Alves, M. G. (2019) · Security and Defence Quarterly · doi:10.35467/sdq/106084 · Cited by 5
“Holton and Bates (Holton et al. 2000, 2007, Bates 2004, Holton 2005, Velada et al. 2009, Bates et al. 2012, Hutchins et al. 2013, Alvelos et al. 2015, Banerjee et al. 2016), two of the most prolific researchers in the field of training transfer, consider that training evaluation policies,…”
Background
Creating Effective eLearning to Help Drive Change
Mulcahy, R. S. (2020) · ACS Chemical Health & Safety · doi:10.1021/acs.chas.0c00091 · Cited by 5
Teaching EndoFLIP Impedance Planimetry to Practicing Endoscopists: An “Into the Fire” Approach to Simulation
Wong, H. J., Su, B., Attaar, M., Kuchta, K. M., Linn, J. G., Haggerty, S. P., Denham, W., & Ujiki, M. B. (2021) · Surgical Innovation · doi:10.1177/15533506211038088 · Cited by 5
Use of Kirkpatrick evaluation model in simulation-based trainings for the mining industry - A case study for froth flotation
Bergamo, P. A. D. S., Izart, C., Streng, E. S., Rosenkranz, J., & Ghorbani, Y. (2022) · Minerals Engineering · doi:10.1016/j.mineng.2022.107825 · Cited by 5
The Role of Evaluation Methods in Health-Related E-learning: A Rapid Review
Stemp, J. D., Ghosh, D., Khan, U. R., & Boyd, J. (2022) · Online Learning · doi:10.24059/olj.v26i4.3115 · Cited by 5
“…student responses about the quality of training; Level 2 (learning) quantifies learning using assessments and exams; Level 3 (behavior) measures the extent to which learning can be applied to the workplace; and Level 4 (results) measures how training has impacted organizational goals (Bates, 2004).”
Background
Evaluation of the Care Pathway in the Context of the Dispensing of Emicizumab (Hemlibra) in Community Pharmacies in France: Protocol for a Cross-sectional Study Based on the Kirkpatrick Model
Fraticelli, L., Freyssenge, J., Promé-Combel, E., Agnellet, E., Dargaud, Y. G., & Chamouard, V. (2023) · JMIR Research Protocols · doi:10.2196/43091 · Cited by 5
“The KM presents limitations [23] because the 4-level implementation can be time-consuming and requires an expensive investment to collect data and analyze learning outcomes.”
Background
The effect of the Educational Scholar Program as a longitudinal faculty development program on the capability of educators as scholars
Keshmiri‬, F. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04682-7 · Cited by 11
“The evaluation of the ESP process and learner-related outcomes of the ESP was conducted based on the Kirk-patrick Model [34].”
Method
E-learning for grass-roots emergency public health personnel: Preliminary lessons from a national program in China
Xu, W., Jiang, Q., Qin, X., Fang, G., & Hu, Z. (2016) · BioScience Trends · doi:10.5582/bst.2016.01083 · Cited by 5
Designing evaluation studies to optimally inform policy: what factors do policy-makers in China consider when making resource allocation decisions on healthcare worker training programmes?
Wu, S., Legido‐Quigley, H., Spencer, J., Coker, R. J., & Khan, M. S. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0292-2 · Cited by 7
An investigation of the effectiveness of research-extension joint human resource development programmes in Iran's agriculture sector
Zamani-Miandashti, N., Mohammadi, I. M., Hoseini, S. M., & Zamani, G. H. (2008) · Human Resource Development International · doi:10.1080/13678860802261694 · Cited by 5
“…and effectiveness, and easily define HRD evaluation as meeting programme goals; some other HRD professionals, like Kirkpatrick (1967, 1987, 1994), have suggested an HRD evaluation framework that is mainly an HRD effectiveness framework, because it only considers programme outcomes (Bates 2004).”
Background
Knowledge Transfer of Educational Programs in Public Management: Transfer-Inhibiting and Transfer-Enhancing Factors in the Belgian Public Sector
Broucker, B. (2010) · Journal of Public Affairs Education · doi:10.1080/15236803.2010.12001595 · Cited by 5
“Factors that influence transfer before, during, and after a program (Holton, 1996; Bates, 2005) exist on the organizational level (Broad & Newstrom, 1992; Awoniyi et al.”
BackgroundInfluential
Strategies for research in healthcare simulation
Nestel, D., & Kelly, M. (2017) · Wiley eBooks · doi:10.1002/9781119061656.ch6 · Cited by 5
Review: The evaluation of a home-based paediatric nursing service: concept and design development using The Kirkpatrick Model
Campbell, S. J., & Mather, C. A. (2018) · Journal of research in nursing · doi:10.1177/1744987118786083 · Cited by 4
“Moreau (2017), Frye and Hemmer (2012), Bates (2004) and Holton (1996) have addressed Kirkpatrick’s original model (1996) and its failure to address specific indicators, such as motivation and baseline knowledge of the health professionals being trained.”
BackgroundInfluential
Evaluation of the comprehensive thematic teaching effectiveness and technique/technology in culinary vocational education
Ma, C.-Y., Wang, K.-C., Liu, D.-Y., & Lai, T.-C. (2023) · Education + Training · doi:10.1108/et-09-2022-0370 · Cited by 4
Evaluation of cost-effectiveness of single-credit traffic safety course based on Kirkpatrick model: a case study of Iran
Golestani, M., Sadeghi‐Bazargani, H., Harzand-Jadidi, S., & Soori, H. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05122-w · Cited by 4
“…before, during, and after participating in the course), (3) behavior level (the type and degree of changes in the behavior of participants as a result of participating in training courses), and (4) results level (the level of achievement of goals directly related to the organization) [9, 10].”
Background
Evaluating a research methodology workshop among postgraduate students using Kirkpatrick’s model
Debata, I., Nayak, S., Ahmed, S., Behera, B. K., & Padhee, S. (2024) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1026_23 · Cited by 4
“[10] This model consists of four evaluation levels which are interdependent, shown in Figure 1.”
Method
Enhancing post-training evaluation of annual performance agreement training: A fusion of fsQCA and artificial neural network approach
Islam, M. Z., Jilani, M. M. A. K., & Karim, M. R. (2024) · PLoS ONE · doi:10.1371/journal.pone.0305916 · Cited by 4
Effectiveness of Agricultural Human Resource Development Interventions in Iran (Three Cases in Fars Province)
Zamani-Miandashti, N., & Malekmohammadi, I. (2012) · Cited by 4
Disrupting the Implementation Gap with Digital Technology in Healthcare Distance Education: Critical Insights from an e-Mentoring Intensional Network Practitioner Research Project.
Singh, G. (2013) · European Journal of Open Distance and E-Learning · Cited by 4
HRD research trends on the application of theories in Korean HRD journals (2007~2016)
Yoon, S. K., Cha, M., 송영수, & Jeong, B. Y. (2017) · The Korean Journal of Human Resource Development Quarterly · doi:10.18211/kjhrdq.2017.19.3.001 · Cited by 4
Creation and Pilot-testing of Virtual Patients for Learning Oncologic Emergency Management
Fawaz, Z. S., Posel, N. H., Royal-Preyra, B., Khriguian, J., & Alfieri, J. (2019) · Cureus · doi:10.7759/cureus.6206 · Cited by 4
“This model can also be integrated in the Kirkpatrick Four-level model of evaluation which includes the reaction of the learner, learning the concepts, behavioral application of the knowledge, and results in terms of performance [3].”
Method
Integration of Instructional Models and Learning Styles for Open and Distance Learning Environment
Zulkifli, Z. A., Sharip, A. A., Zain, S. M. M., Rahid, N. N. A., Saidi, R. M., Rashid, N. A. M., & Geigiana, A. (2022) · World Journal of English Language · doi:10.5430/wjel.v12n2p226 · Cited by 4
Implementation of an educational intervention to improve medical student cost awareness: a prospective cohort study
Tait, S. D., Oshima, S. M., Leraas, H. J., Gunn, A. H., Sarver, M. M., Gunes, F., & Greenup, R. A. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04038-1 · Cited by 4
“Primary outcome measures were based upon Kirkpat-rick’s four-level training evaluation model, [29] and included: acceptability of the educational intervention; student knowledge regarding treatment-related financial hardship; student attitudes regarding cost-conscious practices; and student…”
Method
How, and under what contexts, do academic–practice partnerships collaborate to implement healthcare improvement education into preregistration nursing curriculums: a realist review protocol
Armstrong, L., Moir, C., & Taylor, P. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-077784 · Cited by 4
“31 Ignoring context is considered to be the antithesis of healthcare improvement principles and is instead a prerequisite to understanding why success ensues in one instance and not in another.”
Background
Research protocol of the Laval-ROSA Transilab: a living lab on transitions for people living with dementia
Dufour, I., Arsenault‐Lapierre, G., Guillette, M., Dame, N., Poitras, M.-È., Lussier, M., Fortier, A., Brunet, J., Martin, J., Laverdure, M., et al. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-10248-6 · Cited by 5
“The questionnaires will be constructed using the Kirkpatrick model [35], used to evaluate the results of training and learning programs, and the validated Continuing professional development (CPD)- Reaction tool [36], adapted for the three specific trainings.”
Method
The Business of Corporate Learning: Insights from Practice
Ben‐Hur, S. (2013) · Cited by 4
Implementation fidelity in a loosely coupled system: the challenges of maintaining consistent ‘problem theory’ and ‘programme theory’ in a multi-force training pilot
Giacomantonio, C., & Litmanovitz, Y. (2017) · Policing & Society · doi:10.1080/10439463.2017.1341511 · Cited by 4
“However, as other studies of police training have suggested (e.g. Litmanovitz 2016), positive post-course assessments are not strong predictors of successful outcomes – in this case, actual behavioural or attitudinal changes (see also Webster-Stratton 1990, Bates 2004).”
Background
Evaluating Effectiveness of a Teacher Training MOOC: Industry Perspective
Soorya, M., & Banerjee, G. (2019) · IEEE International Conference on Technology for Education (T4E) · doi:10.1109/t4e.2019.00-42 · Cited by 4
“Data reported in this paper was collected through a postcourse survey adapted from [9].”
BackgroundMethod
Satisfaction with Education
Botek, M. (2013) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2013.12.261 · Cited by 4

Authors: Bates, R.
Affiliations: Louisiana State University (US)
Corresponding author: Bates, Reid
Venue: Evaluation and Program Planning
Publisher: Elsevier BV
Published: 2004-08
Volume: 27
Issue: 3
Pages: 341-347
ISSN: 0149-7189 (ISSN-L)
DOI: 10.1016/j.evalprogplan.2004.04.011
OpenAlex: W2097592370
Cited by: 623
References: 44
Citations by year: 2026: 53 · 2025: 23 · 2024: 42 · 2023: 53 · 2022: 49 · 2021: 39 · 2020: 43 · 2019: 44 · 2018: 32 · 2017: 45 · 2016: 32 · 2015: 32 · 2014: 30 · 2013: 29 · 2012: 22
FWCI: 4.09
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Evaluation and Program Planning (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Social Sciences › Psychology › Applied Psychology › Human Resource Development and Performance Evaluation
Social Sciences › Public Administration › Evaluation and Performance Assessment
Social Sciences › Education › Evaluation of Teaching Practices
Keywords: Kirkpatrick model, training evaluation, beneficence, moral obligation, formative assessment, summative assessment, training effectiveness, human resource development, evaluation limitations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks