The DOI matches its Crossref record.
Objective structured assessment of technical skill (OSATS) for surgical residents
Abstract
BACKGROUND: The technical skill of surgical trainees is not well assessed. This study aimed (1) to compare the reliability of three scoring systems, (2) to compare live and bench formats and (3) to assess construct validity of a test of operative skill. METHODS: Parallel examinations of operative skill, one using live animals and one using simulations, were developed. Performance was graded using operation-specific checklists, detailed global rating forms and pass/fail judgements. Twenty surgical residents each took both formats. RESULTS: Disattenuated correlations between live and bench scores were high (0.69-0.72). Mean interrater reliability across stations ranged from 0.64 to 0.72. Internal consistency was moderate to high (alpha: 0.61-0.74) for the live format using the checklist and for live and bench formats using global ratings. Global ratings discriminated between resident levels for both formats (bench: F(2,17) = 4.45, P < 0.05; live: F(2,17) = 3.55, P < 0.05), checklists did not. CONCLUSION: This preliminary study suggests that the Objective Structured Assessment of Technical Skill can reliably and validly assess surgical skills. Global ratings are a better method of assessment than task-specific checklists. Bench model simulation gives equivalent results to use of live animals for this test format.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Surgical skill and complication rates after bariatric surgery
Birkmeyer, J. D., Finks, J. F., O'Reilly, A., Oerline, M., Carlin, A. M., Nunn, A. R., Dimick, J., Banerjee, M., & Birkmeyer, N. J. O. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1300625 · Cited by 1,504
Training and simulation for patient safety
Aggarwal, R., Mytton, O. T., Derbrew, M., Hananel, D., Heydenburg, M., Issenberg, B., MacAulay, C., Mancini, M. E., Morimoto, T., Soper, N., et al. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2009.038562 · Cited by 739
Self-citationTeaching Surgical Skills: What Kind of Practice Makes Perfect?: A Randomized, Controlled Trial
Moulton, C.-A. E., Dubrowski, A., MacRae, H., Graham, B., Grober, E., & Reznick, R. (2006) · Annals of Surgery · doi:10.1097/01.sla.0000234808.85789.6a · Cited by 653
Self-citationA contemporary approach to validity arguments: a practical guide to Kane's framework
Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. (2015) · Medical Education · doi:10.1111/medu.12678 · Cited by 554
A systematic review of the reliability of objective structured clinical examination scores
Brannick, M. T., Erol‐Korkmaz, H. T., & Prewett, M. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04075.x · Cited by 354
Validation of educational assessments: a primer for simulation and beyond
Cook, D. A., & Hatala, R. (2016) · Advances in Simulation · doi:10.1186/s41077-016-0033-y · Cited by 304
Fazlollahi, A. M., Bakhaidar, M., Alsayegh, A., Yilmaz, R., Winkler-Schwartz, A., Mirchi, N., Langleben, I., Ledwos, N., Sabbagh, A. J., Bajunaid, K., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.49008 · Cited by 285
Bonrath, E. M., Dedy, N. J., Gordon, L. E., & Grantcharov, T. P. (2015) · Annals of Surgery · doi:10.1097/sla.0000000000001214 · Cited by 260
AI in Medical Education: Global situation, effects and challenges
Zhang, W., Cai, M., Lee, H. J., Evans, R., Zhu, C., & Ming, C. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12009-8 · Cited by 148
Simulation-Based Training in Cardiac Surgery
Feins, R. H., Burkhart, H. M., Conte, J. V., Coore, D. N., Fann, J. I., Hicks, G. L., Nesbitt, J. C., Ramphal, P. S., Schiro, S. E., Shen, K. R., et al. (2017) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2016.06.062 · Cited by 148
Video and accelerometer-based motion analysis for automated surgical skills assessment
Zia, A., Sharma, Y., Bettadapura, V., Sarin, E. L., & Essa, I. (2018) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1704-z · Cited by 134
Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating
Azari, D. P., Frasier, L. L., Quamme, S. R. P., Greenberg, C. C., Pugh, C. M., Greenberg, J. A., & Radwin, R. G. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000002478 · Cited by 84
LeBlanc, J., Hutchison, C., Hu, Y., & Donnon, T. (2013) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.k.01284 · Cited by 66
Self-citationShaw, K., Henning, M. A., & Webster, C. S. (2025) · Medical Science Educator · doi:10.1007/s40670-025-02373-0 · Cited by 58
Technical performance score is associated with outcomes after the Norwood procedure
Nathan, M., Sleeper, L. A., Ohye, R. G., Frommelt, P. C., Caldarone, C. A., Tweddell, J. S., Lu, M., Pearson, G. D., Gaynor, J. W., Pizarro, C., et al. (2014) · The Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2014.05.076 · Cited by 54
Accuracy of augmented reality-assisted pedicle screw placement: A systematic review
Youssef, S., McDonnell, J. M., Wilson, K. V., Turley, L., Cunniffe, G., Morris, S., Darwish, S., & Butler, J. S. (2024) · European Spine Journal · doi:10.1007/s00586-023-08094-5 · Cited by 36
“However, it is often considered a low-cost alternative requiring minimum infrastructure compared to certain intraoperative navigation systems and can be employed as a supplement to existing equipment for procedures other than pedicle screw placement, such as operating microscopes [37].”
Educational strategies in performing cesarean section
Madsen, K., Grønbeck, L., Rifbjerg Larsen, C., Østergaard, J., Bergholt, T., Langhoff‐Roos, J., & Sørensen, J. L. (2013) · Acta Obstetricia et Gynecologica Scandinavica · doi:10.1111/aogs.12055 · Cited by 29
Bull, N. B., Silverman, C. D., & Bonrath, E. M. (2020) · Surgery · doi:10.1016/j.surg.2019.08.002 · Cited by 27
Metadata
- Authors
- Martin, J. A., Regehr, G., Reznick, R., Macrae, H., Murnaghan, J., Hutchison, C., & Brown, M.
- Venue
- British Journal of Surgery
- Publisher
- Oxford University Press (OUP)
- Published
- 1997-02
- Volume
- 84
- Issue
- 2
- Pages
- 273-278
- ISSN
- 0007-1323 (ISSN-L)
- 1365-2168
- OpenAlex
- W2109498513
- Cited by
- 2,527
- References
- 0
- FWCI
- 10.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
