CoChat
Sign Up
Cited by 69Explainable Artificial Intelligence (XAI)Surgical Simulation and TrainingArtificial Intelligence in Healthcare and Education

A multi-institutional study using artificial intelligence to provide reliable and fair feedback to surgeons

Open access
Kiyasseh, Dani, Laca, Jasper, Haque, Taseen F., Miles, Brian J., Wagner, Christian, Donoho, Daniel A., Anandkumar, Animashree, Hung, Andrew J.·2023·Communications Medicine
DOI 10.1038/s43856-023-00263-3PMC PMC10063640
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Surgeons who receive reliable feedback on their performance quickly master the skills necessary for surgery. Such performance-based feedback can be provided by a recently-developed artificial intelligence (AI) system that assesses a surgeon's skills based on a surgical video while simultaneously highlighting aspects of the video most pertinent to the assessment. However, it remains an open question whether these highlights, or explanations, are equally reliable for all surgeons. METHODS: Here, we systematically quantify the reliability of AI-based explanations on surgical videos from three hospitals across two continents by comparing them to explanations generated by humans experts. To improve the reliability of AI-based explanations, we propose the strategy of training with explanations -TWIX -which uses human explanations as supervision to explicitly teach an AI system to highlight important video frames. RESULTS: We show that while AI-based explanations often align with human explanations, they are not equally reliable for different sub-cohorts of surgeons (e.g., novices vs. experts), a phenomenon we refer to as an explanation bias. We also show that TWIX enhances the reliability of AI-based explanations, mitigates the explanation bias, and improves the performance of AI systems across hospitals. These findings extend to a training environment where medical students can be provided with feedback today. CONCLUSIONS: Our study informs the impending implementation of AI-augmented surgical training and surgeon credentialing programs, and contributes to the safe and fair democratization of surgery.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s43856-023-00263-3

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI applications in surgery hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision.

    “In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision.”

    From Bias in AI-based models for medical applications: challenges and mitigation strategies (2023) · cites this paper as [3]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 46

  1. [1]Ende J Feedback in clinical medical education JAMA. 1983 250 777 781 10.1001/jama.1983.03340060055026 6876333 · DOI
  2. [2]Roberts KE Bell RL Duffy AJ Evolution of surgical skills training World J. Gastroenterol.: WJG 2006 12 3219 10.3748/wjg.v12.i20.3219 16718842 PMC4087965 · DOI
  3. [3]Karam MD Surgical coaching from head-mounted video in the training of fluoroscopically guided articular fracture surgery JBJS. 2015 97 1031 1039 10.2106/JBJS.N.00748 26085538 · DOI
  4. [4]Singh P Aggarwal R Tahir M Pucher PH Darzi A A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills Annals Surgery. 2015 261 862 869 10.1097/SLA.0000000000000857 25185469 · DOI
  5. [5]Yule S Coaching non-technical skills improves surgical residents’ performance in a simulated operating room J. Surgical Education. 2015 72 1124 1130 10.1016/j.jsurg.2015.06.012 26610355 · DOI · Paper page
  6. [6]Bonrath EM Dedy NJ Gordon LE Grantcharov TP Comprehensive surgical coaching enhances surgical skill in the operating room Annals Surgery. 2015 262 205 212 10.1097/SLA.0000000000001214 25822691 · DOI · Paper page
  7. [7]Hu Y-Y Complementing operating room teaching with video-based coaching JAMA Surgery. 2017 152 318 325 10.1001/jamasurg.2016.4619 27973648 · DOI
  8. [8]Gunning D XAI - explainable artificial intelligence Sci. Robotics. 2019 4 eaay7120 10.1126/scirobotics.aay7120 33137719 · DOI · Paper page
  9. [9]Yuan L In situ bidirectional human-robot value alignment Sci. Robotics 2022 7 eabm4183 10.1126/scirobotics.abm4183 35857532 · DOI
  10. [10]Kiyasseh, D. et al. A vision transformer for decoding surgeon activity from surgical videos. Nat. Biomed. Eng.10.1038/s41551-023-01010-8 (2023). PMC10307635 36997732 · DOI
  11. [15]Vaswani, A. et al. Attention is all you need. Adv. Neural Inform. Process. Sys.30, (2017).
  12. [16]Wiegreffe, S. & Marasovic, A. Teach me to explain: A review of datasets for explainable natural language processing. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) (2021).
  13. [17]Vig, J. et al. Bertology meets biology: Interpreting attention in protein language models. In International Conference on Learning Representations (2020).
  14. [19]Liang W Advances, challenges and opportunities in creating data for trustworthy ai Nat. Machine Intell. 2022 4 669 677 10.1038/s42256-022-00516-1 · DOI
  15. [20]Lundberg SM Explainable machine-learning predictions for the prevention of hypoxaemia during surgery Nature Biomedical Engineering 2018 2 749 760 10.1038/s41551-018-0304-0 31001455 PMC6467492 · DOI · Paper page
  16. [21]Hooker, S., Erhan, D., Kindermans, P.-J. & Kim, B. A benchmark for interpretability methods in deep neural networks. Adv. Neural Inform. Process. Sys.32, 9734–9745 (2019).
  17. [22]Barnett AJ A case-based interpretable deep learning model for classification of mass lesions in digital mammography Nat. Machine Intell. 2021 3 1061 1070 10.1038/s42256-021-00423-x · DOI
  18. [23]Lauritsen SM Explainable artificial intelligence model to predict acute critical illness from electronic health records Nat. Commun. 2020 11 1 11 10.1038/s41467-020-17431-x 32737308 PMC7395744 · DOI · Paper page
  19. [24]Zaidan, O., Eisner, J. & Piatko, C. Using “annotator rationales” to improve machine learning for text categorization. In Conference of the North American Association for Computational Linguistics, 260–267 (2007).
  20. [26]Dai, J., Upadhyay, S., Aivodji, U., Bach, S. H. & Lakkaraju, H. Fairness via explanation quality: Evaluating disparities in the quality of post hoc explanations. Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society 203–214 (2022).
  21. [27]Balagopalan, A. et al. The road to explainability is paved with bias: Measuring the fairness of explanations. 2022 ACM Conference on Fairness, Accountability, and Transparency 1194–1206 (2022).
  22. [28]Haque TF Development and validation of the end-to-end assessment of suturing expertise (EASE) J. Urology. 2022 207 e153 10.1097/JU.0000000000002532.17 PMC9948038 36844996 · DOI
  23. [29]Martinez CH Effect of prostate gland size on the learning curve for robot-assisted laparoscopic radical prostatectomy: does size matter initially? J. Endourol. 2010 24 261 266 10.1089/end.2009.0325 20073551 · DOI
  24. [30]Goldstraw M Overcoming the challenges of robot-assisted radical prostatectomy Prostate Cancer Prostatic Dis. 2012 15 1 7 10.1038/pcan.2011.37 21844888 · DOI
  25. [31]Hung AJ Face, content and construct validity of a novel robotic surgery simulator J. Urology. 2011 186 1019 1025 10.1016/j.juro.2011.04.064 21784469 · DOI
  26. [33]Hind, M. et al. Ted: Teaching AI to explain its decisions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 123–129 (2019).
  27. [34]Kailkhura B Gallagher B Kim S Hiszpanski A Han T Reliable and explainable machine-learning methods for accelerated material discovery NPJ Computational Materials. 2019 5 1 9 10.1038/s41524-019-0248-2 · DOI
  28. [35]Rieger, L., Singh, C., Murdoch, W. & Yu, B. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In International Conference on Machine Learning, 8116–8126 (PMLR, 2020).
  29. [36]Lampinen, A. K. et al. Tell me why! explanations support learning relational and causal structure. In International Conference on Machine Learning, 11868–11890 (PMLR, 2022).
  30. [37]Jacovi, A., Marasović, A., Miller, T. & Goldberg, Y. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 624–635 (2021).
  31. [38]Nam CS Daignault-Newton S Kraft KH Herrel LA Projected us urology workforce per capita, 2020-2060 JAMA Network Open. 2021 4 e2133864 e2133864 10.1001/jamanetworkopen.2021.33864 34783827 PMC8596195 · DOI
  32. [39]Nguyen, G., Kim, D. & Nguyen, A. The effectiveness of feature attribution methods and its correlation with automatic evaluation scores. Adv. Neural Inform. Process. Sys.34, 26422–26436 (2021).
  33. [41]Adebayo J Muelly M Liccardi I Kim B Debugging tests for model explanations Adv. Neural Inform. Process. Sys. 2020 33 700 712
  34. [42]Adebayo, J., Muelly, M., Abelson, H. & Kim, B. Post hoc explanations may be ineffective for detecting unknown spurious correlation. In International Conference on Learning Representations (2021).
  35. [43]Agarwal, C. et al. Openxai: Towards a transparent evaluation of model explanations. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2022).
  36. [44]Zhong, R., Shao, S. & McKeown, K. Fine-grained sentiment analysis with faithful attention. arXiv preprint arXiv:1908.06870 (2019). · arXiv
  37. [45]Abboudi H Learning curves for urological procedures: a systematic review BJU Int. 2014 114 617 629 10.1111/bju.12315 24053179 · DOI
  38. [46]Birkmeyer JD Surgical skill and complication rates after bariatric surgery New England J. Med. 2013 369 1434 1442 10.1056/NEJMsa1300625 24106936 · DOI · Paper page
  39. [47]Stulberg JJ Association between surgeon technical skills and patient outcomes JAMA Surgery. 2020 155 960 968 10.1001/jamasurg.2020.3007 32838425 PMC7439214 · DOI
  40. [48]Noetel M Video improves learning in higher education: A systematic review Rev. Educ. Res. 2021 91 204 236 10.3102/0034654321990713 · DOI
  41. [49]Saedon H Salleh S Balakrishnan A Imray CH Saedon M The role of feedback in improving the effectiveness of workplace based assessments: a systematic review BMC Med. Educ. 2012 12 1 8 10.1186/1472-6920-12-25 22551353 PMC3432628 · DOI
  42. [51]Archer JC State of the science in health professional education: effective feedback Med. Educ. 2010 44 101 108 10.1111/j.1365-2923.2009.03546.x 20078761 · DOI · Paper page
  43. [54]Farquharson A Cresswell A Beard J Chan P Randomized trial of the effect of video feedback on the acquisition of surgical skills J. British Surg. 2013 100 1448 1453 10.1002/bjs.9237 24037563 · DOI
  44. [57]Romero P Halsted’s “see one, do one, and teach one” versus peyton’s four-step approach: a randomized trial for training of laparoscopic suturing and knot tying J. Surg. Educ. 2018 75 510 515 10.1016/j.jsurg.2017.07.025 28801083 · DOI
  45. [58]Ajao OG Alao A Surgical residency training in developing countries: West african college of surgeons as a case study J. Natl Med. Assoc. 2016 108 173 179 27692359
  46. [59]Ng-Kamstra JS Global surgery 2030: a roadmap for high income country actors BMJ Global Health. 2016 1 e000011 10.1136/bmjgh-2015-000011 28588908 PMC5321301 · DOI

Cited by

Sorted by citations

Authors
Kiyasseh, D., Laca, J., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J.
Affiliations
  • California Institute of Technology (US)
  • University of Southern California (US)
  • Houston Methodist (US)
  • St.-Antonius-Hospital Gronau (DE)
  • Children's National (US)
Corresponding author
  • Kiyasseh, Dani
  • Hung, Andrew J.
ORCID
Venue
Communications Medicine
Publisher
Springer Science and Business Media LLC
Published
2023-03-30
Volume
3
Issue
1
Pages
42-42
ISSN
2730-664X (ISSN-L)
DOI
10.1038/s43856-023-00263-3
PMC
PMC10063640
OpenAlex
W4362506180
Cited by
69
References
10
Citations by year
2026: 13 · 2025: 21 · 2024: 24 · 2023: 11
FWCI
6.50
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
U.S. Department of Health & Human Services | NIH | National Cancer Institute (R01CA251579-02)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
surgical skill assessment, surgical video analysis, AI explanations, surgeon skill level, surgical training, surgeon credentialing, algorithmic fairness, medical students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review