CoChat

Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence

Ledwos, Nicole, Mirchi, Nykan, Yilmaz, Recai, Winkler-Schwartz, Alexander, Sawni, Anika, Fazlollahi, Ali M., Bissonnette, Vincent, Bajunaid, Khalid, Sabbagh, Abdulrahman J., Del Maestro, Rolando F. · 2022 · Journal of Neurosurgery
Cited by 36

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

OBJECTIVE: Understanding the variation of learning curves of experts and trainees for a given surgical procedure is important in implementing formative learning paradigms to accelerate mastery. The study objectives were to use artificial intelligence (AI)–derived metrics to determine the learning curves of participants in 4 groups with different expertise levels who performed a series of identical virtual reality (VR) subpial resection tasks and to identify learning curve differences among the 4 groups. METHODS: A total of 50 individuals participated, 14 neurosurgeons, 4 neurosurgical fellows and 10 senior residents (seniors), 10 junior residents (juniors), and 12 medical students. All participants performed 5 repetitions of a subpial tumor resection on the NeuroVR (CAE Healthcare) platform, and 6 a priori–derived metrics selected using the K-nearest neighbors machine learning algorithm were used to assess participant learning curves. Group learning curves were plotted over the 5 trials for each metric. A mixed, repeated-measures ANOVA was performed between the first and fifth trial. For significant interactions (p < 0.05), post hoc Tukey’s HSD analysis was conducted to determine the location of the significance. RESULTS: Overall, 5 of the 6 metrics assessed had a significant interaction (p < 0.05). The 4 groups, neurosurgeons, seniors, juniors, and medical students, showed an improvement between the first and fifth trial on at least one of the 6 metrics evaluated. CONCLUSIONS: Learning curves generated using AI-derived metrics provided novel insights into technical skill acquisition, based on expertise level, during repeated VR-simulated subpial tumor resections, which will allow educators to develop more focused formative educational paradigms for neurosurgical trainees.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3171/2021.12.jns211563

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 19

  1. Hebb AO, Yang T, Silbergeld DL. The sub-pial resection technique for intrinsic tumor surgery. Surg Neurol Int. 2011;2:180.2236878610.4103/2152-7806.90714 · DOI
  2. Sabbagh AJ, Bajunaid KM, Alarifi N, Roadmap for developing complex virtual reality simulation scenarios: subpial neurosurgical tumor resection model. World Neurosurg. 2020;139:e220–e229.3228951010.1016/j.wneu.2020.03.187 · DOI · Paper page
  3. Siyar S, Azarnoush H, Rashidi S, Del Maestro RF. Tremor assessment during virtual reality brain tumor resection. J Surg Educ. 2020;77(3):643–651.3182238910.1016/j.jsurg.2019.11.011 · DOI
  4. Winkler-Schwartz A, Yilmaz R, Mirchi N, Machine learning identification of surgical and operative factors associated with surgical expertise in virtual reality simulation. JAMA Netw Open. 2019;2(8):e198363.3137365110.1001/jamanetworkopen.2019.8363 · DOI · Paper page
  5. Chan J, Pangal DJ, Cardinal T, A systematic review of virtual reality for the assessment of technical skills in neurosurgery. Neurosurg Focus. 2021;51(2):E15.3433347210.3171/2021.5.FOCUS21210 · DOI
  6. Bartlett JD, Lawrence JE, Yan M, The learning curves of a validated virtual reality hip arthroscopy simulator. Arch Orthop Trauma Surg. 2020;140(6):761–767.3198924510.1007/s00402-020-03352-3 · doi:10.1007/s00402-020-03352-3
  7. Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg. 2007;246(5):771–779.1796816810.1097/SLA.0b013e3180f61b09 · DOI · Paper page
  8. Janse JA, Goedegebuure RS, Veersema S, Broekmans FJ, Schreuder HW. Hysteroscopic sterilization using a virtual reality simulator: assessment of learning curve. J Minim Invasive Gynecol. 2013;20(6):775–782.2380656110.1016/j.jmig.2013.04.016 · doi:10.1016/j.jmig.2013.04.016
  9. Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, Winkler-Schwartz A, Del Maestro RF. Artificial intelligence distinguishes surgical training levels in a virtual reality spinal task. J Bone Joint Surg Am. 2019;101(23):e127.10.2106/JBJS.18.01197 · DOI · Paper page
  10. Mirchi N, Bissonnette V, Ledwos N, Artificial neural networks to assess virtual reality anterior cervical discectomy performance. Oper Neurosurg (Hagerstown). 2020;19(1):65–75.10.1093/ons/opz359 · DOI · Paper page
  11. Sewell C, Morris D, Blevins NH, Providing metrics and performance feedback in a surgical simulator. Comput Aided Surg. 2008;13(2):63–81.10.3109/10929080801957712 · doi:10.3109/10929080801957712
  12. Azarnoush H, Siar S, Sawaya R, The force pyramid: a spatial analysis of force application during virtual reality brain tumor resection. J Neurosurg. 2017;127(1):171–181.2768945810.3171/2016.7.JNS16322 · DOI
  13. Sawaya R, Bugdadi A, Azarnoush H, Virtual reality tumor resection: the force pyramid approach. Oper Neurosurg (Hagerstown). 2018;14(6):686–696.10.1093/ons/opx189 · DOI
  14. World Medical A. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–2194.10.1001/jama.2013.281053 · DOI · Paper page
  15. Winkler-Schwartz A, Bissonnette V, Mirchi N, Artificial intelligence in medical education: best practices using machine learning to assess surgical expertise in virtual reality simulation. J Surg Educ. 2019;76(6):1681–1690.3120263310.1016/j.jsurg.2019.05.015 · DOI · Paper page
  16. Brunozzi D, McGuire LS, Alaraj A. NeuroVR Simulator in Neurosurgical Training. In: Alaraj A, ed.Comprehensive Healthcare Simulation: Neurosurgery. Springer International Publishing;2018:211–218.10.1007/978-3-319-75583-0_16 · doi:10.1007/978-3-319-75583-0_16
  17. Alaraj A, Tobin MK, Birk DM, Charbel FT. Simulation in neurosurgery and neurosurgical procedures. In: Levine AI, DeMaria S, Schwartz AD, Sim AJ, eds. The Comprehensive Textbook of Healthcare Simulation. Springer New York;2013:415–423.10.1007/978-1-4614-5993-4_28 · doi:10.1007/978-1-4614-5993-4_28
  18. Mirchi N, Ledwos N, Del Maestro RF. Intelligent tutoring systems: re-envisioning surgical education in response to COVID-19. Can J Neurol Sci. 2021;48(2):198–200.3290764410.1017/cjn.2020.202 · DOI
  19. Tomlinson SB, Hendricks BK, Cohen-Gadol AA. Editorial Innovations in neurosurgical education during the COVID-19 pandemic: is it time to reexamine our neurosurgical training models? J Neurosurg. 2020;133(1):14–15.10.3171/2020.4.JNS201012 · DOI

Cited by

See all 36

Metadata

Authors
Ledwos, N., Mirchi, N., Yilmaz, R., Winkler-Schwartz, A., Sawni, A., Fazlollahi, A. M., Bissonnette, V., Bajunaid, K., Sabbagh, A. J., & Del Maestro, R. F.
Venue
Journal of Neurosurgery
Published
2022-10-01
Volume
137
Issue
4
Pages
1160-1171
ISSN
  • 0022-3085 (ISSN-L)
  • 1933-0693
OpenAlex
W4210345742
Cited by
36
References
2
FWCI
5.51
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence. https://app.cochat.ai/papers/10.3171/2021.12.jns211563
The paper
Ledwos, N., Mirchi, N., Yilmaz, R., Winkler-Schwartz, A., Sawni, A., Fazlollahi, A. M., Bissonnette, V., Bajunaid, K., Sabbagh, A. J., & Del Maestro, R. F. (2022). Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence. Journal of Neurosurgery. https://doi.org/10.3171/2021.12.jns211563

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.