Development and Validation of Objective Performance Metrics for Robot-Assisted Radical Prostatectomy: A Pilot Study
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
PURPOSE: We explore and validate objective surgeon performance metrics using a novel recorder ("dVLogger") to directly capture surgeon manipulations on the da Vinci® Surgical System. We present the initial construct and concurrent validation study of objective metrics during preselected steps of robot-assisted radical prostatectomy. MATERIALS AND METHODS: Kinematic and events data were recorded for expert (100 or more cases) and novice (less than 100 cases) surgeons performing bladder mobilization, seminal vesicle dissection, anterior vesicourethral anastomosis and right pelvic lymphadenectomy. Expert/novice metrics were compared using mixed effect statistical modeling (construct validation). Expert reviewers blindly rated seminal vesicle dissection and anterior vesicourethral anastomosis using GEARS (Global Evaluative Assessment of Robotic Skills). Intraclass correlation measured inter-rater variability. Objective metrics were correlated to corresponding GEARS metrics using Spearman's test (concurrent validation). RESULTS: The performance of 10 experts (mean 810 cases, range 100 to 2,000) and 10 novices (mean 35 cases, range 5 to 80) was evaluated in 100 robot-assisted radical prostatectomy cases. For construct validation the experts completed operative steps faster (p <0.001) with less instrument travel distance (p <0.01), less aggregate instrument idle time (p <0.001), shorter camera path length (p <0.001) and more frequent camera movements (p <0.03). Experts had a greater ratio of dominant-to-nondominant instrument path distance for all steps (p <0.04) except anterior vesicourethral anastomosis. For concurrent validation the median experience of 3 expert reviewers was 300 cases (range 200 to 500). Intraclass correlation among reviewers was 0.6-0.7. For anterior vesicourethral anastomosis and seminal vesicle dissection, kinematic metrics had low associations with GEARS metrics. CONCLUSIONS: Objective metrics revealed experts to be more efficient and directed during preselected steps of robot-assisted radical prostatectomy. Objective metrics had limited associations to GEARS. These findings lay the foundation for developing standardized metrics for surgeon training and assessment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Surgical skill and complication rates after bariatric surgery
Birkmeyer, J. D., Finks, J. F., O'Reilly, A., Oerline, M., Carlin, A. M., Nunn, A. R., Dimick, J., Banerjee, M., & Birkmeyer, N. J. O. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1300625 · Cited by 1,504 · cited 1×
“2018 by AMERICAN UROLOGICAL ASSOCIATION EDUCATION AND RESEARC rates.(1,2) Even after controlling for surgical volume and experience, for radical prostatectomy the differences”
Method
Cited by
Open-Sourced Reinforcement Learning Environments for Surgical Robotics
Richter, F., Orosco, R. K., & Yip, M. C. (2019) · arXiv · doi:10.48550/arxiv.1903.02090 · Cited by 92
“While these systems are already providing great care to patients, they have also opened the door to a variety of research including surgeon performance metrics [6], remote teleoperation [7], and surgical task automation [8].”
BackgroundHuman visual explanations mitigate bias in AI-based assessment of surgeon skills
Kiyasseh, D., Laca, J., Haque, T. F., Otiato, M., Miles, B. J., Wagner, C., Donoho, D. A., Trinh, Q.-D., Anandkumar, A., & Hung, A. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00766-2 · Cited by 53
“To de fi ne novice and expert surgeons, we built on previous literature which uses surgeon caseload, the total number of robotic surgeries performed by a surgeon during their lifetime, as a proxy 54 – 56 .”
MethodSelf-citation
Metadata
- Authors
- Hung, A. J., Chen, J., Jarc, A., Hatcher, D., Djaladat, H., & Gill, I. S.
- Venue
- Journal of Urology
- Published
- 2018-01
- Volume
- 199
- Issue
- 1
- Pages
- 296-304
- ISSN
- 0022-5347 (ISSN-L)
- 1527-3792
- OpenAlex
- W2741187991
- Cited by
- 162
- References
- 1
- FWCI
- 9.85
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Development and Validation of Objective Performance Metrics for Robot-Assisted Radical Prostatectomy: A Pilot Study. https://app.cochat.ai/papers/10.1016/j.juro.2017.07.081
- The paper
- Hung, A. J., Chen, J., Jarc, A., Hatcher, D., Djaladat, H., & Gill, I. S. (2018). Development and Validation of Objective Performance Metrics for Robot-Assisted Radical Prostatectomy: A Pilot Study. Journal of Urology. https://doi.org/10.1016/j.juro.2017.07.081
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.