Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical structure classification and image tagging that features an intrinsic measure of confidence to estimate its own performance with high reliability and which can be applied to both RGB and multispectral imaging (MI) data. METHODS: Organ recognition is performed using a superpixel classification strategy based on textural and reflectance information. Classification confidence is estimated by analyzing the dispersion of class probabilities. Assessment of the proposed technology is performed through a comprehensive in vivo study with seven pigs. RESULTS: When applied to image tagging, mean accuracy in our experiments increased from 65% (RGB) and 80% (MI) to 90% (RGB) and 96% (MI) with the confidence measure. CONCLUSION: Results showed that the confidence measure had a significant influence on the classification accuracy, and MI data are better suited for anatomical structure labeling than RGB data. SIGNIFICANCE: This paper significantly enhances the state of art in automatic labeling of endoscopic videos by introducing the use of the confidence metric, and by being the first study to use MI data for in vivo laparoscopic tissue classification. The data of our experiments will be released as the first in vivo MI dataset upon publication of this paper.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Jungo Reyes [668] | and | 2019 | U-Net | Segmentation | Brain tumor | 2 | Softmax MC dropout, Ensembles”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [668]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
(2014) · Cited by 33,419
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 6,385
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Kolodner, J. L. (2005) · The Cambridge Handbook of the Learning Sciences · doi:10.1017/cbo9780511816833.015 · Cited by 4,247
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
Probability Estimates for Multi-class Classification by Pairwise Coupling
Wu, T., Lin, C., & Weng, R. C. (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1016791 · Cited by 1,468
Loopy Belief Propagation for Approximate Inference: An Empirical Study
Murphy, K. P., Weiss, Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6725 · Cited by 1,397
Weight Uncertainty in Neural Networks
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1505.05424 · Cited by 1,342
Weight Uncertainty in Neural Network
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,335
Probability Estimates for Multi-class Classification by Pairwise Coupling
Wu, T., Lin, C., Weng, R. C., & Singer, Y. (2003) · Cited by 989
Surgical data science for next-generation interventions
Maier-Hein, L., Vedula, S. S., Speidel, S., Navab, N., Kikinis, R., Park, A., Eisenmann, M., Feussner, H., Forestier, G., Giannarou, S., et al. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-017-0132-7 · Cited by 566
Self-citationModelling uncertainty in deep learning for camera relocalization
Kendall, A., & Cipolla, R. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487679 · Cited by 548 · cited 1×
“, [41], [42]), among the most recently proposed solutions, variational Bayes by Backpropagation [43], [44] is is drawing attention of the deep learning community.”
BackgroundSuperpixel segmentation using Linear Spectral Clustering
Li, Z., & Chen, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298741 · Cited by 479
Review of spectral imaging technology in biomedical engineering: achievements and challenges
Li, Q., He, X., Wang, Y., Liu, H., Xu, D., & Guo, F. (2013) · Journal of Biomedical Optics · doi:10.1117/1.jbo.18.10.100901 · Cited by 479
Metrics for evaluating performance and uncertainty of Bayesian network models
Marcot, B. G. (2012) · Ecological Modelling · doi:10.1016/j.ecolmodel.2012.01.013 · Cited by 380
Loopy Belief Propagation: Convergence and Effects of Message Errors
Ihler, A., Fischer, J. W., & Willsky, A. S. (2005) · doi:10.5555/1046920.1088703 · Cited by 286
Toward Detection and Localization of Instruments in Minimally Invasive Surgery
Allan, M., Sebastien Ourselin, A., Thompson, S., Hawkes, D. J., Kelly, J. D., & Stoyanov, D. V. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2012.2229278 · Cited by 157
Multispectral imaging of burn wounds: a new clinical instrument for evaluating burn depth
Afromowitz, M. A., Callis, J. B., Heimbach, D. M., DeSoto, L. A., & Norton, M. (1988) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.7291 · Cited by 150
Validation of Knowledge Acquisition for Surgical Process Models
Neumuth, T., Jannin, P., Strauss, G., Meixensberger, J., & Burgert, O. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2748 · Cited by 112
Implicit Weight Uncertainty in Neural NetworksPreprint
Pawlowski, N., Brock, A., Lee, M. C. H., Rajchl, M., & Glocker, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01297 · Cited by 107
Automatic classification of digestive organs in wireless capsule endoscopy videos
Lee, J. K., Oh, J., Shah, S. K., et al. (2007) · ACM Symposium on Applied Computing · doi:10.1145/1244002.1244230 · Cited by 83
Context-aware Augmented Reality in laparoscopic surgery
Katić, D., Wekerle, A.-L., Görtler, J., Spengler, P., Bodenstedt, S., Röhl, S., Suwelack, S., Kenngott, H., Wagner, M., Müller-Stich, B. P., et al. (2013) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2013.03.003 · Cited by 78
Self-citationGenerating Estimates of Classification Confidence for a Case-Based Spam Filter
Delany, S. J., Cunningham, P., Doyle, D., et al. (2005) · Lecture notes in computer science · doi:10.1007/11536406_16 · Cited by 76
Development of a Protocol for CCD Calibration: Application to a Multispectral Imaging System
Mansouri, A., Marzani, F. S., & Gouton, P. (2005) · International Journal of Robotics and Automation · doi:10.2316/journal.206.2005.2.206-2784 · Cited by 67
Measures of Solution Accuracy in Case-Based Reasoning Systems
Cheetham, W. E., & Price, J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28631-8_9 · Cited by 60
Optimized Anisotropic Rotational Invariant Diffusion Scheme on Cone-Beam CT
Kroon, D.-J., Slump, C. H., & Maal, T. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15711-0_28 · Cited by 60
Fast Part-Based Classification for Instrument Detection in Minimally Invasive Surgery
Sznitman, R., Becker, C., & Fua, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10470-6_86 · Cited by 58
Toward knowledge-based liver surgery: holistic information processing for surgical decision support
März, K., Hafezi, M., Weller, T., Saffari, A., Nolden, M., Fard, N., Majlesara, A., Zelzer, S., Maleshkova, M., Volovyk, M., et al. (2015) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-015-1187-0 · Cited by 30
Self-citationTissue classification for laparoscopic image understanding based on multispectral texture analysis
Zhang, Y., Wirkert, S., Iszatt, J., Kenngott, H., Wagner, M., Mayer, B. F. B., Stock, C., Clancy, N. T., Elson, D. S., & Maier‐Hein, L. (2017) · Journal of medical imaging · doi:10.1117/1.jmi.4.1.015001 · Cited by 30
Self-citationBridging the gap between formal and experience-based knowledge for context-aware laparoscopy
Katić, D., Schuck, J., Wekerle, A.-L., Kenngott, H. G., Müller-Stich, B. P., Dillmann, R., & Speidel, S. (2016) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-016-1379-2 · Cited by 27
Self-citationHyperspectral Imaging: a New Modality in Surgery
Akbari, H., & Kosugi, Y. (2009) · InTech eBooks · doi:10.5772/7478 · Cited by 25
Endoscopic Sheffield Index for Unsupervised In Vivo Spectral Band Selection
Wirkert, S., Clancy, N. T., Stoyanov, D. V., Arya, S., Hanna, G. B., Schlemmer, H.-P., Sauer, P., Elson, D. S., & Maier‐Hein, L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13410-9_11 · Cited by 19
Self-citationAutomatic Detection of the Uterus and Fallopian Tube Junctions in Laparoscopic Images
Prokopetc, K., Collins, T., & Bartoli, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19992-4_43 · Cited by 18
Efficient Multi-organ Segmentation in Multi-view Endoscopic Videos Using Pre-operative Priors
Nosrati, M. S., Peyrat, J., Abinahed, J. A., et al. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10470-6_41 · Cited by 15
Live image parsing in uterine laparoscopy
Chhatkuli, A., Bartoli, A., Malti, A., et al. (2014) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2014.6868106 · Cited by 15
Towards an Efficient Way of Building Annotated Medical Image Collections for Big Data Studies
Gur, Y., Moradi, M., Bulu, H., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67534-3_10 · Cited by 15
Mewes, P. W., Neumann, D., Licegevic, O., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23626-6_18 · Cited by 12
Automatic workflow for narrow-band laryngeal video stitching
Moccia, S., Penza, V., Vanone, G. O., et al. (2016) · doi:10.1109/embc.2016.7590917 · Cited by 12
Self-citationSuperpixel-based structure classification for laparoscopic surgery
Bodenstedt, S., Görtler, J., Wagner, M., Kenngott, H., Müller-Stich, B. P., Dillmann, R., & Speidel, S. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2216750 · Cited by 11
Self-citationMultispectral imaging of organ viability during uterine transplantation surgery
Clancy, N. T., Saso, S., Stoyanov, D. V., Sauvage, V., Corless, D., Boyd, M. B., Noakes, D. E., Thum, M., Ghaem‐Maghami, S., Smith, J., et al. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2040518 · Cited by 10
Tissue classification for laparoscopic image understanding based on multispectral texture analysis
Zhang, Y., Wirkert, S., Iszatt, J., et al. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2216090 · Cited by 10
Self-citationConfidence assessment on eyelid and eyebrow expression recognition
Orozco, J., Rudovic, O. O., Roca, F. X., & Gonzàlez, J. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813454 · Cited by 9
Multispectral tissue analysis and classification towards enabling automated robotic surgery
Triana, B. P., Cha, J., Shademan, A., et al. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2040627 · Cited by 6
Subpopulation-specific confidence designation for more informative biomedical classification
Zhang, C., & Kodell, R. L. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.04.008 · Cited by 6
Research works printed but not linked: 1
- sheffield, 1985, Selecting band combinations from multispectral data, Photogrammetric Eng Remote Sens, 51, 681
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718
Metadata
- Authors
- Moccia, S., Wirkert, S. J., Kenngott, H., Vemuri, A. S., Apitz, M., Mayer, B., De Momi, E., Mattos, L. S., & Maier-Hein, L.
- Affiliations
- Fondazione Politecnico di Milano (IT)
- German Cancer Research Center (DE)
- University Hospital Heidelberg (DE)
- Italian Institute of Technology (IT)
- Venue
- IEEE Transactions on Biomedical Engineering
- Published
- 2018-11
- Volume
- 65
- Issue
- 11
- Pages
- 2649-2659
- ISSN
- 0018-9294 (ISSN-L)
- 1558-2531
- OpenAlex
- W2679674811
- Cited by
- 56
- References
- 45
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 1 · 2023: 4 · 2022: 8 · 2021: 7 · 2020: 13 · 2019: 8 · 2018: 3
- FWCI
- 3.84
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Biomedical Engineering (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion, other-oa)
- Università Politecnica delle Marche (Università Politecnica delle Marche) (submittedVersion)
- CINECA IRIS Institutional Research Information System (Sant'Anna School of Advanced Studies) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (ERC-2015-StG-37960)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- organ classification, laparoscopy, endoscopic video analysis, multispectral imaging, classification confidence, superpixel classification, image tagging, surgical data science, tissue classification
- MeSH terms
- Animals, Digestive System, Image Processing, Computer-Assisted, Laparoscopy, Spleen, Digestive System Surgical Procedures, Swine, Video Recording
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.