Intelligent labeling based on fisher information for medical image segmentation using deep learning
Open accessAbstract
Deep convolutional neural networks (CNN) have recently achieved superior performance at the task of medical image segmentation compared to classic models. However, training a generalizable CNN requires a large amount of training data, which is difficult, expensive, and time-consuming to obtain in medical settings. Active Learning (AL) algorithms can facilitate training CNN models by proposing a small number of the most informative data samples to be annotated to achieve a rapid increase in performance. We proposed a new active learning method based on Fisher information (FI) for CNNs for the first time. Using efficient backpropagation methods for computing gradients together with a novel low-dimensional approximation of FI enabled us to compute FI for CNNs with a large number of parameters. We evaluated the proposed method for brain extraction with a patch-wise segmentation CNN model in two different learning scenarios: universal active learning and active semi-automatic segmentation. In both scenarios, an initial model was obtained using labeled training subjects of a source data set and the goal was to annotate a small subset of new samples to build a model that performs well on the target subject(s). The target data sets included images that differed from the source data by either age group (e.g. newborns with different image contrast) or underlying pathology that was not available in the source data. In comparison to several recently proposed AL methods and brain extraction baselines, the results showed that FI-based AL outperformed the competing methods in improving the performance of the model after labeling a very small portion of target data set (<0.25%).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sourati et al. (2019) when trained with only 0.5% of the training voxels, achieves the same performance as the model that is trained using the entire training set.
“When trained with only 0.5% of the training voxels, this scheme achieves the same performance as the model that is trained using the entire training set.”
From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Sourati et al. (2019)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Fast robust automated brain extraction
Smith, S. M. (2002) · Human Brain Mapping · doi:10.1002/hbm.10062 · Cited by 10,496 · see citing works
Vandenberghe, L., & Boyd, S. (1996) · SIAM Review · doi:10.1137/1038003 · Cited by 4,129 · see citing works
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
Seung, H., Opper, M., & Sompolinsky, H. (1992) · Annual Conference Computational Learning Theory · doi:10.1145/130385.130417 · Cited by 2,193 · see citing works
CVXPY: A Python-Embedded Modeling Language for Convex Optimization.
Diamond, S., & Boyd, S. (2016) · PubMed · Cited by 1,626 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
CVXPY: A Python-Embedded Modeling Language for Convex OptimizationPreprint
Diamond, S., & Boyd, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00943 · Cited by 1,457 · see citing works
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Sener, O., & Savarese, S. (2017) · arXiv · doi:10.48550/arxiv.1708.00489 · Cited by 922 · see citing works
Deep Bayesian Active Learning with Image Data
Gal, Y., Islam, R., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1703.02910 · Cited by 900 · see citing works
Interactive medical image segmentation using deep learning with image-specific fine tuning
Wang, G., Li, W., Zuluaga, M. A., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791721 · Cited by 843 · see citing works
A rewriting system for convex optimization problems
Agrawal, A., Verschueren, R., Diamond, S., & Boyd, S. (2018) · Journal of Control and Decision · doi:10.1080/23307706.2017.1397554 · Cited by 674 · see citing works
Cost-Effective Active Learning for Deep Image Classification
Wang, K., Zhang, D., Li, Y., Zhang, R., & Lin, L. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2589879 · Cited by 669 · see citing works
The Power of Ensembles for Active Learning in Image Classification
Beluch, W. H., Genewein, T., Nurnberger, A., & Kohler, J. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00976 · Cited by 655 · see citing works
volBrain: An Online MRI Brain Volumetry System
Manjón, J. V., & Pierrick Coupé, A. (2016) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2016.00030 · Cited by 632 · see citing works
Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods
Iglesias, J. E., Liu, C., Thompson, P. M., & Tu, Z. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2138152 · Cited by 561 · see citing works
Makropoulos, A., Robinson, E. C., Schuh, A., Wright, R., Fitzgibbon, S. P., Božek, J., Counsell, S. J., Steinweg, J. K., Vecchiato, K., Passerat‐Palmbach, J., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.01.054 · Cited by 561 · see citing works
Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
Yang, L., Zhang, Y., Chen, J., Zhang, S., & Chen, D. Z. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_46 · Cited by 526 · see citing works
Bayesian Active Learning for Classification and Preference Learning
Houlsby, N., Huszár, F., Ghahramani, Z., & Lengyel, M. (2011) · arXiv · doi:10.48550/arxiv.1112.5745 · Cited by 507 · cited 1× · see citing works
“Other information gain measures have been proposed to address shortcomings of US, such as mutual information between unlabeled samples and the model parameters [7], [14], [15], which, however, did not improve uncertainty results for CNNs, and mutual information between query candidates and unlabeled samples [16], whic…”
BackgroundDeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · cited 1× · see citing works
“they either lack any AL component [37], [38], or simply use uncertainty sampling [39], [40].”
BackgroundOptimizing Neural Networks with Kronecker-factored Approximate CurvaturePreprint
Martens, J., & Grosse, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.05671 · Cited by 476 · see citing works
Overcoming Catastrophic Forgetting by Incremental Moment MatchingPreprint
Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W., & Zhang, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08475 · Cited by 472 · see citing works
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Şener, O., & Savarese, S. (2018) · International Conference on Learning Representations · Cited by 460 · see citing works
Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally
Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M. B., & Liang, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.506 · Cited by 430 · see citing works
Batch mode active learning and its application to medical image classification
Hoi, S. C. H., Jin, R., Zhu, J., & Lyu, M. R.-T. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143897 · Cited by 429 · see citing works
Deep Bayesian Active Learning with Image Data
Ghahramani, Z., Gal, Y., & Islam, R. (2017) · PMLR · doi:10.17863/cam.11070 · Cited by 392 · see citing works
Adaptive Active Learning for Image Classification
Li, X., & Guo, Y. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.116 · Cited by 279 · see citing works
Submodularity in Data Subset Selection and Active Learning
Wei, K., Iyer, R., & Bilmes, J. (2015) · International Conference on Machine Learning · Cited by 239 · see citing works
Mahapatra, D., Bozorgtabar, B., Thiran, J., & Reyes, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_65 · Cited by 182 · see citing works
Natural Neural NetworksPreprint
Desjardins, G., Simonyan, K., Pascanu, R., & Kavukcuoglu, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00210 · Cited by 138 · see citing works
Active Learning With Sampling by Uncertainty and Density for Data Annotations
Zhu, J., Wang, H., Tsou, B. K., et al. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2033421 · Cited by 138 · see citing works
(2013) · Liverpool John Moores University · Cited by 133 · see citing works
Interactive Whole-Heart Segmentation in Congenital Heart Disease
Pace, D. F., Dalca, A. V., Geva, T., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_10 · Cited by 124 · see citing works
Active Learning for Interactive 3D Image Segmentation
Top, A., Hamarneh, G., & Rafeef Abugharbieh, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23626-6_74 · Cited by 121 · see citing works
Active Deep Networks for Semi-Supervised Sentiment Classification
Zhou, S., Chen, Q., & Wang, X. (2010) · Cited by 110 · see citing works
Cost-Sensitive Active Learning for Intracranial Hemorrhage Detection
Kuo, W., Häne, C., Yuh, E. L., Mukherjee, P., & Malik, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_82 · Cited by 104 · see citing works
The m-Center Problem: Minimax Facility Location
Garfinkel, R., Neebe, A. W., & Rao, M. R. (1977) · Management Science · doi:10.1287/mnsc.23.10.1133 · Cited by 79 · see citing works
Automatic renal segmentation in DCE-MRI using convolutional neural networks
Haghighi, M., Warfield, S. K., & Kurugol, S. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363865 · Cited by 52 · see citing works
Self-citationLamash, Y., Kurugol, S., Freiman, M., Pérez-Rosselló, J. M., Callahan, M. J., Bousvaros, A., & Warfield, S. K. (2018) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26330 · Cited by 39 · see citing works
Self-citationSemi-Supervised and Active Learning for Automatic Segmentation of Crohn’s Disease
Mahapatra, D., Schüffler, P. J., Tielbeek, J. A. W., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40763-5_27 · Cited by 38 · see citing works
Otálora, S., Perdómo, O., Gonzalez, F. A., & Müller, H. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67534-3_16 · Cited by 33 · see citing works
Active Deep Learning with Fisher Information for Patch-Wise Semantic Segmentation
Sourati, J., Gholipour, A., Dy, J., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_10 · Cited by 29 · see citing works
Self-citationA Probabilistic Active Learning Algorithm Based on Fisher Information Ratio
Sourati, J., Akçakaya, M., Deniz Erdogmus, B., Leen, T. K., & Dy, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2743707 · Cited by 28 · see citing works
Self-citationDeep active learning for nucleus classification in pathology images
Shao, W., Sun, L., & Zhang, D. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363554 · Cited by 28 · see citing works
Active learning guided interactions for consistent image segmentation with reduced user interactions
Veeraraghavan, H., & Miller, J. V. (2011) · IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI) · doi:10.1109/isbi.2011.5872719 · Cited by 26 · see citing works
Guide Me: Interacting with Deep Networks
Rupprecht, C., Laina, I., Navab, N., Hager, G. D., & Tombari, F. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00892 · Cited by 26 · see citing works
Deep Active Self-paced Learning for Accurate Pulmonary Nodule Segmentation
Wang, W., Lu, Y., Wu, B., Chen, T., Chen, D. Z., & Wu, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_80 · Cited by 25 · see citing works
Classification Active Learning Based on Mutual Information
Sourati, J., Akçakaya, M., Dy, J., et al. (2016) · Entropy · doi:10.3390/e18020051 · Cited by 24 · see citing works
Self-citationAsymptotic Analysis of Objectives based on Fisher Information in Active LearningPreprint
Sourati, J., Akçakaya, M., Leen, T. K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08798 · Cited by 16 · see citing works
Self-citationAsymptotic analysis of objectives based on fisher information in active learningPreprint
Sourati, J., Akçakaya, M., Leen, T. K., et al. (2017) · arXiv (Cornell University) · doi:10.5555/3122009.3122043 · Cited by 13 · see citing works
Self-citationLamash, Y., Kurugol, S., & Warfield, S. K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_25 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- [22]Zhang T and Oles F, “ The value of unlabeled data for classification problems,” in Proceedings of the 17th International Conference on Machine Learning, 2000, pp. 1191– 1198.
- [23]Chaudhuri K, Kakade SM, Netrapalli P, and Sanghavi S, “ Convergence rates of active learning for maximum likelihood estimation,” in Advances in Neural Information Processing Systems, 2015, pp. 1090– 1098.
- [29]Grosse R and Martens J, “ A kronecker-factored approximate fisher matrix for convolution layers,” in International Conference on Machine Learning, 2016, pp. 573– 582.
- [32]Roux NL, Manzagol P-A, and Bengio Y, “ Topmoumoute online natural gradient algorithm,” in Advances in neural information processing systems, 2008, pp. 849– 856.
Cited by
Sorted by citations
Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101693 · Cited by 1,095 · see citing works
Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., & Zhang, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102102 · Cited by 126 · see citing works
- Authors
- Sourati, J., Gholipour, A., Dy, J. G., Tomas-Fernandez, X., Kurugol, S., & Warfield, S. K.
- Affiliations
- Boston Children's Hospital (US)
- Northeastern University (US)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2019-11
- Volume
- 38
- Issue
- 11
- Pages
- 2642-2653
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- PMC
- PMC7179938
- OpenAlex
- W2926307165
- Cited by
- 50
- References
- 53
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 7 · 2023: 9 · 2022: 9 · 2021: 11 · 2020: 4 · 2019: 1
- FWCI
- 2.80
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (R01 NS079788, R01 EB019483, R01 DK100404, IDDRC U54 HD090255, R01EB018988, R01NS106030)
- Research Grant from the Boston Children’s Hospital Translational Research Program
- McKnight Foundation
- National Science Foundation (IIS-1546428)
- Crohn's and Colitis Foundation of America
- American Gastroenterological Association
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- active learning (machine learning), Fisher information, medical image segmentation, brain extraction, semi-automatic segmentation, transfer learning, low-dimensional approximation, data-efficient training
- MeSH terms
- Deep Learning, Algorithms, Brain, Child, Preschool, Humans, Image Processing, Computer-Assisted, Infant, Infant, Newborn, Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
