AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Solving the recurrent issue of inadequate labeling during the analysis of medical images can be achieved by feeding the crowd-sourced input labels into the CNN.”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [294]
In the citing paper
“Albarquoni et al. [294] investigated the problem of insufficient labeling. They crowd-sourced the actual mitoses labeling in the histology images of breast cancer (from amateurs online).”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [294]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 2×
“Indeed, deep learning has advanced the field of computer vision the last few years [19] leading to powerful methods for various applications such as object classification [5], detection [20], segmentation [21], robust regression [22] and depth prediction [23].”
BackgroundMethodFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
(2010) · International Conference on Machine Learning · Cited by 13,995
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · cited 1×
“Indeed, deep learning has advanced the field of computer vision the last few years [19] leading to powerful methods for various applications such as object classification [5], detection [20], segmentation [21], robust regression [22] and depth prediction [23].”
MethodMatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444 · cited 1×
“vNet [34] and conducted on an Intel i7 machine with a GeForce”
MethodTowards an integrated crowdsourcing definition
Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012) · Journal of Information Science · doi:10.1177/0165551512437638 · Cited by 1,858 · cited 1×
“C ROWDSOURCING is a type of participative online activity in which an individual, an institution, a non-profit organization or a company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via flexible open call, the voluntary undertaking of a task [1].”
BackgroundMitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
Cireşan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_51 · Cited by 1,536 · cited 1×
“Being inspired by prominent previous work in this field [18], [26], we define the specific contribution of our own work as: i) Learning of a multi-scale CNN model for mitosis detection, ii) Incorporation of aggregation schemes into CNN layers, and iii) Augmentation and retraining of the CNN model with crowd's annotati…”
Method
Research works printed but not linked: 1
- howe, 2006, The rise of crowdsourcing, Wired Mag, 14, 1
Cited by
Items 101–200 of 613
M$^3$Lung-Sys: A Deep Learning System for Multi-Class Lung Pneumonia Screening From CT Imaging
Qian, X., Fu, H., Shi, W., Chen, T., Fu, Y., Shan, F., & Xue, X. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3030853 · Cited by 77
“ferent diseases, such as autism spectrum disorder [16], [17] or Alzheimer’s disease in the brain [18]–[20]; breast cancers [21]– [23]; diabetic retinopathy and Glaucoma in the eye [24]–[26]; and lung cancer [27], [28] or pneumonia [29], [30] in the chest.”
BackgroundCrowdSFL: A Secure Crowd Computing Framework Based on Blockchain and Federated Learning
Li, Z., Liu, J., Hao, J., Wang, H., & Xian, M. (2020) · Electronics · doi:10.3390/electronics9050773 · Cited by 68
“At present, crowdsourcing is actively developing, ranging from the analysis of big data [2] to the location service [3].”
BackgroundBurçak, K. C., Baykan, Ö. K., & Uğuz, H. (2020) · The Journal of Supercomputing · doi:10.1007/s11227-020-03321-y · Cited by 76
A survey of deep learning applications in cryptocurrency
Zhang, J., Cai, K., & Wen, J. (2023) · iScience · doi:10.1016/j.isci.2023.108509 · Cited by 80
Gastric precancerous diseases classification using CNN with a concise model
Zhang, X., Hu, W., Chen, F., Liu, J., Yang, Y., Wang, L., Duan, H., & Si, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185508 · Cited by 70
Jiménez, G., & Racoceanu, D. (2019) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2019.00145 · Cited by 76
“Albarqouni et al. evaluated a method combining crowd-sourcing for data annotation and convolutional neural networks for classification.”
MethodDuan, H., Huang, Y., Liu, L., Dai, H., Chen, L., & Zhou, L. (2019) · BioMedical Engineering OnLine · doi:10.1186/s12938-019-0726-2 · Cited by 73
“Moreover, the CNN architecture has also achieved good performance in several medical detection tasks [23–27].”
MethodSebai, M., Wang, T., & Alfadhli, S. A. (2020) · IEEE Access · doi:10.1109/access.2020.2978754 · Cited by 69
Gupta, R. K., Kaur, M., & Manhas, J. (2019) · Journal of Multimedia Information System · doi:10.33851/jmis.2019.6.2.81 · Cited by 67
Applied Deep Learning in Plastic Surgery: Classifying Rhinoplasty With a Mobile App
Borsting, E. A., DeSimone, R., Ascha, M. S., & Ascha, M. (2019) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000005905 · Cited by 66
Classification of Histopathology Images of Lung Cancer Using Convolutional Neural Network (CNN)
Baranwal, N., Doravari, P., & Kachhoria, R. (2022) · CRC Press eBooks · doi:10.1201/9781003272694-7 · Cited by 75
Guo, F., Huang, G., Zhang, W., Liu, G., Li, T., Ouyang, N., & Zhu, S. (2023) · Measurement · doi:10.1016/j.measurement.2023.113412 · Cited by 70
Deep networks in identifying CT brain hemorrhage
Helwan, A., Fakhri, G. E., Sasanı, H., & Ozsahin, D. U. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-172261 · Cited by 65
“…efficiency when applied to various medical areas such as medical image analysis [9], medical image clas-sification [10, 11], medical organs detection [12], disease detection [13], etc . . . these networks own the paradigms of more biologically inspired structures than the traditional networks [9].”
Classification of Malaria-Infected Cells Using Deep Convolutional Neural Networks
Pan, W. D., Dong, Y., & Wu, D. (2018) · InTech eBooks · doi:10.5772/intechopen.72426 · Cited by 68
An End-to-End Deep Learning Histochemical Scoring System for Breast Cancer TMA
Liu, J., Xu, B., Zheng, C., Gong, Y., Garibaldi, J. M., Soria, D., Green, A., Ellis, I. O., Zou, W., & Qiu, G. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2868333 · Cited by 64
“AggNet with a novel aggregation layer is proposed for mitosis detection in breast cancer histology images [19].”
BackgroundA systematic analysis of deep learning in genomics and histopathology for precision oncology
Unger, M., & Kather, J. N. (2024) · BMC Medical Genomics · doi:10.1186/s12920-024-01796-9 · Cited by 74
Deep Convolutional Neural Networks-Based Automatic Breast Segmentation and Mass Detection in DCE-MRI
Han, J., Jiang, X., Pang, Z., Lin, X., Huang, Y., & Li, L. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/2413706 · Cited by 62
A breast cancer risk predication and classification model with ensemble learning and big data fusion
Jaiswal, V., Saurabh, P., Lilhore, U. K., Pathak, M., Simaiya, S., & Dalal, S. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100298 · Cited by 81
Large-scale medical image annotation with crowd-powered algorithms
Heim, E., Roß, T., Seitel, A., März, K., Stieltjes, B., Eisenmann, M., Lebert, J., Metzger, J., Sommer, G., Sauter, A. W., et al. (2018) · Journal of medical imaging · doi:10.1117/1.jmi.5.3.034002 · Cited by 64
“A similar hybrid crowd-algorithm approach was applied by Albarqouni et al. 24 for breast cancer detection in pathological images.”
MethodBrunetti, A., Carnimeo, L., Trotta, G. F., & Bevilacqua, V. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.06.080 · Cited by 63
A Systematic Review of Applications of Machine Learning in Cancer Prediction and Diagnosis
Sharma, A., & Rani, R. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09556-z · Cited by 67
Stoean, R. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3709-5 · Cited by 60
“Among the recent applications of CNN for medical image diagnosis, histopathological support is one of the main challenges [1, 6, 25].”
BackgroundMusa, I. H., Afolabi, L. O., Zamit, I., Musa, T. H., Musa, H. H., Tassang, A. N., Akintunde, T. Y., & Li, W. (2022) · Cancer Control · doi:10.1177/10732748221095946 · Cited by 64
Wide residual networks for mitosis detection
Zerhouni, E., Lányi, D., Viana, M. P., & Gabrani, M. (2017) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2017.7950667 · Cited by 70
“[14] recently presented a new deep network architecture that accounts for crowdsourcing annotated data.”
BackgroundNateghi, R., Danyali, H., & Helfroush, M. S. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102048 · Cited by 60
Rehman, M. U., Akhtar, S., Zakwan, M., & Mahmood, M. H. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103212 · Cited by 57
Early detection of breast cancer using hybrid of series network and VGG-16
Jahangeer, G. S. B., & Rajkumar, T. D. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09914-2 · Cited by 57
“[2]The study proposed that available ground-truth information was described as the main challenge for the transfer of recent developments in deep learning to the field of biomedical imaging.”
BackgroundLearning from crowds with variational Gaussian processes
Ruiz, P., Morales-Álvarez, P., Molina, R., & Katsaggelos, A. K. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.11.021 · Cited by 58
Deep learning for health informatics: Recent trends and future directions
Srivastava, S., Soman, S., Rai, A., & Srivastava, P. K. (2017) · International Conference on Advances in Computing, Communications and Informatics (ICACCI) · doi:10.1109/icacci.2017.8126082 · Cited by 62
“The techniques have been used to aid clinicians in effective visualization and diagnosis of diseases such as diabetic retinopathy [67] and melanoma [3].”
A Practical Guide to Artificial Intelligence–Based Image Analysis in Radiology
Weikert, T., Cyriac, J., Yang, S., Nešić, I., Parmar, V., & Stieltjes, B. (2019) · Investigative Radiology · doi:10.1097/rli.0000000000000600 · Cited by 58
Beyond confusion matrix: learning from multiple annotators with awareness of instance features
Li, J., Sun, H., & Li, J. (2022) · Machine Learning · doi:10.1007/s10994-022-06211-x · Cited by 63
“Competing strategies: Apart from neural-based LFC baselines, we also compare the LFC-𝐱 to the EM-based LFC methods including: (1) Raykar Raykar et al. (2009): a maximum-likelihood estimator that jointly learns the classifier, the confuse matrices, and the underlying true labels based on EM algorithm; (2) AggNet (Albar…”
MethodBreast Cancer Classification in Histopathological Images using Convolutional Neural Network
Rahhal, M. M. A. (2018) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2018.090310 · Cited by 55
Feng, M., Deng, Y., Yang, L., Jing, Q., Zhang, Z., Xu, L., Wei, X., Zhou, Y., Wu, D., Fei, X., et al. (2020) · Diagnostic Pathology · doi:10.1186/s13000-020-00957-5 · Cited by 57
Wang, X., Fang, Y., Yang, S., Zhu, D., Wang, M., Zhang, J., Tong, R. K., & Han, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101914 · Cited by 60
Cell Detection and Segmentation in Microscopy Images with Improved Mask R-CNN
Fujita, S., & Han, X. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69756-3_5 · Cited by 57
Seamless Lesion Insertion for Data Augmentation in CAD Training
Pezeshk, A. X., Petrick, N., Chen, W.-J., & Sahiner, B. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2640180 · Cited by 53
“annotations through crowd-sourcing, although some recent efforts have been able to use a combination of expert and crowd-sourced truthing in their systems [7].”
A new growing pruning deep learning neural network algorithm (GP-DLNN)
Zemouri, R. M., Omri, N., Fnaiech, F., Noureddine Zerhouni, A., & Fnaiech, N. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04196-8 · Cited by 56
GoogleNet CNN Neural Network towards Chest CT-Coronavirus Medical Image Classification
Alsharman, N., & Jawarneh, I. (2020) · Journal of Computer Science · doi:10.3844/jcssp.2020.620.625 · Cited by 52
“…successfully used to many medical image processing and analysis problems in recent works because the newest neural network architectures often add connections and additional operations to the standard architecture that enable training deeper networks (Albarqouni et al ., 2016; Bao and Chung, 2018).”
MethodSu, Y., Lu, Y., Chen, M., & Liu, A.-A. (2017) · IEEE Access · doi:10.1109/access.2017.2745544 · Cited by 56
“Several methods [18]–[20] have been proposed to detect mitosis with pixel level classification with CNN network on single breast cancer histology image.”
MethodReverse active learning based atrous DenseNet for pathological image classification
Li, Y., Xie, X., Shen, L., & Liu, S. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2979-y · Cited by 51
Pan, X., Lu, Y., Lan, R., Liu, Z., Qin, Z., Wang, H., & Liu, Z. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2021.107038 · Cited by 63
SlideRunner: A Tool for Massive Cell Annotations in Whole Slide Images
Aubreville, M., Bertram, C., Klopfleisch, R., & Maier, A. (2018) · Informatik aktuell · doi:10.1007/978-3-662-56537-7_81 · Cited by 56
“used an aggregation of expert labels and crowdsourcing with non-experts [2], showing effectively the power of an enhanced data set.”
MethodEggerschwiler, B., Canepa, D. D., Pape, H., Casanova, E. A., & Cinelli, P. (2019) · Stem Cell Research & Therapy · doi:10.1186/s13287-019-1170-8 · Cited by 50
Jackson, C. R., Sriharan, A., & Vaickus, L. J. (2020) · Modern Pathology · doi:10.1038/s41379-020-0526-z · Cited by 53
“This approach has already been used in cervical cancer [3], skin lesions [4, 5], breast cancer [6 – 8], kidneys [9], lungs [10], and colon cancers [11].”
MethodCrowdsourcing aggregation with deep Bayesian learning
Li, S.-Y., Huang, S.-J., & Chen, S. (2021) · Science China Information Sciences · doi:10.1007/s11432-020-3118-7 · Cited by 52
Medical Imaging with Deep Learning for COVID- 19 Diagnosis: A Comprehensive ReviewPreprint
Bharati, S., Podder, P., Mondal, M. R. H., & Prasath, V. B. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09602 · Cited by 58
“Object identification is a critical component in video monitoring, image processing systems, and applications in medical diagnostics [77].”
BackgroundAdaptation of Deep Convolutional Neural Networks for Cancer Grading from Histopathological Images
Postavaru, S., Stoean, R., Stoean, C., & Caparrós, G. J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59147-6_4 · Cited by 52
Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers
Xu, J., Zhou, C., Lang, B., & Liu, Q. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-42999-1_6 · Cited by 56
Assessment of Breast Cancer Histology Using Densely Connected Convolutional Networks
Kohl, M., Walz, C., Ludwig, F., Braunewell, S., & Baust, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_103 · Cited by 55
MitosisNet: End-to-End Mitotic Cell Detection by Multi-Task Learning
Alom, M. Z., Aspiras, T. H., Taha, T. M., Bowen, T. J., & Asari, V. K. (2020) · IEEE Access · doi:10.1109/access.2020.2983995 · Cited by 51
“In this implementation, we use our Recurrent Residual U-Net (R2U-Net) model for mitosis region segmentation [21].”
MethodInfluentialCao, S., Zhang, L., Kuang, S., Shi, W., Hu, B., Xie, S., Chen, Y.-N., Liu, H., Chen, S.-M., Jiang, T., et al. (2020) · World Journal of Gastroenterology · doi:10.3748/wjg.v26.i25.3660 · Cited by 48
New Trends in the Use of Artificial Intelligence for the Industry 4.0
Martínez, L. R., Osornio-Rios, R. A., & Delgado-Prieto, M. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.86015 · Cited by 55
Guo, Z., Zhao, L., Yuan, J., & Yu, H. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3131671 · Cited by 47
Singla, N., Dubey, K., & Srivastava, V. (2019) · Journal of Biophotonics · doi:10.1002/jbio.201800255 · Cited by 48
İnik, Ö., Ceyhan, A., Balcıoğlu, E., & Ülker, E. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103350 · Cited by 51
Deep Learning: A Breakthrough in Medical Imaging
Ahmad, H. M., Khan, M. J., Yousaf, A., Ghuffar, S., & Khurshid, K. (2019) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/1573405615666191219100824 · Cited by 49
dos Santos, D. F. D., de Faria, P. R., Travençolo, B. A. N., & do Nascimento, M. Z. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102921 · Cited by 50
Breast cancer detection using deep learning techniques: challenges and future directions
Shahid, M. S., & Imran, A. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20606-7 · Cited by 54
Review of Deep Learning Methods in Mammography, Cardiovascular, and Microscopy Image Analysis
Carneiro, G., Zheng, Y., Xing, F., & Yang, L. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-42999-1_2 · Cited by 51
Kong, F., Li, J., & Lv, Z. (2018) · Journal of Computational Science · doi:10.1016/j.jocs.2018.03.010 · Cited by 49
Massive datasets and machine learning for computational biomedicine: trends and challenges
Андреевич, К. А., Pardalos, P. M., & Karakitsiou, A. (2018) · Annals of Operations Research · doi:10.1007/s10479-018-2891-2 · Cited by 58
“In Albarqouni et al. (2016) the authors used the concept of distributed human intelligence to tackle the problem of low sample size data in biomedicine with help of DDNs: they, as well as the authors in Nguyen et al. (2012), showed that a crowd of untrained individuals can label medical images at…”
BackgroundJin, Y. W., Jia, S., Ashraf, A. B., & Hu, P. (2020) · Cancers · doi:10.3390/cancers12102934 · Cited by 48
Unveiling the Complexity of Medical Imaging through Deep Learning Approaches
Rasool, N., & Bhat, J. I. (2023) · Chaos Theory and Applications · doi:10.51537/chaos.1326790 · Cited by 51
Liu, T., Tian, Y., Zhao, S., Huang, X., & Wang, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2923318 · Cited by 49
A Case for Soft Loss Functions
Uma, A., Fornaciari, T., Hovy, D., Paun, S., Plank, B., & Poesio, M. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7478 · Cited by 39
“A number of methods exist to learn a model directly from the annotations (Raykar et al. 2010; Albarqouni et al. 2016; Guan et al. 2018; Rodrigues and Pereira 2018).”
MethodLearning from Crowds by Modeling Common Confusions
Chu, Z., Ma, J., & Wang, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16730 · Cited by 73
Morriss, N. J., Conley, G. M., Ospina-Mora, S., Meehan, W. P., III, Qiu, J., & Mannix, R. (2020) · Neuroscience · doi:10.1016/j.neuroscience.2020.01.006 · Cited by 45
Feature rearrangement based deep learning system for predicting heart failure mortality
Wang, Z., Zhu, Y., Li, D., Yin, Y., & Zhang, J. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105383 · Cited by 44
Max-MIG: an Information Theoretic Approach for Joint Learning from CrowdsPreprint
Cao, P., Xu, Y., Kong, Y., & Wang, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13436 · Cited by 59
“Albarqouni et al. (2016) extend this method to combine with the deep learning model.”
BackgroundMethodInfluentialAlom, M. Z., Aspiras, T. H., Taha, T. M., Asari, V. K., Bowen, T., Billiter, D., & Arkell, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09075 · Cited by 49
Dong, F., She, R., Cui, C., Shi, S., Hu, X., Zeng, J., Wu, H., Xu, J., & Zhang, Y. (2021) · European Radiology · doi:10.1007/s00330-020-07561-7 · Cited by 43
Lv, Z., Cao, X., Jin, X., Xu, S., & Deng, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-40424-x · Cited by 48
INTRACRANIAL HEMORRHAGE DETECTION IN CT SCAN USING DEEP LEARNING
Tharek, A., Muda, A. S., Hudi, A. B., & Huddin, A. B. (2022) · Asian Journal Of Medical Technology · doi:10.32896/ajmedtech.v2n1.1-18 · Cited by 42
“It also shows great efficiency in various medical areas such as medical image analysis [33], medical organ detection [34], disease detection [35], and many more.”
BackgroundHetz, M. J., Bucher, T.-C., & Brinker, T. J. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103149 · Cited by 47
“…already been successfully tested in various clinical tasks, such as tumor detection [Cruz-Roa et al., 2017], mitosis detection [Saha et al., 2018, Albarqouni et al., 2016], grading of cancer [Bulten et al., 2020], predicting the lymph sentinel status [Kiehl et al., 2021], patient survival…”
MethodCrowd-Algorithm Collaboration for Large-Scale Endoscopic Image Annotation with Confidence
Maier‐Hein, L., Ross, T., Gröhl, J., Glocker, B., Bodenstedt, S., Stock, C., Heim, E., GOETZ, M. A., Wirkert, S., Kenngott, H., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46723-8_71 · Cited by 46
Öztürk, Ş., & Akdemïr, B. (2019) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22309 · Cited by 42
Artificial intelligence and digital pathology: Opportunities and implications for immuno-oncology
Sobhani, F., Robinson, R., Hamidinekoo, A., Roxanis, I., Somaiah, N., & Yuan, Y. (2021) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2021.188520 · Cited by 44
Yin, X.-X., Hadjiloucas, S., Zhang, Y., & Tian, Z. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106510 · Cited by 42
Shi, M. (2021) · Mathematical Problems in Engineering · doi:10.1155/2021/6627114 · Cited by 41
“Specifically, when calculating the output of the current time, the RNN will apply the previously memorized information to the current calculation [19, 20].”
MethodRetractedConvolutional Neural Networks for MRI-Based Brain Tumor Classification
Soewu, T., Singh, D., Rakhra, M., Chakraborty, G. S., & Singh, A. P. (2022) · International Conference on Computation, Automation and Knowledge Management (ICCAKM) · doi:10.1109/iccakm54721.2022.9990173 · Cited by 42
Artificial Intelligence for Cancer Detection—A Bibliometric Analysis and Avenues for Future Research
Karger, E., & Kureljusic, M. (2023) · Current Oncology · doi:10.3390/curroncol30020125 · Cited by 47
“2016 Breast cancer 360 [92] 28 Urban et al.”
BackgroundA systematic review on deep learning‐based automated cancer diagnosis models
Tandon, R., Agrawal, S., Rathore, N. P. S., Mishra, A. K., & Jain, S. K. (2024) · Journal of Cellular and Molecular Medicine · doi:10.1111/jcmm.18144 · Cited by 46
A deep neural network approach to QRS detection using autoencoders
Belkadi, M. A., Daamouche, A., & Melgani, F. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115528 · Cited by 40
Experimental Studies on Rock Thin-Section Image Classification by Deep Learning-Based Approaches
Li, D., Zhao, J., & Ma, J. (2022) · Mathematics · doi:10.3390/math10132317 · Cited by 40
Applications of Artificial Intelligence in Healthcare
Quazi, S., Saha, R. P., & Singh, M. K. (2022) · Journal of Experimental Biology and Agricultural Sciences · doi:10.18006/2022.10(1).211.226 · Cited by 48
Multi-rater Prism: Learning self-calibrated medical image segmentation from multiple raters
Wu, J., Fang, H., Zhu, J., Zhang, Y., Li, X., Liu, Y., Liu, H., Jin, Y., Huang, W., Liu, Q., et al. (2024) · 中国科学通报:英文版 · doi:10.1016/j.scib.2024.06.037 · Cited by 42
Medical Image Analysis Through Deep Learning Techniques: A Comprehensive Survey
Balasamy, K., Seethalakshmi, V., & Suganyadevi, S. (2024) · Wireless Personal Communications · doi:10.1007/s11277-024-11428-1 · Cited by 45
“Self-supervision may be developed in practice using pre-text activities or contrastive learning [44–50].”
InfluentialA Review of Breast Cancer Detection in Medical Images
Lu, Y., Li, J., Su, Y., & Liu, A.-A. (2018) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip.2018.8698732 · Cited by 39
“Method dataset Precision (%) Recall (%) F2 score (%) AUC AggNet [4] AMIDA13 44.”
MethodComputational methods for automated mitosis detection in histopathology images: A review
Mathew, T., Kini, J. R., & Rajan, J. (2020) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2020.11.005 · Cited by 41
Deep Learning-Based Cancer Detection-Recent Developments, Trend and Challenges
Kumar, G., & Alqahtani, H. (2021) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2022.018418 · Cited by 45
Gamification in Online Education: A Visual Bibliometric Network Analysis
Yazdi, A., Karimi, A., & Mystakidis, S. (2024) · Information · doi:10.3390/info15020081 · Cited by 45
“This model’s ability to automate the detection process is demonstrated [89].”
MethodTowards pixel-to-pixel deep nucleus detection in microscopy images
Xing, F., Xie, Y., Shi, X., Chen, P., Zhang, Z., & Yang, L. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3037-5 · Cited by 41
Thyreau, B., & Taki, Y. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101639 · Cited by 40
Li, D., Zhao, J., & Liu, Z. (2022) · Sensors · doi:10.3390/s22041574 · Cited by 39
“…13 December 2021)), face recognition [8], natural language processing [9,10], remote sensing [11,12], autonomous driving perception technologies [13], as well as industrial equipment fault detection, and medical CT analysis [14–16], etc. CNN shows excellent performance on the image process aspect.”
BackgroundCrowdsourcing of Histological Image Labeling and Object Delineation by Medical Students
Grote, A., Schaadt, N. S., Forestier, G., Wemmert, C., & Feuerhake, F. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2883237 · Cited by 37
“of training sets for convolutional neural networks for finding nuclei or mitoses in cancer [8], [9].”
BackgroundLeveraging Unlabeled Whole-Slide-Images for Mitosis Detection
Akram, S. U., Qaiser, T., Graham, S., Kannala, J., Heikkilä, J., & Rajpoot, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00949-6_9 · Cited by 37
Rao, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01788 · Cited by 38
“For example, [13] employed a CNN paired with a crowd-sourced learning architecture and achieved an F-measure of 0.”
MethodDeep Learning for Cardiovascular Risk Stratification
Schlesinger, D. E., & Stultz, C. M. (2020) · Current Treatment Options in Cardiovascular Medicine · doi:10.1007/s11936-020-00814-0 · Cited by 35
Bertram, C. A., Veta, M., Marzahl, C., Stathonikos, N., Maier, A., Klopfleisch, R., & Aubreville, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61166-8_22 · Cited by 34
Joint categorical and ordinal learning for cancer grading in pathology images
Vuong, T. T. L., Kim, K., Song, B., & Kwak, J. T. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102206 · Cited by 39
Metadata
- Authors
- Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., & Navab, N.
- Affiliations
- German Center for Neurodegenerative Diseases (DE)
- Technical University of Munich (DE)
- University of Oxford (GB)
- Johns Hopkins University (US)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2016-05
- Volume
- 35
- Issue
- 5
- Pages
- 1313-1321
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2339885376
- Cited by
- 606
- References
- 38
- Citations by year
- 2026: 14 · 2025: 24 · 2024: 59 · 2023: 59 · 2022: 60 · 2021: 92 · 2020: 89 · 2019: 80 · 2018: 73 · 2017: 46 · 2016: 10
- FWCI
- 57.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- DZNE Pub (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- mitosis detection, breast cancer histopathology, crowdsourced annotation, noisy annotations, data aggregation, convolutional neural networks, CrowdFlower
- MeSH terms
- Machine Learning, Breast Neoplasms, Female, Histocytochemistry, Humans, Image Interpretation, Computer-Assisted, Mitosis, Neural Networks, Computer, Video Games, Internet, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.