The DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning
Open accessAbstract
We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Instead of training an additional model on top of patch embeddings from scratch34,73–80, these whole-slide representation models can be pretrained to distill pathology-specific knowledge from large WSI collections, simplifying clinical endpoint prediction with their off-the-shelf application.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredIn the citing paper
“The remaining studies explored multiple other tasks and many focused on classification of malignant disorders such as breast cancer56,61,88, skin cancer89, and lung cancer59.”
From Self-supervised learning for medical image classification: a systematic review and implementation guidelines (2023) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · cited 2×
“This operation is a kind of non-local operations [49] similar to self-attention [45].”
MethodA simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R., Su, H., Mo, K., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.16 · Cited by 10,029
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
Self-citationSGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Dynamic Routing Between Capsules
Sabour, S., Frosst, N., & Hinton, G. E. (2017) · arXiv · doi:10.48550/arxiv.1710.09829 · Cited by 4,167
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Solving the multiple instance problem with axis-parallel rectangles
Dietterich, T. G., Lathrop, R. H., & Lozano-Pérez, T. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00034-3 · Cited by 2,866
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Attention-based Deep Multiple Instance LearningPreprint
Ilse, M., Tomczak, J. M., & Welling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04712 · Cited by 1,428
A Framework for Multiple-Instance Learning
Maron, O., & Lozano‐Pérez, T. (1997) · Cited by 1,232
A simple framework for contrastive learning of visual representations
Chen, T. (2024) · TIB Data Manager · doi:10.57702/iuuvgtaz · Cited by 1,140
Deep Learning for Identifying Metastatic Breast CancerPreprint
Wang, D., Khosla, A., Gargeya, R., Irshad, H., & Beck, A. H. (2016) · arXiv.org · doi:10.48550/arxiv.1606.05718 · Cited by 1,028
Gland segmentation in colon histology images: The glas challenge contest
Sirinukunwattana, K., Pluim, J. P. W., Chen, H., Qi, X., Heng, P., Guo, Y. B., Wang, L. Y., Matuszewski, B. J., Bruni, E., Sanchez, U., et al. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.08.008 · Cited by 989
Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification
Hou, L., Samaras, D., Kurç, T., Gao, Y., Davis, J., & Saltz, J. H. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.266 · Cited by 921
Is object localization for free? - Weakly-supervised learning with convolutional neural networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298668 · Cited by 886
Asymmetric Non-Local Neural Networks for Semantic Segmentation
Zhu, Z., Xu, M., Bai, S., Huang, T., & Bai, X. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00068 · Cited by 760
Multiple instance learning: A survey of problem characteristics and applications
Carbonneau, M.-A., Cheplygina, V., Granger, E., & Gagnon, G. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.009 · Cited by 740 · cited 1×
“We refer the readers to [4] for a survey.”
BackgroundFrom image-level to pixel-level labeling with Convolutional Networks
Pinheiro, P. O., & Collobert, R. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298780 · Cited by 701
Roa, Á. A. C., Basavanhally, A. N., Gonzalez, F. A., Gilmore, H. L., Feldman, M. D., Ganesan, S., Shih, N. N. C., Tomaszewski, J., & Madabhushi, A. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2043872 · Cited by 621
Revisiting multiple instance neural networks
Wang, X., Yan, Y., Tang, P., Bai, X., & Liu, W. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.026 · Cited by 506
Detecting Cancer Metastases on Gigapixel Pathology Images
Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G. E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P. Q., Corrado, G. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1703.02442 · Cited by 506
Multiple Instance Detection Network with Online Instance Classifier Refinement
Tang, P., Wang, X., Bai, X., & Liu, W. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.326 · Cited by 503
Review of the current state of whole slide imaging in pathology
Pantanowitz, L., Valenstein, P. N., Evans, A. J., Kaplan, K. J., Pfeifer, J. D., Wilbur, D. C., Collins, L. C., & Colgan, T. J. (2011) · Journal of Pathology Informatics · doi:10.4103/2153-3539.83746 · Cited by 435
Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning
Cinbis, R. G., Verbeek, J., & Schmid, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2535231 · Cited by 420
Chen, P.-H. C., Gadepalli, K., MacDonald, R., Liu, Y., Kadowaki, S., Nagpal, K., Kohlberger, T., Dean, J., Corrado, G. S., Hipp, J. D., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0539-7 · Cited by 342
Weakly supervised histopathology cancer image segmentation and classification
Xu, Y. L., Zhu, J.-Y., Chang, E., Lai, M., & Tu, Z. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.01.010 · Cited by 286
Multiple-Instance Learning for Medical Image and Video Analysis
Quellec, G., Cazuguel, G., Cochener, B., & Lamard, M. (2017) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2017.2651164 · Cited by 259
C-MIL: Continuation Multiple Instance Learning for Weakly Supervised Object Detection
Wan, F., Liu, C., Ke, W., Ji, X., Jiao, J., & Ye, Q. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00230 · Cited by 259
Hashimoto, N., Fukushima, D., Koga, R., Takagi, Y., Ko, K., Kohno, K., Nakaguro, M., Nakamura, S., Hontani, H., & Takeuchi, I. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00391 · Cited by 251
Xu, Y. L., Jia, Z., Ai, Y., Zhang, F., Lai, M., & Chang, E. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178109 · Cited by 193
Self-Path: Self-Supervision for Classification of Pathology Images With Limited Annotations
Koohbanani, N. A., Unnikrishnan, B., Khurram, S. A., Krishnaswamy, P., & Rajpoot, N. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3056023 · Cited by 188
Feng, J., & Zhou, Z.-H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10890 · Cited by 161
Attention-based Deep Multiple Instance Learning
Ilse, M., Tomczak, J. M., & Welling, M. (2018) · UvA-DARE (University of Amsterdam) · Cited by 154
Adaptive Weighting Multi-Field-Of-View CNN for Semantic Segmentation in Pathology
Tokunaga, H., Teramoto, Y., Yoshizawa, A., & Bise, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01288 · Cited by 151
Multiple Instance Learning with Center Embeddings for Histopathology Classification
Chikontwe, P., Kim, M., Nam, S. J., Go, H., & Park, S. H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59722-1_50 · Cited by 145
Cornish, T. C., Swapp, R. E., & Kaplan, K. J. (2012) · Advances in Anatomic Pathology · doi:10.1097/pap.0b013e318253459e · Cited by 136
Computer-aided diagnosis from weak supervision: A benchmarking study
Kandemir, M., & Hamprecht, F. A. (2014) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2014.11.010 · Cited by 123
Self-Supervision Closes the Gap Between Weak and Strong Supervision in HistologyPreprint
Dehaene, O., Camara, A., Moindrot, O., de Lavergne, A., & Courtiol, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03583 · Cited by 83
Lu, M. Y., Chen, R. J., Wang, J., Dillon, D., & Mahmood, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10825 · Cited by 80
Automated discrimination of lower and higher grade gliomas based on histopathological image analysis
Mousavi, H. S., Monga, V., Rao, G., & Rao, A. U. K. (2015) · Journal of Pathology Informatics · doi:10.4103/2153-3539.153914 · Cited by 78
Li, B., Keikhosravi, A., Loeffler, A. G., & Eliceiri, K. W. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101938 · Cited by 68
Self-citationBejnordi, B., Litjens, G., Hermsen, M., Karssemeijer, N., & van der Laak, J. A. (2015) · Medical Imaging · doi:10.1117/12.2081768 · Cited by 54
Deep Multi-instance Learning with Dynamic Pooling
Yan, Y., Wang, X., Guo, X., Fang, J., Liu, W., & Huang, J. (2018) · Asian Conference on Machine Learning · Cited by 53
Multiple instance learning with graph neural networksPreprint
Tu, M., Huang, J., He, X., & Zhou, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04881 · Cited by 51
Keikhosravi, A., Li, B., Liu, Y., Conklin, M. W., Loeffler, A. G., & Eliceiri, K. W. (2020) · Communications Biology · doi:10.1038/s42003-020-01151-5 · Cited by 50
Self-citationSOS: Selective Objective Switch for Rapid Immunofluorescence Whole Slide Image Classification
Maksoud, S., Zhao, K., Hobson, P., Jennings, A., & Lovell, B. C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00392 · Cited by 33
Multi-scale learning based segmentation of glands in digital colonrectal pathology images
Gao, Y., Liu, W., Arjun, S., Zhu, L.-J., Ratner, V., Kurç, T., Saltz, J., & Tannenbaum, A. (2016) · SPIE Medical Imaging · doi:10.1117/12.2216790 · Cited by 18
Cluster-Based Learning from Weakly Labeled Bags in Digital PathologyPreprint
Akbar, S., & Martel, A. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00884 · Cited by 7
Flagged first, then by citations
1 of the 60 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
A whole-slide foundation model for digital pathology from real-world data
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., González, J., Gu, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07441-w · Cited by 921
Transformer-based unsupervised contrastive learning for histopathological image classification
Wang, X., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., & Han, X. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102559 · Cited by 751
Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning
Chen, R. J., Chen, C., Li, Y., Chen, T. Y., Trister, A. D., Krishnan, R. G., & Mahmood, F. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01567 · Cited by 621
Self-citationA pathology foundation model for cancer diagnosis and prognosis prediction
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07894-z · Cited by 580
Self-citation
Metadata
- Authors
- Li, B., Li, Y., & Eliceiri, K. W.
- Affiliations
- University of Wisconsin–Madison (US)
- Corresponding author
- Li, Yin
- Eliceiri, Kevin W.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Volume
- 2021
- Pages
- 14313-14323
- ISSN
- 1063-6919 (ISSN-L)
- PMC
- PMC8765709
- OpenAlex
- W3176719058
- Cited by
- 957
- References
- 60
- Citations by year
- 2026: 204 · 2025: 247 · 2024: 241 · 2023: 171 · 2022: 82 · 2021: 12
- FWCI
- 58.33
- Type
- conference paper
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Semiconductor Research Corporation
- Wisconsin Alumni Research Foundation
- National Institutes of Health
- National Institute of General Medical Sciences (P41GM135019)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- whole slide image classification, multiple instance learning, self-supervised contrastive learning, two-stream network, tumor detection, multi-scale features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
