DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotations
Open accessAbstract
Extracting, harvesting and building large-scale annotated radiological image datasets is a greatly important yet challenging problem. It is also the bottleneck to designing more effective data-hungry computing paradigms (e.g., deep learning) for medical image analysis. Yet, vast amounts of clinical annotations (usually associated with disease image findings and marked using arrows, lines, lesion diameters, segmentation, etc.) have been collected over several decades and stored in hospitals' Picture Archiving and Communication Systems. In this paper, we mine and harvest one major type of clinical annotation data - lesion diameters annotated on bookmarked images - to learn an effective multi-class lesion detector via unsupervised and supervised deep Convolutional Neural Networks (CNN). Our dataset is composed of 33,688 bookmarked radiology images from 10,825 studies of 4,477 unique patients. For every bookmarked image, a bounding box is created to cover the target lesion based on its measured diameters. We categorize the collection of lesions using an unsupervised deep mining scheme to generate clustered pseudo lesion labels. Next, we adopt a regional-CNN method to detect lesions of multiple categories, regardless of missing annotations (normally only one lesion is annotated, despite the presence of multiple co-existing findings). Our integrated mining, categorization and detection framework is validated with promising empirical results, as a scalable, universal or multi-purpose CAD paradigm built upon abundant retrospective medical data. Furthermore, we demonstrate that detection accuracy can be significantly improved by incorporating pseudo lesion labels (e.g., Liver lesion/tumor, Lung nodule/tumor, Abdomen lesions, Chest lymph node and others). This dataset will be made publicly available (under the open science initiative).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Detection accuracy could be significantly improved by incorporating pseudo-category labels.
“The results demonstrate that detection accuracy could be significantly improved by incorporating pseudo-category labels.”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · see citing works
Discriminative Clustering by Regularized Information Maximization
Krause, A., Perona, P., & Gomes, R. G. (2010) · neural information processing systems · Cited by 273 · see citing works
Fully Convolutional Network for Liver Segmentation and Lesions Detection
Ben-Cohen, A., Diamant, I., Klang, E., Amitai, M., & Greenspan, H. (2016) · LABELS/DLMIA@MICCAI · doi:10.1007/978-3-319-46976-8_9 · Cited by 236 · see citing works
Tajbakhsh, N., Gotway, M., & Liang, J. (2015) · International Conference on Medical Image Computing and Computer-Assisted Intervention · doi:10.1007/978-3-319-24571-3_8 · Cited by 129 · see citing works
Colitis detection on computed tomography using regional convolutional neural networks
Liu, J., Wang, D., Wei, Z., Lü, L., Kim, L., Turkbey, E. B., & Summers, R. M. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493402 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [3]3. A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature , 542(7639):115-118, 2017.
Cited by
Sorted by citations
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682 · see citing works
“To tackle this problem, one previous study [55] utilized radiologists ’ annotations which are routinely added to radiologists ’ reports (such as circle, arrow, and square).”
MethodDeep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A. X., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2018) · Medical Physics · doi:10.1002/mp.13264 · Cited by 739 · see citing works
Self-citationMedical image super-resolution for smart healthcare applications: A comprehensive survey
Umirzakova, S., Ahmad, S., Khan, L. U., & Whangbo, T. K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102075 · Cited by 155 · see citing works
Rethinking pre-training on medical imaging
Wen, Y., Chen, L., Deng, Y., & Zhou, C. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103145 · Cited by 92 · see citing works
Artificial intelligence-based biomarkers for treatment decisions in oncology
Ligero, M., Nahhas, O. S. M. E., Aldea, M. D., & Kather, J. N. (2025) · Trends in cancer · doi:10.1016/j.trecan.2024.12.001 · Cited by 70 · see citing works
Unsupervised body part regression via spatially self-ordering convolutional neural networks
Yan, K., Lü, L., & Summers, R. M. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363745 · Cited by 58 · see citing works
Self-citationdos Santos, D. P., Brodehl, S., Baeßler, B., Arnhold, G., Dratsch, T., Chon, S., Mildenberger, P., & Jungmann, F. (2019) · Insights into Imaging · doi:10.1186/s13244-019-0777-8 · Cited by 50 · see citing works
Attentive CT Lesion Detection Using Deep Pyramid Inference with Multi-scale Booster
Shao, Q., Gong, L., Ma, K., Liu, H., & Zheng, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_34 · Cited by 49 · see citing works
Wang, Z., Meng, Y., Weng, F., Chen, Y., Lu, F., Liu, X., Hou, M., & Zhang, J. (2019) · Annals of Biomedical Engineering · doi:10.1007/s10439-019-02349-3 · Cited by 41 · see citing works
Wang, Y., Tang, P., Zhou, Y., Shen, W., Fishman, E. K., & Yuille, A. L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3060066 · Cited by 34 · see citing works
Quad-Net: Quad-Domain Network for CT Metal Artifact Reduction
Li, Z., Gao, Q., Wu, Y., Niu, C., Zhang, J., Wang, M., Wang, G., & Shan, H. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3351722 · Cited by 30 · see citing works
Object Detection as a Positive-Unlabeled Problem
Yang, Y., Liang, K. J., & Duke, L. C. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.74 · Cited by 27 · see citing works
An unsupervised reconstruction method for low-dose CT using deep generative regularization prior
Unal, M. O., Ertaş, M., & Yıldırım, İ. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.103598 · Cited by 27 · see citing works
Tam, L., Wang, X., Turkbey, E. B., Lu, K., Wen, Y., & Xu, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59719-1_5 · Cited by 18 · see citing works
Self-citationSelf-Supervised Training For Low-Dose Ct Reconstruction
Unal, M. O., Ertaş, M., & Yıldırım, İ. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9433944 · Cited by 15 · see citing works
UNet with ResNextify and IB modules for low-dose CT image denoising
Chauhan, S., Malik, N., & Vig, R. (2024) · International Journal of Information Technology · doi:10.1007/s41870-024-01898-8 · Cited by 14 · see citing works
Proj2Proj: self-supervised low-dose CT reconstruction
Unal, M. O., Ertaş, M., & Yıldırım, İ. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1849 · Cited by 13 · see citing works
Mao, J., Guo, S., Yin, X., Chang, Y., Nie, B., & Wang, Y. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112536 · Cited by 15 · see citing works
Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction
Wu, J., Jiang, X., Zhong, L., Zheng, W., Li, X., Lin, J., & Li, Z. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad69f7 · Cited by 10 · see citing works
An Improved Confidence Connected Liver Segmentation Method Based on Three Views of CT Images
Song, X., Deng, G., Zhuang, Y., & Zeng, N. (2019) · IEEE Access · doi:10.1109/access.2019.2914856 · Cited by 8 · see citing works
Maryam, T. H., Ru, B., Xie, T., Hadžikadić, M., Wu, Q., & Ge, Y. (2019) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983412 · Cited by 8 · see citing works
FeatSet+: Visual Features Extracted from Public Image Datasets
Cazzolato, M. T., Scabora, L. C., Zabot, G. F., Gutierrez, M. A., Traina, C., & Traina, A. J. M. (2022) · Journal of Information and Data Management · doi:10.5753/jidm.2022.2328 · Cited by 6 · see citing works
Two-stage lesion detection approach based on dimension-decomposition and 3D context
Jiao, J., Pan, H., Chen, C., Jin, T., Yang, D., & Chen, J. (2021) · Tsinghua Science & Technology · doi:10.26599/tst.2021.9010028 · Cited by 4 · see citing works
An effective automatic segmentation of abdominal adipose tissue using a convolution neural network
Micomyiza, C., Zou, B., & Li, Y. (2022) · Diabetes & Metabolic Syndrome Clinical Research & Reviews · doi:10.1016/j.dsx.2022.102589 · Cited by 4 · see citing works
Artificial Intelligence for Scientific Discovery at High-Performance Computing Scales
Lee, K. L. K., & Kumar, N. (2023) · Computer · doi:10.1109/mc.2023.3241692 · Cited by 4 · see citing works
FeatSet: A Compilation of Visual Features Extracted from Public Image Datasets
Cazzolato, M. T., Scabora, L. C., Zabot, G. F., Gutierrez, M. A., Traina, C., & Traina, A. J. M. (2021) · doi:10.5753/dsw.2021.17417 · Cited by 3 · see citing works
Automated detection of lesion in computer tomography images based on Cascade R-CNN
Liu, R., Zhao, Y., Liu, Y., Chen, X., Cui, S., Wang, F., & Lin, Y. (2020) · International Conference on Image, Video Processing and Artificial Intelligence (IVPAI) · doi:10.1117/12.2579644 · Cited by 2 · see citing works
Shapley Value is an Equitable Metric for Data Valuation
Shobeiri, S., & Aajami, M. (2022) · Iraqi Journal for Electrical And Electronic Engineering · doi:10.37917/ijeee.18.2.2 · Cited by 2 · see citing works
A Critical Analysis on Early Stage of Lungs and Colon Cancer Detection Using CNN Algorithm
Ramesh, K., Bhavadharani, C., Narmatha, V. S., Samraj, A., & Nandhakumar, R. (2023) · International Conference on Intelligent Technologies for Sustainable Electric and Communications Systems (iTech SECOM) · doi:10.1109/itechsecom59882.2023.10435341 · Cited by 2 · see citing works
Test-Time Adaptation via Orthogonal Meta-Learning for Medical Imaging
Wang, Z., Lu, Z., Wang, T., Yang, Z., Yu, H. J., Wang, Z., Chen, Y., Lu, J., & Zhang, Y. (2024) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2024.3462542 · Cited by 2 · see citing works
Adaptive Multi-Region Network For Medical Image Analysis
Tao, H., Wang, Z., Gao, Y., Wang, Y., & Khan, L. R. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191155 · Cited by 2 · see citing works
Abdullah, M. F., Sulaiman, S. N., Osman, M. K., Karim, N. K. A., Isa, I. S., & Shuaib, I. L. (2020) · Journal of Electrical & Electronic Systems Research · doi:10.24191/jeesr.v17i1.006 · Cited by 2 · see citing works
LDLCT An Instance-Based Framework for Lesion Detection on Lung CT Scans
Khajuria, T., Badr, E., Al-Mallah, M. H., & Sakr, S. (2019) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2019.00106 · Cited by 2 · see citing works
Non-negative subspace feature representation for few-shot learning in medical imaging
Fan, K., Cai, X., & Niranjan, M. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105334 · Cited by 1 · see citing works
Leveraging Advanced Visual Recognition Classifier For Pneumonia Prediction
Raval, M., Jin, A., Wan, Y., & Gohel, H. (2024) · IEEE International Conference on AI in Cybersecurity (ICAIC) · doi:10.1109/icaic60265.2024.10433800 · Cited by 1 · see citing works
Xiao, C., Li, Z., Tang, J., & Lee, J. J. (2025) · Mathematics · doi:10.3390/math13091528 · Cited by 1 · see citing works
Automatic CT Lesion Detection Based on Feature Pyramid Inference with Multi-scale Response
Tang, Y., Liu, Z., Song, Y., Han, K., Su, J., Wang, W., Hu, F., & Zhang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78609-0_15 · Cited by 1 · see citing works
Jia, H., & Hou, L.-H. (2021) · Aggression and Violent Behavior · doi:10.1016/j.avb.2021.101561 · Cited by 1 · see citing works
RetractedLesion detection method based on double modules for feature extraction
Song, Y., Wu, J., & Liu, Z. (2021) · International Conference on Artificial Intelligence and Information Systems (ICAIIS) · doi:10.1145/3469213.3470267
- Authors
- Yan, K., Wang, X., Lu, L., & Summers, R. M.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.01766
- OpenAlex
- W2763599350
- Cited by
- 40
- References
- 9
- Citations by year
- 2025: 2 · 2024: 9 · 2023: 3 · 2022: 4 · 2021: 8 · 2020: 5 · 2019: 6 · 2018: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- lesion detection, pseudo-labeling, unsupervised clustering, computer-aided detection, PACS, liver lesions, pulmonary nodules, lymph nodes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
