CoChat
Sign Up
PreprintCited by 268Artificial Intelligence in Healthcare and EducationAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?

Open access
Cho, Junghwan, Lee, Kyewook, Shin, Ellie, Choy, Garry, Do, Synho·2015·arXiv
DOI 10.48550/arxiv.1511.06348arXiv 1511.06348
Open this paper in CoChat — free accountOpen PDF

Abstract

The use of Convolutional Neural Networks (CNN) in natural image classification systems has produced very impressive results. Combined with the inherent nature of medical images that make them ideal for deep-learning, further application of such systems to medical image classification holds much promise. However, the usefulness and potential impact of such a system can be completely negated if it does not reach a target accuracy. In this paper, we present a study on determining the optimum size of the training data set necessary to achieve high classification accuracy with low variance in medical image classification systems. The CNN was applied to classify axial Computed Tomography (CT) images into six anatomical classes. We trained the CNN using six different sizes of training data set (5, 10, 20, 50, 100, and 200) and then tested the resulting system with a total of 6000 CT images. All images were acquired from the Massachusetts General Hospital (MGH) Picture Archiving and Communication System (PACS). Using this data, we employ the learning curve approach to predict classification accuracy at a given training sample size. Our research will present a general methodology for determining the training data set size necessary to achieve a certain target classification accuracy that can be easily applied to other problems within such systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1511.06348

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The neural network method critically depends on relatively larger training and test datasets compared to image processing methods.

    “Hence, the neural network method critically depends on relatively larger training and test datasets compared to image processing methods [52-54].”

    From Automated Defect Detection and Decision-Support in Gas Turbine Blade Inspection (2021) · cites this paper as [53]

    The Results and Discussions (p. 6) report that larger training sets improved this CNN’s classification accuracy, supporting the training-data portion. The paper does not establish that neural networks require larger training or test datasets than image-processing methods; it reports using 6,000 test images but makes no comparative claim about test-set requirements. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Neural networks require significant amounts of data to accurately perform a classification, especially with increasing number of classes.

    “However, they require significant amounts of data to accurately perform a classification, especially with increasing number of classes [52-54].”

    From Automated Defect Detection and Decision-Support in Gas Turbine Blade Inspection (2021) · cites this paper as [53]

    The Learning Curve section (p. 4) says classification accuracy depends heavily on training-set size, and Results and Discussions (p. 6) estimates that this classifier needs 4,092 images per class to reach its target accuracy. The study uses six fixed anatomical classes; it does not test whether increasing the number of classes increases the amount of data needed. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 22

  1. Anavi, Yaron, Kogan, Ilya, Gelbart, Elad, Geva, Ofer, and Greenspan, Hayit. A comparative study for chest radiograph image retrieval using binary texture and deep learning classification. In Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE , pp. 2940-2943. IEEE, 2015.
  2. Bar, Yaniv, Diamant, Idit, Wolf, Lior, and Greenspan, Hayit. Deep learning with non-medical training used for chest pathology identification. In SPIE Medical Imaging , pp. 94140V-94140V. International Society for Optics and Photonics, 2015.
  3. Dennis Jr, John E, Gay, David M, and Walsh, Roy E. An adaptive nonlinear least-squares algorithm. ACM Transactions on Mathematical Software (TOMS) , 7(3):348-368, 1981.
  4. Dobbin, Kevin K and Simon, Richard M. Sample size planning for developing classifiers using high-dimensional dna microarray data. Biostatistics , 8(1):101-117, 2007.
  5. Dobbin, Kevin K, Zhao, Yingdong, and Simon, Richard M. How large a training set is needed to develop a classifier for microarray data? Clinical Cancer Research , 14(1):108-114, 2008.
  6. Figueroa, Rosa L, Zeng-Treitler, Qing, Kandula, Sasikiran, and Ngo, Long H. Predicting sample size required for classification performance. BMC medical informatics and decision making , 12 (1):8, 2012.
  7. Havaei, Mohammad, Davy, Axel, Warde-Farley, David, Biard, Antoine, Courville, Aaron, Bengio, Yoshua, Pal, Chris, Jodoin, Pierre-Marc, and Larochelle, Hugo. Brain tumor segmentation with deep neural networks. arXiv preprint arXiv:1505.03540 , 2015. · arXiv
  8. Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross, Guadarrama, Sergio, and Darrell, Trevor. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia , pp. 675-678. ACM, 2014.
  9. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
  10. Margeta, Jan, Criminisi, Antonio, Cabrera Lozoya, R, Lee, Daniel C, and Ayache, Nicholas. Finetuned convolutional neural nets for cardiac mri acquisition plane recognition. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization , pp. 1-11, 2015.
  11. Roth, Holger R, Lee, Christopher T, Shin, Hoo-Chang, Seff, Ari, Kim, Lauren, Yao, Jianhua, Lu, Le, and Summers, Ronald M. Anatomy-specific classification of medical images using deep convolutional nets. arXiv preprint arXiv:1504.04003 , 2015a. · arXiv
  12. Roth, Holger R, Yao, Jianhua, Lu, Le, Stieger, James, Burns, Joseph E, and Summers, Ronald M. Detection of sclerotic spine metastases via random aggregation of deep convolutional neural network classifications. In Recent Advances in Computational Methods and Clinical Applications for Spine Imaging , pp. 3-12. Springer, 2015c.
  13. Shin, Hoo-Chang, Lu, Le, Kim, Lauren, Seff, Ari, Yao, Jianhua, and Summers, Ronald M. Interleaved text/image deep mining on a large-scale radiology database for automated image interpretation. arXiv preprint arXiv:1505.00670 , 2015. · arXiv
  14. Shiraishi, Junji, Li, Qiang, Appelbaum, Daniel, and Doi, Kunio. Computer-aided diagnosis and artificial intelligence in clinical imaging. In Seminars in nuclear medicine , volume 41, pp. 449462. Elsevier, 2011.
  15. Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014. · arXiv
  16. Srivastava, Ruchir, Cheng, Jun, Wong, Damon WK, and Liu, Jiang. Using deep learning for robustness to parapapillary atrophy in optic disc segmentation. In Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on , pp. 768-771. IEEE, 2015.
  17. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. arXiv preprint arXiv:1409.4842 , 2014. · arXiv
  18. Wang, Jiazhuo, MacKenzie, John D, Ramachandran, Rageshree, and Chen, Danny Z. Neutrophils identification by deep learning and voronoi diagram of clusters. In Medical Image Computing and Computer-Assisted Intervention?MICCAI 2015 , pp. 226-233. Springer, 2015.
  19. Xu, Yan, Jia, Zhipeng, Ai, Yuqing, Zhang, Fang, Lai, Maode, Chang, Eric I, et al. Deep convolutional activation features for large scale brain tumor histopathology image classification and segmentation. In Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on , pp. 947-951. IEEE, 2015.
  20. Yan, Zhennan, Zhan, Yiqiang, Peng, Zhigang, Liao, Shu, Shinagawa, Yoshihisa, Metaxas, Dimitris N, and Zhou, Xiang Sean. Bodypart recognition using multi-stage deep learning. In Information Processing in Medical Imaging , pp. 449-461. Springer, 2015.
  21. Ypsilantis, Petros-Pavlos, Siddique, Musib, Sohn, Hyon-Mok, Davies, Andrew, Cook, Gary, Goh, Vicky, and Montana, Giovanni. Predicting response to neoadjuvant chemotherapy with pet imaging using convolutional neural networks. PloS one , 10(9):e0137036, 2015.
  22. Zeiler, Matthew D and Fergus, Rob. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014 , pp. 818-833. Springer, 2014.

Cited by

Sorted by citations

Authors
Cho, J., Lee, K., Shin, E., Choy, G., & Do, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2015
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1511.06348
arXiv
1511.06348
OpenAlex
W2175082661
Cited by
268
References
1
Citations by year
2026: 1 · 2025: 7 · 2024: 39 · 2023: 37 · 2022: 32 · 2021: 48 · 2020: 50 · 2019: 28 · 2018: 15 · 2017: 7 · 2016: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
training data size, medical image classification, convolutional neural networks, learning curve, anatomical classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review