CoChat
Sign Up
Cited by 452Forensic Anthropology and Bioarchaeology StudiesMedical Imaging and AnalysisArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.

Fully Automated Deep Learning System for Bone Age Assessment

Open access
Lee, Hyunkwang, Tajmir, Shahein, Lee, Jenny, Zissen, Maurice, Yeshiwas, Bethel Ayele, Alkasab, Tarik K., Choy, Garry, Do, Synho·2017·Journal of Digital Imaging
DOI 10.1007/s10278-017-9955-8PMC PMC5537090
Open this paper in CoChat — free accountOpen PDF

Abstract

Skeletal maturity progresses through discrete phases, a fact that is used routinely in pediatrics where bone age assessments (BAAs) are compared to chronological age in the evaluation of endocrine and metabolic disorders. While central to many disease evaluations, little has changed to improve the tedious process since its introduction in 1950. In this study, we propose a fully automated deep learning pipeline to segment a region of interest, standardize and preprocess input radiographs, and perform BAA. Our models use an ImageNet pretrained, fine-tuned convolutional neural network (CNN) to achieve 57.32 and 61.40% accuracies for the female and male cohorts on our held-out test images. Female test radiographs were assigned a BAA within 1 year 90.39% and within 2 years 98.11% of the time. Male test radiographs were assigned 94.18% within 1 year and 99.00% within 2 years. Using the input occlusion method, attention maps were created which reveal what features the trained model uses to perform BAA. These correspond to what human experts look at when manually performing BAA. Finally, the fully automated BAA system was deployed in the clinical environment as a decision supporting system for more accurate and efficient BAAs at much faster interpretation time (<2 s) than the conventional method.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10278-017-9955-8

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Overview of a the conventional GP-BAA methodology and b our proposed fully automated BAA deep learning system
Fig. 1 Overview of a the conventional GP-BAA methodology and b our proposed fully automated BAA deep learning systemCC BY · PubMed Central
Fig. 2 Bone age distribution for radiographs of female and male left hands
Fig. 2 Bone age distribution for radiographs of female and male left handsCC BY · PubMed Central
Fig. 4 Overview of a deep CNN-based preprocessing engine to automatically detect a hand, generate a hand mask, and feed that into the vision pipeline to standardize images, making the trained automated BAA model invariant to differences in input images
Fig. 4 Overview of a deep CNN-based preprocessing engine to automatically detect a hand, generate a hand mask, and feed that into the vision pipeline to standardize images, making the trained automated BAA model invariant to differences in input imagesCC BY · PubMed Central
Fig. 3 Examples of input radiographs utilized in this work. All images have varying sizes, but they were resized for the purposes of this figure
Fig. 3 Examples of input radiographs utilized in this work. All images have varying sizes, but they were resized for the purposes of this figureCC BY · PubMed Central
Fig. 5 Finding the optimal combination of image patch sizes and strides for optimal mask generation in the preprocessing engine. a mean Intersection over (mIoU) results were shown for all combinations of strides (2, 4, 8, 16) and image patch sizes (16 × 16, 24 × 24, 32 × 32, 40 × 40, 48 × 48, 56 × 56, 64 × 64). b Representative predicted and ground truth binary maps with the equation for Intersection over Union (IoU) for a single case. mIoU was calculated by taking the arithmetic mean of IoU values for all 280 test cases
Fig. 5 Finding the optimal combination of image patch sizes and strides for optimal mask generation in the preprocessing engine. a mean Intersection over (mIoU) results were shown for all combinations of strides (2, 4, 8, 16) and image patch sizes (16 × 16, 24 × 24, 32 × 32, 40 × 40, 48 × 48, 56 × 56, 64 × 64). b Representative predicted and ground truth binary maps with the equation for Intersection over Union (IoU) for a single case. mIoU was calculated by taking the arithmetic mean of IoU values for all 280 test casesCC BY · PubMed Central
Fig. 6 a GoogLeNet network topology that we used for this study. b The inception module, utilized in GoogLeNet, contains six convolutional layers with different kernel sizes and a pooling layer. All resultant outputs are concatenated into a single output vector
Fig. 6 a GoogLeNet network topology that we used for this study. b The inception module, utilized in GoogLeNet, contains six convolutional layers with different kernel sizes and a pooling layer. All resultant outputs are concatenated into a single output vectorCC BY · PubMed Central
Fig. 7 Ten examples at each stage of preprocessing as described in the “Preprocessing engine” section a Input radiographs. The images have been transformed to a square shape for consistent layout. b Normalized images with consistent grayscale base and image size. c Label maps of hand (white) and non-hand (black) classes. d Generated masks for segmentation. e Final preprocessed images.
Fig. 7 Ten examples at each stage of preprocessing as described in the “Preprocessing engine” section a Input radiographs. The images have been transformed to a square shape for consistent layout. b Normalized images with consistent grayscale base and image size. c Label maps of hand (white) and non-hand (black) classes. d Generated masks for segmentation. e Final preprocessed images.CC BY · PubMed Central
Fig. 8 CNN test accuracy with the real-time data augmentation using different styles of training. The “trained from scratch” method trains a CNN with a random weight initialization. Other methods fine-tune the ImageNet pretrained CNNs by incrementally updating weights of each fully connected (fc) layer from inception5 to conv1, detailed in Fig. 6
Fig. 8 CNN test accuracy with the real-time data augmentation using different styles of training. The “trained from scratch” method trains a CNN with a random weight initialization. Other methods fine-tune the ImageNet pretrained CNNs by incrementally updating weights of each fully connected (fc) layer from inception5 to conv1, detailed in Fig. 6CC BY · PubMed Central
Fig. 9 Performance of four different methods (M1–M4) of training for female (a) and male (b) bone age assessments. M1 trains a CNN from scratch with a random weight initialization on original images down sampled to 224 × 224 pixels. M2 contains images from the automated preprocessing engine. M3 contains synthetically generated images for improving network generalization in addition to M2. M4 fine-tunes an ImageNet pretrained CNN on the preprocessed images with data augmentation turned on. “Correct” corresponds to the case where the prediction of the model is the same as the ground truth. “Within 1 year” and “within 2 years” include the cases where the network’s prediction is within 1 and 2 years, respectively. In addition, root mean squared error (RMSE) and mean average precision (mAP) were reported for the four different models to figure out how robust and well-performing each model is
Fig. 9 Performance of four different methods (M1–M4) of training for female (a) and male (b) bone age assessments. M1 trains a CNN from scratch with a random weight initialization on original images down sampled to 224 × 224 pixels. M2 contains images from the automated preprocessing engine. M3 contains synthetically generated images for improving network generalization in addition to M2. M4 fine-tunes an ImageNet pretrained CNN on the preprocessed images with data augmentation turned on. “Correct” corresponds to the case where the prediction of the model is the same as the ground truth. “Within 1 year” and “within 2 years” include the cases where the network’s prediction is within 1 and 2 years, respectively. In addition, root mean squared error (RMSE) and mean average precision (mAP) were reported for the four different models to figure out how robust and well-performing each model isCC BY · PubMed Central
Fig. 10 Selected examples of attention maps for female (upper rows) and male (lower rows) in the four major skeletal maturity stages: prepuberty, early-and-mid puberty, late puberty, and postpuberty stages [10]. Infant and toddler categories were excluded. Six representative attention maps were carefully chosen to represent the general trend for each category. a Prepuberty: BAAs from 2 to 7 years for females and 3–9 years for males. b Early-and-mid puberty: 7–13 years for females and 9–14 years for males. c Late-puberty: 13–15 years for females and 14–16 years for males. d Postpuberty: 15 and up for females and 17 years and up for males
Fig. 10 Selected examples of attention maps for female (upper rows) and male (lower rows) in the four major skeletal maturity stages: prepuberty, early-and-mid puberty, late puberty, and postpuberty stages [10]. Infant and toddler categories were excluded. Six representative attention maps were carefully chosen to represent the general trend for each category. a Prepuberty: BAAs from 2 to 7 years for females and 3–9 years for males. b Early-and-mid puberty: 7–13 years for females and 9–14 years for males. c Late-puberty: 13–15 years for females and 14–16 years for males. d Postpuberty: 15 and up for females and 17 years and up for malesCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 40 references

Research works printed but not linked: 1

  1. Greulich WW, Pyle SI. Radiographic atlas of skeletal development of the hand and wrist. Am J Med Sci. pdfs.journals. lww.com ; 1959; Available: http://pdfs.journals.lww.com/amjmedsci/1959/09000/Radiographic_Atlas_of_Skeletal_Development_of_the.30.pdf · DOI

Cited by

Sorted by citations

See all 452

Related

Metadata

Authors
Lee, H., Tajmir, S., Lee, J., Zissen, M., Yeshiwas, B. A., Alkasab, T. K., Choy, G., & Do, S.
Affiliations
Harvard University (US)
Corresponding author
Do, Synho
ORCID
Venue
Journal of Digital Imaging
Publisher
Springer Science and Business Media LLC
Published
2017-03-08
Volume
30
Issue
4
Pages
427-441
ISSN
  • 0897-1889 (ISSN-L)
  • 1618-727X
DOI
10.1007/s10278-017-9955-8
PMC
PMC5537090
OpenAlex
W2592765733
Cited by
452
References
40
Citations by year
2026: 21 · 2025: 37 · 2024: 45 · 2023: 57 · 2022: 55 · 2021: 62 · 2020: 75 · 2019: 47 · 2018: 45 · 2017: 8
FWCI
276.78
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Imaging Informatics in Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Topics
  • Social Sciences › Arts and Humanities › Archeology (arts and humanities) › Forensic Anthropology and Bioarchaeology Studies
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
bone age assessment, skeletal maturity, convolutional neural networks, attention maps, decision support system, pediatric endocrine disorders, automated image analysis
MeSH terms
Machine Learning, Adolescent, Adult, Age Determination by Skeleton, Child, Female, Hand, Humans, Male, Software, Neural Networks, Computer, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review