Cited by 1Dental Health and Care UtilizationReliability and Agreement in MeasurementDental materials and restorations

Training Protocol and Calibration of the International Caries Detection and Assessment System in a School-Based Clinical Trial of Elementary School-Age Children

Open access
Yamal, Jose‐Miguel, Mofleh, Dania, Chuang, Ru‐Jye, Wang, Mengxi, Johnson, Kila, Garcia‐Quintana, Alejandra, Titiloye, Tolulope, Nelson, Suchitra +1 more·2025·Journal of Public Health Dentistry

Abstract

OBJECTIVES: This paper demonstrates International Caries Detection and Assessment System (ICDAS) training procedures and inter/intra-examiner reliability for lesion severity, activity, and filling criteria in an elementary school setting. METHODS: ICDAS training was conducted in December 2021 prior to a school-based cluster-randomized controlled trial. A total of 59 children (6-8 years old) in grades 1-2 from one school in Houston, Texas were recruited. Two licensed dentists received a 5-day training from an ICDAS trainer at the participating school. Thirty-eight children were evaluated at least once by each dental trainee, and 18 had repeat examinations. The ICDAS criteria were used to classify dental caries lesion severity (0-6), lesion activity (active/inactive), and filling material (0-9) of all visually available primary and permanent tooth surfaces. Inter and intra-examiner reliability between each dental trainee and the "gold standard" trainer was evaluated using Kappa statistics. RESULTS: For lesion severity, the examination indicated an inter-rater weighted Kappa of 0.77 (95% CI 0.71-0.82) and 0.82 (95% CI 0.77-0.87) for each trainee examiner compared to the gold standard examiner. The Kappas increased slightly from the first to second examinations (lesion-weighted Kappa: 0.76 to 0.84 and 0.82 to 0.84). The intra-rater reliability indicated excellent reliability for lesion and filling (0.83 to 0.94) and moderate for activity (0.58). CONCLUSIONS: ICDAS training/calibration were essential to ensure accuracy and reliability of dental caries measurements. The study demonstrated that ICDAS training of dentists with no prior criteria experience is feasible in a community setting and can lead to high reliability and repeatability.

Integrity signals

Registry

Crossref · 10.1111/jphd.12648

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 9 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Interpretationcitation

    The lesion code weighted Kappas in this study (0.77-0.82) were similar to or higher than those in a similar adult study (0.42-0.83).

    “The lesion code weighted Kappas in the current study (0.77–0.82) were similar or higher than a similar study conducted in adults (0.42–0.83).16”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [16] source

      The cited paper is an adult dental-caries study, and its Abstract > Results gives 0.42–0.83 for weighted kappas and observed-to-maximum kappa ratios taken together. Its Results section reports weighted inter-examiner kappas of 0.34–0.53, so 0.42–0.83 is not the range for weighted kappas alone. No page numbers are available in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Premisecitation

    Using ICDAS helps researchers observe subtle intervention effects and nuanced changes from interventions.

    “Secondly, the adoption of ICDAS facilitates the observation of subtle intervention effects, allowing researchers to discern nuanced changes resulting from the interventions.6,7”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [6] source
    • cites [7] source

      The Introduction describes ICDAS as a seven-category scale of caries severity, and the Discussion says it can provide a more detailed assessment than the criteria used in these trials. Those passages support its ability to distinguish finer clinical differences, but the paper does not show that ICDAS detects subtle intervention effects or changes resulting from interventions. The indexed passages provide section labels but no page numbers. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Interpretationcitation

    The inter-rater Kappa scores in this study were similar to those from a previous study (0.69-0.81) conducted in a university pediatric dental and endodontic clinic.

    “The inter-reliability kappa scores reported in this study were similar to a previous study (0.69–0.81) that conducted ICDAS training in a pediatric dental and endodontic clinic in a university institution,10 a finding similar to prior studies.14”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [14] source

      The Reliability of ICDAS section (page unavailable) reports inter-examiner kappa scores from examiners trained and calibrated with ICDAS, including values that overlap the claimed range. It does not report a 0.69–0.81 range or identify the study setting as a university pediatric dental and endodontic clinic. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Findingown data

    Examiners showed moderate consistency with the ICDAS trainer in identifying lesion activity.

    “We observed moderate consistency among examiners in identifying lesion activity with the ICDAS trainer.”
    Quote in the paperSubstantial claim
  • Findingown data

    Lesion severity inter-rater agreement with the gold standard examiner yielded weighted Kappa values of 0.77 (95% CI 0.71-0.82) for one trainee and 0.82 (95% CI 0.77-0.87) for the other.

    “For lesion severity, the examination indicated an inter-rater weighted Kappa of 0.77 (95% CI 0.71–0.82) and 0.82 (95% CI 0.77–0.87) for each trainee examiner compared to the gold standard examiner.”
    • Also stated in: § Results · plain
      “This resulted in a weighted Kappa=0.77 (95% CI 0.71–0.82) for Examiner A versus B, and a weighted Kappa=0.82 (95% CI 0.77–0.87) for Examiner A versus C (Table 3).”
    Quote in the paperSubstantial claim
  • May combine two claims. 10
  • Classified as probably not a substantial claim. 8
  • Not yet classified. 1
  • May combine two claims.

    Findingown data

    For ICDAS lesion coding, agreement was greater at the >2 cutpoint than at the >1 cutpoint, with Kappa values of 0.72 and 0.77 for the two examiners.

    “Kappas for both cutpoints of >2 or >1 for the ICDAS lesion coding indicated greater agreement for >2 cutpoint (Kappa=0.72 and 0.77 for the two examiners, Table 5).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Intra-rater reliability was excellent for lesion and filling scores (0.83 to 0.94) and moderate for activity (0.58) for both trainee dentists.

    “The intra-rater reliability indicated excellent reliability for lesion and filling (0.83 to 0.94) and moderate for activity (0.58).”
    • Also stated in: § Results · plain
      “The intra-rater reliability for lesion and filling ranged from 0.83 to 0.94, indicating excellent reliability. The intra-rater reliability was moderate for activity at 0.58 for both trainee dentists (Table 4).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Tooth type agreement was 99% between Examiner A and Examiner B and 98% between Examiner A and Examiner C.

    “Agreement of tooth type between Examiners A versus B was 99%, and A versus C was 98% (Supplemental Table 2).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Lesion severity weighted Kappa values rose slightly between the first and second examinations, from 0.77 to 0.84 for one trainee and from 0.82 to 0.84 for the other.

    “The Kappas increased slightly from the first to second examinations (lesion-weighted Kappa: 0.77 to 0.84 and 0.82 to 0.84).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the participating children, tooth decay prevalence was 74% when using a lesion score >1 and 61% when using a lesion score >2.

    “The prevalence of tooth decay in the cohort of participating children was 74% (using a lesion score >1 for at least one surface/tooth) and 61% (using a lesion score >2 for at least one surface/tooth).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The ICDAS trainer was a licensed practicing dentist who had performed over 5000 ICDAS examinations and had over a decade of ICDAS training experience.

    “The ICDAS trainer is a licensed and practicing dentist, has conducted over 5000 dental examinations using ICDAS, and has conducted trainings for clinical trials and observational research studies using ICDAS for over a decade.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study protocol and recruitment materials were approved by the Institutional Review Board at the University of Texas Health Science Center at Houston.

    “The study protocol and recruitment material were approved by the Institutional Review Board Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The dental caries prevalence in the participating children was similar to that reported for Texas children of the same age.

    “This prevalence is similar to that seen across Texas for the same age group.4”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Recording errors could arise because dentists verbally called scores to recorders, and these errors were not measured during training.

    “The ICDAS scoring consisted of the dentist verbally calling the score to the recorder, who would then write the scores in the designated field on the case report form. This could lead to recording errors which was not measured during training.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    ICDAS training for dentists without prior criteria experience is feasible in a community setting and can produce high reliability and repeatability.

    “The study demonstrated that ICDAS training of dentists with no prior criteria experience is feasible in a community setting and can lead to high reliability and repeatability.”
    • Also stated in: § Discussion: · hedged
      “Findings were similar to a study by Nelson et al., 2011 reporting high intra-rater reliability for filling, indicating that dental examiners participating in observational longitudinal or clinical trial studies with no experience in ICDAS criteria can be trained and calibrated.10”
    Quote in the paperSubstantial claim
  • Not yet classified.

    Premise

    Missing teeth were not counted as events because a visual examination cannot determine the reason for missingness and missing teeth were expected to exfoliate naturally in this age range.

    “Missing tooth was not considered as an event since the reason for a missing tooth cannot be determined with a visual examination only, and missing teeth were expected to naturally exfoliate due to the age range of the children assessed.”
    Quote in the paperNot yet classified
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Written parental consent and verbal child assent were obtained at the time of measurement.

    “Written consent was obtained from the parent and verbal assent was obtained from the child at the time of measurement.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The confidence intervals for Kappa did not adjust for correlation among teeth within the same child.

    “No adjustment was made to account for the correlation of the teeth within the same child in the calculation of confidence intervals.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The three examiners independently examined each child at least once, and 18 children were examined twice by each examiner.

    “Days 3 to 5 of the training consisted of a series of assessments where the three examiners (trainer and trainees) independently examined each child at least once (n=38), of which 18 were assessed twice by each examiner (a total of six assessments for each child to assess whether the agreement improved upon re-assessing the child after calibration).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Because air syringes were not used due to COVID-19 restrictions, identifying certain filling stages, such as partial sealant and tooth-colored fillings, may have been more difficult.

    “Restrictions on using air syringes (due to the COVID-19 pandemic) may have caused difficulties in identifying certain stages of tooth fillings, such as partial sealant and tooth-colored fillings.”
    Quote in the paperNot a substantial claim
15 of 15

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Number examined by every dentist differs

    Major

    The abstract identifies 38 children as having been evaluated at least once by each trainee, whereas Results says all 59 children were examined at least once by each dentist. The text does not reconcile those totals.

  • Conflicting counts for children recruited

    Minor

    The paper gives two different recruitment totals. One methods passage says 63 children were recruited and 59 met inclusion criteria, while another says a total of 59 children were recruited.

  • Different first-stage lesion severity kappa for examiner A vs B

    Minor

    One results passage reports examiner A vs B lesion-severity kappa as 0.77, while another reports 0.76 for the same first assessment. This is a direct inconsistency in the reported value.

  • Different totals for the number of recruited children

    Minor

    The paper gives two different recruitment totals. One section says 59 children were recruited, while another says 63 were recruited and 59 met the inclusion criteria.

  • Mismatch in the reported first-round kappa for examiner B

    Minor

    One results passage says examiner B's weighted kappa increased from 0.77 to 0.84, while another says it increased from 0.76 to 0.84. These two statements cannot both describe the same comparison.

  • Initial lesion Kappa differs for trainee B

    Minor

    The abstract gives 0.77 as the first-examination Kappa in its first-to-second comparison, while Results gives 0.76 for the corresponding A-versus-B comparison.

  • Recruitment total differs between Methods and Results

    Minor

    Methods says 63 children were recruited and 59 met the inclusion criteria. Results instead calls 59 the number recruited.

7 of 7

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. 10.1922/CDH_2612Nelson05 · DOI

Cited by

Sorted by citations

Authors
Yamal, J., Mofleh, D., Chuang, R., Wang, M., Johnson, K., Garcia‐Quintana, A., Titiloye, T., Nelson, S., & Sharma, S. V.
Affiliations
  • Department of Biostatistics and Data Science Coordinating Center for Clinical Trials, The University of Texas Health Science Center at Houston (UTHealth Houston) School of Public Health Houston Texas USA
  • The University of Texas Health Science Center at Houston (US)
  • Harris Health System (US)
  • Department of Epidemiology Center for Health Equity, The University of Texas Health Science Center at Houston (UTHealth Houston) School of Public Health Houston Texas USA
  • Case Western Reserve University (US)
Corresponding author
Sharma, Shreela V.
Venue
Journal of Public Health Dentistry
Publisher
Wiley
Published
2024-12-08
Volume
85
Issue
1
Pages
13-20
ISSN
  • 0022-4006 (ISSN-L)
  • 1752-7325
OpenAlex
W4405155953
Cited by
1
References
14
Citations by year
2025: 1
FWCI
0.46
Type
journal article
Language
en
Open access
Locations
  • Journal of Public Health Dentistry (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • Digital Commons-TMC (Texas Medical Center) (submittedVersion)
Funders
National Institute of Dental and Craniofacial Research (4UH3DE029213–02)
Topics
  • Health Sciences › Dentistry › Periodontics › Dental Health and Care Utilization
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
  • Health Sciences › Dentistry › Orthodontics › Dental materials and restorations
Keywords
ICDAS, dental caries assessment, examiner reliability, inter-rater reliability, intra-rater reliability, caries activity, dental fillings, primary teeth, permanent teeth, dental education
MeSH terms
Calibration, Child, Dental Caries, Female, Humans, Male, Texas, Reproducibility of Results, Observer Variation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review