Cited by 29Artificial Intelligence in Healthcare and EducationClinical Laboratory Practices and Quality ControlArtificial Intelligence in Healthcare

Diagnostic Quality Model (DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML

Open access
Lennerz, Jochen K., Salgado, Roberto, Kim, Grace E., Sirintrapun, Sahussapont Joseph, Thierauf, Julia C., Singh, Ankit, Indave, Iciar, Bard, Adam +8 more·2023·Clinical Chemistry and Laboratory Medicine (CCLM)

Abstract

Background: Laboratory medicine has reached the era where promises of artificial intelligence and machine learning (AI/ML) seem palpable. Currently, the primary responsibility for risk-benefit assessment in clinical practice resides with the medical director. Unfortunately, there is no tool or concept that enables diagnostic quality assessment for the various potential AI/ML applications. Specifically, we noted that an operational definition of laboratory diagnostic quality – for the specific purpose of assessing AI/ML improvements – is currently missing. Methods: A session at the 3rd Strategic Conference of the European Federation of Laboratory Medicine in 2022 on “ AI in the Laboratory of the Future” prompted an expert roundtable discussion. Here we present a conceptual diagnostic quality framework for the specific purpose of assessing AI/ML implementations. Results: The presented framework is termed diagnostic quality model (DQM) and distinguishes AI/ML improvements at the test, procedure, laboratory, or healthcare ecosystem level. The operational definition illustrates the nested relationship among these levels. The model can help to define relevant objectives for implementation and how levels come together to form coherent diagnostics. The affected levels are referred to as scope and we provide a rubric to quantify AI/ML improvements while complying with existing, mandated regulatory standards. We present 4 relevant clinical scenarios including multi-modal diagnostics and compare the model to existing quality management systems. Conclusions: A diagnostic quality model is essential to navigate the complexities of clinical AI/ML implementations. The presented diagnostic quality framework can help to specify and communicate the key implications of AI/ML solutions in laboratory diagnostics.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1515/cclm-2022-1151

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Diagnostic Quality Model and Clinical Explainable AI Guidelines provide structured methodologies that could refine the CASoF by integrating rigorous quality assessments and enhancing transparency in AI tools.

    “They provide structured methodologies that could refine the CASoF by integrating rigorous quality assessments and enhancing transparency in AI tools.”

    From Checklist Approach to Developing and Implementing AI in Clinical Settings: Instrument Development Study (2025) · cites this paper as [16]

    The Diagnostic quality aspects section describes a rubric for measuring quality changes associated with AI/ML, supporting the claim that DQM offers a structured approach to quality assessment. The Abstract says the framework can help communicate the implications of AI/ML solutions, but the cited paper does not establish that DQM enhances transparency in AI tools. It also does not present the Clinical Explainable AI Guidelines; the proposed refinement of CASoF is the citing authors’ application, not a finding attributed to this source. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • The Diagnostic Quality Model by Lennerz et al and the Clinical Explainable AI Guidelines by Jin et al address diagnostic quality and explainability within medical imaging.

    “The Diagnostic Quality Model by Lennerz et al [16] and the Clinical Explainable AI Guidelines by Jin et al [17] address diagnostic quality and explainability within medical imaging.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Clinical Laboratory Improvement Amendments (CLIA). Code of Federal Regulations (CFR), Title 42, Chapter IV, Subchapter G Part CFR 493.1-493.2001 42 USC 263a. US 1988.
  2. McPherson RA, Pincus MR, Henry JB. Henry's clinical diagnosis and management by laboratory methods, 21st ed. Philadelphia: Saunders Elsevier; 2007.

Cited by

Sorted by citations

Authors
Lennerz, J. K., Salgado, R., Kim, G. E., Sirintrapun, S. J., Thierauf, J. C., Singh, A., Indave, I., Bard, A., Weissinger, S. E., Heher, Y. K., de Baca, M. E., Cree, I. A., Bennett, S., Carobene, A., Ozben, T., & Ritterhouse, L. L.
Affiliations
  • Harvard University (US)
  • Peter MacCallum Cancer Centre (AU)
  • University of California, San Francisco (US)
  • Memorial Sloan Kettering Cancer Center (US)
  • European Monitoring Centre for Drugs and Drug Addiction (PT)
  • Alb Fils Kliniken (DE)
  • Pacific Pathology Partners , Seattle , WA , USA
  • Centre international de recherche sur le cancer (FR)
  • Mayo Clinic (US)
  • IRCCS San Raffaele Scientific Institute , Milan , Italy
  • Medical Faculty, Dept. of Clinical Biochemistry , Akdeniz University , Antalya , Türkiye
Corresponding author
Lennerz, Jochen K.
Venue
Clinical Chemistry and Laboratory Medicine (CCLM)
Published
2023-01-25
Volume
61
Issue
4
Pages
544-557
ISSN
  • 1434-6621 (ISSN-L)
  • 1437-4331
OpenAlex
W4318021306
Cited by
29
References
114
Citations by year
2026: 6 · 2025: 8 · 2024: 11 · 2023: 4
FWCI
1.16
Type
journal article
Language
en
Open access
Locations
  • Clinical Chemistry and Laboratory Medicine (CCLM) (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Amgen
  • Bristol-Myers Squibb
  • Eli Lilly and Company
  • AstraZeneca
  • Sanofi
  • World Health Organization
  • Centre International de Recherche sur le Cancer
  • National Institutes of Health (CA225655)
  • EMD Serono
  • Loxo Oncology
  • National Cancer Institute (R37CA225655)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
machine learning, laboratory diagnosis, risk-benefit assessment, quality management systems, regulatory standards, expert roundtable
MeSH terms
Machine Learning, Artificial Intelligence, Delivery of Health Care, Humans, Ecosystem

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review