Diagnostic Quality Model (DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML
Open accessAbstract
Background: Laboratory medicine has reached the era where promises of artificial intelligence and machine learning (AI/ML) seem palpable. Currently, the primary responsibility for risk-benefit assessment in clinical practice resides with the medical director. Unfortunately, there is no tool or concept that enables diagnostic quality assessment for the various potential AI/ML applications. Specifically, we noted that an operational definition of laboratory diagnostic quality – for the specific purpose of assessing AI/ML improvements – is currently missing. Methods: A session at the 3rd Strategic Conference of the European Federation of Laboratory Medicine in 2022 on “ AI in the Laboratory of the Future” prompted an expert roundtable discussion. Here we present a conceptual diagnostic quality framework for the specific purpose of assessing AI/ML implementations. Results: The presented framework is termed diagnostic quality model (DQM) and distinguishes AI/ML improvements at the test, procedure, laboratory, or healthcare ecosystem level. The operational definition illustrates the nested relationship among these levels. The model can help to define relevant objectives for implementation and how levels come together to form coherent diagnostics. The affected levels are referred to as scope and we provide a rubric to quantify AI/ML improvements while complying with existing, mandated regulatory standards. We present 4 relevant clinical scenarios including multi-modal diagnostics and compare the model to existing quality management systems. Conclusions: A diagnostic quality model is essential to navigate the complexities of clinical AI/ML implementations. The presented diagnostic quality framework can help to specify and communicate the key implications of AI/ML solutions in laboratory diagnostics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Diagnostic Quality Model and Clinical Explainable AI Guidelines provide structured methodologies that could refine the CASoF by integrating rigorous quality assessments and enhancing transparency in AI tools.
“They provide structured methodologies that could refine the CASoF by integrating rigorous quality assessments and enhancing transparency in AI tools.”
From Checklist Approach to Developing and Implementing AI in Clinical Settings: Instrument Development Study (2025) · cites this paper as [16]
The Diagnostic quality aspects section describes a rubric for measuring quality changes associated with AI/ML, supporting the claim that DQM offers a structured approach to quality assessment. The Abstract says the framework can help communicate the implications of AI/ML solutions, but the cited paper does not establish that DQM enhances transparency in AI tools. It also does not present the Clinical Explainable AI Guidelines; the proposed refinement of CASoF is the citing authors’ application, not a finding attributed to this source. (LLM, conf=0.83)
Experimental — accuracy not yet measuredThe Diagnostic Quality Model by Lennerz et al and the Clinical Explainable AI Guidelines by Jin et al address diagnostic quality and explainability within medical imaging.
“The Diagnostic Quality Model by Lennerz et al [16] and the Clinical Explainable AI Guidelines by Jin et al [17] address diagnostic quality and explainability within medical imaging.”
From Checklist Approach to Developing and Implementing AI in Clinical Settings: Instrument Development Study (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Trastuzumab Deruxtecan in Previously Treated HER2-Low Advanced Breast Cancer
Modi, S., Jacot, W., Yamashita, T., Sohn, J., Vidal, M. J., Tokunaga, E., Tsurutani, J., Ueno, N. T., Prat, A., Chae, Y. S., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2203690 · Cited by 2,764 · see citing works
Standards for Reporting Implementation Studies (StaRI) Statement
Pinnock, H., Barwick, M., Carpenter, C. R., Eldridge, S., Grandes, G., Griffiths, C. J., Rycroft-Malone, J., Meissner, P., Murray, E., Patel, A., et al. (2017) · BMJ · doi:10.1136/bmj.i6795 · Cited by 1,446 · see citing works
Improving Diagnosis in Health Care
Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, & The National Academies of Sciences, E. (2015) · doi:10.17226/21794 · Cited by 1,376 · see citing works
Peters, D. H., Adam, T., Alonge, O., Agyepong, I. A., & Tran, N. (2014) · British Journal of Sports Medicine · doi:10.1136/bmj.f6753 · Cited by 1,109 · see citing works
Unintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066 · see citing works
Privacy and artificial intelligence: challenges for protecting health information in a new era
Murdoch, B. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00687-3 · Cited by 991 · see citing works
Prostate cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up
Parker, C., Castro, E., Fizazi, K., Heidenreich, A., Ost, P., Procopio, G., Tombal, B., & Gillessen, S. (2020) · Annals of Oncology · doi:10.1016/j.annonc.2020.06.011 · Cited by 977 · see citing works
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852 · see citing works
Loh, H. W., Ooi, C. P., Seoni, S., Barua, P. D., Molinari, F., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107161 · Cited by 734 · see citing works
Multimodal machine learning in precision health: A scoping review
Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00712-8 · Cited by 528 · see citing works
Exploring the iceberg of errors in laboratory medicine
Plebani, M. (2009) · Clinica Chimica Acta · doi:10.1016/j.cca.2009.03.022 · Cited by 293 · see citing works
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284 · see citing works
Self-citationTechnology readiness levels for machine learning systems
Lavin, A., Lee, C. M., Visnjic, A., Ganju, S., Newman, D. J., Ganguly, S., Lange, D., Baydin, A. G., Sharma, A., Gibson, A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33128-9 · Cited by 247 · see citing works
Validation of digital pathology imaging for primary histopathological diagnosis
Snead, D. R. J., Tsang, Y., Meskiri, A., Kimani, P. K., Crossman, R., Rajpoot, N. M., Blessing, E., Chen, K., Gopalakrishnan, K., Matthews, P., et al. (2016) · Histopathology · doi:10.1111/his.12879 · Cited by 238 · see citing works
Self-citationSahu, M., Gupta, R. V., Ambasta, R. K., & Kumar, P. (2022) · Progress in molecular biology and translational science · doi:10.1016/bs.pmbts.2022.03.002 · Cited by 202 · see citing works
Fintech Risk Management: A Research Challenge for Artificial Intelligence in Finance
Giudici, P. (2018) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2018.00001 · Cited by 198 · see citing works
Prostate cancer screening with prostate-specific antigen (PSA) test: a clinical practice guideline
Tikkinen, K. A. O., Dahm, P., Lytvyn, L., Heen, A. F., Vernooij, R. W. M., Siemieniuk, R. A. C., Wheeler, R., Vaughan, B., Fobuzi, A. C., Blanker, M. H., et al. (2018) · BMJ · doi:10.1136/bmj.k3581 · Cited by 176 · see citing works
The effects of implementing synoptic pathology reporting in cancer diagnosis: a systematic review
Sluijter, C. E., van Lonkhuijzen, L. R. C. W., van Slooten, H.-J., Nagtegaal, I. D., & Overbeek, L. I. H. (2016) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-016-1935-8 · Cited by 173 · see citing works
Cabitza, F., Campagner, A., Ferrari, D., Resta, C. D., Ceriotti, D., Sabetta, E., Colombini, A., Vecchi, E. D., Banfi, G., Locatelli, M., et al. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-1294 · Cited by 168 · see citing works
Self-citationAdvantages and limitations of total laboratory automation: a personal overview
Lippi, G., & Rin, G. D. (2019) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2018-1323 · Cited by 165 · see citing works
A Rich Array of Prostate Cancer Molecular Biomarkers: Opportunities and Challenges
Kohaar, I., Petrovics, G., & Srivastava, S. K. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20081813 · Cited by 152 · see citing works
Beckers, R., Kwade, Z., & Zanca, F. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.02.011 · Cited by 141 · see citing works
Fonseka, T. M., Bhat, V., & Kennedy, S. H. (2019) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867419864428 · Cited by 126 · see citing works
Car, J., Sheikh, A., Wicks, P., & Williams, M. S. (2019) · BMC Medicine · doi:10.1186/s12916-019-1382-x · Cited by 126 · see citing works
Raciti, P., Sue, J., Retamero, J. A., Ceballos, R., Godrich, R., Kunz, J. D., Casson, A., Thiagarajan, D., Ebrahimzadeh, Z., Viret, J., et al. (2023) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2022-0066-oa · Cited by 119 · see citing works
Automation and artificial intelligence in the clinical laboratory
Naugler, C., & Church, D. L. (2019) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2018.1561640 · Cited by 112 · see citing works
FDA Approval Summary: Abemaciclib With Endocrine Therapy for High-Risk Early Breast Cancer
Royce, M., Osgood, C., Mulkey, F., Bloomquist, E., Pierce, W. F., Roy, A., Kalavar, S., Ghosh, S., Philip, R., Rizvi, F., et al. (2022) · Journal of Clinical Oncology · doi:10.1200/jco.21.02742 · Cited by 107 · see citing works
Lima-Oliveira, G., Lippi, G., Salvagno, G. L., Picheth, G., & Guidi, G. C. (2015) · Journal of Medical Biochemistry · doi:10.2478/jomb-2014-0043 · Cited by 102 · see citing works
Medical Diagnostic Tests: A Review of Test Anatomy, Phases, and Statistical Treatment of Data
Bolboaca, S. D. (2019) · Computational and Mathematical Methods in Medicine · doi:10.1155/2019/1891569 · Cited by 101 · see citing works
Machine learning in laboratory medicine: waiting for the flood?
Cabitza, F., & Banfi, G. (2017) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2017-0287 · Cited by 99 · see citing works
Synoptic Reporting: Evidence-Based Review and Future Directions
Renshaw, A. A., Mena-Allauca, M., Gould, E. W., & Sirintrapun, S. J. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00088 · Cited by 92 · see citing works
Self-citationStenzinger, A., Alber, M., Allgäuer, M., Jurmeister, P. S., Bockmayr, M. L., Budczies, J., Lennerz, J. K., Eschrich, J., Kazdal, D. N., Schirmacher, P., et al. (2021) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2021.02.011 · Cited by 91 · see citing works
Self-citationQuality checkpoints in the MRI-directed prostate cancer diagnostic pathway
Barrett, T., de Rooij, M., Giganti, F., ALLEN, C. M., Barentsz, J. O., & Padhani, A. R. (2022) · Nature Reviews Urology · doi:10.1038/s41585-022-00648-4 · Cited by 89 · see citing works
Evaluating Imaging and Computer-aided Detection and Diagnosis Devices at the FDA
Gallas, B. D., Chan, H.-P., D’Orsi, C. J., Dodd, L. E., Giger, M. L., Gur, D., Krupinski, E. A., Metz, C. E., Myers, K. J., Obuchowski, N. A., et al. (2012) · Academic Radiology · doi:10.1016/j.acra.2011.12.016 · Cited by 81 · see citing works
Boon, I. S., Yong, T. P. T. A., & Boon, C. S. (2018) · Medicines · doi:10.3390/medicines5040131 · Cited by 80 · see citing works
Unraveling Human Complexity and Disease With Systems Biology and Personalized Medicine
Naylor, S., & Chen, J. Y. (2010) · Personalized Medicine · doi:10.2217/pme.10.16 · Cited by 74 · see citing works
Thakur, V., Akerele, O. A., & Randell, E. W. (2022) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2022.2106544 · Cited by 71 · see citing works
Ninatti, G., Kirienko, M., Neri, E., Sollini, M., & Chiti, A. (2020) · Diagnostics · doi:10.3390/diagnostics10060359 · Cited by 70 · see citing works
Sheffield, K. M., Peachey, J., Method, M. W., Grimes, B. R., Brown, J., Saverno, K. R., Sugihara, T., Cui, Z. L., & Lee, K. T. (2022) · Future Oncology · doi:10.2217/fon-2022-0310 · Cited by 68 · see citing works
Agile Processes in Software Engineering and Extreme Programming
Wohlin, C. (2012) · Lecture notes in business information processing · doi:10.1007/978-3-642-30350-0 · Cited by 65 · see citing works
Adversarial attacks against supervised machine learning based network intrusion detection systems
Alshahrani, E., Alghazzawi, D., Alotaibi, R., & Rabie, O. B. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275971 · Cited by 65 · see citing works
The International Collaboration for Cancer Classification and Research
Cree, I. A., Ruiz, B. I. I., Zavadil, J., McKay, J., Olivier, M., Kozlakidis, Z., Lazar, A. J., Hyde, C., Holdenrieder, S., Hastings, R. J., et al. (2020) · International Journal of Cancer · doi:10.1002/ijc.33260 · Cited by 64 · see citing works
Self-citationClinlabomics: leveraging clinical laboratory data by data mining strategies
Wen, X., Leng, P., Wang, J., Yang, G., Zu, R., Jia, X., Zhang, K., Mengesha, B. A., Huang, J. H., Wang, D., et al. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04926-1 · Cited by 63 · see citing works
Mazzucca, S., Tabak, R. G., Pilar, M. R., Ramsey, A. T., Baumann, A. A., Kryzer, E., Lewis, E. M., Padek, M. M., Powell, B. J., & Brownson, R. C. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00032 · Cited by 62 · see citing works
Artificial Intelligence in Medicine
(2022) · Springer eBooks · doi:10.1007/978-3-030-64573-1 · Cited by 61 · see citing works
Harmonization and Standardization: Where Are We Now?
Miller, W. G., & Greenberg, N. L. (2020) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfaa189 · Cited by 59 · see citing works
Quality in laboratory diagnostics: from theory to practice
Lippi, G., Plebani, M., & Šimundić, A.-M. (2010) · Biochemia Medica · doi:10.11613/bm.2010.014 · Cited by 55 · see citing works
Identification of human errors and influencing factors: A machine learning approach
de Morais, C. P. M., Yung, K. L., Johnson, K., Moura, R., Beer, M., & Patelli, E. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105528 · Cited by 50 · see citing works
Rethinking Algorithm Performance Metrics for Artificial Intelligence in Diagnostic Medicine
Reyna, M. A., Nsoesie, E. O., & Clifford, G. D. (2022) · JAMA · doi:10.1001/jama.2022.10561 · Cited by 49 · see citing works
Ghosh, P., Tamboli, P., Vikram, R., & Rao, A. U. K. (2015) · Journal of medical imaging · doi:10.1117/1.jmi.2.4.041009 · Cited by 47 · see citing works
Clinical Utility of Genomic Testing in Cancer Care
Pritchard, D. E., Goodman, C., & Nadauld, L. D. (2022) · JCO Precision Oncology · doi:10.1200/po.21.00349 · Cited by 47 · see citing works
Flowing through laboratory clinical data: the role of artificial intelligence and big data
Padoan, A., & Plebani, M. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0653 · Cited by 42 · see citing works
Carobene, A., Cabitza, F., Bernardini, S., Gopalan, R., Lennerz, J. K., Weir, C., & Cadamuro, J. (2023) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-1030 · Cited by 42 · see citing works
Self-citationSelecting patients with HER2-low breast cancer: Getting out of the tangle
Baez‐Navarro, X., Salgado, R., Denkert, C., Lennerz, J. K., Penault-Llorca, F. M., Viale, G., Bartlett, J. M. S., & van Deurzen, C. H. M. (2022) · European Journal of Cancer · doi:10.1016/j.ejca.2022.08.022 · Cited by 42 · see citing works
Self-citationCarobene, A., Milella, F., Famiglini, L., & Cabitza, F. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0182 · Cited by 37 · see citing works
Self-citationGillan, C., Milne, E., Harnett, N., Purdie, T. G., Jaffray, D. A., & Hodges, B. D. (2018) · Journal of Radiotherapy in Practice · doi:10.1017/s1460396918000468 · Cited by 35 · see citing works
Marble, H. D., Huang, R., Dudgeon, S. N., Lowe, A., Herrmann, M. D., Blakely, S., Leavitt, M. O., Isaacs, M., Hanna, M. G., Sharma, A., et al. (2020) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_27_20 · Cited by 35 · see citing works
Self-citationSvoboda, E., Bořil, T., Rusz, J., Tykalová, T., Horáková, D., Guttmann, C. R. G., Blagoev, K. B., Hatabu, H., & Valtchinov, V. I. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105853 · Cited by 35 · see citing works
Constantinescu, G. A., Schulze, M., Peitzsch, M., Hofmockel, T., Scholl, U. I., Williams, T. A., Lenders, J. W. M., & Eisenhofer, G. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0470 · Cited by 34 · see citing works
Shen, T., Liu, L., Li, W., Fu, P., Xu, K., Jiang, Y., Pan, F., Guo, Y., & Zhang, M.-C. (2019) · Cancer Imaging · doi:10.1186/s40644-019-0221-9 · Cited by 33 · see citing works
Sørensen, P. D., Christensen, H., Laursen, S. W., Hardahl, C., Brandslund, I., & Madsen, J. S. (2021) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2021-1015 · Cited by 32 · see citing works
COMMITTEE ON DIAGNOSTIC ERROR IN HEALTH CARE
Balogh, E. P., Miller, B. T., & Ball, J. R. (2015) · Cited by 31 · see citing works
External validation of Machine Learning models for COVID-19 detection based on Complete Blood Count
Campagner, A., Carobene, A., & Cabitza, F. (2021) · Health Information Science and Systems · doi:10.1007/s13755-021-00167-3 · Cited by 29 · see citing works
Self-citationThakur, V., Akerele, O. A., Brake, N., Wiscombe, M., Broderick, S., Campbell, E., & Randell, E. W. (2022) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2022.12.001 · Cited by 29 · see citing works
Digumarthy, S. R., Mendoza, D. P., Lin, J. J., Chen, T., Rooney, M. M., Chin, E. N., Sequist, L. V. D., Lennerz, J. K., Gainor, J. F., & Shaw, A. T. (2019) · Clinical Lung Cancer · doi:10.1016/j.cllc.2019.10.006 · Cited by 28 · see citing works
Self-citationA robust and parsimonious machine learning method to predict ICU admission of COVID-19 patients
Famiglini, L., Campagner, A., Carobene, A., & Cabitza, F. (2022) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-022-02543-x · Cited by 28 · see citing works
Self-citationHenriksen, K., Dymek, C., Harrison, M. I., Brady, P., & Arnold, S. B. (2017) · Diagnosis · doi:10.1515/dx-2017-0016 · Cited by 27 · see citing works
Tracking Progress in Improving Diagnosis: A Framework for Defining Undesirable Diagnostic Events
Olson, A. P. J., Graber, M. L., & Singh, H. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4304-2 · Cited by 27 · see citing works
The Case for Laboratory Developed Procedures
Kaul, K. L., Sabatini, L. M., Tsongalis, G. J., Caliendo, A. M., Olsen, R. J., Ashwood, E. R., Bale, S. J., Benirschke, R. C., Carlow, D. C., Funke, B., et al. (2017) · Academic Pathology · doi:10.1177/2374289517708309 · Cited by 25 · see citing works
A survey on Artificial Intelligence and Big Data utilisation in Italian clinical laboratories
Bellini, C., Padoan, A., Carobene, A., & Guerranti, R. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0680 · Cited by 25 · see citing works
Self-citationHow Might Artificial Intelligence Applications Impact Risk Management?
Banja, J. D. (2020) · The AMA Journal of Ethic · doi:10.1001/amajethics.2020.945 · Cited by 24 · see citing works
Tajiri, A., Ishihara, R., Kato, Y., Inoue, T., Matsueda, K., Miyake, M., Waki, K., Shimamoto, Y., Fukuda, H., Matsuura, N., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-10739-2 · Cited by 24 · see citing works
Artificial Intelligence Approach for Variant Reporting
Zomnir, M. G., Lipkin, L., Pacula, M., Meneses, E. D., Macleay, A., Duraisamy, S., Nadhamuni, N., Turki, S. H. A., Zheng, Z., Rivera, M. N., et al. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.16.00079 · Cited by 22 · see citing works
Self-citationEvaluating the Effectiveness of Diagnostic Tests
Kennedy, A. G. (2022) · JAMA · doi:10.1001/jama.2022.4463 · Cited by 22 · see citing works
Diagnostic Value of MAML2 Rearrangements in Mucoepidermoid Carcinoma
Thierauf, J., Farahani, A. A., Indave, B. I., Bard, A. Z., White, V. A., Smith, C., Marble, H. D., Hyrcza, M. D., Chan, J. K. C., Bishop, J. A., et al. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23084322 · Cited by 22 · see citing works
Self-citationHealth Care Infrastructure for Financially Sustainable Clinical Genomics
Lennerz, J. K., McLaughlin, H. M., Baron, J. M., Rasmussen, D., Shin, M. S., Berners-Lee, N., Batten, J. M., Swoboda, K. J., Gala, M. K., Winter, H. S., et al. (2016) · Journal of Molecular Diagnostics · doi:10.1016/j.jmoldx.2016.04.003 · Cited by 20 · see citing works
Self-citationVidali, M., Carobene, A., Esposito, S. A., Napolitano, G., Caracciolo, A., Seghezzi, M., Previtali, G., Lippi, G., & Buoro, S. (2020) · International Journal of Laboratory Hematology · doi:10.1111/ijlh.13379 · Cited by 19 · see citing works
Self-citationPrivacy-Preserving Artificial Intelligence: Application to Precision Medicine
Vizitiu, A., Nita, C., Puiu, A., Suciu, C., & Lucian Mihai, I. T. U. (2019) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857960 · Cited by 19 · see citing works
Lans, A., Pierik, R., Bales, J. R., Fourman, M. S., Shin, D., Kanbier, L. N., Rifkin, J., DiGiovanni, W., Chopra, R. R., Moeinzad, R., et al. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102396 · Cited by 19 · see citing works
The harmonization issue in laboratory medicine: the commitment of CCLM
Zaninotto, M., Graziani, M. S., & Plebani, M. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-1111 · Cited by 19 · see citing works
NICE: a step forward in the quality of NHS care
Horton, R. (1999) · The Lancet · doi:10.1016/s0140-6736(99)00098-7 · Cited by 18 · see citing works
Raheem, F., Ofori, H., Simpson, L., & Shah, V. S. (2022) · Annals of Pharmacotherapy · doi:10.1177/10600280211073322 · Cited by 18 · see citing works
Stahl, A.-C., Tietz, A.-S., Kendziora, B., & Dewey, M. (2022) · European Radiology · doi:10.1007/s00330-022-09008-7 · Cited by 18 · see citing works
Current status of clinical research using artificial intelligence techniques
Karekar, S. R., & Vazifdar, A. K. (2021) · Perspectives in Clinical Research · doi:10.4103/picr.picr_25_20 · Cited by 17 · see citing works
Everything is varied: The surprising impact of instantial variation on ML reliability
Campagner, A., Famiglini, L., Carobene, A., & Cabitza, F. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110644 · Cited by 17 · see citing works
Self-citationAlmalawi, A., Khan, A. I., Alsolami, F. J., Abushark, Y. B., Alfakeeh, A. S., & Mekuriyaw, W. D. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4048197 · Cited by 15 · see citing works
Kearney, S. J., Lowe, A., Lennerz, J. K., Parwani, A. V., Bui, M. M., Wack, K., Giannini, G., & Abels, E. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.765385 · Cited by 14 · see citing works
Self-citationRepresentational ethical model calibration
Carruthers, R., Straw, I., Ruffle, J. K., Herron, D., Nelson, A., Bzdok, D., Fernández-Reyes, D., Rees, G. E., & Nachev, P. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00716-4 · Cited by 14 · see citing works
Envisioning the Future Morbidity and Mortality Conference: A Vehicle for Systems Change
Cifra, C. L., & Miller, M. R. (2016) · Pediatric Quality and Safety · doi:10.1097/pq9.0000000000000003 · Cited by 13 · see citing works
Huang, R., Lasiter, L., Bard, A. Z., Quinn, B., Young, C., Salgado, R., Allen, J. D., & Lennerz, J. K. (2021) · JCO Oncology Practice · doi:10.1200/op.20.00862 · Cited by 12 · see citing works
Self-citationRakovic, K., Colling, R. T., Browning, L., Dolton, M., Horton, M. R., Protheroe, A. S., Lamb, A. D. G., Bryant, R. J., Scheffer, R., Crofts, J., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12051225 · Cited by 12 · see citing works
Artificial intelligence in pathology and laboratory medicine
Pillay, T. S. (2021) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2021-207682 · Cited by 11 · see citing works
Adversarial Data Hiding in Digital Images
Wang, D., Li, M., & Zhang, Y. (2022) · Entropy · doi:10.3390/e24060749 · Cited by 11 · see citing works
McGenity, C., Bossuyt, P. M., & Treanor, D. E. (2022) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2022.100091 · Cited by 10 · see citing works
The gaps between the new EU legislation on in vitro diagnostics and the on-the-ground reality
Horgan, D., Plebani, M., Orth, M., Macintyre, E. A., Jackson, S., Lal, J. A., Dube, F., Kozaric, M., Tumiene, B., Salgado, R., et al. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-1051 · Cited by 10 · see citing works
Self-citationIalongo, C., & Bernardini, S. (2019) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2019-0226 · Cited by 9 · see citing works
Using Agile Methods to Implement a Laboratory for Software Product Quality Evaluation
Verdugo, J., Rodríguez, M., & Piattini, M. G. (2014) · Lecture notes in business information processing · doi:10.1007/978-3-319-06862-6_10 · Cited by 9 · see citing works
Gamification Use in Agile Project Management: An Experience Report
Pereira, I. M., Amorim, V. J. P., Cota, M. A., & Gonçalves, G. D. C. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-55907-0_3 · Cited by 9 · see citing works
Diagnosis: Fundamental Principles and Methods
Gale, M. (2022) · Cureus · doi:10.7759/cureus.28730 · Cited by 9 · see citing works
Gottlieb, S. L., & McClellan, M. (2022) · JAMA Health Forum · doi:10.1001/jamahealthforum.2022.4449 · Cited by 8 · see citing works
Marble, H. D., Bard, A. Z., Mizrachi, M., & Lennerz, J. K. (2021) · Journal of Molecular Diagnostics · doi:10.1016/j.jmoldx.2021.07.011 · Cited by 7 · see citing works
Self-citationFDA fosters innovative approaches in research, resources and collaboration
Gallas, B. D., Badano, A. G., Dudgeon, S. N., Elfer, K. N., Garcia, V. L., Lennerz, J. K., Myers, K., Petrick, N., & Margerrison, E. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00450-2 · Cited by 7 · see citing works
Self-citationMorbidity and Mortality Conferences: Change You Can Believe In?
Pierluissi, E. (2012) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-12-00252.1 · Cited by 6 · see citing works
Gao, J., Zhang, Q., Zhang, C., Chen, M., Li, D., Fu, Y. S., Lv, X., Zhang, B., & Guo, H. (2019) · Clinical Radiology · doi:10.1016/j.crad.2019.06.012 · Cited by 6 · see citing works
Routine blood tests as an active surveillance to monitor COVID-19 prevalence. A retrospective study.
Ferrari, D., Cabitza, F., Carobene, A., & Locatelli, M. (2020) · Acta bio-medica dell'Ateneo parmense/Acta biomedica · doi:10.23750/abm.v91i3.10218 · Cited by 6 · see citing works
Self-citationAll in for patient safety: a team approach to quality improvement in our laboratories
Weiss, V. L., Heher, Y., Seegmiller, A. C., VanderLaan, P. A., & Nishino, M. (2021) · Journal of the American Society of Cytopathology · doi:10.1016/j.jasc.2021.12.001 · Cited by 6 · see citing works
Self-citationHarris, C. K., Chen, Y., Jensen, K. C., Hornick, J. L., Kilfoyle, C., Lamps, L. W., & Heher, Y. (2022) · Academic Pathology · doi:10.1016/j.acpath.2022.100048 · Cited by 6 · see citing works
Self-citationPasotti, F., Pellegrinelli, L., Liga, G., Rizzetto, M., Azzarà, G., MOLIN, S. D., Lungu, O. L., Greco, S., Galli, C., Bubba, L., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12061483 · Cited by 5 · see citing works
Harris, C. K., Chen, Y., Yarsky, B., Haspel, R. L., & Heher, Y. (2022) · Academic Pathology · doi:10.1016/j.acpath.2022.100049 · Cited by 5 · see citing works
Self-citationArtificial intelligence in medicine creates real risk management and litigation issues
Keris, M. P. (2020) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21445 · Cited by 4 · see citing works
Do not sell regulatory science short
Lennerz, J. K., Marble, H. D., Lasiter, L., Poste, G. H., Sirintrapun, S. J., & Salgado, R. (2021) · Nature Medicine · doi:10.1038/s41591-021-01298-6 · Cited by 3 · see citing works
Self-citationShetty, O. A., Shet, T. M., Iyer, R., Gogte, P., Gurav, M., Joshi, P., Karnik, N., Pai, T., Epari, S., & Desai, S. B. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274089 · Cited by 2 · see citing works
Khan, Z. U. N., Jafri, L., Hall, P., Schultz, M. J., Ahmed, S., Khan, A. H., & Majid, H. (2022) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2022.104651 · Cited by 2 · see citing works
Everything is Varied: The Surprising Impact of Individual Variation on ML Robustness in MedicinePreprint
Campagner, A., Famiglini, L., Carobene, A., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.04555
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- Clinical Laboratory Improvement Amendments (CLIA). Code of Federal Regulations (CFR), Title 42, Chapter IV, Subchapter G Part CFR 493.1-493.2001 42 USC 263a. US 1988.
- McPherson RA, Pincus MR, Henry JB. Henry's clinical diagnosis and management by laboratory methods, 21st ed. Philadelphia: Saunders Elsevier; 2007.
Cited by
Sorted by citations
Cai, Y., Cai, Y.-Q., Tang, L.-Y., Wang, Y.-H., Gong, M., Jing, T.-C., Li, H.-J., Li-Ling, J., Hu, W., Yin, Z., et al. (2024) · BMC Medicine · doi:10.1186/s12916-024-03273-7 · Cited by 122 · see citing works
Jacob, C., Brasier, N., Laurenzi, E., Heuss, S., Mougiakakou, S. G., Çöltekin, A., & Peter, M. K. (2024) · Journal of Medical Internet Research · doi:10.2196/67485 · Cited by 113 · see citing works
Computational pathology in cancer diagnosis, prognosis, and prediction – present day and prospects
Verghese, G. E., Lennerz, J. K., Ruta, D. A., Ng, W., Thavaraj, S., Siziopikou, K. P., Naidoo, T., Rane, S. U., Salgado, R., Pinder, S. E., et al. (2023) · The Journal of Pathology · doi:10.1002/path.6163 · Cited by 90 · see citing works
Self-citationCai, Y., Gong, D.-X., Tang, L.-Y., Cai, Y. H., Li, H., Jing, T.-C., Gong, M., Hu, W., Zhang, Z.-W., Zhang, X., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/47645 · Cited by 48 · see citing works
Advances in computational nephropathology
Hölscher, D. L., Bülow, R. D., Strauch, M., & Boor, P. (2025) · Kidney International · doi:10.1016/j.kint.2025.06.029 · Cited by 18 · see citing works
Total laboratory automation: fit for its intended purposes?
Plebani, M. (2025) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2025-0855 · Cited by 17 · see citing works
Khokhar, M., Yadav, D., & Sharma, P. (2025) · Indian Journal of Clinical Biochemistry · doi:10.1007/s12291-025-01315-2 · Cited by 16 · see citing works
Kahles, A., Volckmar, A., Goldschmid, H., Salto‐Tellez, M., Vogeser, M., Stone, J., Brüggemann, M., Budczies, J., Kazdal, D. N., Salgado, R., et al. (2025) · ESMO Open · doi:10.1016/j.esmoop.2025.105909 · Cited by 14 · see citing works
Self-citationOwoyemi, A., Osuchukwu, J., Salwei, M. E., & Boyd, A. (2025) · JMIRx Med · doi:10.2196/65565 · Cited by 18 · see citing works
“The Diagnostic Quality Model by Lennerz et al [16] and the Clinical Explainable AI Guidelines by Jin et al [17] address diagnostic quality and explainability within medical imaging.”
Artificial intelligence and laboratory data in rheumatic diseases
Galozzi, P., Basso, D., Plebani, M., & Padoan, A. (2023) · Clinica Chimica Acta · doi:10.1016/j.cca.2023.117388 · Cited by 12 · see citing works
Deep learning applications in digital pathology
Boor, P. (2024) · Nature Reviews Nephrology · doi:10.1038/s41581-024-00870-w · Cited by 11 · see citing works
Özben, T. (2023) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-0091 · Cited by 12 · see citing works
Jafri, L., Farooqui, A. J., Grant, J., Omer, U., Gale, R., Ahmed, S., Khan, A. H., Siddiqui, I., Ghani, F., & Majid, H. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05078-x · Cited by 10 · see citing works
Bogdanoski, G., Lucas, F. M., Kern, W., & Czechowska, K. (2024) · Cytometry Part B Clinical Cytometry · doi:10.1002/cyto.b.22167 · Cited by 12 · see citing works
Bredella, M. A., Fintelmann, F. J., Iafrate, A. J., Dagogo‐Jack, I., Dreyer, K. J., Louis, D. N., Brink, J. A., & Lennerz, J. K. (2024) · Radiology · doi:10.1148/radiol.240335 · Cited by 8 · see citing works
Self-citationFan, L., Zhang, W., Li, H., Zeng, X., Teng, Y.-E., Gong, Y., Jin, X., Zhao, S., Sun, T., Chen, W., et al. (2025) · Cancer Cell · doi:10.1016/j.ccell.2025.11.003 · Cited by 6 · see citing works
Andersen, E. S., Röttger, R., Brasen, C. L., & Brandslund, I. (2024) · Clinical Chemistry · doi:10.1093/clinchem/hvae019 · Cited by 5 · see citing works
Checklist to Support the Development and Implementation of AI in Clinical SettingsPreprint
Owoyemi, A., Osuchukwu, J., Salwei, M. E., & Boyd, A. (2024) · medRxiv · doi:10.1101/2024.08.08.24311701 · Cited by 5 · see citing works
Hou, H. X., Li, A., Thierauf, J., & Lennerz, J. K. (2023) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfad079 · Cited by 3 · see citing works
Self-citationAmato, C. L., Yang, C., Badihi, N., Bernhard, L., Ratib, O. M., Seeliger, B., & Wilhelm, D. (2024) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-024-03278-w · Cited by 3 · see citing works
Damaiyanti, D. W., Rizal, M. B., & Liu, P. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-33030-8.00001-9 · Cited by 2 · see citing works
Penteado, B. E., Fornazin, M., Castro, L., Rachid, R., Gonçalves, L. H. D. N., & Falcão, M. Z. (2025) · Health and Technology · doi:10.1007/s12553-025-01015-0 · Cited by 2 · see citing works
Jacob, C., Brasier, N., Laurenzi, E., Heuss, S., Mougiakakou, S. G., Çöltekin, A., & Peter, M. K. (2024) · doi:10.2196/preprints.67485 · Cited by 1 · see citing works
Artificial Intelligence in Pulmonary Infectious Diseases
Chunkhare, M., Madire, B., & Deshmukh, V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7759-9.ch003
Unified definition of integrated diagnostics: a position statement of the EFLM
Lennerz, J. K., Ben, F. D., Stenzinger, A., Frauenknecht, K. B. M., Carobene, A., Berishvili, N., Guimarães, J. T., Martí‐Bonmatí, L., Buettner, R., Haselmann, V., et al. (2026) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2026-0949
Self-citationRegulations and ethical considerations in digital pathology and artificial intelligence
Gullapalli, R. R., & Parwani, A. V. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40383-5.00018-2
Alshreef, B. S. (2026) · Journal of Medical Biochemistry · doi:10.5937/jomb0-65319
AI Model Reporting Standards in Health Care: Systematic Review and Consolidation
Akhmad, E. S., Slob, D., Gomes, A. L., Mingels, S., Dekker, A. L. A. J., & van Soest, J. (2026) · JMIR AI · doi:10.2196/92329
Yuji, K., & Yuji, W. (2026) · Expert Review of Medical Devices · doi:10.1080/17434440.2026.2753843
- Authors
- Lennerz, J. K., Salgado, R., Kim, G. E., Sirintrapun, S. J., Thierauf, J. C., Singh, A., Indave, I., Bard, A., Weissinger, S. E., Heher, Y. K., de Baca, M. E., Cree, I. A., Bennett, S., Carobene, A., Ozben, T., & Ritterhouse, L. L.
- Affiliations
- Harvard University (US)
- Peter MacCallum Cancer Centre (AU)
- University of California, San Francisco (US)
- Memorial Sloan Kettering Cancer Center (US)
- European Monitoring Centre for Drugs and Drug Addiction (PT)
- Alb Fils Kliniken (DE)
- Pacific Pathology Partners , Seattle , WA , USA
- Centre international de recherche sur le cancer (FR)
- Mayo Clinic (US)
- IRCCS San Raffaele Scientific Institute , Milan , Italy
- Medical Faculty, Dept. of Clinical Biochemistry , Akdeniz University , Antalya , Türkiye
- Corresponding author
- Lennerz, Jochen K.
- ORCID
- Venue
- Clinical Chemistry and Laboratory Medicine (CCLM)
- Publisher
- Walter de Gruyter GmbH
- Published
- 2023-01-25
- Volume
- 61
- Issue
- 4
- Pages
- 544-557
- ISSN
- 1434-6621 (ISSN-L)
- 1437-4331
- OpenAlex
- W4318021306
- Cited by
- 29
- References
- 114
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 11 · 2023: 4
- FWCI
- 1.16
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Clinical Chemistry and Laboratory Medicine (CCLM) (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Amgen
- Bristol-Myers Squibb
- Eli Lilly and Company
- AstraZeneca
- Sanofi
- World Health Organization
- Centre International de Recherche sur le Cancer
- National Institutes of Health (CA225655)
- EMD Serono
- Loxo Oncology
- National Cancer Institute (R37CA225655)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- machine learning, laboratory diagnosis, risk-benefit assessment, quality management systems, regulatory standards, expert roundtable
- MeSH terms
- Machine Learning, Artificial Intelligence, Delivery of Health Care, Humans, Ecosystem
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review