Nuclear Medicine and Artificial Intelligence: Best Practices for Algorithm Development
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
The nuclear medicine field has seen a rapid expansion of academic and commercial interest in developing artificial intelligence (AI) algorithms. Users and developers can avoid some of the pitfalls of AI by recognizing and following best practices in AI algorithm development. In this article, recommendations on technical best practices for developing AI algorithms in nuclear medicine are provided, beginning with general recommendations and then continuing with descriptions of how one might practice these principles for specific topics within nuclear medicine. This report was produced by the AI Task Force of the Society of Nuclear Medicine and Molecular Imaging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · cited 1×
“For some tasks, having more labelers per sample can produce greater performance gains than using a larger dataset but with fewer labelers (15,16).”
BackgroundFederated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · cited 1×
“In federated learning, data cohorts reside within their respective institutional boundaries but models and weights are shared across institutions (24).”
BackgroundA survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522 · cited 1×
“However, for limited datasets, cross validation produces a less biased estimate of a method’s generalization performance than using one-time partitions (i.e., holdout testing) ( 23 ).”
MethodCalculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917 · cited 1×
“No formal guidelines exist for estimating the size of the training set, although some practical approaches have been described (10), and trial and error are therefore often necessary (11).”
BackgroundWarfield, S. K., Zou, K. H., & Wells, W. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.828354 · Cited by 2,335
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · cited 1×
“To avoid unnecessary complexity, investigators using large models are encouraged to also evaluate simpler models as a baseline comparison (e.g., logistic regression ( 20 )).”
MethodThe preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · cited 1×
“Prospective studies should use preregistered statistical analysis plans (9).”
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · cited 2×
“There are also valid concerns about the reproducibility of claims made in many published AI studies (2) and the generalizability of trained algorithms (3).”
BackgroundLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · cited 1×
“The hype that propels the development of AI algorithms in nuclear medicine is counterbalanced by concerns about certain pitfalls of AI (1).”
BackgroundAI in Medical Imaging Informatics: Current Challenges and Future Directions
Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2991043 · Cited by 713 · cited 1×
“AI can be used to prepopulate radiology reports, assist in real-time report generation, help standardize reporting, and perform structured synoptic reporting (58).”
BackgroundReporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701
Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist.
Norgeot, B., Quer, G., Beaulieu-Jones, B. K., Torkamani, A., Dias, R., Gianfrancesco, M., Arnaout, R., Kohane, I. S., Saria, S., Topol, E., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1041-y · Cited by 665 · cited 2×
“This can be accomplished by using reporting checklists such as MI-CLAIM ( 29 ).”
MethodTransparency and reproducibility in artificial intelligence
Haibe-Kains, B., Adam, G. A., Hosny, A., Khodakarami, F., Massive Analysis Quality Control (MAQC) Society Board of Directors, Shraddha, T., Kusko, R., Sansone, S.-A., Tong, W., Wolfinger, R. D., et al. (2020) · Nature · doi:10.1038/s41586-020-2766-y · Cited by 540 · cited 1×
“There are also valid concerns about the reproducibility of claims made in many published AI studies (2) and the generalizability of trained algorithms (3).”
BackgroundCruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., Ashrafian, H., Beam, A. L., Collins, G. S., Darzi, A., Deeks, J. J., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30219-3 · Cited by 486
Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J. P. A., & Shah, N. H. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa088 · Cited by 410
Bluemke, D. A., Moy, L., Bredella, M. A., Ertl-Wagner, B. B., Fowler, K. J., Goh, V. J., Halpern, E. F., Hess, C. P., Schiebler, M. L., & Weiss, C. R. (2020) · Radiology · doi:10.1148/radiol.2019192515 · Cited by 343
An algorithmic approach to reducing unexplained pain disparities in underserved populations
Pierson, E., Cutler, D. M., Leskovec, J., Mullainathan, S., & Obermeyer, Z. (2021) · Nature Medicine · doi:10.1038/s41591-020-01192-7 · Cited by 268
Pineau, J., Vincent-Lamarre, P., Sinha, K., Larivière, V., Beygelzimer, A., d'Alché-Buc, F., Fox, E., & Larochelle, H. (2020) · arXiv · doi:10.48550/arxiv.2003.12206 · Cited by 232 · cited 1×
“This could consist of reproducibility checklists (35) and/or dedicated “data expert” reviewers, similar to statistics reviewers that are solicited for articles involving sophisticated statistical analyses.”
BackgroundInterpretation and visualization techniques for deep learning models in medical imaging
Huff, D. T., Weisman, A. J., & Jeraj, R. (2021) · Physics in Medicine & Biology · doi:10.1088/1361-6560/abcd17 · Cited by 172
Cheplygina, V., de Bruijne, M., & Pluim, J. P. W. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.03.009 · Cited by 93
Cited by
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
“Initial efforts have focused on providing recommendations for the reporting of AI studies for different medical domains or clinical tasks (eg, TRIPOD+AI, 11 CLAIM, 12 CONSORT-AI, 13 DECIDE-AI, 14 PROBAST-AI, 15 CLEAR 16 ).”
Self-citationEthical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance
Herington, J., McCradden, M. D., Creel, K., Boellaard, R., Jones, E. C., Jha, A. K., Rahmim, A., Scott, P. J. H., Sunderland, J. J., Wahl, R. L., et al. (2023) · Journal of Nuclear Medicine · doi:10.2967/jnumed.123.266110 · Cited by 61
Self-citationCurrent State and Prospects of Artificial Intelligence in Allergy
van Breugel, M., Fehrmann, R. S. N., Bügel, M., Rezwan, F. I., Holloway, J. W., Nawijn, M. C., Fontanella, S., Custovic, A., & Koppelman, G. H. (2023) · Allergy · doi:10.1111/all.15849 · Cited by 43
Metadata
- Authors
- Bradshaw, T. J., Boellaard, R., Dutta, J., Jha, A. K., Jacobs, P., Li, Q., Liu, C., Sitek, A., Saboury, B., Scott, P. J. H., Slomka, P. J., Sunderland, J. J., Wahl, R. L., Yousefirizi, F., Zuehlsdorff, S., Rahmim, A., & Buvat, I.
- Venue
- Journal of Nuclear Medicine
- Publisher
- Society of Nuclear Medicine
- Published
- 2021-11-05
- Volume
- 63
- Issue
- 4
- Pages
- 500-510
- ISSN
- 0161-5505 (ISSN-L)
- 2159-662X
- OpenAlex
- W3209261377
- Cited by
- 94
- References
- 21
- FWCI
- 3.63
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.