Cited by 67Artificial Intelligence in Healthcare and EducationAnatomy and Medical TechnologyArtificial Intelligence in Education

Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases

Open access
Temsah, Mohamad-Hani, Alhuzaimi, Abdullah N., Almansour, Mohammed, Aljamaan, Fadi, Alhasan, Khalid, Batarfi, Munirah A., Altamimi, Ibraheem, Alharbi, Amani +11 more·2024·Journal of Medical Systems

Abstract

Artificial Intelligence (AI), particularly AI-Generated Imagery, has the potential to impact medical and patient education. This research explores the use of AI-generated imagery, from text-to-images, in medical education, focusing on congenital heart diseases (CHD). Utilizing ChatGPT's DALL·E 3, the research aims to assess the accuracy and educational value of AI-created images for 20 common CHDs. In this study, we utilized DALL·E 3 to generate a comprehensive set of 110 images, comprising ten images depicting the normal human heart and five images for each of the 20 common CHDs. The generated images were evaluated by a diverse group of 33 healthcare professionals. This cohort included cardiology experts, pediatricians, non-pediatric faculty members, trainees (medical students, interns, pediatric residents), and pediatric nurses. Utilizing a structured framework, these professionals assessed each image for anatomical accuracy, the usefulness of in-picture text, its appeal to medical professionals, and the image's potential applicability in medical presentations. Each item was assessed on a Likert scale of three. The assessments produced a total of 3630 images' assessments. Most AI-generated cardiac images were rated poorly as follows: 80.8% of images were rated as anatomically incorrect or fabricated, 85.2% rated to have incorrect text labels, 78.1% rated as not usable for medical education. The nurses and medical interns were found to have a more positive perception about the AI-generated cardiac images compared to the faculty members, pediatricians, and cardiology experts. Complex congenital anomalies were found to be significantly more predicted to anatomical fabrication compared to simple cardiac anomalies. There were significant challenges identified in image generation. Based on our findings, we recommend a vigilant approach towards the use of AI-generated imagery in medical education at present, underscoring the imperative for thorough validation and the importance of collaboration across disciplines. While we advise against its immediate integration until further validations are conducted, the study advocates for future AI-models to be fine-tuned with accurate medical data, enhancing their reliability and educational utility.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10916-024-02072-0

Cites

51 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs may deal with citations and their bibliographic identifiers as text, making them vulnerable to paraphrasing and other linguistic manipulations and perhaps as a watermark so that AI-generated output could be identified versus human-produced output to reduce AI misuse.

    “LLMs may deal with citations and their bibliographic identifiers as text, making them vulnerable to paraphrasing and other linguistic manipulations and perhaps as a watermark so that the output that is generated by AI could be identified versus that produced by humans to reduce AI misuse [29,31,32].”

    From Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study (2024) · cites this paper as [32]

    Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Johnson-Laird, P.N. Mental models: Towards a cognitive science of language, inference, and consciousness; Harvard University Press: 1983. .
  2. Deci, E.L.; Ryan, R.M. Intrinsic motivation and self-determination in human behavior; Springer Science & Business Media: 2013.
  3. Ramesh, A.; Pavlov, M.; Goh, G.; Gray, S.; Voss, C.; Radford, A.; Chen, M.; Sutskever, I. Zero-shot text-to- image generation. 2021; pp. 8821-8831.

Cited by

Sorted by citations

Authors
Temsah, M.-H., Alhuzaimi, A. N., Almansour, M., Aljamaan, F., Alhasan, K., Batarfi, M. A., Altamimi, I., Alharbi, A., Alsuhaibani, A. A., Alwakeel, L., Alzahrani, A. A., Alsulaim, K. B., Jamal, A., Khayat, A., Alghamdi, M. H., Halwani, R., Khan, M. K., Al-Eyadhy, A., & Nazer, R.
Affiliations
  • King Saud Medical City (SA)
  • King Saud University (SA)
  • King Faisal Specialist Hospital & Research Centre (SA)
  • King Saud bin Abdulaziz University for Health Sciences (SA)
  • Prince Sultan Military College of Health Sciences (SA)
  • University of Sharjah (AE)
Corresponding author
  • Temsah, Mohamad-Hani
  • Alhuzaimi, Abdullah N.
  • Almansour, Mohammed
  • Aljamaan, Fadi
  • Alhasan, Khalid
  • Batarfi, Munirah A.
  • Altamimi, Ibraheem
  • Alharbi, Amani
  • Alsuhaibani, Adel Abdulaziz
  • Alwakeel, Leena
  • Alzahrani, Abdulrahman Abdulkhaliq
  • Alsulaim, Khaled B.
  • Jamal, Amr
  • Khayat, Afnan
  • Alghamdi, Mohammed Hussien
  • Halwani, Rabih
  • Khan, Muhammad Khurram
  • Al-Eyadhy, Ayman
  • Nazer, Rakan
Venue
Journal of Medical Systems
Published
2024-05-23
Volume
48
Issue
1
Pages
54-54
ISSN
  • 0148-5598 (ISSN-L)
  • 1573-689X
OpenAlex
W4398277600
Cited by
66
References
52
Citations by year
2026: 21 · 2025: 32 · 2024: 13
FWCI
6.18
Type
journal article
Language
en
Open access
Locations
  • Journal of Medical Systems (publishedVersion)
  • Research Square (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Engineering › Biomedical Engineering › Anatomy and Medical Technology
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
AI-generated images, congenital heart disease, text-to-image generation, medical education, anatomical accuracy, image evaluation, healthcare professionals' perceptions, pediatric cardiology, image validation
MeSH terms
Artificial Intelligence, Heart Defects, Congenital, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review