CoChat
Sign Up
Cited by 640Artificial Intelligence in Healthcare and EducationExplainable Artificial Intelligence (XAI)Radiology practices and education

ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports

Open access
Jeblick, Katharina, Schachtner, Balthasar, Dexl, Jakob, Mittermeier, Andreas, Stüber, Anna Theresa, Topalis, Johanna, Weber, Tobias, Wesp, Philipp +3 more·2023·European Radiology
DOI 10.1007/s00330-023-10213-1PMC PMC11126432
Open this paper in CoChat — free accountOpen PDF

Abstract

Objectives: To assess the quality of simplified radiology reports generated with the large language model (LLM) ChatGPT and to discuss challenges and chances of ChatGPT-like LLMs for medical text simplification. Methods: In this exploratory case study, a radiologist created three fictitious radiology reports which we simplified by prompting ChatGPT with “Explain this medical report to a child using simple language.” In a questionnaire, we tasked 15 radiologists to rate the quality of the simplified radiology reports with respect to their factual correctness, completeness, and potential harm for patients. We used Likert scale analysis and inductive free-text categorization to assess the quality of the simplified reports. Results: Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient. Nevertheless, instances of incorrect statements, missed relevant medical information, and potentially harmful passages were reported. Conclusion: While we see a need for further adaption to the medical field, the initial insights of this study indicate a tremendous potential in using LLMs like ChatGPT to improve patient-centered care in radiology and other medical domains. Clinical relevance statement: Patients have started to use ChatGPT to simplify and explain their medical reports, which is expected to affect patient-doctor interaction. This phenomenon raises several opportunities and challenges for clinical routine. Key Points: • Patients have started to use ChatGPT to simplify their medical reports, but their quality was unknown. • In a questionnaire, most participating radiologists overall asserted good quality to radiology reports simplified with ChatGPT. However, they also highlighted a notable presence of errors, potentially leading patients to draw harmful conclusions. • Large language models such as ChatGPT have vast potential to enhance patient-centered care in radiology and other medical domains. To realize this potential while minimizing harm, they need supervision by medical experts and adaption to the medical field. Graphical Abstract:

AI snapshot

Written from the paper’s full text.

TL;DR

In this exploratory case study, ChatGPT was prompted to simplify three fictitious radiology reports, and 15 radiologists generally rated the outputs as factually correct, complete, and not potentially harmful. However, they also identified incorrect statements, missing relevant information, and potentially harmful passages in a considerable number of simplified reports.

Key findings

  • Across all 45 simplified reports, median ratings were 2 (“Agree”) for factual correctness and completeness, and 4 (“Disagree”) for potential harm; no radiologist selected “Strongly disagree” for correctness/completeness, and none selected “Strongly agree” for potential harm.
  • Free-text review found incorrect text passages in 23/45 simplified reports (51%), missing relevant medical information in 10/45 (22%), and potentially harmful conclusions in 16/45 (36%).
  • Error categories included misinterpretation of medical terms (e.g., “DD”/differential diagnosis mistaken as final diagnosis; “thyroid struma” described as “infection in their thyroid gland” or “extra thyroid gland”), hallucinations (e.g., “no signs of cancer in the thyroid gland”; “brain does not seem to be damaged”), imprecise language, odd language, unmentioned findings, and unspecific location information.
  • The Oncol. CT report had the highest mean factual-correctness rating (2.5, SD 1.1) and potential-harm mean 3.3 (SD 1.1); Head MRI had a slightly lower potential-harm median (3) than Knee MRI and Oncol. CT (both median 4).

Methods

  • Prospective exploratory case study using three fictitious radiology reports (Knee MRI, Head MRI, Oncol. CT) written by a radiologist with 10 years of experience and intended to be of intermediate complexity.
  • The ChatGPT online interface (version December 15th, 2022) was prompted with “Explain this medical report to a child using simple language:” followed by the original report; the model was restarted and prompted 15 times per report, generating 45 simplified reports.
  • Fifteen radiologists with median experience of 5 years (IQR [1–10]) rated one randomly selected simplified version per original report on 5-point Likert scales for factual correctness, completeness, and potential harm, with free-text follow-up; analysis used descriptive statistics and inductive categorization.

Limitations

  • Stated: small number of original reports (n = 3) and experts (n = 15); fictitious reports were written to represent intermediate medical complexity and translated to English by a non-native-speaking radiologist; only one prompt phrasing was used after heuristic selection; the study did not measure the quality of simplification.
  • Evident: the radiologist sample came from a single clinic (“from our clinic”), and ChatGPT outputs were non-deterministic, while each questionnaire included only one randomly selected simplified version per original report.
  • Evident: the study used fictitious reports and no patient information, so it does not validate performance on real clinical reports or patient outcomes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00330-023-10213-1

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The abbreviation "DD" (differential diagnosis) was often mistaken as a final diagnosis in the simplified reports.

    “The abbreviation “DD” (differential diagnosis) was often mistaken as a final diagnosis (B.2.6, B.2.8)”
    Quote in the paperNot a substantial claim
  • Findingown data

    Most participating radiologists agreed that the simplified radiology reports were factually correct (median rating 2 on a 5-point scale where 1=strongly agree).

    “Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.”
    • Also stated in: § Likert scale analysis · plain
      “The participants generally agreed (median = 2) with the statements that the simplified reports are factually correct and complete, respectively.”
    Quote in the paperSubstantial claim
  • Findingown data

    Misinterpretation of differential diagnosis ("DD") as final diagnosis was considered potentially harmful for the patient.

    “For instance, the misinterpretation of differential diagnosis (“DD”) as final diagnosis was considered potentially harmful for the patient, e.g., one radiologist commented: “GBM is one (likely) DD which implies that other DDs exist (e.g. radionecrosis).” (B.2.7).”
    Quote in the paperSubstantial claim
  • Findingown data

    For potential harm, the Head MRI report had a slightly lower median rating (median = 3) than the Knee MRI and Oncol. CT reports (median = 4).

    “For potential harm, the Head MRI report had a slightly lower median (median = 3) compared to the other two reports (median = 4).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Most participating radiologists agreed that the simplified radiology reports were complete (median rating 2).

    “Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.”
    • Also stated in: § Likert scale analysis · plain
      “The participants generally agreed (median = 2) with the statements that the simplified reports are factually correct and complete, respectively.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 17
  • May combine two claims. 15

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The incorrect text passages were categorized into misinterpretation of medical terms, hallucinations, imprecise language, and odd language.

    “In the case of incorrect text passages, we identified the inductive categories misinterpretation of medical terms, hallucinations, imprecise language, and odd language.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Potentially harmful unmentioned findings were found, e.g., the sentence "the parts that are staying the same size are changing" was found to be an unclear statement missing interpretation.

    “We also found potentially harmful unmentioned findings in the simplified report. The sentence “the parts that are staying the same size are changing” (B.3.5) was found to be an unclear statement missing interpretation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In all cases, ChatGPT generated plausible-sounding outputs and did not indicate that the response could be incorrect.

    “Even though the model is able to state to not know the answer [1], in all cases ChatGPT generated plausible-sounding outputs, while it did not indicate that the response could be incorrect.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    ChatGPT's output is non-deterministic, i.e., applying the same prompt multiple times results in different responses, hindering reproducibility.

    “Furthermore, ChatGPT’s output is non-deterministic, i.e., applying the same prompt multiple times results in different responses, hindering reproducibility.”
    • Also stated in: § Simplification of radiology reports using ChatGPT · plain
      “The version of the interface of ChatGPT used in this study does not allow changing any model settings, resulting in non-deterministic outputs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants rated the hallucination "brain does not seem to be damaged" as potentially harmful to the patient, as the original report describes a growing mass.

    “For the category of hallucinations, participants rated the passage “brain does not seem to be damaged” (B.2.9) as potentially harmful to the patient, as the original report describes a growing mass.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    The wording "small growth" was deemed a harmful conclusion because the original report states progression in size, which "almost doubled".

    “Furthermore, the wording “small growth” (B.2.3) was deemed a harmful conclusion, as the original report (B.2.0) states a progression in size, which “almost doubled.””
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    ChatGPT described "thyroid struma" as "infection in their thyroid gland" in one instance and as "extra thyroid gland" in another.

    “Furthermore, “thyroid struma” was described by ChatGPT in one instance as “infection in their thyroid gland” (B.3.13) and in another as “extra thyroid gland” (B.3.7).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Missing relevant medical information was categorized into unmentioned findings and unspecific location.

    “Missing relevant medical information was categorized into unmentioned findings and unspecific location.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Imprecise language highlighted as incorrect was sometimes also graded as potentially harmful to the patient, e.g., "not clear that spots are pulmonal metastases" and "some extra fluid" is not a significant increasing edema.

    “Additionally, we observed that imprecise language highlighted as incorrect was sometimes also graded as potentially harmful to the patient. For example, participants listed that it is “not clear that spots are pulmonal metastases” (B.3.5) and that “some extra fluid” is not a significant increasing edema (B.2.13).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Lymph nodes were simplified as "they might have cancer" although the original report stated "no evidence of recurrence or new lymph node metastases".

    “Lymph nodes were simplified as “they might have cancer” (B.3.9), but the original report stated that there was “no evidence of recurrence or new lymph node metastases.””
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Grammatical errors such as "CT scan" instead of "CT scanner" were the exception.

    “Grammatical errors such as “CT scan” instead of “CT scanner” (B.3.7) were the exception.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Written informed consent was obtained from participating radiologists.

    “Written informed consent was obtained from participating radiologists.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The fictitious original reports were written to represent an intermediate level of medical complexity and translated to English by a non-native-speaking radiologist.

    “The fictitious original reports were written to represent an intermediate level of medical complexity and translated to English by a non-native-speaking radiologist.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    There is a huge need for text simplification in the medical domain.

    “In the medical domain, there is a huge need for text simplification [8, 9].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [8] Barrett SK, Patrie J, Kitts AB, Hanley M, Swanson CM (2021) Vitzthum von Eckstaedt H, et al Patient-centered reporting in radiology: a single-site survey study of lung cancer screening results. J Thorac Imaging 36(6):367 source

      Not checked in the free run.

    • cites [9] Keselman A, Slaughter L, Arnott-Smith C et al (2007) Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, p 399

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    According to the rules of the Ethics Committee of the Faculty of Medicine, LMU Munich, no ethical consultation was necessary for the approval of this study.

    “According to the rules of the Ethics Committee of the Faculty of Medicine, LMU Munich, no ethical consultation was therefore necessary for the approval of this study.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    LLMs like ChatGPT have intrinsic biases, including stereotypical associations, negative sentiment towards specific groups, and biases due to imbalanced training data.

    “Also, it is well established that LLMs like ChatGPT have intrinsic biases, including stereotypical associations, or negative sentiment towards specific groups, as well as biases due to imbalanced training data [6, 34, 35].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [6] Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901

      Not checked in the free run.

    • cites [34] Bender EM, Gebru T, McMillan-Major A, Shmitchell S (2021) On the dangers of stochastic parrots: can language models be too big?. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, p 610–623 source

      Not checked in the free run.

    • cites [35] Bommasani R, Hudson DA, Adeli E et al (2021) On the opportunities and risks of foundation models. ArXiv Prepr ArXiv210807258.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The radiologists' answers were collected in anonymized questionnaires.

    “Their answers were collected in anonymized questionnaires.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Simplified and thus more comprehensible reports would enable patients to better understand and oversee their own health situation and empower them to make informed and active decisions throughout the medical treatment process.

    “Simplified and thus more comprehensible reports would enable patients to better understand and oversee their own health situation and empower them to make informed and active decisions throughout the medical treatment process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We used only one phrasing of the prompt for report simplification after a heuristic selection process and did not measure the quality of simplification.

    “We used only one phrasing of the prompt for report simplification after a heuristic selection process and did not measure the quality of simplification.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Radiological findings might have immediate consequences for patients, but they are typically only communicated in a free-text report in specialized medical jargon, targeting a clinician or doctor as the recipient.

    “Radiological findings might have immediate consequences for patients. However, they are typically only communicated in a free-text report in specialized medical jargon, targeting a clinician or doctor as the recipient.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [9] Keselman A, Slaughter L, Arnott-Smith C et al (2007) Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, p 399

      Not checked in the free run.

    • cites [10] Yi PH, Golden SK, Harringa JB, Kliewer MA (2019) Readability of lumbar spine MRI reports: will patients understand? AJR Am J Roentgenol 212(3):602–606 source

      Not checked in the free run.

    • cites [11] Kindig DA, Panzer AM, Nielsen-Bohlman L, editors (2004) Health literacy: a prescription to end confusion. National Academies Press, Washington, DC source

      Not checked in the free run.

    • cites [12] Martin-Carreras T, Cook TS, Kahn CE (2019) Readability of radiology reports: implications for patient-centered care. Clin Imaging 54:116–120 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    These errors can be grouped into the same error categories that were found for factual correctness and completeness.

    “These errors can be grouped into the same error categories that were found for factual correctness and completeness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Misinterpretation of medical terms hinting at recurrence of cancer or understating progression of a disease is likely to cause harm for patients.

    “For example, misinterpretation of medical terms hinting at recurrence of cancer or understating progression of a disease is likely to cause harm for patients.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    ChatGPT can generate plausible-sounding text, but the content does not need to be true.

    “As known for other LLMs [13, 14], ChatGPT can generate plausible-sounding text, but the content does not need to be true (Fig. 1).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Lin S, Hilton J, Evans O (2022) TruthfulQA: measuring how models mimic human falsehoods. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL. Association for Computational Linguistics, p 3214–52 source

      Not checked in the free run.

    • cites [14] Taylor R, Kardas M, Cucurull G et al (2022) Galactica: a large language model for science. ArXiv Prepr ArXiv221109085.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Efforts to adapt LLMs to the medical field have been made, e.g., by Lee et al, Gu et al, and Singhal et al.

    “Efforts in this direction have been made, e.g., by Lee et al, Gu et al, and Singhal et al [20–22].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [20] Lee J, Yoon W, Kim S et al (2020) BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4):1234–1240 source

      Not checked in the free run.

    • cites [21] Gu Y, Tinn R, Cheng H et al (2021) Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthc. 3(1):2:1–2:23 source

      Not checked in the free run.

    • cites [22] Singhal K, Azizi S, Tu T et al (2022) Large language models encode clinical knowledge [Internet]. arXiv; [cited 2023 Jan 13].

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Previous approaches for the simplification of radiology reports mostly replaced recurrent phrases or augmented radiology reports with lay-language definitions.

    “Previous approaches for the simplification of radiology reports mostly replaced recurrent phrases [29–32] or augmented radiology reports with lay-language definitions [33].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [29] Kvist M, Velupillai S (2013) Professional language in Swedish radiology reports–characterization for patient-adapted text simplification. In: Scandinavian Conference on Health Informatics. Linköping University Electronic Press, p 55–59

      Not checked in the free run.

    • cites [30] Abrahamsson E, Forni T, Skeppstedt M, Kvist M (2014) Medical text simplification using synonym replacement: adapting assessment of word difficulty to a compounding language. In: Proceedings of the 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR). p 57–65 source

      Not checked in the free run.

    • cites [31] Ramadier L, Lafourcade M (2017) Radiological text simplification using a general knowledge base. In: International Conference on Computational Linguistics and Intelligent Text Processing. Springer, p 617–627 source

      Not checked in the free run.

    • cites [32] Qenam B, Kim TY, Carroll MJ, Hogarth M (2017) Text simplification using consumer health vocabulary to generate patient-centered radiology reporting: translation and evaluation. J Med Internet Res 19(12):e417 source

      Not checked in the free run.

    • cites [33] Oh SC, Cook TS, Kahn CE (2016) PORTER: a prototype system for patient-oriented radiology reporting. J Digit Imaging 29(4):450–454 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The remaining inductive categories might be attributed to the fact that ChatGPT was trained on general data and not tuned to the task of radiology report simplification.

    “The remaining inductive categories might be attributed to the fact that ChatGPT was trained on general data and not tuned to the task of radiology report simplification.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Hallucinations are difficult to remedy.

    “Hallucinations are an intrinsic problem of generative models like LLMs [15] and difficult to remedy [16, 17].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [16] Wang C, Sennrich R (2020) On exposure bias, hallucination and domain shift in neural machine translation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics [Internet]. Online: Association for Computational Linguistics; [cited 2023 Jan 24]. p 3544–52 source

      Not checked in the free run.

    • cites [17] Guerreiro NM, Voita E, Martins AFT (2022) Looking for a needle in a haystack: a comprehensive study of hallucinations in neural machine translation [Internet]. arXiv; [cited 2023 Jan 24]. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Hallucinations are an intrinsic problem of generative models like LLMs.

    “Hallucinations are an intrinsic problem of generative models like LLMs [15] and difficult to remedy [16, 17].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Zhou C, Neubig G, Gu J et al (2021) Detecting hallucinated content in conditional neural sequence generation. In: Findings of the association for computational linguistics: ACL/IJCNLP. Association for Computational Linguistics, p 1393–404 source

      Not checked in the free run.

28 of 28

  • The paper relies on this without support.

    Patients have started to use ChatGPT to simplify and explain their medical reports.

    “Patients have started to use ChatGPT to simplify and explain their medical reports, which is expected to affect patient-doctor interaction.”

  • The paper relies on this without support.

    ChatGPT was not explicitly trained for medical text simplification and is not intended to be used for this critical task.

    “At the same time, ChatGPT was not explicitly trained for medical text simplification and is not intended to be used for this critical task.”

  • The paper relies on this without support.

    The training data of ChatGPT is static, i.e., new research findings are not considered in ChatGPT’s output.

    “The training data is static, i.e., new research findings are not considered in ChatGPT’s output.”

  • The paper relies on this without support.

    The resulting models lack the accessibility and quality of ChatGPT.

    “However, the resulting models lack the accessibility and quality of ChatGPT.”

  • The paper relies on this without support.

    Uploading protected health information to a proprietary service such as ChatGPT might compromise patients’ privacy.

    “Finally, uploading protected health information to a proprietary service, such as ChatGPT, might compromise patients’ privacy.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ChatGPT has great potential to become a daily assistant for purposes of general or professional consulting.

    “Both versions are popular and remarkably successful so that ChatGPT has great potential to become a daily assistant for purposes of general or professional consulting (Jeblick et al., 2023).”

    From Teachers vs. Chat GPT: identifying and naming acts of online sexual abuse (2025) · cites this paper as Jeblick et al., 2023

    The Introduction describes ChatGPT’s growing public popularity, and the Discussion sees great potential for it to help patients understand radiology reports. But the paper does not establish that both versions are successful or that ChatGPT is suited to be a daily assistant for general or professional consulting. Its proposed clinical use requires a medically adapted model and radiologist review. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The ability of ChatGPT to help streamline the clinical workflow appears promising, with possible cost savings and increased efficiency in health care delivery.

    “The ability of ChatGPT to help in streamlining the clinical workflow appears promising, with possible cost savings and increased efficiency in health care delivery [31,37,39,77].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]

    The Discussion envisages integrating ChatGPT-like tools into radiology centers to generate simplified reports automatically, subject to radiologist review. This supports a possible role in clinical workflow, but the paper does not establish or predict cost savings or increased efficiency in health care delivery. The indexed passages do not provide page numbers. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Ethical issues including risk of bias and transparency issues appeared as recurring major concerns in health care settings.

    “Ethical issues including the risk of bias and transparency issues appeared as recurring major concerns [51,68,69,77].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]

    The Discussion (page not available in our copy) identifies intrinsic bias in ChatGPT and describes potentially harmful errors as a medical-ethics concern. It does not identify transparency as a concern or establish that these issues recur across healthcare settings. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • ChatGPT hallucination could be risky if the generated content is not thoroughly evaluated by researchers and health providers with proper expertise.

    “Importantly, the concept of ChatGPT hallucination could be risky if the generated content is not thoroughly evaluated by researchers and health providers with proper expertise [37,56,73,77,79].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]

  • ChatGPT plays a crucial role in pandemic management by aiding in the swift and accurate dissemination of critical information.

    “ChatGPT plays a crucial role in pandemic management by aiding in the swift and accurate dissemination of critical information.”

    From A Systematic Review and Meta-Analysis of Artificial Intelligence Tools in Medicine and Healthcare: Applications, Considerations, Limitations, Motivation and Challenges (2024) · cites this paper as [7]

Cites

Flagged first, then by citations

Research works printed but not linked: 8

  1. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
  2. Agarwal A (2022) Explain to me like I am five–sentence simplification using transformers. ArXiv Prepr ArXiv221204595
  3. Taylor R, Kardas M, Cucurull G et al (2022) Galactica: a large language model for science. ArXiv Prepr ArXiv221109085.
  4. Singhal K, Azizi S, Tu T et al (2022) Large language models encode clinical knowledge [Internet]. arXiv; [cited 2023 Jan 13].
  5. Shardlow M (2014) A survey of automated text simplification. Int J Adv Comput Sci Appl 4(1):58–70
  6. Al-Thanyyan SS, Azmi AM (2021) Automated text simplification: a survey. ACM Comput Surv CSUR 54(2):1–36
  7. Kvist M, Velupillai S (2013) Professional language in Swedish radiology reports–characterization for patient-adapted text simplification. In: Scandinavian Conference on Health Informatics. Linköping University Electronic Press, p 55–59
  8. Bommasani R, Hudson DA, Adeli E et al (2021) On the opportunities and risks of foundation models. ArXiv Prepr ArXiv210807258.

Cited by

Sorted by citations

Related

Authors
Jeblick, K., Schachtner, B., Dexl, J., Mittermeier, A., Stüber, A. T., Topalis, J., Weber, T., Wesp, P., Sabel, B. O., Ricke, J., & Ingrisch, M.
Affiliations
LMU Klinikum (DE)
Corresponding author
Jeblick, Katharina
ORCID
Venue
European Radiology
Publisher
Springer Science and Business Media LLC
Published
2023-10-05
Volume
34
Issue
5
Pages
2817-2825
ISSN
  • 0938-7994 (ISSN-L)
  • 1432-1084
DOI
10.1007/s00330-023-10213-1
PMC
PMC11126432
OpenAlex
W4387356888
Cited by
549
References
22
Citations by year
2026: 105 · 2025: 152 · 2024: 157 · 2023: 133 · 2022: 1
FWCI
21.84
Type
journal article
Language
en
Open access
Licence
Locations
  • European Radiology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by)
  • PUBLISSO (German National Library of Medicine) (submittedVersion, cc-by)
Funders
  • Deutsche Forschungsgemeinschaft (460037581)
  • Universitätsklinik München
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
Keywords
ChatGPT, large language models, radiology reports, medical text simplification, patient-centered care, factual accuracy, patient harm, radiologist evaluation, patient-doctor interaction, artificial intelligence in radiology
MeSH terms
Radiologists, Humans, Physician-Patient Relations, Surveys and Questionnaires, Radiology, Radiology Information Systems, Patient-Centered Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review