The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index; none of the 30 sources its claims cite is checked for support yet.
ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports
Open accessAbstract
Objectives: To assess the quality of simplified radiology reports generated with the large language model (LLM) ChatGPT and to discuss challenges and chances of ChatGPT-like LLMs for medical text simplification. Methods: In this exploratory case study, a radiologist created three fictitious radiology reports which we simplified by prompting ChatGPT with “Explain this medical report to a child using simple language.” In a questionnaire, we tasked 15 radiologists to rate the quality of the simplified radiology reports with respect to their factual correctness, completeness, and potential harm for patients. We used Likert scale analysis and inductive free-text categorization to assess the quality of the simplified reports. Results: Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient. Nevertheless, instances of incorrect statements, missed relevant medical information, and potentially harmful passages were reported. Conclusion: While we see a need for further adaption to the medical field, the initial insights of this study indicate a tremendous potential in using LLMs like ChatGPT to improve patient-centered care in radiology and other medical domains. Clinical relevance statement: Patients have started to use ChatGPT to simplify and explain their medical reports, which is expected to affect patient-doctor interaction. This phenomenon raises several opportunities and challenges for clinical routine. Key Points: • Patients have started to use ChatGPT to simplify their medical reports, but their quality was unknown. • In a questionnaire, most participating radiologists overall asserted good quality to radiology reports simplified with ChatGPT. However, they also highlighted a notable presence of errors, potentially leading patients to draw harmful conclusions. • Large language models such as ChatGPT have vast potential to enhance patient-centered care in radiology and other medical domains. To realize this potential while minimizing harm, they need supervision by medical experts and adaption to the medical field. Graphical Abstract:
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
In this exploratory case study, ChatGPT was prompted to simplify three fictitious radiology reports, and 15 radiologists generally rated the outputs as factually correct, complete, and not potentially harmful. However, they also identified incorrect statements, missing relevant information, and potentially harmful passages in a considerable number of simplified reports.
Key findings
- Across all 45 simplified reports, median ratings were 2 (“Agree”) for factual correctness and completeness, and 4 (“Disagree”) for potential harm; no radiologist selected “Strongly disagree” for correctness/completeness, and none selected “Strongly agree” for potential harm.
- Free-text review found incorrect text passages in 23/45 simplified reports (51%), missing relevant medical information in 10/45 (22%), and potentially harmful conclusions in 16/45 (36%).
- Error categories included misinterpretation of medical terms (e.g., “DD”/differential diagnosis mistaken as final diagnosis; “thyroid struma” described as “infection in their thyroid gland” or “extra thyroid gland”), hallucinations (e.g., “no signs of cancer in the thyroid gland”; “brain does not seem to be damaged”), imprecise language, odd language, unmentioned findings, and unspecific location information.
- The Oncol. CT report had the highest mean factual-correctness rating (2.5, SD 1.1) and potential-harm mean 3.3 (SD 1.1); Head MRI had a slightly lower potential-harm median (3) than Knee MRI and Oncol. CT (both median 4).
Methods
- Prospective exploratory case study using three fictitious radiology reports (Knee MRI, Head MRI, Oncol. CT) written by a radiologist with 10 years of experience and intended to be of intermediate complexity.
- The ChatGPT online interface (version December 15th, 2022) was prompted with “Explain this medical report to a child using simple language:” followed by the original report; the model was restarted and prompted 15 times per report, generating 45 simplified reports.
- Fifteen radiologists with median experience of 5 years (IQR [1–10]) rated one randomly selected simplified version per original report on 5-point Likert scales for factual correctness, completeness, and potential harm, with free-text follow-up; analysis used descriptive statistics and inductive categorization.
Limitations
- Stated: small number of original reports (n = 3) and experts (n = 15); fictitious reports were written to represent intermediate medical complexity and translated to English by a non-native-speaking radiologist; only one prompt phrasing was used after heuristic selection; the study did not measure the quality of simplification.
- Evident: the radiologist sample came from a single clinic (“from our clinic”), and ChatGPT outputs were non-deterministic, while each questionnaire included only one randomly selected simplified version per original report.
- Evident: the study used fictitious reports and no patient information, so it does not validate performance on real clinical reports or patient outcomes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
25 Own result · 10 Not checked yet · 7 Own interpretation · 6 Own method · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The abbreviation “DD” (differential diagnosis) was often mistaken as a final diagnosis (B.2.6, B.2.8)”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.”
- Also stated in: § Likert scale analysis · plain
“The participants generally agreed (median = 2) with the statements that the simplified reports are factually correct and complete, respectively.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Likert scale analysis · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For instance, the misinterpretation of differential diagnosis (“DD”) as final diagnosis was considered potentially harmful for the patient, e.g., one radiologist commented: “GBM is one (likely) DD which implies that other DDs exist (e.g. radionecrosis).” (B.2.7).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
hedgedSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For potential harm, the Head MRI report had a slightly lower median (median = 3) compared to the other two reports (median = 4).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Likert scale analysis
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.”
- Also stated in: § Likert scale analysis · plain
“The participants generally agreed (median = 2) with the statements that the simplified reports are factually correct and complete, respectively.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Likert scale analysis · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, we found instances of unsuited and odd language. The idiom “wear and tear” (B.1.4) was used to describe the degeneration of tendons and a radiological examination was circumscribed as “to see how the cancer is doing” (B.3.8).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some findings in the simplified reports were made up by ChatGPT. Examples for these hallucinations were “no signs of cancer in the thyroid gland” (B.3.3) or “brain does not seem to be damaged” (B.2.9).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In line with the findings for factual correctness and completeness, the participants disagreed (median = 4) on the potential of wrong conclusions drawn from the simplified reports resulting in physical and/or psychological harm.”
- Also stated in: § Likert scale analysis · plain
“No radiologist chose “Strongly agree.””
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Likert scale analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Likert scale analysis · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants highlighted incorrect text passages in 23 simplified reports (51%), listed missing relevant information for 10 simplified reports (22%), and listed potentially harmful conclusions for 16 simplified reports (36%).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants highlighted incorrect text passages in 23 simplified reports (51%), listed missing relevant information for 10 simplified reports (22%), and listed potentially harmful conclusions for 16 simplified reports (36%).”
- Also stated in: § Discussion · hedged
“In about one-third of all simplified reports, participating radiologists found errors, which might lead patients to draw wrong conclusions, potentially resulting in physical and/or psychological harm.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Discussion · hedged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, we observed passages of imprecise language. For instance, the medial compartment of the knee was described as the “middle part of your leg” (B.1.5), or the brain as the “head” (B.2.7).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, we observed missing or unspecific location information, e.g., imprecise localization resulted in ambiguous assignment of a growing mass to present or excised tumor (B.2.11).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants highlighted incorrect text passages in 23 simplified reports (51%), listed missing relevant information for 10 simplified reports (22%), and listed potentially harmful conclusions for 16 simplified reports (36%).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, a growing mass of “currently max. 22 mm” was incorrectly simplified as a “small, abnormal growth that has gotten bigger” (B.2.3).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The fact that the solid portions of the pulmonary metastases are decreasing—consistent with therapy response—was not included in the simplified report B.3.5.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, instances of incorrect text passages and missing relevant medical information were identified in a considerable number of cases, which could lead patients to draw harmful conclusions.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this exploratory study, most participating radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient, indicating that ChatGPT is in principle able to simplify radiology reports.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of original radiology reports (n = 3) and experts (n = 15) for assessing the quality of the simplified reports is small.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, these model-related challenges discourage the use of ChatGPT for the task of simplification of radiology reports in an autonomous setting.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The training data of ChatGPT is static, i.e., new research findings are not considered in ChatGPT’s output.
- Rests on an assumption: Uploading protected health information to a proprietary service such as ChatGPT might compromise patients’ privacy.
None of the sources cited for this claim have been checked yet.
In the paper
“For patients without a medical background, these reports are often inaccessible [9–12].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Introduction
What the sources say
- Not checked yetcites [9] Keselman A, Slaughter L, Arnott-Smith C et al (2007) Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, p 399
- Not checked yetcites [10] Yi PH, Golden SK, Harringa JB, Kliewer MA (2019) Readability of lumbar spine MRI reports: will patients understand? AJR Am J Roentgenol 212(3):602–606 source
- Not checked yetcites [11] Kindig DA, Panzer AM, Nielsen-Bohlman L, editors (2004) Health literacy: a prescription to end confusion. National Academies Press, Washington, DC source
- Not checked yetcites [12] Martin-Carreras T, Cook TS, Kahn CE (2019) Readability of radiology reports: implications for patient-centered care. Clin Imaging 54:116–120 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“So far, most authors focused on summarization of radiology reports, i.e., on the creation of a shorter version that includes all important aspects at the same level of text complexity [23–26].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [23] Zhang Y, Ding DY, Qian T, Manning CD, Langlotz CP (2018) Learning to summarize radiology findings. In: Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, Louhi@EMNLP. Association for Computational Linguistics, p 204–213 source
- Not checked yetcites [24] Cai X, Liu S, Han J, Yang L, Liu Z, Liu T (2021) ChestXRayBERT: a pretrained language model for chest radiology report summarization. IEEE Trans Multimed 25:845–855 source
- Not checked yetcites [25] Liang S, Kades K, Fink M et al (2022) Fine-tuning BERT models for summarizing german radiology findings. In: Proceedings of the 4th Clinical Natural Language Processing Workshop, p 30–40 source
- Not checked yetcites [26] Chaves A, Kesiku C, Garcia-Zapirain B (2022) Automatic text summarization of biomedical text data: a systematic review. Information 13(8):393 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 15
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the case of incorrect text passages, we identified the inductive categories misinterpretation of medical terms, hallucinations, imprecise language, and odd language.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found potentially harmful unmentioned findings in the simplified report. The sentence “the parts that are staying the same size are changing” (B.3.5) was found to be an unclear statement missing interpretation.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though the model is able to state to not know the answer [1], in all cases ChatGPT generated plausible-sounding outputs, while it did not indicate that the response could be incorrect.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, ChatGPT’s output is non-deterministic, i.e., applying the same prompt multiple times results in different responses, hindering reproducibility.”
- Also stated in: § Simplification of radiology reports using ChatGPT · plain
“The version of the interface of ChatGPT used in this study does not allow changing any model settings, resulting in non-deterministic outputs.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Simplification of radiology reports using ChatGPT · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the category of hallucinations, participants rated the passage “brain does not seem to be damaged” (B.2.9) as potentially harmful to the patient, as the original report describes a growing mass.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the wording “small growth” (B.2.3) was deemed a harmful conclusion, as the original report (B.2.0) states a progression in size, which “almost doubled.””
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, “thyroid struma” was described by ChatGPT in one instance as “infection in their thyroid gland” (B.3.13) and in another as “extra thyroid gland” (B.3.7).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Missing relevant medical information was categorized into unmentioned findings and unspecific location.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, we observed that imprecise language highlighted as incorrect was sometimes also graded as potentially harmful to the patient. For example, participants listed that it is “not clear that spots are pulmonal metastases” (B.3.5) and that “some extra fluid” is not a significant increasing edema (B.2.13).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Lymph nodes were simplified as “they might have cancer” (B.3.9), but the original report stated that there was “no evidence of recurrence or new lymph node metastases.””
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Grammatical errors such as “CT scan” instead of “CT scanner” (B.3.7) were the exception.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Results > Free-text analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Written informed consent was obtained from participating radiologists.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The fictitious original reports were written to represent an intermediate level of medical complexity and translated to English by a non-native-speaking radiologist.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In the medical domain, there is a huge need for text simplification [8, 9].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Introduction
What the sources say
- Not checked yetcites [8] Barrett SK, Patrie J, Kitts AB, Hanley M, Swanson CM (2021) Vitzthum von Eckstaedt H, et al Patient-centered reporting in radiology: a single-site survey study of lung cancer screening results. J Thorac Imaging 36(6):367 source
- Not checked yetcites [9] Keselman A, Slaughter L, Arnott-Smith C et al (2007) Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, p 399
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“According to the rules of the Ethics Committee of the Faculty of Medicine, LMU Munich, no ethical consultation was therefore necessary for the approval of this study.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Also, it is well established that LLMs like ChatGPT have intrinsic biases, including stereotypical associations, or negative sentiment towards specific groups, as well as biases due to imbalanced training data [6, 34, 35].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [6] Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Not checked yetcites [34] Bender EM, Gebru T, McMillan-Major A, Shmitchell S (2021) On the dangers of stochastic parrots: can language models be too big?. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, p 610–623 source
- Not checked yetcites [35] Bommasani R, Hudson DA, Adeli E et al (2021) On the opportunities and risks of foundation models. ArXiv Prepr ArXiv210807258.
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Their answers were collected in anonymized questionnaires.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Simplified and thus more comprehensible reports would enable patients to better understand and oversee their own health situation and empower them to make informed and active decisions throughout the medical treatment process.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We used only one phrasing of the prompt for report simplification after a heuristic selection process and did not measure the quality of simplification.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Radiological findings might have immediate consequences for patients. However, they are typically only communicated in a free-text report in specialized medical jargon, targeting a clinician or doctor as the recipient.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Introduction
What the sources say
- Not checked yetcites [9] Keselman A, Slaughter L, Arnott-Smith C et al (2007) Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, p 399
- Not checked yetcites [10] Yi PH, Golden SK, Harringa JB, Kliewer MA (2019) Readability of lumbar spine MRI reports: will patients understand? AJR Am J Roentgenol 212(3):602–606 source
- Not checked yetcites [11] Kindig DA, Panzer AM, Nielsen-Bohlman L, editors (2004) Health literacy: a prescription to end confusion. National Academies Press, Washington, DC source
- Not checked yetcites [12] Martin-Carreras T, Cook TS, Kahn CE (2019) Readability of radiology reports: implications for patient-centered care. Clin Imaging 54:116–120 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These errors can be grouped into the same error categories that were found for factual correctness and completeness.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, misinterpretation of medical terms hinting at recurrence of cancer or understating progression of a disease is likely to cause harm for patients.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“As known for other LLMs [13, 14], ChatGPT can generate plausible-sounding text, but the content does not need to be true (Fig. 1).”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Introduction
What the sources say
- Not checked yetcites [13] Lin S, Hilton J, Evans O (2022) TruthfulQA: measuring how models mimic human falsehoods. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL. Association for Computational Linguistics, p 3214–52 source
- Not checked yetcites [14] Taylor R, Kardas M, Cucurull G et al (2022) Galactica: a large language model for science. ArXiv Prepr ArXiv221109085.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Efforts in this direction have been made, e.g., by Lee et al, Gu et al, and Singhal et al [20–22].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [20] Lee J, Yoon W, Kim S et al (2020) BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4):1234–1240 source
- Not checked yetcites [21] Gu Y, Tinn R, Cheng H et al (2021) Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthc. 3(1):2:1–2:23 source
- Not checked yetcites [22] Singhal K, Azizi S, Tu T et al (2022) Large language models encode clinical knowledge [Internet]. arXiv; [cited 2023 Jan 13].
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Previous approaches for the simplification of radiology reports mostly replaced recurrent phrases [29–32] or augmented radiology reports with lay-language definitions [33].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [29] Kvist M, Velupillai S (2013) Professional language in Swedish radiology reports–characterization for patient-adapted text simplification. In: Scandinavian Conference on Health Informatics. Linköping University Electronic Press, p 55–59
- Not checked yetcites [30] Abrahamsson E, Forni T, Skeppstedt M, Kvist M (2014) Medical text simplification using synonym replacement: adapting assessment of word difficulty to a compounding language. In: Proceedings of the 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR). p 57–65 source
- Not checked yetcites [31] Ramadier L, Lafourcade M (2017) Radiological text simplification using a general knowledge base. In: International Conference on Computational Linguistics and Intelligent Text Processing. Springer, p 617–627 source
- Not checked yetcites [32] Qenam B, Kim TY, Carroll MJ, Hogarth M (2017) Text simplification using consumer health vocabulary to generate patient-centered radiology reporting: translation and evaluation. J Med Internet Res 19(12):e417 source
- Not checked yetcites [33] Oh SC, Cook TS, Kahn CE (2016) PORTER: a prototype system for patient-oriented radiology reporting. J Digit Imaging 29(4):450–454 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining inductive categories might be attributed to the fact that ChatGPT was trained on general data and not tuned to the task of radiology report simplification.”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Hallucinations are an intrinsic problem of generative models like LLMs [15] and difficult to remedy [16, 17].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [16] Wang C, Sennrich R (2020) On exposure bias, hallucination and domain shift in neural machine translation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics [Internet]. Online: Association for Computational Linguistics; [cited 2023 Jan 24]. p 3544–52 source
- Not checked yetcites [17] Guerreiro NM, Voita E, Martins AFT (2022) Looking for a needle in a haystack: a comprehensive study of hallucinations in neural machine translation [Internet]. arXiv; [cited 2023 Jan 24]. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Hallucinations are an intrinsic problem of generative models like LLMs [15] and difficult to remedy [16, 17].”
§ ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports > Discussion
What the sources say
- Not checked yetcites [15] Zhou C, Neubig G, Gu J et al (2021) Detecting hallucinated content in conditional neural sequence generation. In: Findings of the association for computational linguistics: ACL/IJCNLP. Association for Computational Linguistics, p 1393–404 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Patients have started to use ChatGPT to simplify and explain their medical reports.
“Patients have started to use ChatGPT to simplify and explain their medical reports, which is expected to affect patient-doctor interaction.”
The paper relies on this without support.
ChatGPT was not explicitly trained for medical text simplification and is not intended to be used for this critical task.
“At the same time, ChatGPT was not explicitly trained for medical text simplification and is not intended to be used for this critical task.”
The paper relies on this without support.
The training data of ChatGPT is static, i.e., new research findings are not considered in ChatGPT’s output.
“The training data is static, i.e., new research findings are not considered in ChatGPT’s output.”
The paper relies on this without support.
The resulting models lack the accessibility and quality of ChatGPT.
“However, the resulting models lack the accessibility and quality of ChatGPT.”
The paper relies on this without support.
Uploading protected health information to a proprietary service such as ChatGPT might compromise patients’ privacy.
“Finally, uploading protected health information to a proprietary service, such as ChatGPT, might compromise patients’ privacy.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Within the realm of surgical procedures, such as cosmetic orthognathic surgery, ChatGPT can function as a virtual assistant, offering patients crucial preoperative information.”
From A Systematic Review and Meta-Analysis of Artificial Intelligence Tools in Medicine and Healthcare: Applications, Considerations, Limitations, Motivation and Challenges (2024) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“ChatGPT’s GPT-3.5 iteration has already shown promise in several routine medical tasks and medical research [4-7], even raising ethical concerns in the literature [2,3,8].”
From Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study (2024) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Both versions are popular and remarkably successful so that ChatGPT has great potential to become a daily assistant for purposes of general or professional consulting (Jeblick et al., 2023).”
From Teachers vs. Chat GPT: identifying and naming acts of online sexual abuse (2025) · cites this paper as Jeblick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The ability of ChatGPT to help in streamlining the clinical workflow appears promising, with possible cost savings and increased efficiency in health care delivery [31,37,39,77].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Ethical issues including the risk of bias and transparency issues appeared as recurring major concerns [51,68,69,77].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]
Experimental — accuracy not yet measured
The cited source says what the claim attributes to it.
In the citing paper
“Importantly, the concept of ChatGPT hallucination could be risky if the generated content is not thoroughly evaluated by researchers and health providers with proper expertise [37,56,73,77,79].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [77]
In the citing paper
“ChatGPT plays a crucial role in pandemic management by aiding in the swift and accurate dissemination of critical information.”
In the citing paper
“Its natural language processing capabilities enable it to generate coherent responses, making it a valuable tool for healthcare organisations.”
In the citing paper
“Specific advantages include its ability to provide rapid updates, preventive measures, and medical guidelines to the public.”
In the citing paper
“ChatGPT can offer standardised and accurate responses, which prepare patients for their consultations and help them make informed decisions [5,6,7,8,9,10,42,43].”
In the citing paper
“It was used in all areas: healthcare and management [5,6,7,8,9], aiding cosmetic orthognathic surgery consultation [10], revolutionising dental practices [8,11,12], medical scientific articles and education fields [6,9,13,14,15,16,17,18,19,20,21,22,23], support in disease diagnosis [24], revolutionising radiology, sonar imaging and writing assessment [13,25,26,27,28,29,30,31,32,33], pharmaceutical”
Citations
Cites
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 3×
“However, ChatGPT - like most LLMs - is trained on data with a certain timestamp [40].”
BackgroundMethodInfluentialTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Health Literacy: A Prescription to End Confusion
Committee on Health Literacy, Board on Neuroscience and Behavioral Health, & Institute of Medicine (2004) · doi:10.17226/10883 · Cited by 3,444
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 1×
“Given that non-maleficence is a major principle of medical ethics, also in the context of AI applications [18, 19], we do see a need for further adaption of ChatGPT and similar LLMs to the medical field, in order to mitigate the risk for harmful errors.”
BackgroundThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 1×
“Given that non-maleficence is a major principle of medical ethics, also in the context of AI applications [18, 19], we do see a need for further adaption of ChatGPT and similar LLMs to the medical field, in order to mitigate the risk for harmful errors.”
BackgroundTruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1×
“As known for other LLMs [13, 14], Chat-GPT can generate plausible-sounding text, but the content does not need to be true (Fig.”
BackgroundPrinciples of Biomedical Ethics
Athar, S. (2012) · Journal of the Islamic Medical Association of North America · doi:10.5915/43-3-8476 · Cited by 335
Learning to Summarize Radiology Findings
Zhang, Y., Ding, D. Y., Qian, T., Manning, C. D., & Langlotz, C. P. (2018) · Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis · doi:10.18653/v1/w18-5623 · Cited by 126
Research works printed but not linked: 8
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Agarwal A (2022) Explain to me like I am five–sentence simplification using transformers. ArXiv Prepr ArXiv221204595
- Taylor R, Kardas M, Cucurull G et al (2022) Galactica: a large language model for science. ArXiv Prepr ArXiv221109085.
- Singhal K, Azizi S, Tu T et al (2022) Large language models encode clinical knowledge [Internet]. arXiv; [cited 2023 Jan 13].
- Shardlow M (2014) A survey of automated text simplification. Int J Adv Comput Sci Appl 4(1):58–70
- Al-Thanyyan SS, Azmi AM (2021) Automated text simplification: a survey. ACM Comput Surv CSUR 54(2):1–36
- Kvist M, Velupillai S (2013) Professional language in Swedish radiology reports–characterization for patient-adapted text simplification. In: Scandinavian Conference on Health Informatics. Linköping University Electronic Press, p 55–59
- Bommasani R, Hudson DA, Adeli E et al (2021) On the opportunities and risks of foundation models. ArXiv Prepr ArXiv210807258.
Cited by
Sorted by citations
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
Yeo, Y. H., Samaan, J. S., Ng, W. H., Ting, P., Trivedi, H. D., Vipani, A., Ayoub, W. S., Yang, J. D., Liran, O., Spiegel, B. M. R., et al. (2023) · Clinical and Molecular Hepatology · doi:10.3350/cmh.2023.0089 · Cited by 669
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628
Metadata
- Authors
- Jeblick, K., Schachtner, B., Dexl, J., Mittermeier, A., Stüber, A. T., Topalis, J., Weber, T., Wesp, P., Sabel, B. O., Ricke, J., & Ingrisch, M.
- Affiliations
- LMU Klinikum (DE)
- Corresponding author
- Jeblick, Katharina
- ORCID
- Venue
- European Radiology
- Published
- 2023-10-05
- Volume
- 34
- Issue
- 5
- Pages
- 2817-2825
- ISSN
- 0938-7994 (ISSN-L)
- 1432-1084
- PMC
- PMC11126432
- OpenAlex
- W4387356888
- Cited by
- 549
- References
- 22
- Citations by year
- 2026: 105 · 2025: 152 · 2024: 157 · 2023: 133
- FWCI
- 21.84
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- European Radiology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by)
- PUBLISSO (German National Library of Medicine) (submittedVersion, cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (460037581)
- Universitätsklinik München
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Keywords
- ChatGPT, large language models, radiology reports, medical text simplification, patient-centered care, factual accuracy, patient harm, radiologist evaluation, patient-doctor interaction, artificial intelligence in radiology
- MeSH terms
- Radiologists, Humans, Physician-Patient Relations, Surveys and Questionnaires, Radiology, Radiology Information Systems, Patient-Centered Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
