The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models (LLMs)

Probably not a substantial claim: may not pass S3 (load-bearing).

AI-driven infodemics could also inundate scientific journals with low-quality, excessively produced content.

“While infodemic issues in social media and grey literature are noted, AI-driven infodemics could also inundate scientific journals with low-quality, excessively produced content46.”

• cites [46] de Angelis, L. et al. Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health. Frontiers in public health 11, 1166120 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

De Angelis et al. present the concept of an AI-driven infodemic in which the overwhelming spread of imprecise, unclear, or false information leads to disorientation and potentially harmful behavior among recipients.

“De Angelis et al. take this concern one step further, presenting the concept of an AI-driven infodemic46 in which the overwhelming spread of imprecise, unclear, or false information leads to disorientation and potentially harmful behavior among recipients.”

• cites [46] de Angelis, L. et al. Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health. Frontiers in public health 11, 1166120 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Schmälzle & Wilcox highlight the risks of a dual use of LLMs: while they might further equal access to information, malicious actors can and seem to be using LLMs to spread fake information and devise health messages at an unprecedented scale that is harmful to societies.

“Schmälzle & Wilcox highlight the risks of a dual use of LLMs50. While they might further equal access to information, malicious actors can and seem to be using LLMs to spread fake information and devise health messages at an unprecedented scale that is harmful to societies50,51,75.”

• cites [50] Schmälzle, R. & Wilcox, S. Harnessing artificial intelligence for health message generation: The folic acid message engine. Journal of Medical Internet Research 24, e28858 (2022). sourceNot checked yet

  Not checked in the free run.

• cites [51] Shahriar, S. & Hayawi, K. Let’s have a chat! a conversation with chatgpt: Technology, applications, and limitations. Preprint at https://doi.org/10.48550/arXiv.2302.13817 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [75] Temsah, O. et al. Overview of early chatgpt’s presence in medical literature: Insights from a hybrid literature review by chatgpt and human experts. Cureus 15, e37281 (2023).Not checked yet

  Not checked in the free run.

May combine two claims.

The included records comprised 23 original articles, 11 letters, six editorials, four reviews, three comments, one report, and five unspecified articles.

“53 records were included in the dataset, encompassing 23 original articles25,35–56, including theoretical or empirical work, 11 letters57–67, six editorials68–73, four reviews8,74–76, three comments24,77,78, one report79 and five unspecified articles80–84.”

May combine two claims.

In patient consultation and communication, LLMs can offer a novel approach to patient-provider interaction by facilitating informational exchange and bridging gaps between clinical and preclinical settings.

“In patient consultation and communication, LLMs can offer a novel approach to patient-provider interaction by facilitating informational exchange and bridging gaps between clinical and preclinical settings, such as self-management measures or community aids8.”

Probably not a substantial claim: may not pass S3 (load-bearing).

After removal of duplicates, 738 records underwent title/abstract screening.

“After removal of duplicates, 738 records went through title/abstract screening.”

May combine two claims.

Advantages of using LLMs are attributed to their capacity in data analysis, information provisioning, support in decision-making, or mitigating information loss and enhancing information accessibility.

“Advantages of using LLMs are attributed to their capacity in data analysis, information provisioning, support in decision-making or mitigating information loss and enhancing information accessibility.”

May combine two claims.

Research by Knebel et al. on self-triaging in ophthalmologic emergencies indicates that ChatGPT's responses can't reliably prioritize urgency, reducing their usefulness.

“For instance, research by Knebel et al. on self-triaging in ophthalmologic emergencies indicates that ChatGPT’s responses can’t reliably prioritize urgency, reducing their usefulness45.”

• cites [45] Knebel, D., Priglinger, S., Scherer, N., Siedlecki, J. & Schworm, B. Assessment of chatgpt in the preclinical management of ophthalmological emergencies – an analysis of ten fictional case vignettes. Preprint at https://doi.org/10.1101/2023.04.16.23288645 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Several authors in the dataset discuss the use of LLMs for initial diagnosis and triaging as a potentially transformative application.

“To support initial diagnosis and triaging of patients39,52, several authors discuss the use of LLMs in the context of predictive patient analysis and risk assessment in or prior to clinical situations as a potentially transformative application74,80.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Biases and perpetuating existing stereotypes and disparities are a constantly discussed theme in treatment applications.

“However, as with diagnostic applications, biases and perpetuating existing stereotypes and disparities are a constantly discussed theme60,61,67.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The dataset encompasses studies that explore the systemic implications of LLMs, especially from a public health perspective.

“The dataset encompasses studies that explore the systemic implications of LLMs, especially from a public health perspective50,61,75.”

May combine two claims.

This highlights the potential for accelerating research and relieving researchers of workload, leading to more efficient research workflows.

“This highlights the potential for accelerating research46,79 and relieving researchers of workload8,40,64,74,75,83, leading to more efficient research workflows and allowing researchers to spend less time on burdensome routine work8,80.”

Probably not a substantial claim: may not pass S3 (load-bearing).

158 full-texts were assessed.

“158 full-texts were assessed.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A number of authors raise doubts about overly positive expectations.

“However, a number of authors raise doubts about overly positive expectations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

An AI-driven infodemic could exacerbate future health threats.

“However, the authors caution that an AI-driven infodemics could exacerbate future health threats.”

• cites [46] de Angelis, L. et al. Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health. Frontiers in public health 11, 1166120 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Yeo et al. highlight that LLMs could provide cross-lingual translation and thus contribute to equalizing healthcare and racial disparities.

“In their work on the use of ChatGPT 4 in overcoming language barriers, Yeo et al. highlight an additional benefit, as LLMs could provide cross-lingual translation and thus contribute to equalizing healthcare and racial disparities56.”

• cites [56] Yeo, Y. H. et al. Gpt-4 outperforms chatgpt in answering non-english questions related to cirrhosis. Preprint at https://doi.org/10.1101/2023.05.04.23289482 (2023). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Problems in clinical applications are tightly connected to inherent biases in LLMs, their tendency to hallucinate, and their intransparency.

“These problems are tightly connected to inherent biases in LLMs, their tendency to “hallucinate” and their intransparency52.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Four general themes emerged from the dataset: clinical applications, patient support applications, support of health professionals, and public health perspectives.

“During analysis, four general themes emerged in our dataset, which we used to structure our reporting. These themes include clinical applications, patient support applications, support of health professionals, and public health perspectives.”

May combine two claims.

LLMs can offer laypersons personalized guidance, such as lifestyle adjustments during illness, self-assessment of symptoms, self-triaging, and emergency management steps.

“LLMs can offer laypersons personalized guidance, such as lifestyle adjustments during illness80, self-assessment of symptoms61,63, self-triaging, and emergency management steps8,57.”

May combine two claims.

The inability of most current LLMs to seek clarifications by asking questions and their lack of sensitivity to query variations can lead to imprecise answers.

“The inability of most current LLMs to seek clarifications by asking questions and their lack of sensitivity to query variations can lead to imprecise answers45,63.”

May combine two claims.

LLMs are not able to replicate a process of clinical reasoning in general and fail to comprehend the complexity of the process.

“This is in line with others noting that LLMs are not able to replicate a process of clinical reasoning in general and, hence, fail to comprehend the complexity of the process44,59,75.”

• cites [44] Hosseini, M. et al. An exploratory survey about using chatgpt in education, healthcare, and research. Preprint at https://doi.org/10.1101/2023.03.31.23287979 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [59] Beltrami, E. J. & Grant-Kels, J. M. Consulting chatgpt: Ethical dilemmas in language model artificial intelligence. Journal of the American Academy of Dermatology 90, 879–880 (2024). sourceNot checked yet

  Not checked in the free run.

• cites [75] Temsah, O. et al. Overview of early chatgpt’s presence in medical literature: Insights from a hybrid literature review by chatgpt and human experts. Cureus 15, e37281 (2023).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Concerns about overreliance and deskilling are raised, as LLMs might diminish researchers' skills and overly shape research outcomes.

“Additionally, concerns about overreliance and deskilling are raised, as LLMs might diminish researchers’ skills and overly shape research outcomes46.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It remains unclear in the dataset what patient consultation applications would look like in practice-specifically where, when and how LLMs could be integrated.

“However, it remains unclear in our dataset what such applications would look like in practice — specifically where, when and how LLMs could actually be integrated.”

May combine two claims.

The consensus is that LLMs could streamline clinical workflows, offering time savings for health professionals.

“The consensus is that LLMs could streamline clinical workflows8,36,43,51,52,60,68,74,80,81,83, offering time savings for health professionals currently burdened with extensive administrative duties68,83.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Such access can improve laypersons' health literacy by providing needs-oriented access to comprehensible medical information.

“Such access can improve laypersons’ health literacy by providing a needs-oriented access to comprehensible medical information68, which is regarded as an important precondition of autonomy to allow more independent, health-related decisions8,74.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some authors fear that using LLMs could compromise research integrity by disrupting traditional trust factors like source traceability, factual consistency, and process transparency.

“Some authors fear that using LLMs could compromise research integrity by disrupting traditional trust factors like source traceability, factual consistency, and process transparency24.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The systematic review's search yielded 796 database hits.

“Our search yielded a total of 796 database hits.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Tang et al. emphasize ChatGPT's tendency to produce attribution and misinterpretation errors, potentially distorting original source information.

“However, LLMs capacities are also critically examined, with Tang et al. emphasizing ChatGPT’s tendency to produce attribution and misinterpretation errors, potentially distorting original source information.”

May combine two claims.

Ethical acceptability varies across fields; otolaryngology and infectious disease studies find ChatGPT's responses to patients lack detail but aren't harmful, whereas pharmacology and mental health indicate greater potential risks.

“However, ethical acceptability varies across fields; for instance, otolaryngology and infectious disease studies find ChatGPT’s responses to patients lack detail but aren’t harmful53, whereas pharmacology and mental health indicate greater potential risks67,68.”

May combine two claims.

An ethical problem emerges with potentially negative effects on patient outcomes due to biases in the training data, especially with the lack of diverse datasets risking the underrepresentation of marginalized or vulnerable groups.

“An ethical problem emerges with potentially negative effects on patient outcomes due to biases in the training data36,39,41,74,75,78, especially with the lack of diverse datasets risking the underrepresentation of marginalized or vulnerable groups.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In health research, LLMs are suggested to support text, evidence or data summarization, identify research targets, design experiments or studies, facilitate knowledge sharing, and communicate results.

“In health research, LLMs are suggested to support text, evidence or data summarization54,64,82, identify research targets8,61,72,83, designing experiments or studies72,83, facilitate knowledge sharing between collaborators37,70,80, and to communicate results74.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All authors note that clinical applications of LLMs might be problematic and require close human oversight.

“All authors note, however, that such applications might also be problematic and require close human oversight39,44,51,80.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

State-of-the-art LLMs carry the risks of biases, hallucinations, and challenges in validity assessment, reliability testing, and reproducibility.

“Inherently, state-of-the-art LLMs carry the risks of biases, hallucinations, and challenges in validity assessment, reliability testing, and reproducibility.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The review highlights a vivid testing phase of LLMs across various healthcare domains.

“Our review highlights a vivid testing phase of LLMs across various healthcare domains12.”

• cites [12] Li, J., Dada, A., Kleesiek, J. & Egger, J. Chatgpt in healthcare: A taxonomy and systematic review. Comput Methods Programs Biomed 245, 108013 (2024). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There is an overarching sentiment of the promise LLMs hold despite the lack of real-world applications, especially in the clinic.

“Despite the lack of real-world applications, especially in the clinic, there is an overarching sentiment of the promise LLMs hold.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

A portion of current research seems propelled more by experimental curiosity than by well-defined methodological rigor, at times pushing the boundaries of ethical acceptability, particularly when sensitive real patient data are utilized to explore capabilities of systems like ChatGPT.

“Notably, a portion of current research seems propelled more by a sense of experimental curiosity than by well-defined methodological rigor, at times pushing the boundaries of ethical acceptability, particularly when sensitive real patient data are utilized to explore capabilities of systems like ChatGPT.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The dataset exemplifies a diversity of perspectives, methodologies, and applications of LLMs, revealing a significant degree of ambiguity and uncertainty about the appropriate engagement with this technology.

“Our dataset exemplifies a diversity of perspectives, methodologies, and applications of LLMs, revealing a significant degree of ambiguity and uncertainty about the appropriate engagement with this technology.”

May combine two claims.

The ethical concerns echo wider AI ethics discourse, particularly in healthcare, and touch on fairness, bias, non-maleficence, transparency, and privacy.

“These concerns echo the wider discourse on AI ethics86–88, particularly in healthcare89, and touch on issues of fairness, bias, non-maleficence, transparency, and privacy.”

• cites [86] Kazim, E. & Koshiyama, A. S. A high-level overview of ai ethics. Patterns 2, 100314 (2021). sourceNot checked yet

  Not checked in the free run.

• cites [87] Hagendorff, T. The ethics of ai ethics: An evaluation of guidelines. Minds and Machines 30, 99–120 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [88] Jobin, A., Ienca, M. & Vayena, E. The global landscape of ai ethics guidelines. Nature Machine Intelligence 1, 389–399 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [89] Morley, J. et al. The ethics of ai in health care: A mapping review. Social Science & Medicine 260, 113172 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

By automating repetitive tasks, professionals could dedicate more time to high-quality medical tasks.

“By automating these repetitive tasks, professionals could dedicate more time to high-quality medical tasks83.”

May combine two claims.

Biased models may result in unfair treatment of disadvantaged groups, leading to disparities in access, exacerbating existing inequalities, or harming persons through selective accuracy.

“Biased models may result in unfair treatment of disadvantaged groups, leading to disparities in access, exacerbating existing inequalities, or harming persons through selective accuracy41.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Such applications would require the large-scale integration of LLMs into existing clinical data systems.

“Crucially, such applications would require the large-scale integration of LLMs into existing clinical data systems49.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Low- and middle-income countries often have a huge treatment gap driven by a deficit in professionals or inequitable resource distribution.

“These countries often have a huge treatment gap driven by a deficit in professionals or inequitable resource distribution.”

May combine two claims.

Using LLMs could mitigate accessibility and affordability issues, potentially offering a more favorable alternative to the current lack of access.

“Using LLMs could mitigate accessibility and affordability issues, potentially offering a more favorable alternative to the current lack of access71.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Inclusions were screened and extracted in a two-staged process following a modified rapid review approach.

“Inclusions were screened and extracted in a two-staged process following a modified rapid review approach97.”

• cites [97] Garritty, C. et al. Updated recommendations for the cochrane rapid review methods guidance for rapid reviews of effectiveness. BMJ 384, e076335 (2024). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors independently screened titles and abstracts of 10% of all database hits (73 records) to test and refine inclusion and exclusion criteria.

“Subsequently, the authors independently screened titles and abstracts of 10% of all database hits (73 records) to test and refine inclusion and exclusion criteria.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors descriptively report on procedural quality criteria to distinguish material that underwent processual quality control from other material.

“We descriptively report on procedural quality criteria (see Table 1) to distinguish material that underwent processual quality control (such as peer review) from other material.”

Classified as probably not a substantial claim.

A final synthesis was conducted following a meta-aggregative approach.

“A final synthesis was conducted following a meta-aggregative approach.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ChatGPT presents a unique instance where a general-purpose technology has rapidly permeated healthcare and research to an unprecedented extent.

“This presents a rather unique instance where a general-purpose technology has rapidly permeated the sector of healthcare and research to an unprecedented extent.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The overview of ethical issues is not exhaustive.

“With this in mind, we do not understand our overview of ethical issues as exhaustive.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The effectiveness of LLM usage hinges on various situational factors, including the user, their level of expertise and epistemic position, the specific domain of application, the risk profile of the application, and potential alternatives that the LLM is compared against.

“Contextually, the effectiveness of LLM usage hinges on various situational factors, including the user utilizing LLMs, their level of expertise as well as their epistemic position (e.g. expert versus layperson), the specific domain of application, the risk profile of the application, and potential alternatives that the LLM is compared against.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There is a lack of more critical perspectives potentially informed by non-Western experience and exploration of LLMs.

“However, a lack of more critical perspectives potentially informed by non-Western experience and exploration of LLMs needs to be noted.”

May combine two claims.

The surge in LLMs was largely caused by the advent and ease of use of ChatGPT, a platform not originally tailored for professional healthcare settings, yet widely adopted within it.

“This surge in LLMs was largely caused by the advent and ease of use of ChatGPT, a platform not originally tailored for professional healthcare settings, yet widely adopted within it12,83.”

• cites [12] Li, J., Dada, A., Kleesiek, J. & Egger, J. Chatgpt in healthcare: A taxonomy and systematic review. Comput Methods Programs Biomed 245, 108013 (2024). sourceNot checked yet

  Not checked in the free run.

• cites [83] Harrer, S. Attention is not all you need: The complicated case of ethically using large language models in healthcare and medicine. eBioMedicine 90, 104512 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Introducing LLM technologies through an iterative process facilitates a gradual understanding of the actual consequences of LLM use, thereby mitigating uncertainties.

“This approach facilitates a gradual understanding of the actual consequences of LLM use, thereby mitigating uncertainties.”

Probably not a substantial claim: may not pass S2 (checkable).

The real challenge lies not in articulating the need for ethical guidance but in comprehending the scope of what this entails.

“While we concur with the need for such guidance, our analysis suggests that the real challenge lies not in the articulation of such a need but in comprehending the scope of what this entails.”

Classified as probably not a substantial claim.

According to van der Poel, the full benefits, risks, and ethical issues of a technology become evident only after its widespread adoption.

“We employ this concept in a descriptive sense to denote a situation in which – according to van der Poel – the full benefits, risks, and ethical issues of a technology become evident only after its widespread adoption95.”

• cites [95] van de Poel, I. An ethical framework for evaluating experimental technology. Science and Engineering Ethics 22, 667–686 (2016). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A review protocol focusing on practical applications and ethical considerations grounded in experience was designed by the authors and registered in PROSPERO.

“A review protocol focusing on practical applications and ethical considerations grounded in experience was designed by the authors and registered in the international prospective register of systematic reviews96.”

• cites [96] Ranisch, R. & Haltaufderheide, J. Ethics of chatgpt: A systemic review of large language models in healthcare and medicine. PROSPERO CRD42023431326. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023431326.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Ethical examination of LLMs in healthcare is still nascent and struggles to keep pace with rapid technical advancements.

“Ethical examination of LLMs in healthcare is still nascent and struggles to keep pace with rapid technical advancements.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Categories were iteratively refined and aggregated through additional coding until saturation was reached.

“These categories were, then, iteratively refined and aggregated through additional coding until saturation was reached.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The underrepresentation of non-Western perspectives might impact the scope of ethical issues discussed and how certain issues are addressed and evaluated.

“This might have an impact on the scope of ethical issues discussed as well as on how certain issues are addressed and evaluated.”

May combine two claims.

A significant portion of the source material originated from preprint servers and did not undergo rigorous peer review, which can lead to limitations in quality and generalisability.

“A significant portion of the source material originated from preprint servers and did not undergo rigorous peer review, which can lead to limitations in quality and generalisability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors critically engage with the findings during reporting to appraise comprehensiveness and validity of the extracted information pieces.

“In addition, we critically engage with the findings during reporting to appraise comprehensiveness and validity of the extracted information pieces.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The issues of medical education, academic writing, authorship and plagiarism are affected by the use of LLMs in significant ways.

“While we recognize that these issues are affected by the use of LLMs in significant ways6,98,99 these, challenges are not specific to health-related applications.”

• cites [6] Clusmann, J. et al. The future landscape of large language models in medicine. Communications Medicine 3, 141 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [98] Abd-alrazaq, A. et al. Large language models in medical education: Opportunities, challenges, and future directions. JMIR Medical Education 9, e48291 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [99] Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D. & Smith, A. Generating scholarly content with chatgpt: Ethical challenges for medical publishing. The Lancet Digital Health 5, e105–e106 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Results were iteratively discussed in three joint coding sessions.

“Results were iteratively discussed in three joint coding sessions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The effects of misinformation are particularly severe in healthcare, where the outcome could be dire.

“The effects of such misinformation are particularly severe in healthcare, where the outcome could be dire.”

Probably not a substantial claim: may not pass S2 (checkable).

There are inherent and contextual limitations and benefits associated with LLMs that warrant consideration.

“There are inherent and contextual limitations and benefits associated with LLMs that warrant consideration.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The social experiment perspective acknowledges the inherent uncertainties associated with the deployment of LLMs in medicine and healthcare due to their novelty, complexity, and opacity.

“This perspective acknowledges the inherent uncertainties associated with the deployment of LLMs in medicine and healthcare due to their novelty, complexity, and opacity.”

Classified as probably not a substantial claim.

The inclination of LLMs to output erroneous information underscores the need for human oversight and continual validation of machine-generated output.

“The inclination of LLMs to output erroneous information underscores the need for human oversight and continual validation of machine-generated output, as our dataset demonstrates.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Framing current developments as social experiment reinforces the need to establish and respect ethical limits, especially within the healthcare domain, where professional duties and responsibilities towards patients are foundational.

“Furthermore, framing the current developments as social experiment also reinforces the need to establish and respect ethical limits – especially within the healthcare domain, where professional duties and responsibilities towards patients are foundational.”

May combine two claims.

The profit-driven nature of the field can lead to concentrations of power among a limited number of companies and a lack of transparency.

“The profit-driven nature of the field can lead to concentrations of power among a limited number of companies and a lack of transparency.”

• cites [24] Li, H. et al. Ethics of large language models in medicine and medical research. The Lancet Digital Health 5, e333–e335 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [36] Ali, H., Qadir, J. & Shah, Z. Chatgpt and large language models (llms) in healthcare: Opportunities and risks. Preprint at https://doi.org/10.36227/techrxiv.22579852.v2 (2023). sourceNot checked yet

  Not checked in the free run.

• cites [43] Harskamp, R. E. & de Clercq, L. Performance of chatgpt as an ai-assisted decision support tool in medicine: A proof-of-concept study for interpreting symptoms and management of common cardiac conditions (amstelheart-2). Preprint at https://doi.org/10.1101/2023.03.25.23285475 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Both authors independently coded 10% of the material to develop and refine the coding scheme.

“Both authors independently coded 10% of the material to develop and refine the coding scheme in more detail.”

Classified as probably not a substantial claim. May combine two claims.

LLMs stand to significantly bolster healthcare quality by mitigating information loss and enhancing medical information accessibility.

“Moreover, by mitigating information loss and enhancing medical information accessibility, LLMs stand to significantly bolster healthcare quality.”

May combine two claims.

Power concentration means pricing control lies with LLM companies, with revenues predominantly directed towards them.

“Power concentration also means pricing control lies with LLM companies, with revenues predominantly directed towards them44.”

• cites [44] Hosseini, M. et al. An exploratory survey about using chatgpt in education, healthcare, and research. Preprint at https://doi.org/10.1101/2023.03.31.23287979 (2023). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The selection of training data and knowledge bases typically encompasses knowledge from well-funded, English-speaking countries and significantly underrepresents knowledge from other regions.

“These questions are also mirrored in the selection of training data and knowledge bases24 which typically encompass knowledge from well-funded, English speaking countries and, thus, significantly underrepresents knowledge from other regions.”

• cites [24] Li, H. et al. Ethics of large language models in medicine and medical research. The Lancet Digital Health 5, e333–e335 (2023). sourceNot checked yet

  Not checked in the free run.