CoChat
Sign Up
Cited by 192Topic ModelingArtificial Intelligence ApplicationsNatural Language Processing Techniques

Decoding ChatGPT: A taxonomy of existing research, current challenges, and possible future directions

Open access
Sohail, Shahab Saquib, Farhat, Faiza, Himeur, Yassine, Nadeem, Mohammad, Madsen, Dag Øivind, Singh, Yashbir, Atalla, Shadi, Mansoor, Wathiq·2023·Journal of King Saud University - Computer and Information Sciences
DOI 10.1016/j.jksuci.2023.101675
Open this paper in CoChat — free account

Abstract

Chat Generative Pre-trained Transformer (ChatGPT) has gained significant interest and attention since its launch in November 2022. It has shown impressive performance in various domains, including passing exams and creative writing. However, challenges and concerns related to biases and trust persist. In this work, we present a comprehensive review of over 100 Scopus-indexed publications on ChatGPT, aiming to provide a taxonomy of ChatGPT research and explore its applications. We critically analyze the existing literature, identifying common approaches employed in the studies. Additionally, we investigate diverse application areas where ChatGPT has found utility, such as healthcare, marketing and financial services, software engineering, academic and scientific writing, research and education, environmental science, and natural language processing. Through examining these applications, we gain valuable insights into the potential of ChatGPT in addressing real-world challenges. We also discuss crucial issues related to ChatGPT, including biases and trustworthiness, emphasizing the need for further research and development in these areas. Furthermore, we identify potential future directions for ChatGPT research, proposing solutions to current challenges and speculating on expected advancements. By fully leveraging the capabilities of ChatGPT, we can unlock its potential across various domains, leading to advancements in conversational AI and transformative impacts in society.

Verified 2026-10-06 · 6 references not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

This review synthesizes 109 Scopus-indexed ChatGPT publications up to March 25, 2023, proposing a taxonomy across healthcare, marketing/finance, software engineering, academic/scientific writing, research/education, environmental science, and NLP. It identifies intrinsic and usage-related challenges—hallucination, bias, misinformation, copyright, over-reliance—and future directions in conversational ability, personalization, multimodal design, and trustworthiness.

Key findings

  • Of 109 included articles from 349 authors across 53 nations, medicine was the largest subject area at 23%, followed by social sciences at 20% and computer science at 11%; the United States contributed 33 publications, the United Kingdom 10, and Australia and China 9 each.
  • The corpus comprised 68 evaluations, 39 prediction articles, and 10 reviews; most evaluations focused on scientific writing (30 publications), followed by education and research ability.
  • Applications span healthcare, marketing/financial services, software engineering, academic/scientific writing, research/education, environmental science, and NLP; for example, ChatGPT’s bug-fixing was comparable to CoCoNut and Codex and superior to standard approaches, and in one abstract-generation experiment 68% of generated abstracts were correctly identified while 14% of real abstracts were mistakenly identified as generated.
  • Challenges are categorized as intrinsic (hallucination, biased content, not real-time, misinformation, inexplicability) and usage-related (ethical issues, copyright violation, over-reliance); future directions include improving conversational capabilities, personalization, multimodal design, and trustworthiness.

Methods

  • Systematic literature review following Kitchenham (2004), searching Scopus with inclusion criteria: English, peer-reviewed, keyword “chatgpt” or “chat-gpt” in abstract/title, discusses ChatGPT, and published by March 25, 2023; exclusions included short papers, duplicates, and earlier versions with errata.
  • A total of 109 articles were included and analyzed by subject area, country/collaboration, publication trends, taxonomy, applications, challenges, and future directions.

Limitations

  • The paper states ChatGPT’s own limitations—intrinsic issues such as hallucination, biased content, not being real-time, misinformation, and inexplicability, plus usage-related issues such as ethical concerns, copyright violation, and over-reliance—and notes that early ChatGPT research still faces limitations.
  • The paper does not explicitly label limitations of the review itself; evident from the methodology, the search is limited to Scopus, English, peer-reviewed publications up to March 25, 2023, excluding short papers, duplicates, and errata, making coverage database-specific and time-bound.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.jksuci.2023.101675

Cites

201 of 207 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 117 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Medicine had the highest representation among the included publications, accounting for 23% of the total.

    “Notably, the field of medicine had the highest representation, accounting for 23% of the total publications.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.
  • Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Computer science accounted for 11% of the included publications.

    “This was followed by social sciences (20%) and computer science (11%) (see Figure 1).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.
  • Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    The United Kingdom had 10 publications in the corpus.

    “It was followed by the United Kingdom with 10 publications, and Australia and China with 9 publications each.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.
  • Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Social sciences accounted for 20% of the included publications.

    “This was followed by social sciences (20%) and computer science (11%) (see Figure 1).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.
  • Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    The United States ranked first as the country of origin for published articles, with 33 publications in the corpus.

    “The United States ranked first as the country of origin for published articles, with a total of 33 publications in the corpus.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.
  • Classified as probably not a substantial claim. 80
  • May combine two claims. 39

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Multidisciplinary subjects made up 8% of the total corpus.

    “Multidisciplinary subjects and health professionals made up 8% and 7% of the total corpus, respectively.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • May combine two claims.

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    The 109 included articles involved contributions from 349 authors representing 53 different nations.

    “These articles involved contributions from 349 authors representing 53 different nations, indicating a wide range of international participation in the literature on ChatGPT.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Health professionals made up 7% of the total corpus.

    “Multidisciplinary subjects and health professionals made up 8% and 7% of the total corpus, respectively.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    Australia had 9 publications in the corpus.

    “It was followed by the United Kingdom with 10 publications, and Australia and China with 9 publications each.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingreasoning
    § IV. RECENT TREND OF PUBLICATIONS RELATED TO CHATGPT · p. 4

    Many authors have expressed concerns regarding the ethical and scientific implications of ChatGPT's features and their potential negative effects on scientific academia.

    “As a result, many authors have expressed concerns regarding the ethical and scientific implications of ChatGPT's features and their potential negative effects on scientific academia.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation
    § SUBJECT AREA · p. 5

    Some authors have asked questions based on their extensive understanding of various fields, including parasitology, clinical diagnosis, environmental sciences, and public health.

    “Some have also asked questions based on their extensive understanding of various fields, including parasitology Huh (2023 b ), ˇ Slapeta (2023), clinical diagnosis Hirosawa et al. (2023), environmental sciences Rillig et al. (2023), public health Biswas (2023 e ), etc.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Huh (2023 b )

      Not checked in the free run.

    • cites ˇ Slapeta (2023)

      Not checked in the free run.

    • cites Hirosawa et al. (2023)

      Not checked in the free run.

    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given
    § IV. RECENT TREND OF PUBLICATIONS RELATED TO CHATGPT · p. 4

    Several articles have explored how ChatGPT could impact learners and educators, as well as the ethical considerations that must be taken into account when using it.

    “Meanwhile, several articles have explored how ChatGPT could impact learners and educators, as well as the ethical considerations that must be taken into account when using it.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § III. DIVERSITY OF PUBLICATION ON CHATGPT · p. 4

    Iraqi Journal for Computer Science and Mathematics published 2 documents.

    “Iraqi Journal for Computer Science and Mathematics has also published 2 documents as well.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § IV. RECENT TREND OF PUBLICATIONS RELATED TO CHATGPT · p. 4

    The majority of publications center around either ChatGPT's feature assessment as a scientific writing assistant or predictions about the future of the education system.

    “Furthermore, it has been noted that the majority of publications center around either ChatGPT's feature assessment as a scientific writing assistant or predictions about the future of the education system.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation
    § IV. RECENT TREND OF PUBLICATIONS RELATED TO CHATGPT · p. 4

    The different assessments included those for bias, for study, for mental health, and for public health.

    “The different assessments included those for bias Wang (2023 b ), Wang, Miao, Li, Wang & Lin (2023), for study Cooper (2023), for mental health Prada et al. (2023), for public health Jungwirth & Haluza (2023 a ); Biswas (2023 e ), etc.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Wang (2023 b )

      Not checked in the free run.

    • cites Cooper (2023) source

      Not checked in the free run.

    • cites Prada et al. (2023)

      Not checked in the free run.

    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § SUBJECT AREA · p. 5

    Clinicians and medical professionals probed ChatGPT with hypothetical or real-world patient symptoms to determine whether or not it would aid in the diagnosis or treating patients.

    “Clinicians and medical professionals probed ChatGPT with hypothetical or real-world patient symptoms to determine whether or not it would aid in the diagnosis or treating patients.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § III. DIVERSITY OF PUBLICATION ON CHATGPT · p. 4

    Accountability in Research published 4 articles.

    “Next, Accountability in Research has published 4 articles.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    The United Kingdom had 18 collaborations.

    “Switzerland ranked second in terms of collaboration, with 20 collaborative countries, followed by Australia with 19 collaborations, and the United Kingdom with 18 collaborations (see Figure 2).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • May combine two claims.

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    The United States had the most extensive collaboration network, collaborating with 24 different countries, accounting for over 18% of the total corpus.

    “In terms of collaborations, the United States had the most extensive network, collaborating with 24 different countries, accounting for over 18% of the total corpus.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning
    § IV. RECENT TREND OF PUBLICATIONS RELATED TO CHATGPT · p. 4

    The interpretations in most articles can be broadly categorized into two types: those that predict the future of the topic in the context of ChatGPT and those that assess ChatGPT's features and their potential impacts.

    “These interpretations can be broadly categorized into two types: those that predict the future of the topic in the context of ChatGPT and those that assess ChatGPT's features and their potential impacts.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § II. SURVEY METHODOLOGY · p. 3

    China had 9 publications in the corpus.

    “It was followed by the United Kingdom with 10 publications, and Australia and China with 9 publications each.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § SUBJECT AREA · p. 6

    Ethical considerations are discussed in the majority of the papers either in detail or briefly.

    “Ethical considerations are discussed in the majority of the papers either in detail or briefly.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.
    • Rests on an assumption: The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given
    § VIII. CONCLUSION · p. 22

    Early ChatGPT research still faces some limitations.

    “Despite its potential, the early ChatGPT researches still face some limitations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § VIII. CONCLUSION · p. 22

    Ethical concerns exist in ChatGPT research.

    “Additionally, we have identified and discussed ethical concerns.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given
    § VIII. CONCLUSION · p. 22

    The issues facing early ChatGPT research can be classified as intrinsic and usage-centric.

    “We have identified certain issues that may need to be addressed, which are classified as intrinsic and usage-centric.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § B. Progress of ChatGPT research · p. 2

    Subsequent versions GPT-2, GPT-3, and GPT-3.5 significantly augmented the number of parameters, facilitating more accurate and human-like responses.

    “Subsequent versions, such as GPT-2, GPT3, and the latest release, GPT-3.5, have experienced notable advancements by significantly augmenting the number of parameters. This augmentation has facilitated the generation of responses that are even more accurate and human-like.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    ChatGPT utilizes a transformer architecture consisting of encoder-decoder layers that collaborate to process and generate natural language text.

    “ChatGPT utilizes a transformer architecture consisting of encoder-decoder layers that collaborate to process and generate natural language text Chen, Yu, Yang, Guo, Bian & Wu (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chen, Yu, Yang, Guo, Bian & Wu (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    The transformer architecture of ChatGPT consists of two main components: the encoder and the decoder.

    “The transformer architecture of ChatGPT consists of two main components: the encoder and the decoder Budzianowski & Vulic (2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Budzianowski & Vulic (2019)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § V. APPLICATIONS OF CHATGPT · p. 6

    ChatGPT is turning out to be a Code Assistant and Debugger, helping software developers by providing code suggestions, debugging assistance, and answering programming-related questions.

    “ChatGPT is turning out to be a Code Assistant and Debugger. It helps software developers by providing code suggestions, debugging assistance, and answering programming-related questions Aljanabi, Ghazi, Ali, Abed et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aljanabi, Ghazi, Ali, Abed et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § B. Progress of ChatGPT research · p. 2

    The initial iteration of ChatGPT, GPT-1, had 117 million parameters and was trained on a substantial corpus of text data.

    “The initial iteration of ChatGPT referred to as GPT-1, was equipped with 117 million parameters and underwent training on a substantial corpus of text data Ernst & Bavota (2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ernst & Bavota (2022) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    ChatGPT incorporates reinforcement learning from human feedback (RLHF), allowing it to learn and understand human preferences through extended dialogues.

    “To continually improve the reliability and accuracy of the model, ChatGPT incorporates reinforcement learning from human feedback (RLHF), allowing it to learn and understand human preferences through extended dialogues Christiano et al. (2017), Stiennon et al. (2020).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Christiano et al. (2017)

      Not checked in the free run.

    • cites Stiennon et al. (2020)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A. Architecture of ChatGPT · p. 1

    ChatGPT has the ability to generate new responses or utilize pre-existing ones.

    “ChatGPT has the ability to generate new responses or utilize pre-existing ones Salvagno et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Salvagno et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    Researchers have highlighted potential harms associated with ChatGPT, including the propagation of stereotypes, biased responses, and dissemination of misleading information.

    “Researchers have been exploring and highlighting the potential harms associated with ChatGPT, including the propagation of stereotypes, biased responses, and dissemination of misleading information Liang et al. (2021), Nadeem et al. (2020).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Liang et al. (2021)

      Not checked in the free run.

    • cites Nadeem et al. (2020)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § V. APPLICATIONS OF CHATGPT · p. 6

    In marketing, ChatGPT works as a conversational chatbot to handle customer inquiries, provide product recommendations, and assist with order tracking.

    “In marketing, ChatGPT works as a conversational chatbot to handle customer inquiries, provide product recommendations, and assist with order tracking.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cascella et al. (2023 a )

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    The attention mechanism allows the model to selectively focus on different parts of the input text while generating the output.

    “An essential feature of the transformer architecture is the attention mechanism, which allows the model to selectively focus on different parts of the input text while generating the output Jain et al. (n.d.).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jain et al. (n.d.)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Architecture of ChatGPT · p. 2

    There are concerns that ChatGPT-generated content may infringe upon copyright and intellectual property rights.

    “Moreover, there are concerns that ChatGPT-generated content may infringe upon copyright and intellectual property rights Deng & Lin (2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Deng & Lin (2022)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § III. DIVERSITY OF PUBLICATION ON CHATGPT · p. 4

    Following its launch, ChatGPT quickly gained a lot of popularity in a variety of disciplines, including academia and science.

    “Following its launch, ChatGPT quickly gained a lot of popularity in a variety of disciplines Cox (2023), Dwivedi et al. (2023), Tlili et al. (2023 a ), including academia Chen (2023 b ) and science Morreel et al. (2023).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Cox (2023)

      Not checked in the free run.

    • cites Dwivedi et al. (2023) source

      Not checked in the free run.

    • cites Tlili et al. (2023 a )

      Not checked in the free run.

    • Rests on an assumption: The rapid dissemination of ChatGPT research resulted in a diverse research landscape due to its wide publicity and acceptance at different levels in daily life.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT occasionally noticed and corrected errors in repeated questions.

    “although it occasionally noticed and corrected them.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT had deficits in consistency.

    “ChatGPT also had deficits in situational awareness, inference, and consistency”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT did not repeat the same advice in a single response.

    “and it did not repeat the same advice in a single response.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT's responses were coherent.

    “ChatGPT's responses were coherent”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT's answers included a summary of its understanding of the scenario and question, management options, and disclaimers that reflected its information sources, similar to the format of patient information websites.

    “The model's answers included a summary of its understanding of the scenario and question, management options, and disclaimers that reflected its information sources, which were similar to the format of patient information websites.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT had deficits in situational awareness.

    “ChatGPT also had deficits in situational awareness, inference, and consistency”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § C. Software Engineering · p. 13

    This feature can enhance the coding experience for less-skilled programmers, making it more user-friendly and intuitive.

    “This feature can enhance the coding experience for less-skilled programmers, making it more user-friendly and intuitive Aljanabi, Ghazi, Ali, Abed et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aljanabi, Ghazi, Ali, Abed et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    Source control was often disregarded by ChatGPT.

    “source control was often disregarded.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § II. SURVEY METHODOLOGY · p. 3

    The literature review adopted the methodology proposed in Kitchenham (2004).

    “To conduct our literature review, we adopted the methodology proposed in Kitchenham (2004).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kitchenham (2004)

      Not checked in the free run.

    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT's antimicrobial spectra and regimens were appropriate.

    “while its antimicrobial spectra and regimens were appropriate”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT had deficits in inference.

    “ChatGPT also had deficits in situational awareness, inference, and consistency”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § II. SURVEY METHODOLOGY · p. 3

    The articles were screened by going through the abstract and assessing its relevance to the title under consideration.

    “We have also performed a screening of the articles by going through the abstract and assessing its relevance to the title under consideration.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.

  • May combine two claims.

    Premisecitation
    § III. DIVERSITY OF PUBLICATION ON CHATGPT · p. 4

    Research scholars have been using ChatGPT in a variety of ways, including writing scientific literature, gathering data, drafting abstracts for research papers, and looking for medical diagnostic advice.

    “Research scholars have been using ChatGPT in a variety of ways, including writing scientific literature, gathering data, drafting abstracts for research papers Else (2023), and looking for medical diagnostic advice Hirosawa et al. (2023).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Else (2023) source

      Not checked in the free run.

    • cites Hirosawa et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT's spelling and grammar were appropriate.

    “their spelling and grammar were appropriate”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Healthcare · p. 6

    In telemedicine, the available information is typically limited to language only, making it a potential candidate for interventions using LLM.

    “Clinicians often rely on complex information to make decisions, but in telemedicine, the available information is typically limited to language only, making it a potential candidate for interventions using LLM Roosan et al. (2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roosan et al. (2016)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    The information provided by ChatGPT was consistent.

    “The information provided by ChatGPT was consistent”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Software Engineering · p. 13

    ChatGPT has a significant benefit in coding, as it can process human inputs, allowing software developers to supply code snippets or commands in a conversational way rather than providing specific keywords or phrases.

    “ChatGPT has a significant benefit in coding, as it can process human inputs, allowing software developers to supply code snippets or commands in a conversational way rather than providing specific keywords or phrases.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aljanabi, Ghazi, Ali, Abed et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Healthcare · p. 6

    ChatGPT often repeated questions verbatim, including any errors.

    “ChatGPT often repeated questions verbatim, including any errors”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Howard et al. (2023)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § II. SURVEY METHODOLOGY · p. 3

    Inclusion and exclusion criteria were created to filter the retrieved articles from the Scopus database.

    “After running search queries, inclusion (IC) and exclusion (EC) criteria were created to filter out the retrieved articles from the Scopus database.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The Scopus database and the search queries used capture the relevant literature on ChatGPT.

  • May combine two claims.

    Premisenone given
    § C. Research Questions and Prime Contributions · p. 3

    Existing review works on Large Language Models and AI Generated Content are very broad and do not take into consideration the specificities related to ChatGPT.

    “There are existing review works on Large Language Models (LLM) Melis et al. (2017), Chen et al. (2021) and AI Generated Content (AIGC) Cao et al. (2023) but they are very broad and do not take into consideration the specificities related to ChatGPT.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Melis et al. (2017)

      Not checked in the free run.

    • cites Chen et al. (2021)

      Not checked in the free run.

    • cites Cao et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Marketing and financial services · p. 9

    ChatGPT has the potential to contribute to the advancement of gastroenterology by generating high-quality research questions.

    “to contribute to the advance- ment of gastroenterology by generating high-quality re- search questions”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lahat et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § B. Marketing and financial services · p. 11

    NLP technologies are being used to create conversational chatbots to mimic the conversational proficiency of humans.

    “NLP technologies are being used to create conversational chatbots to mimic the con- versational proficiency of hu- mans.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § G. Natural Language Processing · p. 14

    Task-specific models like RoBERTa outperformed ChatGPT in specialized downstream NLP tasks, though ChatGPT demonstrated satisfactory performance compared to baseline models in a variety of tasks.

    “The findings revealed that task-specific models like RoBERTa outperformed ChatGPT, particularly in specialized downstream tasks. However, ChatGPT still demonstrated satisfactory performance compared to baseline models in a variety of tasks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Amin et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § B. Marketing and financial services · p. 12

    ChatGPT's information can be updated and corrected, but it may not always reflect the latest knowledge.

    “Use ChatGPT's information can be updated and cor- rected, but it may not always reflect the latest knowledge.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Usage-related · p. 15

    Few publishers have recognized the use of ChatGPT for academic writing to promote its ethical usage practices.

    “In fact, few publishers have recognized the use of ChatGPT for academic writing to promote its ethical usage practices Editorialge (Jan, 2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Editorialge (Jan, 2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § B. Marketing and financial services · p. 9

    ChatGPT has the potential to support decision-making in Japanese healthcare settings, but caution is needed due to performance improvements required.

    “ChatGPT potential to support decision-making in Japanese healthcare settings, but cau- tion is needed due to per- formance improvements re- quired.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § B. Usage-related · p. 15

    Unethical use includes deliberate generation of manipulated content that can promote misinformation and provoke violence, creating damage at an individual or organizational level.

    “Unethical use also includes the deliberate generation of manipulated content that can promote misinformation and provoke violence, creating damage at an individual or organizational level Elali & Rachid (2023), Cao et al. (2023).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Elali & Rachid (2023)

      Not checked in the free run.

    • cites Cao et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § VI. CHALLENGES AND ISSUES OF CHATGPT · p. 14

    Researchers have identified several issues regarding ChatGPT that can be broadly categorized into intrinsic limitations and usage-related concerns.

    “Researchers have identified several issues regarding ChatGPT, which can be broadly categorized into two groups: intrinsic limitations and usage-related concerns.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § G. Natural Language Processing · p. 14

    Larger models like ChatGPT can perform NLP tasks without the need for specific data adaptation.

    “Qin et al. (2023) argued that larger models like ChatGPT can perform NLP tasks without the need for specific data adaptation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Qin et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Marketing and financial services · p. 10

    Prolonged questioning could be beneficial in enhancing and improving both the kinds of human reasoning and argumentation imitated by current LLMs.

    “Prolonged questioning could be beneficial in enhanc- ing and improving both the kinds of human reasoning and argumentation imitated by current LLMs.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cox & Tzoc (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § B. Marketing and financial services · p. 12

    ChatGPT can act as a user assistance tool, providing guidance, explanations, and educational resources to users interacting with robots.

    “ChatGPT can act as a user assistance tool, provid- ing guidance, explanations, and educational resources to users interacting with robots.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Marketing and financial services · p. 10

    The knowledge and ability to analyze results of parasitology test by ChatGPT were not yet on par with those of Korean medical students.

    “The knowledge and ability to analyze results of para- sitology test by ChatGPT were not yet on par with those of Korean medical stu- dents”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Huh (2023 c , a )

      Not checked in the free run.

    • cites ˇ Slapeta (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Usage-related · p. 15

    There are concerns that ChatGPT-generated content may lead to violations of copyright and intellectual property rights.

    “There are also concerns that ChatGPT-generated content may lead to violations of copyright and intellectual property rights Sallam (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sallam (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § B. Marketing and financial services · p. 9

    ChatGPT's knowledge and interpretation in the Chinese National Medical Licensing Examination (NMLE) are below medical students' level.

    “ChatGPT's knowl- edge and interpretation in the Chinese Chinese Na- tional Medical Licensing Ex- amination (NMLE) are be- low medical students' level”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Marketing and financial services · p. 9

    The correct diagnoses within ten differential-diagnosis lists generated by ChatGPT was more than 90%.

    “The correct diagnoses within ten differential-diagnosis lists generated by ChatGPT was more than 90%.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hirosawa et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § VI. CHALLENGES AND ISSUES OF CHATGPT · p. 14

    These limitations make the usage and deployment of ChatGPT difficult in real-world scenarios.

    “These limitations make the usage and deployment of ChatGPT difficult in real-world scenarios.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § B. Marketing and financial services · p. 11

    ChatGPT can recognize when a word, phrase, or sentence has multiple possible interpretations, leading to potential confusion or miscommunication.

    “It can recognize when a word, phrase, or sentence has mul- tiple possible interpretations, leading to potential confu- sion or miscommunication.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § E. Research and Education · p. 13

    ChatGPT can be used to solve both technical problems, such as engineering and computer programming, and non-technical problems, such as language and literature.

    “Rahman & Watanobe (2023) has experimentally shown that ChatGPT can be used to solve both technical problems, such as engineering and computer programming, and non-technical problems, such as language and literature.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rahman & Watanobe (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § B. Marketing and financial services · p. 12

    ChatGPT can aid in diagnosing issues and providing troubleshooting guidance for robotic systems.

    “ChatGPT can aid in diag- nosing issues and providing troubleshooting guidance for robotic systems.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given
    § A. Intrinsic · p. 14

    Intrinsic issues include five major limitations: hallucination, biased content, not real-time, misinformation, and inexplicability.

    “It includes five major limitations, namely, hallucination, biased content, not real-time, misinformation and inexplicability.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § C. Ethical concerns · p. 15

    Detection of artificially generated fake text and meaningless information is not a new challenge.

    “It is worth noting that the detection of artificially generated fake text and meaningless information is not a new challenge Amancio (2015), Jawahar et al. (2020).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Amancio (2015)

      Not checked in the free run.

    • cites Jawahar et al. (2020)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § B. Marketing and financial services · p. 12

    ChatGPT can help establish shared understandings, allocate tasks, and assist in real-time communication, thereby improving the overall efficiency and performance of robotic teams.

    “ChatGPT can help establish shared un- derstandings, allocate tasks, and assist in real-time com- munication, thereby improv- ing the overall efficiency and performance of robotic teams.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § G. Natural Language Processing · p. 14

    The application of ChatGPT is rapidly expanding across various domains, and this growing trend is expected to continue in the foreseeable future.

    “Additionally, the application of ChatGPT is rapidly expanding across various domains Mijwil et al. (2023), Aljanabi, Ghazi, Ali, Abed et al. (2023), indicating a growing trend that is expected to continue in the foreseeable future.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Mijwil et al. (2023)

      Not checked in the free run.

    • cites Aljanabi, Ghazi, Ali, Abed et al. (2023)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § A. Intrinsic · p. 14

    Hallucination and misinformation can lead to counterfactual or meaningless responses, which can seriously threaten the reliability of generated content.

    “Both issues can lead to the creation of counterfactual or meaningless responses, which can seriously threaten the reliability of the generated content.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § D. Academic and scientific writing · p. 13

    In an experiment using 50 scientific journal abstracts, 68% of ChatGPT-generated abstracts were correctly identified as generated, and 14% of real abstracts were mistakenly identified as ChatGPT-generated.

    “68% of the generated abstracts were correctly identified as such, and 14% of the real abstracts were mistakenly identified as generated by ChatGPT.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gao, Tong, Wu, Chen, Zhu & Wang (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § B. Usage-related · p. 15

    Usage-related issues include unethical usage of the tool, copyright-infringed content, and overreliance on ChatGPT.

    “The category of usage-related issues includes unethical usage of the tool, copyright-infringed content, and overreliance on ChatGPT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § C. Ethical concerns · p. 15

    Identifying fake text falls within a well-established domain, with many state-of-the-art algorithms and techniques available to tag fake text with significant accuracy.

    “The latter process, which is fake text identification, falls within a well-established domain, with many state-of-the-art algorithms and techniques available to tag fake text with significant accuracy Zhou & Zafarani (2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou & Zafarani (2020) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § G. Natural Language Processing · p. 14

    ChatGPT excelled in NLP tasks requiring reasoning skills, including arithmetic reasoning, while facing challenges in tasks such as sequence tagging.

    “The evaluation results indicated that ChatGPT excelled in tasks that required reasoning skills, including arithmetic reasoning while facing challenges in tasks such as sequence tagging.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Qin et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § B. Marketing and financial services · p. 11

    ChatGPT helps identify instances of linguistic ambiguity in text or speech.

    “ChatGPT helps identify in- stances of linguistic ambi- guity in text or speech.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § B. Marketing and financial services · p. 8

    ChatGPT can help assess the risk level of individuals experiencing mental health crises by analyzing conversations.

    “ChatGPT can help assess the risk level of individuals experiencing mental health crises by analyzing conversations.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § E. Research and Education · p. 13

    ChatGPT performs well in structured tasks such as translating code and explaining well-known concepts but struggles with nuanced tasks such as explaining less familiar terms and creating code from scratch.

    “ChatGPT performs well in structured tasks such as translating code and explaining wellknown concepts but struggles with nuanced tasks such as explaining less familiar terms and creating code from scratch.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Megahed et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § G. Natural Language Processing · p. 14

    Findings from NLP studies can be extrapolated to other related areas, including assessment of news article accuracy, especially detection of fake news.

    “Given the similarities among various NLP tasks, the findings from the aforementioned studies can be extrapolated to other related areas, including the assessment of news article accuracy, especially in the detection of fake news.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § G. Natural Language Processing · p. 14

    GPT models could generate translation outputs that were highly competitive for languages with abundant resources, but exhibited limitations for low-resource languages.

    “The results demonstrated that GPT models could generate translation outputs that were highly competitive for languages with abundant resources. However, for low-resource languages, the current state of GPT models exhibited limitations, indicating the need for further improvements.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hendy et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § B. Usage-related · p. 15

    There is a fear of overreliance on ChatGPT, which may make humans lazy and apathetic and make them always rely on generated information.

    “Lastly, there is a fear of overreliance on the tool, which may make humans lazy and apathetic and make them always rely on the generated information Sallam et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sallam et al. (2023) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Ethical concerns · p. 16

    Major publishers like Elsevier have stated that AI cannot be listed as an author and that its use must be properly acknowledged.

    “Major publishers like Elsevier, who publish the Lancet family of journals, have already stated that AI cannot be listed as an author and that its use must be properly acknowledged Sallam et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sallam et al. (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § C. Ethical concerns · p. 16

    ChatGPT's availability, ease of use, and multi-language capabilities could significantly boost scholarly output, thereby democratizing knowledge dissemination.

    “Our view is that ChatGPT's availability, ease of use, and multi-language capabilities could significantly boost scholarly output, thereby democratizing knowledge dissemination.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § A. Improving Conversational Capabilities · p. 16

    The quality of training data is crucial, and data used for training should be accurate, relevant, and diverse enough to represent the full range of language and speech patterns.

    “The quality of the data is also crucial, and it is important to ensure that the data used for training is accurate, relevant, and diverse enough to represent the full range of language and speech patterns.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § C. Ethical concerns · p. 15

    Differentiating between human-generated and ChatGPT-generated reviews is more challenging for a machine-learning model when using rephrased text.

    “By comparing their model with an approach based on perplexity scores, they found that differentiating between human-generated and ChatGPT-generated reviews is more challenging for the machine-learning model when using rephrased text.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mitrovic et al. (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § A. Improving Conversational Capabilities · p. 18

    Adding feelings to AI language models runs the risk of unintended consequences and risks like bias and discrimination, manipulation, privacy evasion, and inappropriate or offensive responses.

    “As a result, adding feelings to AI language models runs the risk of unintended consequences and risks like bias and discrimination, manipulation, privacy evasion, and inappropriate or offensive responses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § A. Improving Conversational Capabilities · p. 16

    Other factors such as model architecture, training methods, and hyperparameters can also impact the model's performance.

    “Additionally, other factors such as the model architecture, training methods, and hyperparameters can also impact the model's performance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § C. Ethical concerns · p. 15

    Specifically identifying texts generated by ChatGPT is relatively new, and researchers are actively working to address this issue.

    “However, the former step, which focuses on specifically identifying texts generated by ChatGPT, is relatively new, and researchers are actively working to address this issue Dugan et al. (2022), Mitchell et al. (2023), Curtis (2023), Mitrovic et al. (2023).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Dugan et al. (2022)

      Not checked in the free run.

    • cites Mitchell et al. (2023)

      Not checked in the free run.

    • cites Curtis (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § A. Improving Conversational Capabilities · p. 17

    Incorporating human feedback is an effective way to improve the performance of ChatGPT.

    “Incorporating human feedback is an effective way to improve the performance of ChatGPT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § A. Improving Conversational Capabilities · p. 18

    Effective conversational AI systems like ChatGPT require a combination of various techniques and considerations to deliver a satisfying and human-like conversational experience.

    “Arguably, effective conversational AI systems like ChatGPT require a combination of various techniques and considerations to deliver a satisfying and human-like conversational experience, for example, considering factors such as context understanding, coherence, and response relevance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § A. Improving Conversational Capabilities · p. 17

    Fine-tuning allows ChatGPT to learn and understand the nuances of natural language conversations, enabling it to generate more human-like responses.

    “In the case of ChatGPT, fine-tuning allows the model to learn and understand the nuances of natural language conversations, enabling it to generate more human-like responses.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Bakker et al. (2022)

      Not checked in the free run.

    • cites Himeur et al. (2022 b )

      Not checked in the free run.

    • cites Dwivedi et al. (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § A. Improving Conversational Capabilities · p. 16

    Simply increasing the volume and variety of data alone may not always result in improved performance.

    “However, it is also important to note that simply increasing the volume and variety of data alone may not always result in improved performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationnone given
    § VIII. CONCLUSION · p. 22

    Improving Conversational Capabilities, Personalization, Multimodal design, and Trustworthiness are key areas that hold significant promise for the future of ChatGPT.

    “Improving Conversational Capabilities, Personalization, Multimodal design, and Trustworthiness are some of the key areas that hold significant promise for the future of ChatGPT.”
    Quote in the paperNot a substantial claim
98 of 98

  • The paper relies on this without support.

    The Scopus database and the search queries used capture the relevant literature on ChatGPT.

    “After running search queries, inclusion (IC) and exclusion (EC) criteria were created to filter out the retrieved articles from the Scopus database.”

  • The paper relies on this without support.

    The rapid dissemination of ChatGPT research resulted in a diverse research landscape due to its wide publicity and acceptance at different levels in daily life.

    “The rapid dissemination of ChatGPT research resulted in a diverse research landscape due to its wide publicity and acceptance at different levels in our daily lives.”

  • The paper relies on this without support.

    The 109 articles included from Scopus are representative of the broader literature on ChatGPT.

    “After applying the specified criteria, a total of 109 articles were included in the analysis.”

  • The paper relies on this without support.

    ChatGPT has the capability to automatically generate responses by drawing information from numerous internet sources, often without requiring further input from the user.

    “ChatGPT has the capability to automatically generate responses by drawing information from numerous internet sources, often without requiring further input from the user.”

  • The paper relies on this without support.

    ChatGPT is unaware of copyright materials.

    “Since ChatGPT is unaware of copyright materials, verifying the generated content before using/publishing and tagging by humans in case of copyright violation might help to resolve this issue.”

  • The paper relies on this without support.

    Continuous training and updating of ChatGPT requires ongoing resources and infrastructure.

    “Additionally, it is important to note that continuous training and updating of ChatGPT requires ongoing resources and infrastructure to support it.”

  • The paper relies on this without support.

    Speech recognition technology enables ChatGPT to comprehend and recognize spoken words.

    “By processing aural input and translating it into text that software or other devices may use, speech recognition technology enables ChatGPT to comprehend and recognize spoken words.”

  • The paper relies on this without support.

    Video content analysis allows a multimodal ChatGPT to analyze and process video footage, detect objects, and track motions.

    “By incorporating video content analysis, a multimodal ChatGPT can analyze and process video footage, detect objects, and track motions within the video.”

  • The paper relies on this without support.

    Image captioning can be used to add descriptive or explanatory text to images.

    “Similarly, image captioning can be used to add descriptive or explanatory text to images, which can help to make them more accessible and informative for viewers.”

  • The paper relies on this without support.

    Audio captioning can help make audio content more accessible.

    “It would be beneficial for persons who are deaf or hard of hearing, as well as nonnative speakers, to add text subtitles to audio information or audio captioning because it can help make audio content more accessible.”

  • The paper relies on this without support.

    Video captioning can provide closed captions or subtitles during conversations.

    “Video captioning can be added to ChatGPT to provide closed captions or subtitles during conversations, making the video content more accessible.”

  • The paper relies on this without support.

    Video indexing allows users to search for specific content within a video using keywords or timestamps.

    “Additionally, video indexing allows users to search for specific content within a video using keywords or timestamps, making it easier to find relevant information quickly.”

  • The paper relies on this without support.

    ChatGPT is designed to generate responses based on the input it receives.

    “As an AI language model, ChatGPT is designed to generate responses based on the input it receives.”

  • The paper relies on this without support.

    ChatGPT does not have the ability to actively disclose information about itself or the data it uses to generate responses.

    “While it can provide information and insights, it does not have the ability to actively disclose information about itself or the data it uses to generate responses.”

  • The paper relies on this without support.

    ChatGPT may not always provide the most nuanced or complete answer to a question, and it may occasionally make mistakes or provide incorrect information.

    “As such, it may not always provide the most nuanced or complete answer to a question, and it may occasionally make mistakes or provide incorrect information.”

  • The paper relies on this without support.

    Transparent AI systems are simpler to comprehend and analyze, they encourage trust and make it more likely that the system will make ethical and legal choices.

    “Because transparent AI systems are simpler to comprehend and analyze, they encourage trust and make it more likely that the system will make ethical and legal choices.”

  • The paper relies on this without support.

    Humans can comprehend and validate an explainable AI's decision-making process, which is crucial for building confidence in AI systems.

    “As a result, humans can comprehend and validate the system's decision-making process, which is crucial for building confidence in AI systems.”

  • The paper relies on this without support.

    Human-Centered Design principles are usually applied in the research and development process of AI language models.

    “In the case of developing AI language models, including ChatGPT, Human-Centered Design principles are usually applied in the research and development process, such as conducting user research to understand the needs and behaviors of people who use language models, testing different design prototypes, and iterating on the design based on user feedback.”

  • The paper relies on this without support.

    ChatGPT can provide information and insights.

    “While it can provide information and insights, it does not have the ability to actively disclose information about itself or the data it uses to generate responses.”
19 of 19

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The majority of existing work on ChatGPT in academia consists of conceptual and opinion-based discussions and quantitative surveys aimed at specific groups.

    “The majority of existing work consists of conceptual and opinion-based discussions (e.g., Farrelly & Baker, 2023;Hsu & Ching., 2023;Sohail et al., 2023;Yu, 2023), along with quantitative surveys aimed at specific groups (e.g., Hammour et al., 2023;Tangadulrat et al., 2023).”

    From Widen the debate: What is the academic community’s perception on ChatGPT? (2024) · cites this paper as Sohail et al., 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Section roadmap misnumbers the applications section

    Minor

    The organizational roadmap assigns the applications discussion to Section 4, but the applications section is headed Section V. The subsequent sections are likewise numbered one higher than the roadmap states.

1 of 1

Cites

3 retracted

Flagged first, then by citations

Research works printed but not linked: 34

  1. Abdel-Messih, 2023, ‘Chatgpt in clinical toxicology’, JMIR, Med. Educ., 9
  2. Aljanabi, 2023, Chatgpt: Open possibilities, Iraqi J. Comput. Sci. Mathe., 4, 62
  3. Arif, T.B., Munaf, U., Ul-Haque, I., 2023b. The future of medical education and research: Is chatgpt a blessing or blight in disguise?. · DOI · Paper page
  4. Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P.S., Sun, L., 2023. A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt, arXiv preprint arXiv:2303.04226. · arXiv
  5. Cascella, 2023, Evaluating the feasibility of chatgpt in healthcare: An analysis of multiple clinical and research scenarios, J. Med. Syst., 47, 10.1007/s10916-023-01925-4 · DOI · Paper page
  6. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G. et al., 2021. Evaluating large language models trained on code, arXiv preprint arXiv:2107.03374. · arXiv
  7. Domnich, A., Anbarjafari, G., 2021. Responsible ai: Gender bias assessment in emotion recognition, ArXiv abs/2103.11436. · arXiv
  8. Dönmez, İ., Sahin, I., Gülen, S., 2023. Conducting academic research with the ai interface chatgpt: Challenges and opportunities. J. STEAM Educ. 6(2), 101–118.
  9. Editorialge, Jan, 2023. Elsevier and Cambridge university allow use of chatgpt for academic writing, https://editorialge.com/elsevier-and-cambridge-university-allow-use-chatgpt/. [Online; accessed 29-March-2023].
  10. Fijacko, 2023, Can chatgpt pass the life support exams without entering the american heart association course?, Resuscitation, 185, 10.1016/j.resuscitation.2023.109732 · DOI · Paper page
  11. Gao, M., Ruan, J., Sun, R., Yin, X., Yang, S., Wan, X., 2023. Human-like summarization evaluation with chatgpt, arXiv preprint arXiv:2304.02554. · arXiv
  12. Gaševic, 2023, ‘Empowering learners for the age of artificial intelligence’, Computers and Education, Artif. Intell.
  13. Gilson, 2023, How does chatgpt perform on the united states medical licensing examination? the implications of large language models for medical education and knowledge assessment, JMIR Med. Educ., 9, e45312, 10.2196/45312 · DOI · Paper page
  14. Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y.J., Afify, M., Awadalla, H.H., 2023. How good are gpt models at machine translation? a comprehensive evaluation, arXiv preprint arXiv:2302.09210. · arXiv
  15. Jungwirth, 2023, Artificial intelligence and public health: An exploratory study, Int. J. Environ. Res. Public Health, 20, 4541, 10.3390/ijerph20054541 · DOI
  16. Kasai, J., Kasai, Y., Sakaguchi, K., Yamada, Y., Radev, D., 2023. Evaluating gpt-4 and chatgpt on japanese medical licensing examinations, arXiv preprint arXiv:2303.18027. · arXiv
  17. Lamichhane, B., 2023. Evaluation of chatgpt for nlp-based mental health applications, arXiv preprint arXiv:2303.15727. · arXiv
  18. Lee, 2023, The rise of chatgpt: Exploring its potential in medical education, Anatom. Sci. Educ.
  19. Mann, 2023, Artificial intelligence discusses the role of artificial intelligence in translational medicine: A jacc: Basic to translational science interview with chatgpt, JACC: Basic Translat. Sci., 8, 221
  20. Melis, G., Dyer, C., Blunsom, P., 2017. On the state of the art of evaluation in neural language models, arXiv preprint arXiv:1707.05589. · arXiv
  21. Mitrovic, S., Andreoletti, D., Ayoub, O., 2023. Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text, arXiv preprint arXiv:2301.13852. · arXiv
  22. Ortega-Martín, M., García-Sierra, Ó., Ardoiz, A., Álvarez, J., Armenteros, J.C., Alonso, A., 2023. Linguistic ambiguity analysis in chatgpt, arXiv preprint arXiv:2302.06426. · arXiv
  23. Qiu, H., He, H., Zhang, S., Li, A., Lan, Z., 2023. Smile: Single-turn to multi-turn inclusive language expansion via chatgpt for mental health support, arXiv preprint arXiv:2305.00450. · arXiv
  24. Rathore, 2023, Future of textile: Sustainable manufacturing & prediction via chatgpt, Eduzone: Int. Peer Rev./Refereed Multidisc. J., 12, 52
  25. Tlili, 2023, What if the devil is my guardian angel: Chatgpt as a case study of using chatbots in education, Smart Learn. Environ., 10, 15, 10.1186/s40561-023-00237-x · DOI · Paper page
  26. Van Bulck, 2023, Response to the letter to the editor on: Dr. chatgpt in cardiovascular nursing: A deeper dive into trustworthiness, value, and potential risks, Eur. J. Cardiovasc. Nurs., zvad049
  27. Wang, C., Li, M., Smola, A.J., 2019. Language models with transformers, arXiv preprint arXiv:1904.09408. · arXiv
  28. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., Zhou, D., 2022. Chain of thought prompting elicits reasoning in large language models, arXiv preprint arXiv:2201.11903. · arXiv
  29. White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., Schmidt, D.C., 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt, arXiv preprint arXiv:2302.11382. · arXiv
  30. Wu, J., Wu, X., Qiu, Z., Li, M., Zheng, Y., Yang, J., 2023. Qualifying chinese medical licensing examination with knowledge enhanced generative pre-training model, arXiv preprint arXiv:2305.10163. · arXiv
  31. Yang, K., Ji, S., Zhang, T., Xie, Q., Ananiadou, S., 2023. On the evaluations of chatgpt and emotion-enhanced prompting for mental health analysis, arXiv preprint arXiv:2304.03347. · arXiv
  32. Zheng, O., Abdel-Aty, M., Wang, D., Wang, Z., Ding, S., 2023. Chatgpt is on the horizon: Could a large language model be all we need for intelligent transportation?, arXiv preprint arXiv:2303.05382. · arXiv
  33. Zhuo, T.Y., Huang, Y., Chen, C., Xing, Z., 2023. Exploring ai ethics of chatgpt: A diagnostic analysis, arXiv preprint arXiv:2301.12867. · arXiv
  34. Šlapeta, 2023, Are chatgpt and other pretrained language models good parasitologists?, Trends Parasitol., 10.1016/j.pt.2023.02.006 · DOI

Cited by

Sorted by citations

Related

Authors
Sohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W.
Affiliations
  • Jamia Hamdard (IN)
  • Aligarh Muslim University (IN)
  • University of Dubai (AE)
  • University of South-Eastern Norway (NO)
  • Mayo Clinic (US)
Corresponding author
  • Sohail, Shahab Saquib
  • Farhat, Faiza
  • Himeur, Yassine
  • Nadeem, Mohammad
  • Madsen, Dag Øivind
  • Singh, Yashbir
  • Atalla, Shadi
  • Mansoor, Wathiq
ORCID
Venue
Journal of King Saud University - Computer and Information Sciences
Publisher
Springer Science and Business Media LLC
Published
2023-09
Volume
35
Issue
8
Pages
101675
ISSN
1319-1578 (ISSN-L)
DOI
10.1016/j.jksuci.2023.101675
OpenAlex
W4385494594
Semantic Scholar
4f4058c6c0333a5bab7eba59a4b28bd7a8b51620
Cited by
192
References
201
Citations by year
2026: 31 · 2025: 58 · 2024: 84 · 2023: 19
FWCI
18.02
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of King Saud University - Computer and Information Sciences (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
ChatGPT, large language models, research taxonomy, algorithmic bias, trustworthiness, conversational AI, health care, financial services, software engineering, academic writing, natural language processing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review