Decoding ChatGPT: A taxonomy of existing research, current challenges, and possible future directions

In the paper

“The different assessments included those for bias Wang (2023 b ), Wang, Miao, Li, Wang & Lin (2023), for study Cooper (2023), for mental health Prada et al. (2023), for public health Jungwirth & Haluza (2023 a ); Biswas (2023 e ), etc.”

What the sources say

• Not checked yetcites Wang (2023 b )
• Supported

  The citation to Cooper (2023) supports the ‘for study’ category, not the separate categories of bias, mental health, or public health. The Methodology section describes an exploratory evaluation of ChatGPT’s responses about science education and its potential uses in teaching, including designing a classroom rubric. (LLM, conf=0.87)

  cites Cooper (2023) source
• Not checked yetcites Prada et al. (2023)

In the paper

“Multidisciplinary subjects and health professionals made up 8% and 7% of the total corpus, respectively.”

In the paper

“These articles involved contributions from 349 authors representing 53 different nations, indicating a wide range of international participation in the literature on ChatGPT.”

In the paper

“Multidisciplinary subjects and health professionals made up 8% and 7% of the total corpus, respectively.”

In the paper

“It was followed by the United Kingdom with 10 publications, and Australia and China with 9 publications each.”

In the paper

“As a result, many authors have expressed concerns regarding the ethical and scientific implications of ChatGPT's features and their potential negative effects on scientific academia.”

In the paper

“Some have also asked questions based on their extensive understanding of various fields, including parasitology Huh (2023 b ), ˇ Slapeta (2023), clinical diagnosis Hirosawa et al. (2023), environmental sciences Rillig et al. (2023), public health Biswas (2023 e ), etc.”

What the sources say

• Not checked yetcites Huh (2023 b )
• Not checked yetcites ˇ Slapeta (2023)
• Not checked yetcites Hirosawa et al. (2023)

In the paper

“Meanwhile, several articles have explored how ChatGPT could impact learners and educators, as well as the ethical considerations that must be taken into account when using it.”

In the paper

“Iraqi Journal for Computer Science and Mathematics has also published 2 documents as well.”

In the paper

“Furthermore, it has been noted that the majority of publications center around either ChatGPT's feature assessment as a scientific writing assistant or predictions about the future of the education system.”

In the paper

“Clinicians and medical professionals probed ChatGPT with hypothetical or real-world patient symptoms to determine whether or not it would aid in the diagnosis or treating patients.”

In the paper

“Next, Accountability in Research has published 4 articles.”

In the paper

“Switzerland ranked second in terms of collaboration, with 20 collaborative countries, followed by Australia with 19 collaborations, and the United Kingdom with 18 collaborations (see Figure 2).”

In the paper

“In terms of collaborations, the United States had the most extensive network, collaborating with 24 different countries, accounting for over 18% of the total corpus.”

In the paper

“These interpretations can be broadly categorized into two types: those that predict the future of the topic in the context of ChatGPT and those that assess ChatGPT's features and their potential impacts.”

In the paper

“It was followed by the United Kingdom with 10 publications, and Australia and China with 9 publications each.”

In the paper

“Ethical considerations are discussed in the majority of the papers either in detail or briefly.”

In the paper

“Despite its potential, the early ChatGPT researches still face some limitations.”

In the paper

“Additionally, we have identified and discussed ethical concerns.”

In the paper

“We have identified certain issues that may need to be addressed, which are classified as intrinsic and usage-centric.”

In the paper

“Subsequent versions, such as GPT-2, GPT3, and the latest release, GPT-3.5, have experienced notable advancements by significantly augmenting the number of parameters. This augmentation has facilitated the generation of responses that are even more accurate and human-like.”

In the paper

“ChatGPT utilizes a transformer architecture consisting of encoder-decoder layers that collaborate to process and generate natural language text Chen, Yu, Yang, Guo, Bian & Wu (2023).”

What the sources say

• Not checked yetcites Chen, Yu, Yang, Guo, Bian & Wu (2023)

In the paper

“The transformer architecture of ChatGPT consists of two main components: the encoder and the decoder Budzianowski & Vulic (2019).”

What the sources say

• Not checked yetcites Budzianowski & Vulic (2019)

In the paper

“ChatGPT is turning out to be a Code Assistant and Debugger. It helps software developers by providing code suggestions, debugging assistance, and answering programming-related questions Aljanabi, Ghazi, Ali, Abed et al. (2023).”

What the sources say

• Not checked yetcites Aljanabi, Ghazi, Ali, Abed et al. (2023)

In the paper

“The initial iteration of ChatGPT referred to as GPT-1, was equipped with 117 million parameters and underwent training on a substantial corpus of text data Ernst & Bavota (2022).”

What the sources say

• Not checked yetcites Ernst & Bavota (2022) source

In the paper

“To continually improve the reliability and accuracy of the model, ChatGPT incorporates reinforcement learning from human feedback (RLHF), allowing it to learn and understand human preferences through extended dialogues Christiano et al. (2017), Stiennon et al. (2020).”

What the sources say

• Not checked yetcites Christiano et al. (2017)
• Not checked yetcites Stiennon et al. (2020)

In the paper

“ChatGPT has the ability to generate new responses or utilize pre-existing ones Salvagno et al. (2023).”

What the sources say

• Not checked yetcites Salvagno et al. (2023)

In the paper

“Researchers have been exploring and highlighting the potential harms associated with ChatGPT, including the propagation of stereotypes, biased responses, and dissemination of misleading information Liang et al. (2021), Nadeem et al. (2020).”

What the sources say

• Not checked yetcites Liang et al. (2021)
• Not checked yetcites Nadeem et al. (2020)

In the paper

“In marketing, ChatGPT works as a conversational chatbot to handle customer inquiries, provide product recommendations, and assist with order tracking.”

What the sources say

• Not checked yetcites Cascella et al. (2023 a )

In the paper

“An essential feature of the transformer architecture is the attention mechanism, which allows the model to selectively focus on different parts of the input text while generating the output Jain et al. (n.d.).”

What the sources say

• Not checked yetcites Jain et al. (n.d.)

In the paper

“Moreover, there are concerns that ChatGPT-generated content may infringe upon copyright and intellectual property rights Deng & Lin (2022).”

What the sources say

• Not checked yetcites Deng & Lin (2022)

In the paper

“Following its launch, ChatGPT quickly gained a lot of popularity in a variety of disciplines Cox (2023), Dwivedi et al. (2023), Tlili et al. (2023 a ), including academia Chen (2023 b ) and science Morreel et al. (2023).”

What the sources say

• Not checked yetcites Cox (2023)
• Not checked yetcites Dwivedi et al. (2023) source
• Not checked yetcites Tlili et al. (2023 a )

In the paper

“although it occasionally noticed and corrected them.”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“ChatGPT also had deficits in situational awareness, inference, and consistency”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“and it did not repeat the same advice in a single response.”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“ChatGPT's responses were coherent”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“The model's answers included a summary of its understanding of the scenario and question, management options, and disclaimers that reflected its information sources, which were similar to the format of patient information websites.”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“ChatGPT also had deficits in situational awareness, inference, and consistency”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“This feature can enhance the coding experience for less-skilled programmers, making it more user-friendly and intuitive Aljanabi, Ghazi, Ali, Abed et al. (2023).”

What the sources say

• Not checked yetcites Aljanabi, Ghazi, Ali, Abed et al. (2023)

In the paper

“source control was often disregarded.”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“To conduct our literature review, we adopted the methodology proposed in Kitchenham (2004).”

What the sources say

• Not checked yetcites Kitchenham (2004)

In the paper

“while its antimicrobial spectra and regimens were appropriate”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“ChatGPT also had deficits in situational awareness, inference, and consistency”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“We have also performed a screening of the articles by going through the abstract and assessing its relevance to the title under consideration.”

In the paper

“Research scholars have been using ChatGPT in a variety of ways, including writing scientific literature, gathering data, drafting abstracts for research papers Else (2023), and looking for medical diagnostic advice Hirosawa et al. (2023).”

What the sources say

• Not checked yetcites Else (2023) source
• Not checked yetcites Hirosawa et al. (2023)

In the paper

“their spelling and grammar were appropriate”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“Clinicians often rely on complex information to make decisions, but in telemedicine, the available information is typically limited to language only, making it a potential candidate for interventions using LLM Roosan et al. (2016).”

What the sources say

• Not checked yetcites Roosan et al. (2016)

In the paper

“The information provided by ChatGPT was consistent”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“ChatGPT has a significant benefit in coding, as it can process human inputs, allowing software developers to supply code snippets or commands in a conversational way rather than providing specific keywords or phrases.”

What the sources say

• Not checked yetcites Aljanabi, Ghazi, Ali, Abed et al. (2023)

In the paper

“ChatGPT often repeated questions verbatim, including any errors”

What the sources say

• Not checked yetcites Howard et al. (2023)

In the paper

“After running search queries, inclusion (IC) and exclusion (EC) criteria were created to filter out the retrieved articles from the Scopus database.”

In the paper

“There are existing review works on Large Language Models (LLM) Melis et al. (2017), Chen et al. (2021) and AI Generated Content (AIGC) Cao et al. (2023) but they are very broad and do not take into consideration the specificities related to ChatGPT.”

What the sources say

• Not checked yetcites Melis et al. (2017)
• Not checked yetcites Chen et al. (2021)
• Not checked yetcites Cao et al. (2023)

In the paper

“to contribute to the advance- ment of gastroenterology by generating high-quality re- search questions”

What the sources say

• Not checked yetcites Lahat et al. (2023)

In the paper

“NLP technologies are being used to create conversational chatbots to mimic the con- versational proficiency of hu- mans.”

In the paper

“The findings revealed that task-specific models like RoBERTa outperformed ChatGPT, particularly in specialized downstream tasks. However, ChatGPT still demonstrated satisfactory performance compared to baseline models in a variety of tasks.”

What the sources say

• Not checked yetcites Amin et al. (2023)

In the paper

“Use ChatGPT's information can be updated and cor- rected, but it may not always reflect the latest knowledge.”

In the paper

“In fact, few publishers have recognized the use of ChatGPT for academic writing to promote its ethical usage practices Editorialge (Jan, 2023).”

What the sources say

• Not checked yetcites Editorialge (Jan, 2023)

In the paper

“ChatGPT potential to support decision-making in Japanese healthcare settings, but cau- tion is needed due to per- formance improvements re- quired.”

In the paper

“Unethical use also includes the deliberate generation of manipulated content that can promote misinformation and provoke violence, creating damage at an individual or organizational level Elali & Rachid (2023), Cao et al. (2023).”

What the sources say

• Not checked yetcites Elali & Rachid (2023)
• Not checked yetcites Cao et al. (2023)

In the paper

“Researchers have identified several issues regarding ChatGPT, which can be broadly categorized into two groups: intrinsic limitations and usage-related concerns.”

In the paper

“Qin et al. (2023) argued that larger models like ChatGPT can perform NLP tasks without the need for specific data adaptation.”

What the sources say

• Not checked yetcites Qin et al. (2023)

In the paper

“Prolonged questioning could be beneficial in enhanc- ing and improving both the kinds of human reasoning and argumentation imitated by current LLMs.”

What the sources say

• Not checked yetcites Cox & Tzoc (2023)

In the paper

“ChatGPT can act as a user assistance tool, provid- ing guidance, explanations, and educational resources to users interacting with robots.”

In the paper

“The knowledge and ability to analyze results of para- sitology test by ChatGPT were not yet on par with those of Korean medical stu- dents”

What the sources say

• Not checked yetcites Huh (2023 c , a )
• Not checked yetcites ˇ Slapeta (2023)

In the paper

“There are also concerns that ChatGPT-generated content may lead to violations of copyright and intellectual property rights Sallam (2023).”

What the sources say

• Not checked yetcites Sallam (2023) source

In the paper

“ChatGPT's knowl- edge and interpretation in the Chinese Chinese Na- tional Medical Licensing Ex- amination (NMLE) are be- low medical students' level”

In the paper

“The correct diagnoses within ten differential-diagnosis lists generated by ChatGPT was more than 90%.”

What the sources say

• Not checked yetcites Hirosawa et al. (2023)

In the paper

“These limitations make the usage and deployment of ChatGPT difficult in real-world scenarios.”

In the paper

“It can recognize when a word, phrase, or sentence has mul- tiple possible interpretations, leading to potential confu- sion or miscommunication.”

In the paper

“Rahman & Watanobe (2023) has experimentally shown that ChatGPT can be used to solve both technical problems, such as engineering and computer programming, and non-technical problems, such as language and literature.”

What the sources say

• Not checked yetcites Rahman & Watanobe (2023)

In the paper

“ChatGPT can aid in diag- nosing issues and providing troubleshooting guidance for robotic systems.”

In the paper

“It includes five major limitations, namely, hallucination, biased content, not real-time, misinformation and inexplicability.”

In the paper

“It is worth noting that the detection of artificially generated fake text and meaningless information is not a new challenge Amancio (2015), Jawahar et al. (2020).”

What the sources say

• Not checked yetcites Amancio (2015)
• Not checked yetcites Jawahar et al. (2020)

In the paper

“ChatGPT can help establish shared un- derstandings, allocate tasks, and assist in real-time com- munication, thereby improv- ing the overall efficiency and performance of robotic teams.”

In the paper

“Additionally, the application of ChatGPT is rapidly expanding across various domains Mijwil et al. (2023), Aljanabi, Ghazi, Ali, Abed et al. (2023), indicating a growing trend that is expected to continue in the foreseeable future.”

What the sources say

• Not checked yetcites Mijwil et al. (2023)
• Not checked yetcites Aljanabi, Ghazi, Ali, Abed et al. (2023)

In the paper

“Both issues can lead to the creation of counterfactual or meaningless responses, which can seriously threaten the reliability of the generated content.”

In the paper

“68% of the generated abstracts were correctly identified as such, and 14% of the real abstracts were mistakenly identified as generated by ChatGPT.”

What the sources say

• Not checked yetcites Gao, Tong, Wu, Chen, Zhu & Wang (2023)

In the paper

“The category of usage-related issues includes unethical usage of the tool, copyright-infringed content, and overreliance on ChatGPT.”

In the paper

“The latter process, which is fake text identification, falls within a well-established domain, with many state-of-the-art algorithms and techniques available to tag fake text with significant accuracy Zhou & Zafarani (2020).”

What the sources say

• Not checked yetcites Zhou & Zafarani (2020) source

In the paper

“The evaluation results indicated that ChatGPT excelled in tasks that required reasoning skills, including arithmetic reasoning while facing challenges in tasks such as sequence tagging.”

What the sources say

• Not checked yetcites Qin et al. (2023)

In the paper

“ChatGPT helps identify in- stances of linguistic ambi- guity in text or speech.”

In the paper

“ChatGPT can help assess the risk level of individuals experiencing mental health crises by analyzing conversations.”

In the paper

“ChatGPT performs well in structured tasks such as translating code and explaining wellknown concepts but struggles with nuanced tasks such as explaining less familiar terms and creating code from scratch.”

What the sources say

• Not checked yetcites Megahed et al. (2023)

In the paper

“Given the similarities among various NLP tasks, the findings from the aforementioned studies can be extrapolated to other related areas, including the assessment of news article accuracy, especially in the detection of fake news.”

In the paper

“The results demonstrated that GPT models could generate translation outputs that were highly competitive for languages with abundant resources. However, for low-resource languages, the current state of GPT models exhibited limitations, indicating the need for further improvements.”

What the sources say

• Not checked yetcites Hendy et al. (2023)

In the paper

“Lastly, there is a fear of overreliance on the tool, which may make humans lazy and apathetic and make them always rely on the generated information Sallam et al. (2023).”

What the sources say

• Not checked yetcites Sallam et al. (2023) source

In the paper

“Major publishers like Elsevier, who publish the Lancet family of journals, have already stated that AI cannot be listed as an author and that its use must be properly acknowledged Sallam et al. (2023).”

What the sources say

• Not checked yetcites Sallam et al. (2023) source

In the paper

“Our view is that ChatGPT's availability, ease of use, and multi-language capabilities could significantly boost scholarly output, thereby democratizing knowledge dissemination.”

In the paper

“The quality of the data is also crucial, and it is important to ensure that the data used for training is accurate, relevant, and diverse enough to represent the full range of language and speech patterns.”

In the paper

“By comparing their model with an approach based on perplexity scores, they found that differentiating between human-generated and ChatGPT-generated reviews is more challenging for the machine-learning model when using rephrased text.”

What the sources say

• Not checked yetcites Mitrovic et al. (2023)

In the paper

“As a result, adding feelings to AI language models runs the risk of unintended consequences and risks like bias and discrimination, manipulation, privacy evasion, and inappropriate or offensive responses.”

In the paper

“Additionally, other factors such as the model architecture, training methods, and hyperparameters can also impact the model's performance.”

In the paper

“However, the former step, which focuses on specifically identifying texts generated by ChatGPT, is relatively new, and researchers are actively working to address this issue Dugan et al. (2022), Mitchell et al. (2023), Curtis (2023), Mitrovic et al. (2023).”

What the sources say

• Not checked yetcites Dugan et al. (2022)
• Not checked yetcites Mitchell et al. (2023)
• Not checked yetcites Curtis (2023)

In the paper

“Incorporating human feedback is an effective way to improve the performance of ChatGPT.”

In the paper

“Arguably, effective conversational AI systems like ChatGPT require a combination of various techniques and considerations to deliver a satisfying and human-like conversational experience, for example, considering factors such as context understanding, coherence, and response relevance.”

In the paper

“In the case of ChatGPT, fine-tuning allows the model to learn and understand the nuances of natural language conversations, enabling it to generate more human-like responses.”

What the sources say

• Not checked yetcites Bakker et al. (2022)
• Not checked yetcites Himeur et al. (2022 b )
• Not checked yetcites Dwivedi et al. (2023) source

In the paper

“However, it is also important to note that simply increasing the volume and variety of data alone may not always result in improved performance.”

In the paper

“Improving Conversational Capabilities, Personalization, Multimodal design, and Trustworthiness are some of the key areas that hold significant promise for the future of ChatGPT.”