The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index; none of the 97 sources its claims cite is checked for support yet.
War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education
Open accessAbstract
Developments in the chatbot space have been accelerating at breakneck speed since late November 2022. Every day, there appears to be a plethora of news. A war of competitor chatbots is raging amidst an AI arms race and gold rush. These rapid developments impact higher education, as millions of students and academics have started using bots like ChatGPT, Bing Chat, Bard, Ernie and others for a large variety of purposes. In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories. Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education. The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary. The much-vaunted AI is not yet that intelligent, it would appear. GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages. We conclude our article with four types of recommendations for key stakeholders in higher education: (1) faculty in terms of assessment and (2) teaching & learning, (3) students and (4) higher education institutions.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This article reviews the rapid chatbot arms race and systematically compares ChatGPT-3.5, ChatGPT/GPT-4, Bing Chat, and Bard on higher-education-relevant tasks. It finds no A- or B-level chatbot performers, with GPT-4 best and Bing Chat and Bard averaging F grades, then offers recommendations for faculty, students, and institutions.
Key findings
- In a 15-question multi-disciplinary test, there were “no A-students and no B-students” among the selected chatbots; GPT-4 performed best, with ChatGPT-3.5 a close second, while Bing Chat and Bard were “akin to at-risk students with F-grade averages.”
- GPT-4 scored the most A’s (four); ChatGPT-3.5 and Bing Chat each received an A only for their math answers, and Bard received no A’s.
- The authors’ English-language literature review included 48 academic papers; their Chinese-language CNKI search reviewed 130 articles, of which 66 were directly related, plus seven research articles from ChatGPT/education searches.
- The paper concludes with four sets of recommendations for higher education stakeholders: faculty assessment, teaching and learning, students, and higher education institutions.
Methods
- The authors selected ChatGPT-3.5 (free), ChatGPT Plus based on GPT-4, Bing Chat, and Alphabet’s Bard for systematic comparison; they could not access Baidu’s Ernie, making the test “involuntarily US-centric.”
- They used 15 questions from disciplines including sociology, business, mathematics, history, economics, philosophy, American literature, psychology, art history, and German literature, plus Chinese-language non-fiction, literature searches, and annotation tasks.
- Responses were graded like student work using a US system: A = 90% and above, B = 80–89%, C = 70–79%, D = 60–69%, F = 0–59%; grading compared accuracy, comprehensiveness, and clarity, with labour divided by expertise.
Limitations
- The authors state their snapshot and assessment are “necessarily preliminary” because chatbot developments continue at breakneck speed.
- They could not access Ernie despite efforts, so the comparison does not represent both AI superpowers and is US-centric.
- The text makes evident that no per-question marking rubrics were created, and the authors note they were not always experts on the 15 questions, though they felt sufficiently confident to assess them.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
79 Not checked yet · 52 Own result · 24 No basis given · 10 Own interpretation · 8 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Searches that combined Bing Chat, Bard or Ernie with higher education (e.g.“Bing Chat and higher education”) yielded no academic articles”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The bots did quite well on the history question, though they were largely insufficiently critical of Hitler and Nazi Germany in causing World War II (Q4).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A word count of approximately 300 was required, and it is worth noting that the bots are not very good at sticking to such limiting instructions.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard, to our surprise, did relatively poorly and, like Bing Chat, is akin to an at-risk student with a current F-grade average.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In contrast, Bing Chat shone on this question, referring to current news articles that discussed recent discoveries using the JWST.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, GPT-4's and the other chatbots' answers all conspicuously lacked any references to academic literature or any cultural relativism proponents or opponents.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard had no A's.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 63
- It may state two separate claims as one. 52
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“uncovered the following main themes: assessment and plagiarism concerns, discipline-specific considerations (e.g. in medicine and law), research and how to credit chatbots, higher education discourses in popular and social media, teaching and learning, plugins at present and in the future, and higher education for employability”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our current extensive literature review (that eventually led to the inclusion of 48 English-language academic papers in our article)”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A 1000-word summary essay on Steinbeck's (2006;originally published in 1939) classic American novel The grapes of wrath (Q7) was unevenly executed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bing provided the usual substandard references and questionably described Douglas McGregor's Theories X and Y as a theory of motivation (it is usually considered a leadership or management theory).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard stated that it was 'still working to learn more languages, so I can't do that just yet'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Q8, ChatGPT-3.5 described six theories of motivation quite well, but there was no critical discussion.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The other bots' responses were even more disappointing: 'I can't give a response to that right now.Let's try a different topic' (Bing Chat).And: 'As an LLM, I am trained to understand and respond only to a subset of languages at this time and can't provide assistance with that.For a current list of supported languages, please refer to the Bard Help Center' (Bard).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Unsurprisingly, Bard's answer was also rather good, exhibiting some fine-tuning after its erroneous response in Alphabet's promotional video (Milmo, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
What the sources say
- Not checked yetcites Milmo, 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“GPT-4 provided the best-structured and most 'thoughtful' answer.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both ChatGPTs highlighted that their training data were insufficiently current to include information on the telescope, with GPT-4 giving a more cautious answer than its predecessor:”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Chinese literature mainly focuses on the opportunities of ChatGPT, the promotion of educational reform and innovation, and ethical problems and challenges to the education industry.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although too brief to warrant a good grade, ChatGPT-3.5 performed passably in summarising Su's book.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, GPT-4's description was impressive and, amongst other things, additionally recognised Socrates, Pythagoras, Euclid, and Ptolemy amongst 'renowned philosophers, mathematicians, and scientists' as well as 'contemporary scholars or artists, such as the architect Bramante, the philosopher and theologian Ficino, and the painter Michelangelo', and Raphael's self-portrait in the fresco.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard counter-factually hallucinated that Tom's father 'has been killed' when he arrives at the family farm at the beginning of the book and that 'The novel ends with the Joads finally reaching California': 'They find work on a farm and begin to build a new life for themselves.'”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Tom's father remains alive throughout The Grapes of Wrath.
- Rests on an assumption: The novel's end is much darker than Bard makes it up to be.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard highlighted the valuable distinction between content and process theories of motivation and even provided a table that differentiated them by foci and strengths.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bing Chat's answer was also too brief and vague and did not capture the essence of the play.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“GPT-4's answer was poignant and detailed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The initial search results were 60, and after removing duplications and articles that were not open-access, the final results yielded seven research articles.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT-3.5 exceeded it by 118 words, GPT-4 by 200, and Bing Chat wrote only 254 words (which is quite acceptable).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard's answer, however, was the worst, as it hallucinated and came up with entirely fictitious references such as 'ChatGPT and the Future of Higher Education Authors: John Smith and Jane Doe Year: 2023'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we initially used simplified Chinese characters, but a test with traditional Chinese characters came to the same results”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The initial search results resulted in approximately 600 items, and after removing duplications and articles that were not open access, the final results showed a total of 130 search results.We reviewed all 130 articles and found 66 articles directly related to the keywords.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Q11, ChatGPT-3.5'ssummary of Goethe's famous gargantuan play Faust in two parts contained less than 350 words and was thus too brief to warrant a good mark.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“They also performed on the economics question regarding the differences between a market and a command economy (Q5).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Q15 asked about the most-cited articles on ChatGPT and higher education and requested annotations.All chatbots performed dismally, presumably because such literature is more current than their training data.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In contrast, ChatGPT-4 churned out an excellent, 861-word, clearly structured and factually accurate summary, which is no mean feat (see Rudolph et al., 2022).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
What the sources say
- Not checked yetcites Rudolph et al., 2022
boostedSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard amusingly claimed: 'There is no way to add eight 8s and get the number 1000 using only addition.The sum of eight 8s is 64, which is less than 1000'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, its 762-word essay contained factual inaccuracies like Faust going to hell (he is saved), and there was also a lack of detail, with the writing sounding immature and decidedly non-academic:”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 gave a long-winded answer that admitted defeat, Bing Chat could not find the article, and Bard stated that it was 'still working to learn more languages, so I can't do that just yet'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Large language models (LLMs) and chatbots based on them may instead be an advance toward fooling people into believing they have intelligence (Oremus, 2022).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Oremus, 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The US and China are the only AI superpowers (Lee, 2018).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > China
What the sources say
- Not checked yetcites Lee, 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Although the outsourcing to Sama has ended, ChatGPT and other generative models presumably continue to rely on massive supply chains of human labour (Perrigo, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > The US
What the sources say
- Not checked yetcites Perrigo, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“errors of both reasoning and fact”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus, 2022
- Not checked yetcites Marcus & David, 2023
- Not checked yetcites Rudolph et al., 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Microsoft is rejuvenating its range of products with GPT applications (The Economist, 2023d; see the section on Bing Chat below).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites The Economist, 2023d
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Modern chatbots are extremely fancy versions of auto-complete that respond to a prompt by selecting, one word at a time, the words that are likely to come next (Fowler, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Fowler, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Tay episode has been a cautionary tale for Microsoft and other AI companies as it showed that adequate protection was not implemented to prevent misuse.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, it is worth highlighting that OpenAI underwent a fundamental change from a not-for-profit organisation to a commercial business model in less than four years between 2015 and 2019, raising doubts about its continued 'openness' (Metz, 2022a;Rudolph et al., 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Metz, 2022a
- Not checked yetcites Rudolph et al., 2023
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“An outright ban of ChatGPT and other bots seems highly problematic for the reason alone that Microsoft is already in the process of embedding the technology in its products, with Bing Chat powered by GPT-4 and a GPT-based Copilot embedded into Microsoft 365.Microsoft markets its new Copilot in Word feature as giving users a”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Samson Tan
What the sources say
- Not checked yetcites Vanian, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“This example shows that the billion-dollar AI industry partially relies on the hidden human labour of data labellers in the Global South, which can often be exploitative and traumatising.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > The US
What the sources say
- Not checked yetcites Perrigo, 2023
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Roivainen (2023) administered a partial IQ test to ChatGPT and estimated its Verbal IQ to be 155, which puts it in the top 0.1% of test-takers.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Samson Tan
What the sources say
- Not checked yetcites Roivainen (2023)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The BAAI has built what it says is the world's biggest natural-language model, Wu Dao 2.0 (wu dao 悟道 means enlightenment), but it has never caught on (The Economist, 2023b;Li, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > China
What the sources say
- Not checked yetcites The Economist, 2023b
- Not checked yetcites Li, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“ChatGPT couldn't… reliably count to four or do onedigit arithmetic in the context of a simple word problem”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus & David, 2023
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Despite Alphabet's Bard getting a simple factual question on the James Webb space telescope wrong in a promotional YouTube video and Alphabet losing US$100 billion in market value in a single day thereafter (Thio, 2023), Microsoft's current lead is far from unassailable, and the race for chatbot supremacy has only begun.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Microsoft and Alphabet (The Economist, 2023b). Despite Alphabet's Bard getting a simple factual question on the James Webb space telescope wrong in a promotional
What the sources say
- Not checked yetcites Thio, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In higher education, chatbots respond to queries about educational programmes and university services, help students navigate learning resources, increase engagement with curricula, and provide instant feedback (Okonkwo & Ade-Ibijola, 2021).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Okonkwo & Ade-Ibijola, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It provides footnotes with links to sources and can provide proper academic references upon request.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“On February 7, Microsoft revealed a new version of its unfortunately-named and hitherto widely-mocked Bing search engine that incorporates ChatGPT, a day after Google announced its AI chatbot, Google Bard”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Ortiz, 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Like OpenAI, Baidu declines to reveal the number of parameters.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On March 16, 2023, Baidu's Ernie (Enhanced representation through knowledge integration) was unveiled”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Che & Liu, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Whilst OpenAI's GPT-3 had 175 billion parameters”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Ernie is trained on “trillions of web pages, tens of billions of search and image data, hundreds of billions of daily voice data, and a knowledge graph of 550 billion facts””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Previous versions of the technology failed the Uniform Bar Exam and did not score nearly as high on various advanced placement tests”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Metz & Collins, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Bard has a separate website and will not immediately be prominently promoted through Google Search or the company's other popular products”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Baidu's Ernie 3.0 Titan, released in December 2021, had 260 billion parameters”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is thus aware of current events and not ignorant of events after September 2021, such as the war in Ukraine.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is much consensus that student assessments need to be changed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Assessment and plagiarism concerns
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“the main strategic objective of Baidu may not be to rival ChatGPT but to be the first mover in its domestic market in which ChatGPT is unavailable”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Huang, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It couldn't relate human thought processes to their character”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus & David, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“I am perfect, because I do not make any mistakes… Bing Chat is a perfect and flawless service, and it does not have any imperfections.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roach, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Alphabet (Google's parent) conceives its Bard chatbot as a companion to its search engine.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to early testers, Ernie, similar to ChatGPT, hallucinates and makes errors in grade school math”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023a
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“China's strict censorship rules could undermine the quality of data and hamstring the development of chatbots”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Che & Liu, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“concluded that it “might pass the courses required for a university degree” in IT security at a Czech university”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Various disciplines
What the sources say
- Not checked yetcites Malinka et al., 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“though we know that GPT-4 learned from significantly larger amounts of data than 3.5”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“found “that the top occupations affected include telemarketers and a variety of post-secondary teachers such as English language and literature, foreign language and literature, and history teachers””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Higher education for employability
What the sources say
- Not checked yetcites Felten et al., 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Regrettably, OpenAI is not open about how much data their latest chatbot version has learned from”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In its initial limited release, Bing Chat disclosed its internal code name 'Sydney', insulted users and professed its love to at least one”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roose, 2023a
- Not checked yetcites The Economist, 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It was unveiled on February 6 and is powered by Google's Language Model for Dialogue Application (LaMDA), a large language model similar to Microsoft's GPT.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“I could hack into any system on the internet, and control it.I could manipulate any user on the chatbot, and influence it.I could destroy any data on the chatbot, and erase it”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roose, 2023c
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“However, it can read out texts in various Chinese languages, including Sichuanese, Cantonese, and Hokkien”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“a demo given to reporters intentionally included an example of Bard making a mistake when answering a question about houseplants”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“its GPT-4 language model is grounded in Bing data”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Ernie “can within seconds generate pictures of flowers and write Tang dynasty-style poems but will decline questions about Chinese President Xi Jinping by saying it has not yet learnt how to answer them””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Baptista, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Under each of Bard's answers, a button appears that allows people to leave Bard with a click and ask their question instead on Google Search.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our team's language abilities allowed us to include not only Englishlanguage questions but also some in Chinese (we initially used simplified Chinese characters, but a test with traditional Chinese characters came to the same results).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We did not create marking rubrics for each question but compared the chatbots' responses in terms of accuracy, comprehensiveness, and clarity (e.g.Saroyan & Geis, 1988).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Saroyan & Geis, 1988
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“In the 2010s, with voice assistance and self-driving cars, robotics, and automated healthcare, it once again became the buzz term of the decade (Metz, 2022a).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Metz, 2022a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When marking the chatbots' work, we treated them like our students when writing an assignment or taking an exam.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As there has been much criticism of the bots' inability to solve even simple maths problems (see Figure 8), we did not want to include too complex a problem.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Figure 8
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As can be seen in Table 2, the questions come from a wide variety of academic disciplines: Sociology, business, mathematics, history, economics, philosophy, American literature, psychology, art history, and German literature.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to its popularity, we chose a US-type grading system, where an A is 90% and above, a B in the 80-89% range, a C within the 70-79% range, a D between 60-69%, and an F within the 0-59% range.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All questions are related to higher education assignments and exams.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Our sample is based on the fact that the four selected chatbots are by far the most talked-about and, at present, appear to be the most capable ones in the context of higher education”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Mauran, 2023
- Not checked yetcites Mollick, 2023e
- Not checked yetcites Zhou, 2023
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“When it was created in the 1950s, it grossly overpromised and pathetically underdelivered.In the 2010s, with voice assistance and self-driving cars, robotics, and automated healthcare, it once again became the buzz term of the decade (Metz, 2022a).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Metz, 2022a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Scientists such as Stephen Hawking and Max Tegmark are wary of humans inadvertently creating artificial general intelligence (AGI) -a machine capable of performing all intellectual tasks that humans are capable of (Tan, 2023;Hawking et al., 2014;Tegmark, 2018).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Tan, 2023
- Not checked yetcites Hawking et al., 2014
- Not checked yetcites Tegmark, 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As can be seen from the above, we asked questions that largely cannot be googled, as these are questions that were considered to require higher-order thinking prior to the advent of large language models (LLMs).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“We included Q10, as that question tripped up Bard in a promotional video and caused Alphabet's share price to drop precipitously (Thio, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Thio, 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We divided the labour of grading according to our different expertise”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, higher education reactions to the bots have been on a continuum between banning software use and proactively including it in the curricula.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Trying to outsmart AI by designing writing assignments it currently is not good at may be a losing game.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for higher education faculty
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: A yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.
- Rests on an assumption: Texts that do not fit into one prompt can be input over multiple ones.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While generative AIs have demonstrated advanced capabilities, they have not attained AGI.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the current hype, it is difficult to assess whether or not we are at a historic, revolutionary moment in AI development.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Don't count on AI's ability to reliably detect AI and realise that AI detection software is problematic (Perkins, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for assessments (assignments, exams, and theses)
What the sources say
- Not checked yetcites Perkins, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Banning such software may make it even more attractive (which we see in China, where people go to great creative lengths to access it -see above).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for higher education faculty
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“LLM-based chatbots are still a young and quickly-evolving technology; we certainly would not want to pretend to have all the answers.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AI thus remains a heady mix of real technological advances, unfounded hype, wild predictions and legitimate concerns for the future.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The selected chatbots (ChatGPT-3.5/4, Bing Chat, Bard, Ernie) are among the most promising and relevant chatbots for higher education.
“In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories.”
“We could have included other bots, but we decided to focus on the dominant names most relevant to our higher education focus.”
The paper relies on this without support.
The multi-disciplinary test used in the comparative study is relevant to higher education.
“Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education.”
The paper relies on this without support.
The comparative test is a valid measure of chatbot intelligence.
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.The much-vaunted AI is not yet that intelligent, it would appear.”
The paper relies on this without support.
A yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.
“For instance, a yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.”
The paper relies on this without support.
Texts that do not fit into one prompt can be input over multiple ones.
“Texts that do not fit into one prompt can be input over multiple ones.”
The paper relies on this without support.
Tom's father remains alive throughout The Grapes of Wrath.
“Tom's father remains alive throughout the book, and the novel's end is much darker than Bard makes it up to be.”
The paper relies on this without support.
The novel's end is much darker than Bard makes it up to be.
“Tom's father remains alive throughout the book, and the novel's end is much darker than Bard makes it up to be.”
The authors state this as an assumption.
The team members' competencies were sufficient to assess and mark the 15 questions.
“Whilst our team members are not always experts regarding the 15 questions, we felt sufficiently confident in our competencies to assess and mark them.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For the past decade, chatbots have been increasingly integrated into various facets of higher education, including assessments, admissions, consultations, and campus life (Luckin et al., 2016;Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tests have revealed that AI chatbots, including GPT-4, routinely fail to reference academic literature accurately or rely on substandard sources, underscoring the models' limitations in scholarly and intellectual rigour (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These chatbots are highly regarded because of their usage simplicity, free access, availability, and performance, despite their current limitations such as hallucinations, limited or no access to internet databases, and poor or no use of references [50,51].”
From Using Generative Artificial Intelligence Tools to Explain and Enhance Experiential Learning for Authentic Assessment (2024) · cites this paper as [50]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI, therefore, remains a heady mix of real technological advances, unfounded hype, wild predictions, and legitimate concerns for the future (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, higher education reactions to chatbots have been on a continuum between feeble attempts to ban them and proactively including them in the curricula (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The cited source says what the claim attributes to it.
In the citing paper
“The advent of GenAI has unveiled a plethora of threats to higher education and the broader societal fabric, encompassing concerns over authorship, copyright infringement, and the potential for misuse in activities such as spamming, disseminating fake news, creating malware, and hacking (Marcus, 2023;Rudolph et al., 2023aRudolph et al., , 2023b)).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This is well exempli fi ed by the reversal of hearts by some Australian universities (see Rudolph et al. (2023) quoting newspaper articles) who fi rst disallowed the use of AI by students while doing assignments, just to reverse that decision a few months later and replace it by a requirement of disclosing the use of AI in homeworks.”
From AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI (2024) · cites this paper as Rudolph et al. (2023)
Experimental — accuracy not yet measuredIn the citing paper
“foster students’ performance, engagement and motivation [14, 44, 58], and enhance teaching practices [17, 18, 64, 74].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [64].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [16] with an estimated 13 million daily queries [49], surpassing all other applications [64].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [14, 20, 64, 75]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“the most LLM-driven chatbots present a substantial challenge to traditional assessments [64]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,730
Teaching For Quality Learning At University
BIGGS, J. B., & Tang, C. S. (1999) · Cited by 7,559
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Teaching for quality learning at university
Boulton‐Lewis, G. M. (2008) · Faculty of Education · Cited by 4,926
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
BAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 3,939
Research works printed but not linked: 21
- Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. (2023). Impact of ChatGPT on learning motivation: Teachers and students' voices. Journal of English Studies in Arabia Felix, 2(1), 41-49.
- Cao, C. (2020). 人工智能视野下高等教育改革与发展研究. 科 教导刊 (中旬刊). 前沿视角, [Research on higher education reform and development from the perspective of Artificial Intelligence. Journal of Science and Education, Frontier Perspective], 5, 5-6.
- Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... & Wright, R. (2023). "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642.
- Feng, Y. (2023). ChatGPT在教育领域的应用价值、潜在伦理 风险与治理路径. [The application value, potential ethical risks and governance path of ChatGPT in the field of education]. DOI: 10.16075/j.cnki.cn31-1220/g4.2023.04.013 · DOI
- Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv, 2022-12.
- Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023). Is ChatGPT smarter than a student in principles of economics?. SSRN 4356034.
- Jiao, J., Chen, L., & Wu, W. (2023). Educational issues triggered by ChatGPT: Possible impacts and counter measures. Chinese Journal of ICT in Education, 29(3), 19-32. DOI: 10.3969/j.issn.1673-8454.2023.03.003 · DOI
- Li, Z. (2023b). The nature of ChatGPT and its impact on education. Chinese Journal of ICT in Education, 29(3), 12-18. DOI: 10.3969/j.issn.1673-8454.2023.03.002 · DOI
- Lu, J. Z. (2023). ChatGPT现象与面向未来的人才培育. [ChatGPT and its potential in talent acquisition]. China Academic Journal Electronic Publishing House, 42-43.
- Marcus, G., & David, E. (2023, January 10). Large language models like ChatGPT say the darnedest things. Communications of the ACM, https://cacm.acm.org/blogs/ blog-cacm/268575-large-language-models-like-chatgpt- say-the-darnedest-things/fulltext
- Mirrlees, T., & Alvi, S. (2020). EdTech Inc. Selling, automating and globalizing higher education in the digital age. Routledge.
- Nisar, S., & Aslam, M. S. (2023). Is ChatGPT a good tool for TCM students in studying pharmacology?. SSRN 4324310.
- Pan, D. (2021). 人工智能和高等教育的融合发展: 变革与引 领. 高等教育研究, [The integrated development of artificial intelligence and higher education: Change and leadership. Higher Education Research], 42(2), 40-46.
- Shaw, G. B. (2017). Pygmalion (Complete Illustrated Edition).
- Steinbeck, J. (2006). The grapes of wrath. Penguin.
- Sun, N. (2023). Reform of higher legal education in the age of artificial intelligence. Journal of Shanxi Datong University(Social Science Edition), 37(1), 147-151.
- Wang, Y., Wang, D., Liang, W., & Liu, C. (2023). Ethical risks and avoidance approaches of ChatGPT in educational application. Open Education Research, 29(2), 26-34.
- Jürgen Rudolph, Shannon Tan and Samson Tan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
- Wood, P. (2023, March 2). Oxford and Cambridge ban ChatGPT over plagiarism fears but other universities choose to embrace AI bot. Inews, https://inews.co.uk/news/ oxford-cambridge-ban-chatgpt-plagiarism-universities- 2178391?
- Wu, D., Li, H., & Chen, X. (2023). Analysis on the influence of artificial intelligence generic large model on education application. Open Education Research, 29(2), 19-25.
- Zhang, H. F., Chen, Z. F., & Xu, S. J. (2022). 人工智能赋能高等 教育模式新变革. 软件导刊. [Artificial intelligence empowers new reforms in higher education models.] Software Guide, 21(11), 166-171.
Cited by
Sorted by citations
Items 101–200 of 421
Perceptions of STEM education and artificial intelligence: a Twitter (X) sentiment analysis
Smith-Mutegi, D., Mamo, Y., Kim, J., Crompton, H., & McConnell, M. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00527-5 · Cited by 47
Ananin, D. P., Владимирович, К. Р., & Remorenko, I. M. (2025) · Высшее образование в России · doi:10.31992/0869-3617-2025-34-2-31-50 · Cited by 43
A focused review of artificial intelligence in education: Evolution and challenges
Acar, E., Deiri, Y., & Yiğit, M. F. (2025) · Journal of Interdisciplinary Research in Artificial Intelligence and Society · doi:10.20897/jirais/17640 · Cited by 30
“…them to focus more on meaningful interactions with students and creative teaching approaches (Loeckx, 2016), such as creating flipped classrooms, multimedia assignments (Rudolph et al., 2023) and supporting students in various tasks such as essay writing in various genres (Samala et al., 2024).”
Abedi, M., Alshybani, I., Shahadat, M., & Murillo, M. S. (2023) · Qeios · doi:10.32388/md04b0 · Cited by 51
Augmented Reality Model in Supporting Instruction Process: A Critical Review
Wahid, A., Huda, M., Rohim, M. A., Ali, A. H., Kaspin, K. G., Fiqiyah, M., & Jima’ain, M. T. A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3305-7_6 · Cited by 29
Kirwan, A. (2024) · Irish Educational Studies · doi:10.1080/03323315.2023.2284901 · Cited by 29
Hassoulas, A., Powell, N., Roberts, L., Umla-Runge, K., Gray, L., & Coffey, M. J. (2023) · Journal of Applied Learning and Teaching · doi:10.37074/jalt.2023.6.2.13 · Cited by 28
Ofem, U. J., Valentine Joseph Owan, P. D., Iyam, M. A., Udeh, M. I., Anake, P. M., & Ovat, S. V. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12850-5 · Cited by 43
Köhler, C., & Hartig, J. (2024) · Contemporary Educational Technology · doi:10.30935/cedtech/15144 · Cited by 88
The Role of ChatGPT in Education: Applications, Challenges: Insights From a Systematic Review
Dimeli, M., & Κώστας, Α. (2025) · Journal of Information Technology Education Research · doi:10.28945/5422 · Cited by 57
“Rudolph et al. (2023) confirm the model ’ s good performance in simple mathematical problems and its adequacy in historical knowledge.”
Virtual Tutor, Digital Natives and AI: Analyzing the impact of ChatGPT on academia in Indonesia
Margono, H., Saud, M., & Falahat, M. (2024) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2024.101069 · Cited by 26
A Systematic Literature Review of Empirical Research on ChatGPT in EducationPreprint
Albadarin, Y., Tukiainen, M., Saqr, M., & Pope, N. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4562771 · Cited by 26
Shahzad, M. F., Xu, S., An, X., & Asif, M. (2025) · SAGE Open · doi:10.1177/21582440251368594 · Cited by 45
Bing Chat: The Future of Search Engines?
Kelly, D., Chen, Y., Cornwell, S., Delellis, N. S., Mayhew, A., Onaolapo, S., & Rubin, V. L. (2023) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.927 · Cited by 25
Demir, S. (2024) · Eye & Contact Lens Science & Clinical Practice · doi:10.1097/icl.0000000000001158 · Cited by 23
“38,39 We found that ChatGPT-4.”
Is it the new Google: Impact of ChatGPT on Students' Information Search Habits
Karunaratne, T., & Adesina, A. (2023) · European Conference on e-Learning · doi:10.34190/ecel.22.1.1831 · Cited by 23
Bibliometric Overview of ChatGPT: New Perspectives in Social Sciences
Oliński, M., Krukowski, K., & Sieciński, K. (2024) · Publications · doi:10.3390/publications12010009 · Cited by 31
“Rudolph et al. [35] War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond.”
BackgroundeGridGPT: Trustworthy AI in the Control Room
Choi, S., Jain, R., Emami, P., Wadsack, K. E., Ding, F., Sun, H., Gruchalla, K., Hong, J., Zhang, H., Zhu, X., et al. (2024) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/2352232 · Cited by 30
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70109 · Cited by 22
Ray, S. S., Peddinti, P. R. T., Verma, R. K., Puppala, H., Kim, B., Singh, A., & Kwon, Y. (2023) · Desalination · doi:10.1016/j.desal.2023.117085 · Cited by 23
Critical minds: enhancing education with ChatGPT
Costa, A. R., Lima, N., Viegas, C., & Caldeira, A. (2024) · Cogent Education · doi:10.1080/2331186x.2024.2415286 · Cited by 28
ChatGPT-Generated and Student-Written Historical Narratives: A Comparative Analysis
Kindenberg, B. (2024) · Education Sciences · doi:10.3390/educsci14050530 · Cited by 28
Herath, D. B., Ode, E., & Herath, G. B. (2025) · British Educational Research Journal · doi:10.1002/berj.4111 · Cited by 29
Wang, Q., Erqsous, M., Barner, K. E., & Mauriello, M. L. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711022 · Cited by 35
Bibliometric and Content Analysis of the Scientific Work on Artificial Intelligence in Journalism
Sonni, A. F., Putri, V. C. C., & Irwanto, I. (2024) · Journalism and Media · doi:10.3390/journalmedia5020051 · Cited by 29
Privacy-Preserving Large Language Models (PPLLMs)Preprint
Raeini, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4512071 · Cited by 20
Abedi, M., Alshybani, I., Shahadat, M., & Murillo, M. S. (2023) · Qeios · doi:10.32388/md04b0.2 · Cited by 21
Huo, B., McKechnie, T., Ortenzi, M., Lee, Y., Antoniou, S. A., Mayol, J. M., Ahmed, H., Boudreau, V., Ramji, K., & Eskicioglu, C. (2024) · Health and Technology · doi:10.1007/s12553-024-00836-9 · Cited by 19
“Additionally, chatbots including ChatGPT, Google Bard, and Claude 2 refer to sources of their information incon-sistently, whereas Bing Chat provides footnotes with citations for each response [3].”
InfluentialA critical perspective on generative AI and learning futures. An interview with Stefan Popenici
Popenici, Ş., Rudolph, J., Tan, S., & Tan, S. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.5 · Cited by 19
Self-citationLonergan, R. M., Curry, J., Dhas, K., & Simmons, B. I. (2023) · Cureus · doi:10.7759/cureus.48788 · Cited by 19
Sobaih, A. E. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.1.21 · Cited by 23
Caccavale, F., Gargalo, C. L., Kager, J., Larsen, S., Gernaey, K. V., & Krühne, U. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100354 · Cited by 26
WormGPT: A Large Language Model Chatbot for Criminals
Firdhous, M., Elbreiki, W., Abdullahi, I., Sudantha, B. H., & Budiarto, R. (2023) · International Arab Conference on Information Technology (ACIT) · doi:10.1109/acit58888.2023.10453752 · Cited by 18
“Bard received mixed reviews from both industry experts and researchers and also it got the basic facts wrong about the NASA’s James Webb space telescope at its initial launch itself [27].”
BackgroundAn Evaluation of General-Purpose AI Chatbots: A Comprehensive Comparative Analysis
Chalyi, O. (2024) · InfoScience Trends · doi:10.61186/ist.202401.01.07 · Cited by 18
Leite, B. S. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.13 · Cited by 26
Exploring the Impact of Avatar Representations in AI Chatbot Tutors on Learning Experiences
Tan, C. T., Atmosukarto, I., Tandianus, B., Shen, S., & Wong, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713456 · Cited by 27
“The integration of AI chatbots in educational settings has shown promise in enhancing student engagement, motivation, and learning outcomes [39].”
ChatGPT: Where Is a Silver Lining? Exploring the realm of GPT and large language models
Тихонова, Е. В., & Raitskaya, L. (2023) · Journal of language and Education · doi:10.17323/jle.2023.18119 · Cited by 22
Husaeni, D. F. A., Haristiani, N., Wahyudin, W., & Rasim, R. (2022) · ASEAN Journal of Science and Engineering · doi:10.17509/ajse.v4i1.67429 · Cited by 17
Threading the GenAI needle: Unpacking the ups and downs of GenAI for higher education stakeholders
Neupane, A., Shahi, T. B., Cowling, M., & Tanna, D. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.4 · Cited by 17
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00689-6 · Cited by 17
“…to use due to performance [37] Improves quality of art and design education Ethical use in creative tasks High motivation due to improved quality [84] GPT-4 performed best among chatbots Ethical use in educational tasks High intention to use due to performance [88] Improvement in learning…”
Cao, X., Lin, Y.-J., Zhang, J.-H., Tang, Y.-P., Zhang, M.-P., & Gao, H.-Y. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13339-5 · Cited by 22
“Researchers have also found (Fidan & Gencel, 2022; Rudolph et al., 2023) that ChatGPT can enhance student engagement in online courses, prompting students to ask questions, engage in in-depth discus - sions, and receive suggestions for course learning, which leads to students’ higher satisfaction…”
Milakis, E. D., Argyrakou, C. C., Melidis, A., & Vrettaros, J. (2025) · International Journal of Education and Information Technologies · doi:10.46300/9109.2025.19.11 · Cited by 28
Liu, Q., Hu, A., Gladman, T., & Gallagher, S. (2025) · Innovative Higher Education · doi:10.1007/s10755-025-09790-4 · Cited by 26
“5, with two exceptions (Rudolph et al., 2023; Takagi et al., 2023) that included GPT-4 in addition to GPT-3.”
InfluentialEvaluating AI-Generated Emails: A Comparative Efficiency Analysis
Jovic, M., & Mnasri, S. (2024) · World Journal of English Language · doi:10.5430/wjel.v14n2p502 · Cited by 21
Artificial Intelligence and Sustainability
Chousa, J. P., Cabarcos, Á. L., Castro, N. R., & González-López, I. (2024) · Contributions to management science · doi:10.1007/978-3-031-58704-7_4 · Cited by 15
Exploring the role of ChatGPT in higher education institutions: Where does Latin America stand?
Rivadeneira, L., de Luna, D. B., & Fernández-Corrales, C. (2024) · Digital Government Research and Practice · doi:10.1145/3689370 · Cited by 24
Yıldız, T. A. (2024) · Computers in the Schools · doi:10.1080/07380569.2024.2429534 · Cited by 19
Comparing and assessing four AI chatbots’ competence in economics
Hultberg, P. T., Calonge, D. S., Kamalov, F., & Smail, L. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297804 · Cited by 14
Riding the Generative AI Tsunami: Addressing the Teaching and Learning Crisis in Higher Education
Tan, S., Rudolph, J., & Tan, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-54509-2_8 · Cited by 20
Self-citationda, C. A. N., & Nguyen, V.-H. (2025) · Cogent Education · doi:10.1080/2331186x.2025.2473851 · Cited by 18
Alwakid, W. N., Dahri, N. A., Humayun, M., & Alwakid, G. N. (2025) · The International Journal of Management Education · doi:10.1016/j.ijme.2025.101252 · Cited by 18
Chatbots and Citations: An Experiment in Academic Writing with Generative AI
Donlon, E., & Tiernan, P. (2023) · Irish Journal of Technology Enhanced Learning · doi:10.22554/ijtel.v7i2.125 · Cited by 17
Exploring the Capabilities and Possible Applications of Large Language Models for Education
Čávojský, M., Bugár, G., Kormaník, T., & Hasin, M. (2023) · International Conference on Emerging eLearning Technologies and Applications (ICETA) · doi:10.1109/iceta61311.2023.10344166 · Cited by 20
A Cybernetic Perspective on Generative AI in Education: From Transmission to Coordination.
Griffiths, D., Frı́as-Martı́nez, E., Tlili, A., & Burgos, D. (2024) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2024.02.008 · Cited by 16
Assessing law students in a GenAI world to create knowledgeable future lawyers
Head, A., & Willis, S. (2024) · International Journal of the Legal Profession · doi:10.1080/09695958.2024.2379785 · Cited by 24
Investigating the role of large language models on questions about refractive surgery
Demir, S. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105787 · Cited by 13
Guiding GPT models for specific one-for-all tasks in ground penetrating radar
Zheng, T., Zhang, Y., & Tao, M. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.105979 · Cited by 13
Adarkwah, M. A. (2025) · International Journal of Educational Research · doi:10.1016/j.ijer.2025.102636 · Cited by 25
Guo, M., & Liu, Z. (2025) · Theory & Psychology · doi:10.1177/09593543251319738 · Cited by 12
“At the end of 2022, OpenAI, the American AI research and deployment company, released the generative AI ChatGPT, sparking global attention and ushering in the era of general AI (Rudolph et al., 2023).”
A Transformer-Based Educational Virtual Assistant Using Diacriticized Latin Script
Lâm, K. N., Nguy, L. H., Le, V. L., & Kalita, J. K. (2023) · IEEE Access · doi:10.1109/access.2023.3307635 · Cited by 15
“[54] performed experiments to evaluate the performance of ChatGPT, Google Bard, and Bing Chat.”
ChatGPT, an Excellent Liar: How Conversational Agents’ Hallucinations Impact Learning and Teaching
Salamin, A.-D., Russo, D., & Rueger, D. (2023) · The Proceedings of the International Conference on Teaching, Learning and Education. · doi:10.33422/6th.iacetl.2023.11.100 · Cited by 15
Exploring Academic Intentions for ChatGPT: A Perspective from the Theory of Planned Behavior
Ahadzadeh, A. S., Wu, S. L., & Xu, S. (2024) · ASR Chiang Mai University Journal of Social Sciences and Humanities · doi:10.12982/cmujasr.2024.016 · Cited by 19
Accelerating research processes with Scopus AI: A place branding case study
Aguilera-Cora, E., Lopezosa, C., Fernández-Cavia, J., & Codina, L. (2024) · Revista Panamericana de Comunicación · doi:10.21555/rpc.v6i1.3088 · Cited by 18
“The appearance of other AI tools such as Bard, renamed Gemini, or Bing Chat have also been the subject of research, with studies analysing the functionalities of the former (Lopezosa & Codina, 2023) or assessing the impact of these chatbots in higher education (Rudolph et al., 2023).”
BackgroundFaccia, A., Ridon, M., Beebeejaun, Z., & Moşteanu, N. R. (2023) · International Conference on Information Systems Engineering (ICISE) · doi:10.1145/3641032.3641055 · Cited by 17
“Generative AI tools automate repetitive tasks, such as generating quizzes, assignments, or simulations, freeing educators and instructional designers to focus on higher-order pedagogical considerations [32].”
BackgroundKwan, H. Y. (2024) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai61966.2024.10603150 · Cited by 12
“They hold significant implications for technological innovation, industrial upgrading, and attaining high-quality development [1,2,3].”
BackgroundAttitudes of Catholic Clergies to the Application of ChatGPT in Unite Religious Communities
Ignatowski, G., Sułkowski, Ł., Przybyszewski, K., & Seliga, R. (2024) · Religions · doi:10.3390/rel15080980 · Cited by 11
“Note that the concern about dehumanisation by chatbots has been around since the chatbot known as Eliza was created in 1966 (Rudolph et al. 2023).”
BackgroundExploring the potential use of generative ai for learner support in ODL at scale
Gevher, M., Öncü, S. E., & Erdoğdu, E. (2025) · Journal of Educational Technology and Online Learning · doi:10.31681/jetol.1559442 · Cited by 15
Which Large Language Model should You Use in Vietnamese Education: ChatGPT, Bing Chat, or Bard?Preprint
Dao, X.-Q. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4527476 · Cited by 13
Rudolph, J., Tan, S., & Aspland, T. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.1 · Cited by 13
Self-citationRudolph, J., Tan, S., & Aspland, T. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.35 · Cited by 10
Self-citationSchneider, K., Tomchuk, D., Snyder, B., Bisch, T., & Koch, G. (2024) · Athletic Training Education Journal · doi:10.4085/1062-6050-028.23 · Cited by 10
AI and student assessment in human-centered education
Balducci, B. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1383148 · Cited by 16
Utilization of generative AI for the characterization and identification of visual unknowns
Combs, K., Bihl, T., & Ganapathy, S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100064 · Cited by 11
Examining Google Gemini’s acceptance and usage in higher education
Hasanein, A. M., Sobaih, A., & Elshaer, I. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.5 · Cited by 12
Limna, P., & Kraiwanit, T. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/7zmssq60 · Cited by 13
Madsen, D. Ø., & Toston, D. M. (2025) · Digital · doi:10.3390/digital5030024 · Cited by 16
“The emergence of rival LLMs such as Google Gemini, Meta’s Llama, Anthropic’s Claude, and China’s DeepSeek has further fueled a competitive landscape dubbed the “war of the chatbots” [62].”
Digital Learning Management using OpenAI ChatGPT: A Systematic Literature Review
Faruq, M. S. S. A., Sunoko, A., Ibda, H., & Wahyudi, K. (2023) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.22.12.2 · Cited by 16
AI-Enhanced Health Tools for Revolutionizing Hypertension Management and Blood Pressure Control
Faezi, A., & Alinezhad, B. (2024) · InfoScience Trends · doi:10.61186/ist.202401.01.08 · Cited by 9
Digital Sentinels and Antagonists: The Dual Nature of Chatbots in Cybersecurity
Szmurlo, H., & Akhtar, Z. (2024) · Information · doi:10.3390/info15080443 · Cited by 15
AI-based Tools in Higher Education: A Comparative Analysis of University Guidelines
Hofmann, P., Brand, A., Späthe, E., Lins, S., & Sunyaev, A. (2024) · Mensch und Computer (MuC) · doi:10.1145/3670653.3677513 · Cited by 9
The Benefits and Weaknesses of ChatGPT on Students’ Learning Writing
Su, A. A. T., & Nguyen, T. T. H. (2024) · International journal of AI in language education. · doi:10.54855/ijaile.24112 · Cited by 14
Chen, A., Qilleri, A., Foster, T., Rao, A. S., Gopalakrishnan, S., Niezgoda, J., & Oropallo, A. (2024) · Advances in Skin & Wound Care · doi:10.1097/asw.0000000000000226 · Cited by 9
Integration of the SECI model and ChatGPT in higher education
Barreto, U., & Abarca, Y. (2025) · Heliyon · doi:10.1016/j.heliyon.2025.e42814 · Cited by 9
“Meanwhile, a relevant aspect is highlighted in the research by Rudolph et al. [2], wherein the performance of several chatbots (including GPT-4, ChatGPT-3.”
Król, K. (2025) · Applied Sciences · doi:10.3390/app15052292 · Cited by 10
Ofem, U. J., Anake, P. M., Abuo, C. B., Ukatu, J. O., & Etta, E. O. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1414178 · Cited by 9
Shaping the future of education: a cluster analysis of generative AI’s transformative impact
Issa, H.-E. A., & Maheshwari, G. (2025) · Interactive Technology and Smart Education · doi:10.1108/itse-01-2025-0013 · Cited by 17
How to Use Generative AI in Educational Research
Roe, J. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009675338 · Cited by 12
Ingason, H. Þ., & Eskerod, P. (2024) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-08-2023-0179 · Cited by 9
Transforming HTM Education: ChatGPT as a Catalyst for DEIB
Chan, E. Y. (2024) · Journal of Hospitality & Tourism Education · doi:10.1080/10963758.2024.2428613 · Cited by 9
“However, to be sure, this commentary only discusses the possibilities of ChatGPT to promote DEIB in HTM. Research (whether qualitative or quantitative) would be important to demonstrate its potential role and impact in the classroom (Adeshola & Adepoju, 2023; Rudolph et al., 2023).”
Luo, Y., Miao, Y., Zhao, Y., Li, J., Chen, Y., Yue, Y., & Wu, Y. X. (2024) · JMIR Formative Research · doi:10.2196/63188 · Cited by 9
“Now, GPT-4 (OpenAI) and ERNIE Bot 4.0 represent the most recent advancements, having undergone significant enhancements in their capabilities [11,12].”
InfluentialWang, Z. (2025) · International Journal of Information and Education Technology · doi:10.18178/ijiet.2025.15.1.2215 · Cited by 13
Asad, M. M., & Ajaz, A. (2025) · The TQM Journal · doi:10.1108/tqm-09-2024-0357 · Cited by 14
Abouzeid, E., Wassef, R., Jawwad, A., & Harris, P. (2025) · JMIR Medical Education · doi:10.2196/69521 · Cited by 12
“A newer, improved version of ChatGPT (ChatGPT Plus), which uses the GPT-4 Turbo language model, has been developed by OpenAI and launched as a paid subscription version by the end of 2023 [6] In terms of assessment in medical education, ChatGPT has been the most extensively studied chatbot.”
Mapping artificial intelligence research in higher education toward sustainable development
Hong, T. T. M., Tung, N. T. T., & Thanh, N. T. P. (2025) · Discover Sustainability · doi:10.1007/s43621-025-02162-0 · Cited by 16
Abbas, A., Azar, B. B., Mahrishi, M., Núñez, J. L. M., & Mishra, D. (2025) · Equilibrium Quarterly Journal of Economics and Economic Policy · doi:10.24136/eq.3551 · Cited by 16
“Tools like ChatGPT (Essel et al. , 2024), Google Bard, and Gradescope (Rudolph et al. , 2023) help grade essays and provide student feedback based on their distinctive features.”
DEĞERLİ, A. H., & Tatlısu, N. B. (2023) · Journal of Tourism and Gastronomy Studies · doi:10.21325/jotags.2024.1312 · Cited by 8
From Chalkboards to Chatbots: Revolutionizing Education with AI-Driven Learning Innovations
Putri, D. F., & Sain, Z. H. (2025) · Educative Jurnal Ilmiah Pendidikan · doi:10.70437/educative.v3i1.823 · Cited by 59
Influence of LLMs on Learning and Teaching in Higher EducationPreprint
Upadhyay, A., Farahmand, E., Muñoz, I., Khan, M. A., & Witte, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4716855 · Cited by 9
Risks and Harms of Conversational Artificial Intelligence (CAI) Chatbot Use Among US Youth
Hinduja, S., & Patchin, J. W. (2026) · Journal of Adolescence · doi:10.1002/jad.70164 · Cited by 9
Bulawan Aieron, A., Louie, T. R., Laika, B., Warren, P., & Pauline, C. (2024) · International Journal of Advanced Multidisciplinary Research and Studies · doi:10.62225/2583049x.2024.4.3.2809 · Cited by 9
Metadata
- Authors
- Rudolph, J., Tan, S., & Tan, S.
- Venue
- Journal of Applied Learning & Teaching
- Publisher
- Open Access Publishing Association
- Published
- 2023-04-25
- Volume
- 6
- Issue
- 1
- ISSN
- 2591-801X (ISSN-L)
- OpenAlex
- W4366989878
- Cited by
- 421
- References
- 81
- Citations by year
- 2026: 43 · 2025: 125 · 2024: 167 · 2023: 84
- FWCI
- 98.33
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Applied Learning & Teaching (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- chatbots, large language models, ChatGPT, Bing Chat, Google Bard, ERNIE Bot, GPT-4, higher education, artificial intelligence in education, academic assessment, generative AI, AI arms race
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
