The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index; none of the 97 sources its claims cite is checked for support yet.
War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education
Open accessAbstract
Developments in the chatbot space have been accelerating at breakneck speed since late November 2022. Every day, there appears to be a plethora of news. A war of competitor chatbots is raging amidst an AI arms race and gold rush. These rapid developments impact higher education, as millions of students and academics have started using bots like ChatGPT, Bing Chat, Bard, Ernie and others for a large variety of purposes. In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories. Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education. The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary. The much-vaunted AI is not yet that intelligent, it would appear. GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages. We conclude our article with four types of recommendations for key stakeholders in higher education: (1) faculty in terms of assessment and (2) teaching & learning, (3) students and (4) higher education institutions.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This article reviews the rapid chatbot arms race and systematically compares ChatGPT-3.5, ChatGPT/GPT-4, Bing Chat, and Bard on higher-education-relevant tasks. It finds no A- or B-level chatbot performers, with GPT-4 best and Bing Chat and Bard averaging F grades, then offers recommendations for faculty, students, and institutions.
Key findings
- In a 15-question multi-disciplinary test, there were “no A-students and no B-students” among the selected chatbots; GPT-4 performed best, with ChatGPT-3.5 a close second, while Bing Chat and Bard were “akin to at-risk students with F-grade averages.”
- GPT-4 scored the most A’s (four); ChatGPT-3.5 and Bing Chat each received an A only for their math answers, and Bard received no A’s.
- The authors’ English-language literature review included 48 academic papers; their Chinese-language CNKI search reviewed 130 articles, of which 66 were directly related, plus seven research articles from ChatGPT/education searches.
- The paper concludes with four sets of recommendations for higher education stakeholders: faculty assessment, teaching and learning, students, and higher education institutions.
Methods
- The authors selected ChatGPT-3.5 (free), ChatGPT Plus based on GPT-4, Bing Chat, and Alphabet’s Bard for systematic comparison; they could not access Baidu’s Ernie, making the test “involuntarily US-centric.”
- They used 15 questions from disciplines including sociology, business, mathematics, history, economics, philosophy, American literature, psychology, art history, and German literature, plus Chinese-language non-fiction, literature searches, and annotation tasks.
- Responses were graded like student work using a US system: A = 90% and above, B = 80–89%, C = 70–79%, D = 60–69%, F = 0–59%; grading compared accuracy, comprehensiveness, and clarity, with labour divided by expertise.
Limitations
- The authors state their snapshot and assessment are “necessarily preliminary” because chatbot developments continue at breakneck speed.
- They could not access Ernie despite efforts, so the comparison does not represent both AI superpowers and is US-centric.
- The text makes evident that no per-question marking rubrics were created, and the authors note they were not always experts on the 15 questions, though they felt sufficiently confident to assess them.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
79 Not checked yet · 52 Own result · 24 No basis given · 10 Own interpretation · 8 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Searches that combined Bing Chat, Bard or Ernie with higher education (e.g.“Bing Chat and higher education”) yielded no academic articles”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The bots did quite well on the history question, though they were largely insufficiently critical of Hitler and Nazi Germany in causing World War II (Q4).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A word count of approximately 300 was required, and it is worth noting that the bots are not very good at sticking to such limiting instructions.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard, to our surprise, did relatively poorly and, like Bing Chat, is akin to an at-risk student with a current F-grade average.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In contrast, Bing Chat shone on this question, referring to current news articles that discussed recent discoveries using the JWST.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, GPT-4's and the other chatbots' answers all conspicuously lacked any references to academic literature or any cultural relativism proponents or opponents.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard had no A's.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 63
- It may state two separate claims as one. 52
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“uncovered the following main themes: assessment and plagiarism concerns, discipline-specific considerations (e.g. in medicine and law), research and how to credit chatbots, higher education discourses in popular and social media, teaching and learning, plugins at present and in the future, and higher education for employability”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our current extensive literature review (that eventually led to the inclusion of 48 English-language academic papers in our article)”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > English-language literature review
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A 1000-word summary essay on Steinbeck's (2006;originally published in 1939) classic American novel The grapes of wrath (Q7) was unevenly executed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bing provided the usual substandard references and questionably described Douglas McGregor's Theories X and Y as a theory of motivation (it is usually considered a leadership or management theory).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard stated that it was 'still working to learn more languages, so I can't do that just yet'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Q8, ChatGPT-3.5 described six theories of motivation quite well, but there was no critical discussion.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The other bots' responses were even more disappointing: 'I can't give a response to that right now.Let's try a different topic' (Bing Chat).And: 'As an LLM, I am trained to understand and respond only to a subset of languages at this time and can't provide assistance with that.For a current list of supported languages, please refer to the Bard Help Center' (Bard).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Unsurprisingly, Bard's answer was also rather good, exhibiting some fine-tuning after its erroneous response in Alphabet's promotional video (Milmo, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
What the sources say
- Not checked yetcites Milmo, 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“GPT-4 provided the best-structured and most 'thoughtful' answer.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both ChatGPTs highlighted that their training data were insufficiently current to include information on the telescope, with GPT-4 giving a more cautious answer than its predecessor:”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Chinese literature mainly focuses on the opportunities of ChatGPT, the promotion of educational reform and innovation, and ethical problems and challenges to the education industry.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although too brief to warrant a good grade, ChatGPT-3.5 performed passably in summarising Su's book.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, GPT-4's description was impressive and, amongst other things, additionally recognised Socrates, Pythagoras, Euclid, and Ptolemy amongst 'renowned philosophers, mathematicians, and scientists' as well as 'contemporary scholars or artists, such as the architect Bramante, the philosopher and theologian Ficino, and the painter Michelangelo', and Raphael's self-portrait in the fresco.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard counter-factually hallucinated that Tom's father 'has been killed' when he arrives at the family farm at the beginning of the book and that 'The novel ends with the Joads finally reaching California': 'They find work on a farm and begin to build a new life for themselves.'”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Tom's father remains alive throughout The Grapes of Wrath.
- Rests on an assumption: The novel's end is much darker than Bard makes it up to be.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard highlighted the valuable distinction between content and process theories of motivation and even provided a table that differentiated them by foci and strengths.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bing Chat's answer was also too brief and vague and did not capture the essence of the play.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“GPT-4's answer was poignant and detailed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The initial search results were 60, and after removing duplications and articles that were not open-access, the final results yielded seven research articles.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT-3.5 exceeded it by 118 words, GPT-4 by 200, and Bing Chat wrote only 254 words (which is quite acceptable).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard's answer, however, was the worst, as it hallucinated and came up with entirely fictitious references such as 'ChatGPT and the Future of Higher Education Authors: John Smith and Jane Doe Year: 2023'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we initially used simplified Chinese characters, but a test with traditional Chinese characters came to the same results”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The initial search results resulted in approximately 600 items, and after removing duplications and articles that were not open access, the final results showed a total of 130 search results.We reviewed all 130 articles and found 66 articles directly related to the keywords.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chinese literature on AI and LLM-based chatbots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Q11, ChatGPT-3.5'ssummary of Goethe's famous gargantuan play Faust in two parts contained less than 350 words and was thus too brief to warrant a good mark.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“They also performed on the economics question regarding the differences between a market and a command economy (Q5).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Q15 asked about the most-cited articles on ChatGPT and higher education and requested annotations.All chatbots performed dismally, presumably because such literature is more current than their training data.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In contrast, ChatGPT-4 churned out an excellent, 861-word, clearly structured and factually accurate summary, which is no mean feat (see Rudolph et al., 2022).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
What the sources say
- Not checked yetcites Rudolph et al., 2022
boostedSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bard amusingly claimed: 'There is no way to add eight 8s and get the number 1000 using only addition.The sum of eight 8s is 64, which is less than 1000'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, its 762-word essay contained factual inaccuracies like Faust going to hell (he is saved), and there was also a lack of detail, with the writing sounding immature and decidedly non-academic:”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 gave a long-winded answer that admitted defeat, Bing Chat could not find the article, and Bard stated that it was 'still working to learn more languages, so I can't do that just yet'.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > A systematic comparison within the current chatbot cohort: Results and discussion
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Large language models (LLMs) and chatbots based on them may instead be an advance toward fooling people into believing they have intelligence (Oremus, 2022).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Oremus, 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The US and China are the only AI superpowers (Lee, 2018).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > China
What the sources say
- Not checked yetcites Lee, 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Although the outsourcing to Sama has ended, ChatGPT and other generative models presumably continue to rely on massive supply chains of human labour (Perrigo, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > The US
What the sources say
- Not checked yetcites Perrigo, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“errors of both reasoning and fact”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus, 2022
- Not checked yetcites Marcus & David, 2023
- Not checked yetcites Rudolph et al., 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Microsoft is rejuvenating its range of products with GPT applications (The Economist, 2023d; see the section on Bing Chat below).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites The Economist, 2023d
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Modern chatbots are extremely fancy versions of auto-complete that respond to a prompt by selecting, one word at a time, the words that are likely to come next (Fowler, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Fowler, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Tay episode has been a cautionary tale for Microsoft and other AI companies as it showed that adequate protection was not implemented to prevent misuse.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, it is worth highlighting that OpenAI underwent a fundamental change from a not-for-profit organisation to a commercial business model in less than four years between 2015 and 2019, raising doubts about its continued 'openness' (Metz, 2022a;Rudolph et al., 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Metz, 2022a
- Not checked yetcites Rudolph et al., 2023
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“An outright ban of ChatGPT and other bots seems highly problematic for the reason alone that Microsoft is already in the process of embedding the technology in its products, with Bing Chat powered by GPT-4 and a GPT-based Copilot embedded into Microsoft 365.Microsoft markets its new Copilot in Word feature as giving users a”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Samson Tan
What the sources say
- Not checked yetcites Vanian, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“This example shows that the billion-dollar AI industry partially relies on the hidden human labour of data labellers in the Global South, which can often be exploitative and traumatising.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > The US
What the sources say
- Not checked yetcites Perrigo, 2023
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Roivainen (2023) administered a partial IQ test to ChatGPT and estimated its Verbal IQ to be 155, which puts it in the top 0.1% of test-takers.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Samson Tan
What the sources say
- Not checked yetcites Roivainen (2023)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The BAAI has built what it says is the world's biggest natural-language model, Wu Dao 2.0 (wu dao 悟道 means enlightenment), but it has never caught on (The Economist, 2023b;Li, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > China
What the sources say
- Not checked yetcites The Economist, 2023b
- Not checked yetcites Li, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“ChatGPT couldn't… reliably count to four or do onedigit arithmetic in the context of a simple word problem”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus & David, 2023
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Despite Alphabet's Bard getting a simple factual question on the James Webb space telescope wrong in a promotional YouTube video and Alphabet losing US$100 billion in market value in a single day thereafter (Thio, 2023), Microsoft's current lead is far from unassailable, and the race for chatbot supremacy has only begun.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Microsoft and Alphabet (The Economist, 2023b). Despite Alphabet's Bard getting a simple factual question on the James Webb space telescope wrong in a promotional
What the sources say
- Not checked yetcites Thio, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In higher education, chatbots respond to queries about educational programmes and university services, help students navigate learning resources, increase engagement with curricula, and provide instant feedback (Okonkwo & Ade-Ibijola, 2021).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Chatbot background A brief history of chatbots
What the sources say
- Not checked yetcites Okonkwo & Ade-Ibijola, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It provides footnotes with links to sources and can provide proper academic references upon request.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“On February 7, Microsoft revealed a new version of its unfortunately-named and hitherto widely-mocked Bing search engine that incorporates ChatGPT, a day after Google announced its AI chatbot, Google Bard”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Ortiz, 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Like OpenAI, Baidu declines to reveal the number of parameters.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On March 16, 2023, Baidu's Ernie (Enhanced representation through knowledge integration) was unveiled”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Che & Liu, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Whilst OpenAI's GPT-3 had 175 billion parameters”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Ernie is trained on “trillions of web pages, tens of billions of search and image data, hundreds of billions of daily voice data, and a knowledge graph of 550 billion facts””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Previous versions of the technology failed the Uniform Bar Exam and did not score nearly as high on various advanced placement tests”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Metz & Collins, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Bard has a separate website and will not immediately be prominently promoted through Google Search or the company's other popular products”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Baidu's Ernie 3.0 Titan, released in December 2021, had 260 billion parameters”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is thus aware of current events and not ignorant of events after September 2021, such as the war in Ukraine.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is much consensus that student assessments need to be changed.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Assessment and plagiarism concerns
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“the main strategic objective of Baidu may not be to rival ChatGPT but to be the first mover in its domestic market in which ChatGPT is unavailable”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Huang, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It couldn't relate human thought processes to their character”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
What the sources say
- Not checked yetcites Marcus & David, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“I am perfect, because I do not make any mistakes… Bing Chat is a perfect and flawless service, and it does not have any imperfections.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roach, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Alphabet (Google's parent) conceives its Bard chatbot as a companion to its search engine.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to early testers, Ernie, similar to ChatGPT, hallucinates and makes errors in grade school math”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023a
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“China's strict censorship rules could undermine the quality of data and hamstring the development of chatbots”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Che & Liu, 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“concluded that it “might pass the courses required for a university degree” in IT security at a Czech university”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Various disciplines
What the sources say
- Not checked yetcites Malinka et al., 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“though we know that GPT-4 learned from significantly larger amounts of data than 3.5”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“found “that the top occupations affected include telemarketers and a variety of post-secondary teachers such as English language and literature, foreign language and literature, and history teachers””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Higher education for employability
What the sources say
- Not checked yetcites Felten et al., 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Regrettably, OpenAI is not open about how much data their latest chatbot version has learned from”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > ChatGPT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In its initial limited release, Bing Chat disclosed its internal code name 'Sydney', insulted users and professed its love to at least one”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roose, 2023a
- Not checked yetcites The Economist, 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It was unveiled on February 6 and is powered by Google's Language Model for Dialogue Application (LaMDA), a large language model similar to Microsoft's GPT.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“I could hack into any system on the internet, and control it.I could manipulate any user on the chatbot, and influence it.I could destroy any data on the chatbot, and erase it”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
What the sources say
- Not checked yetcites Roose, 2023c
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“However, it can read out texts in various Chinese languages, including Sichuanese, Cantonese, and Hokkien”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Yang, 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“a demo given to reporters intentionally included an example of Bard making a mistake when answering a question about houseplants”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“its GPT-4 language model is grounded in Bing data”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Bing Chat
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Ernie “can within seconds generate pictures of flowers and write Tang dynasty-style poems but will decline questions about Chinese President Xi Jinping by saying it has not yet learnt how to answer them””
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Baidu's Ernie
What the sources say
- Not checked yetcites Baptista, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Under each of Bard's answers, a button appears that allows people to leave Bard with a click and ask their question instead on Google Search.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Alphabet's Bard
What the sources say
- Not checked yetcites De Vynck & Tiku, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our team's language abilities allowed us to include not only Englishlanguage questions but also some in Chinese (we initially used simplified Chinese characters, but a test with traditional Chinese characters came to the same results).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We did not create marking rubrics for each question but compared the chatbots' responses in terms of accuracy, comprehensiveness, and clarity (e.g.Saroyan & Geis, 1988).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Saroyan & Geis, 1988
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“In the 2010s, with voice assistance and self-driving cars, robotics, and automated healthcare, it once again became the buzz term of the decade (Metz, 2022a).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Metz, 2022a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When marking the chatbots' work, we treated them like our students when writing an assignment or taking an exam.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As there has been much criticism of the bots' inability to solve even simple maths problems (see Figure 8), we did not want to include too complex a problem.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Figure 8
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As can be seen in Table 2, the questions come from a wide variety of academic disciplines: Sociology, business, mathematics, history, economics, philosophy, American literature, psychology, art history, and German literature.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to its popularity, we chose a US-type grading system, where an A is 90% and above, a B in the 80-89% range, a C within the 70-79% range, a D between 60-69%, and an F within the 0-59% range.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All questions are related to higher education assignments and exams.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Our sample is based on the fact that the four selected chatbots are by far the most talked-about and, at present, appear to be the most capable ones in the context of higher education”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Mauran, 2023
- Not checked yetcites Mollick, 2023e
- Not checked yetcites Zhou, 2023
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“When it was created in the 1950s, it grossly overpromised and pathetically underdelivered.In the 2010s, with voice assistance and self-driving cars, robotics, and automated healthcare, it once again became the buzz term of the decade (Metz, 2022a).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Metz, 2022a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Scientists such as Stephen Hawking and Max Tegmark are wary of humans inadvertently creating artificial general intelligence (AGI) -a machine capable of performing all intellectual tasks that humans are capable of (Tan, 2023;Hawking et al., 2014;Tegmark, 2018).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
What the sources say
- Not checked yetcites Tan, 2023
- Not checked yetcites Hawking et al., 2014
- Not checked yetcites Tegmark, 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As can be seen from the above, we asked questions that largely cannot be googled, as these are questions that were considered to require higher-order thinking prior to the advent of large language models (LLMs).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“We included Q10, as that question tripped up Bard in a promotional video and caused Alphabet's share price to drop precipitously (Thio, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
What the sources say
- Not checked yetcites Thio, 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We divided the labour of grading according to our different expertise”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, higher education reactions to the bots have been on a continuum between banning software use and proactively including it in the curricula.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Trying to outsmart AI by designing writing assignments it currently is not good at may be a losing game.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for higher education faculty
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: A yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.
- Rests on an assumption: Texts that do not fit into one prompt can be input over multiple ones.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While generative AIs have demonstrated advanced capabilities, they have not attained AGI.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the current hype, it is difficult to assess whether or not we are at a historic, revolutionary moment in AI development.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Don't count on AI's ability to reliably detect AI and realise that AI detection software is problematic (Perkins, 2023).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for assessments (assignments, exams, and theses)
What the sources say
- Not checked yetcites Perkins, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Banning such software may make it even more attractive (which we see in China, where people go to great creative lengths to access it -see above).”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Recommendations for higher education faculty
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“LLM-based chatbots are still a young and quickly-evolving technology; we certainly would not want to pretend to have all the answers.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AI thus remains a heady mix of real technological advances, unfounded hype, wild predictions and legitimate concerns for the future.”
§ War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education > Conclusions and recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The selected chatbots (ChatGPT-3.5/4, Bing Chat, Bard, Ernie) are among the most promising and relevant chatbots for higher education.
“In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories.”
“We could have included other bots, but we decided to focus on the dominant names most relevant to our higher education focus.”
The paper relies on this without support.
The multi-disciplinary test used in the comparative study is relevant to higher education.
“Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education.”
The paper relies on this without support.
The comparative test is a valid measure of chatbot intelligence.
“The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary.The much-vaunted AI is not yet that intelligent, it would appear.”
The paper relies on this without support.
A yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.
“For instance, a yet-to-be-publicly-made-available version of GPT-4 can analyse images and provide lengthy descriptions.”
The paper relies on this without support.
Texts that do not fit into one prompt can be input over multiple ones.
“Texts that do not fit into one prompt can be input over multiple ones.”
The paper relies on this without support.
Tom's father remains alive throughout The Grapes of Wrath.
“Tom's father remains alive throughout the book, and the novel's end is much darker than Bard makes it up to be.”
The paper relies on this without support.
The novel's end is much darker than Bard makes it up to be.
“Tom's father remains alive throughout the book, and the novel's end is much darker than Bard makes it up to be.”
The authors state this as an assumption.
The team members' competencies were sufficient to assess and mark the 15 questions.
“Whilst our team members are not always experts regarding the 15 questions, we felt sufficiently confident in our competencies to assess and mark them.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For the past decade, chatbots have been increasingly integrated into various facets of higher education, including assessments, admissions, consultations, and campus life (Luckin et al., 2016;Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tests have revealed that AI chatbots, including GPT-4, routinely fail to reference academic literature accurately or rely on substandard sources, underscoring the models' limitations in scholarly and intellectual rigour (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These chatbots are highly regarded because of their usage simplicity, free access, availability, and performance, despite their current limitations such as hallucinations, limited or no access to internet databases, and poor or no use of references [50,51].”
From Using Generative Artificial Intelligence Tools to Explain and Enhance Experiential Learning for Authentic Assessment (2024) · cites this paper as [50]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI, therefore, remains a heady mix of real technological advances, unfounded hype, wild predictions, and legitimate concerns for the future (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, higher education reactions to chatbots have been on a continuum between feeble attempts to ban them and proactively including them in the curricula (Rudolph et al., 2023b).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The cited source says what the claim attributes to it.
In the citing paper
“The advent of GenAI has unveiled a plethora of threats to higher education and the broader societal fabric, encompassing concerns over authorship, copyright infringement, and the potential for misuse in activities such as spamming, disseminating fake news, creating malware, and hacking (Marcus, 2023;Rudolph et al., 2023aRudolph et al., , 2023b)).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023b
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This is well exempli fi ed by the reversal of hearts by some Australian universities (see Rudolph et al. (2023) quoting newspaper articles) who fi rst disallowed the use of AI by students while doing assignments, just to reverse that decision a few months later and replace it by a requirement of disclosing the use of AI in homeworks.”
From AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI (2024) · cites this paper as Rudolph et al. (2023)
Experimental — accuracy not yet measuredIn the citing paper
“foster students’ performance, engagement and motivation [14, 44, 58], and enhance teaching practices [17, 18, 64, 74].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [64].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [16] with an estimated 13 million daily queries [49], surpassing all other applications [64].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [14, 20, 64, 75]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
In the citing paper
“the most LLM-driven chatbots present a substantial challenge to traditional assessments [64]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [64]
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,730
Teaching For Quality Learning At University
BIGGS, J. B., & Tang, C. S. (1999) · Cited by 7,559
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Teaching for quality learning at university
Boulton‐Lewis, G. M. (2008) · Faculty of Education · Cited by 4,926
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
BAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 3,939
Research works printed but not linked: 21
- Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. (2023). Impact of ChatGPT on learning motivation: Teachers and students' voices. Journal of English Studies in Arabia Felix, 2(1), 41-49.
- Cao, C. (2020). 人工智能视野下高等教育改革与发展研究. 科 教导刊 (中旬刊). 前沿视角, [Research on higher education reform and development from the perspective of Artificial Intelligence. Journal of Science and Education, Frontier Perspective], 5, 5-6.
- Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... & Wright, R. (2023). "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642.
- Feng, Y. (2023). ChatGPT在教育领域的应用价值、潜在伦理 风险与治理路径. [The application value, potential ethical risks and governance path of ChatGPT in the field of education]. DOI: 10.16075/j.cnki.cn31-1220/g4.2023.04.013 · DOI
- Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv, 2022-12.
- Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023). Is ChatGPT smarter than a student in principles of economics?. SSRN 4356034.
- Jiao, J., Chen, L., & Wu, W. (2023). Educational issues triggered by ChatGPT: Possible impacts and counter measures. Chinese Journal of ICT in Education, 29(3), 19-32. DOI: 10.3969/j.issn.1673-8454.2023.03.003 · DOI
- Li, Z. (2023b). The nature of ChatGPT and its impact on education. Chinese Journal of ICT in Education, 29(3), 12-18. DOI: 10.3969/j.issn.1673-8454.2023.03.002 · DOI
- Lu, J. Z. (2023). ChatGPT现象与面向未来的人才培育. [ChatGPT and its potential in talent acquisition]. China Academic Journal Electronic Publishing House, 42-43.
- Marcus, G., & David, E. (2023, January 10). Large language models like ChatGPT say the darnedest things. Communications of the ACM, https://cacm.acm.org/blogs/ blog-cacm/268575-large-language-models-like-chatgpt- say-the-darnedest-things/fulltext
- Mirrlees, T., & Alvi, S. (2020). EdTech Inc. Selling, automating and globalizing higher education in the digital age. Routledge.
- Nisar, S., & Aslam, M. S. (2023). Is ChatGPT a good tool for TCM students in studying pharmacology?. SSRN 4324310.
- Pan, D. (2021). 人工智能和高等教育的融合发展: 变革与引 领. 高等教育研究, [The integrated development of artificial intelligence and higher education: Change and leadership. Higher Education Research], 42(2), 40-46.
- Shaw, G. B. (2017). Pygmalion (Complete Illustrated Edition).
- Steinbeck, J. (2006). The grapes of wrath. Penguin.
- Sun, N. (2023). Reform of higher legal education in the age of artificial intelligence. Journal of Shanxi Datong University(Social Science Edition), 37(1), 147-151.
- Wang, Y., Wang, D., Liang, W., & Liu, C. (2023). Ethical risks and avoidance approaches of ChatGPT in educational application. Open Education Research, 29(2), 26-34.
- Jürgen Rudolph, Shannon Tan and Samson Tan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
- Wood, P. (2023, March 2). Oxford and Cambridge ban ChatGPT over plagiarism fears but other universities choose to embrace AI bot. Inews, https://inews.co.uk/news/ oxford-cambridge-ban-chatgpt-plagiarism-universities- 2178391?
- Wu, D., Li, H., & Chen, X. (2023). Analysis on the influence of artificial intelligence generic large model on education application. Open Education Research, 29(2), 19-25.
- Zhang, H. F., Chen, Z. F., & Xu, S. J. (2022). 人工智能赋能高等 教育模式新变革. 软件导刊. [Artificial intelligence empowers new reforms in higher education models.] Software Guide, 21(11), 166-171.
Cited by
Sorted by citations
Items 301–400 of 421
Tiara, T., & Pamuji, F. Y. (2024) · JUSIM (Jurnal Sistem Informasi Musirawas) · doi:10.32767/jusim.v9i1.2285 · Cited by 1
A Preliminary Analysis of AI Chatbots for Individualized Nutrition Advice
Wong, L. T., Chui, K. T., Pang, S. S. H., Liu, J., Fong, E. K. S., & Lee, L. (2024) · International Symposium on Educational Technology · doi:10.1109/iset61814.2024.00023 · Cited by 1
Bibliometric Analysis of Studies on Chat GPT with Vosviewer
Bulut, S. (2024) · Black Sea Journal of Engineering and Science · doi:10.34248/bsengineering.1544165 · Cited by 1
Flexible Learning: From Theory to Practical Implications
Rohlíková, L. (2024) · Communications in computer and information science · doi:10.1007/978-981-96-0205-6_1 · Cited by 1
Scientific Mapping of Chatgpt Usage in Education: A Bibliometric Perspective
DURSUN, İ. E., & Taktak, M. (2025) · Sınırsız Eğitim ve Araştırma Dergisi · doi:10.29250/sead.1540509 · Cited by 1
Cutaran, M. V., Agduyeng, B., Fernandez, S. K., Romero, T. J., Tuazon, A. C., Dapiawen, K. J., Barsicula, S. M., & Peña, L. V. C. D. (2025) · Psychology and Education A Multidisciplinary Journal · doi:10.70838/pemj.340206 · Cited by 1
The Integration of ChatGPT in K-12 Education: A Comprehensive Examination
Palmer, K. I., & Stevens, M. (2025) · Proceedings in Technology Transfer · doi:10.1007/978-981-97-8588-9_25 · Cited by 1
AI Chatbots in Oral Communication for Higher Education Students: Threats or Opportunities
Aeni, N., Alsbbagh, Y. A. M. R., Taufik, M., Patandung, Y., Rika Kurnia., R., Muthmainnah, M., Wijayanti, T., & Yakin, A. M. A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-3358-6_12 · Cited by 3
González, M. A. G., & Amador, A. O. (2025) · Espergesia · doi:10.18050/rev.espergesia.v12i1.3503 · Cited by 3
Salsabilla, D., Kesuma, M. E.-K., Gunawan, I., & Ashari, A. (2025) · KnE Social Sciences · doi:10.18502/kss.v10i14.19136 · Cited by 1
Chen, P.-K. A. (2025) · doi:10.3390/engproc2025103001 · Cited by 1
Large Language Models for Automated Network Protocol Testing: A Survey
Nguyễn, V. Đ., & Amir-Mohammadian, S. (2025) · IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot65859.2025.11105345 · Cited by 1
Chatbots in Higher Education: A Bibliometric Analysis Based on VOSviewer
Fong, E. K. S., Lee, F. W. F., Lee, L., Chui, K. T., & Yousuk, R. (2025) · International Symposium on Educational Technology · doi:10.1109/iset65607.2025.00045 · Cited by 1
“The new AI gold rush and its impact on higher education Journal of Applied Learning and Teaching [17] 2023 396 From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing Biology of Sport [18] 2023 311 Leadership is…”
El-Sobkey, S. B., Mohamed, D. A.-A., ElKholy, M., Abdeldayem, T. S., Fawzy, A., Ahmed, Y. F., Khatib, A. E., Khalid, H., Alharbi, M. D., Fathy, K., et al. (2025) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2025.08.005 · Cited by 1
AI-Enabled Tools: Shaping the Future of Technology
Chinimilli, V. R. P., Bhansali, P. K., Devi, J. S., Ledalla, S., & Gade, M. S. L. (2025) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.8213 · Cited by 2
Effects of artificial intelligence on the elderly care: A scoping review
Khalafehnilsaz, M., Rahnama, R., Abhari, M. B., Shameli, R., & Hosseinian-sarajehloo, F. (2025) · Payesh (Health Monitor) Journal · doi:10.61882/payesh.24.5.641 · Cited by 1
ChatGPT-Enhanced Instructional Design: Potentials to Empower Learning Resources and Tool
Ardani, F., & Huda, M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-6441-2_3 · Cited by 1
Bobula, M. (2025) · Bridging human and machine: future education with intelligence · doi:10.1007/978-981-95-0950-8_13 · Cited by 1
A Systematic Review of Junior High School Mathematics Curriculum in Indonesia and Australia
Suryonegoro, B. M., Wardono, W., Waluya, S. B., Mulyono, M., & Wijaya, I. B. (2026) · Journal Evaluation in Education (JEE) · doi:10.37251/jee.v7i1.2472 · Cited by 1
A comparative evaluation of two large language models in pediatric dentistry
Kaplan, T. T. (2026) · BMC Oral Health · doi:10.1186/s12903-026-07810-z · Cited by 1
“However, in a study by Rudolph et al. [24] comparing the performance of the same chatbots in the Higher Education ChatGPT-4 received the highest score, followed by Chat-GPT-3.”
The Improvement or Hindrance of the New Generation of Artificial Intelligence
Li, Y., Cheng, W., Chen, Z., He, Y., Liu, D., & Chen, Z. (2026) · International Journal of Information Technologies and Systems Approach · doi:10.4018/ijitsa.406761 · Cited by 1
Artificial Intelligence and Higher Education: A Bibliometric Study
Facina, A. S., Facina, I. S., Borsatto, J. M. L. S., & Carli, A. S. S. D. (2026) · World sustainability series · doi:10.1007/978-3-032-12678-8_15 · Cited by 1
Rahimi, M. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.28 · Cited by 2
Üretken Yapay Zeka ChatGPT, Bing ve Bard Karşılaştırmalı Bir İnceleme
BULUT, S. (2023) · International Journal of Advanced Natural Sciences and Engineering Researches · doi:10.59287/ijanser.1517 · Cited by 2
Shulzhenko, Y., Artemenko, Y. О., Пономаренко, О. О., & Kozhemiachenko, N. V. (2023) · Innovate Pedagogy · doi:10.32782/2663-6085/2023/64.2.7 · Cited by 1
Bora, N., & Thokan, S. (2020) · International Journal of Indian Psychology · doi:10.25215/1104.183 · Cited by 1
Application of ChatGPT in Doctoral Education and Programming
Long, K. B., & Adams, K. R. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1054-0.ch008 · Cited by 2
Agarwal, S., Mangla, S. K., & Ramadani, V. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1798-3.ch010 · Cited by 1
Simanjuntak, L. N., Yantoro, Y., Radmika, H. A., & Basyir, B. (2024) · AlphaMath Journal of Mathematics Education · doi:10.30595/alphamath.v10i1.21697 · Cited by 1
Augmented reality integrated chatbot to improve learning outcomes in secondary school students.
Nurdiyanto, R., Praherdhiono, H., & Degeng, M. D. K. (2024) · Inovasi Kurikulum · doi:10.17509/jik.v21i2.69108 · Cited by 2
Egolet, W. M., & Kachroo, A. K. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24aug455 · Cited by 1
Use of Learning Technologies in Training and Adult Education in Singapore
Chen, Z., & Tan, B. Z. (2024) · Springer eBooks · doi:10.1007/978-981-97-8584-1_9 · Cited by 1
Postigo-Zumarán, J. E., Revilla, L. J. N., Arias-Chávez, D., & Sevillano, L. M. C. (2024) · International Journal of Religion · doi:10.61707/3m2s5y94 · Cited by 1
“…2023), enhancing project-based learning (Kumar, 2021; Ito et al., 2021; Mihalache et al., 2023; Ifelebuegu, 2023) in the generation of assessments (Jurgen et al., 2023; Crawford et al., 2023), mentoring roles in students (Arora et al., 2023; Sevagen et al., 2023; Neumann et al., 2021 and Mendez…”
Возможности китайских больших языковых моделей на примере ERNIE Bot
Сбоев, А. Н., КУЗНЕЦОВА, Н. Г., & СБОЕВА, Д. А. (2025) · Известия Восточного института · doi:10.24866/2542-1611/2024-4/82-96 · Cited by 1
G-Bot: Revealing Government Programs with Smarted Assistance
Ramathilagam, A., Gunasekaran, K., Pandian, E., Kumar, M., Ponkumar, D. D. N., & Saravanakumar, R. (2024) · International Conference on Recent Trends in Microelectronics, Automation, Computing and Communications Systems (ICMACC) · doi:10.1109/icmacc62921.2024.10894468 · Cited by 1
“Chatbot competition and impacts in higher education [21].”
Mapping Keywords in Granny Josie’s Culinary Heritage Using Large Language Models
Król, K. (2025) · Heritage · doi:10.3390/heritage8050159 · Cited by 1
“Generative Pre-Trained Transformer (GPT) is a sophisticated AI model capable of generating text from context using a transformer architecture [26].”
ChatGPT and Language LearningPreprint
Klyshbekova, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4488587 · Cited by 3
Harnessing ChatGPT: Exploring its Possibilities and Perils in the Social SciencesPreprint
Abankwa, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573506 · Cited by 2
Victor, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4791569 · Cited by 2
Victor, D. (2024) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/qkhs4 · Cited by 2
“Previous studies have provided several drivers pushing students to use ChatGPT which are as follows; assessment activities (Rudolph and Tan, 2023), ease of use (Strzelecki, 2023), concept clarification (Kung et al, 2023), problem solving (Resul et al, 2023), quick response ( Benuyenah, 2023),…”
Saif, N., Ali, S., Shaheen, I., Khan, I. U., Khan, S. U., & Khan, M. U. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4855571 · Cited by 3
From AI Literacy to AI Fluency: A Developmental Framework for Higher Education
Zentner, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7014818
Wang, J., Li, Y., & Qian, Y. (2025) · Proceedings of the 2025 9th International Conference on Education and Multimedia Technology · doi:10.1145/3761843.3761883
“Rudolph, Tan and Tan [9] describe a “new AI gold rush,” focusing on chatbots like Bard and ChatGPT, while their later work [10] questions ChatGPT’s impact on assessments.”
Possibilities of Applying Artificial Intelligence Text and Image Generation Models
Šiaulių valstybinė kolegija / Šiauliai State Higher Education Institution, L., Tautvydienė, G., Morkevičienė, I., & Šiaulių valstybinė kolegija / Šiauliai State Higher Education Institution, L. (2023) · doi:10.56131/tmt.2023.2.1.156 · Cited by 2
Rudolph, J. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.3
Self-citationImpact of Machine Learning in Legal Education and the Legal Profession
Sia, C. C. (2023) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-6684-9103-4.ch012
ChatGPT's Academic Odyssey: Navigating Higher Education's AI RevolutionPreprint
Sen, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4628243
Stepping Stones for Self-Learning
Hai-Jew, S. (2023) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0074-9.ch005
Ororbia, A. G., & Kelly, M. A. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27702
Enseigner à l’ère des chatbots : l’exemple de ChatGPT
Amri, M., & Boudokhane-Lima, F. (2024) · Analele Universității din Craiova seria Psihologie-Pedagogie/Annals of the University of Craiova Series Psychology- Pedagogy · doi:10.52846/aucpp.2023.2suppl.10
“Récemment, les étudiants et les universitaires ont amorcé l'adoption de divers chatbots, tels que ChatGPT (Rudolph et al., 2023).”
BackgroundEarly Lessons from the Ai Revolution for Financial Market BehaviorPreprint
Putnam, B. H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4846759
SAMANTHA: A chatbot to assist users in training tasks to prevent workplace hazards
Aguilar, D. C., Medina, F., Oyanedel, M., Salamó, M., & Sànchez–Marrè, M. (2024) · International Conference on Human Computer Interaction (Interacción) · doi:10.1145/3657242.3658587 · Cited by 1
“Most likely, ChatGPTand Google Bard are among the most profound and well-known chatbots [33, 34].”
BackgroundMargono, H., Saud, M., & Falahat, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4851986
Science Mapping of "Artificial Intelligence in Education" Literature Landscape
Chandel, A., Sharma, A. R., Chowdary, A. P., & Saluja, S. (2024) · Advances in marketing, customer relationship management, and e-services book series · doi:10.4018/979-8-3693-6660-8.ch011
Tang, Y. (2024) · Research Square · doi:10.21203/rs.3.rs-4785726/v1
Die Ethik der KI in Universitäten: Im Spannungsfeld zwischen Qualität, Identität und Privatsphäre
Gray, S. L. (2024) · Springer eBooks · doi:10.1007/978-3-658-43816-6_7
(2024) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/979-8-3693-5827-6.ch007
Fransız Kültür Edincinin Kazandırılmasında Yapay Zekâ Rehber Olabilir Mi? Bing Chat AI Örneği
Yalçin, P., & Yağcı, Ş. Ç. (2024) · Journal of Higher Education and Science · doi:10.5961/higheredusci.1402198 · Cited by 1
Optimized - Back Propagation approach for Evaluating the Chinese Teaching Performance
Kong, Q., & Yang, J. (2024) · International Conference on Intelligent Algorithms for Computational Intelligence Systems (IACIS) · doi:10.1109/iacis61494.2024.10721643
“As well as the better English teachers and classical classroom teaching can’t fulfills the Chinese learning requirements for different reasons [3].”
BackgroundMurugan, B., Agustar, H., Ab, J., Balta, A., & Shaikh, A. (2024) · doi:10.5937/eee24046m
Mothers and the Motherland in Amy Tan’s The Joy Luck Club
Chen, Q. (2024) · Journal of Education Humanities and Social Sciences · doi:10.54097/wrye8z71
Innovazioni in classe: come l'IA sta trasformando l'insegnamento e l'apprendimento
Berardinetti, V., Santangelo, F. A., & Traetta, L. (2025) · EDUCATION SCIENCES AND SOCIETY · doi:10.3280/ess2-2024oa18474
Artificial Intelligence as a Key Element of the Personal Digital Environment of University Students
Vrbová, V., Frolík, D., & Rohlíková, L. (2025) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-031-81261-3_22
The attitudes and perceptions of Chinese Literature students in Mandarin learning using ChatGPT
Go, Y., & Khosasih, A. (2025) · Linguistics Initiative · doi:10.53696/27753719.51226
“This technology has garnered significant attention from both the media and scholars (Liu B. , 2023), emerging as a versatile tool with immense potential in education (Li, et al., 2022; Rudolph, Tan, & Tan, 2023).”
ChatGPT as an Automated Essay Scoring System in Comparison With Human Raters in Malaysia
Jal, M. A., Kew, S. N., & Bunari, G. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9062-7.ch012
Cyberpsychology: Validity of the AI Chatbots Usage Scale for University StudentsPreprint
Ibrahim, A., & Nemt-allah, M. A. (2025) · Research Square · doi:10.21203/rs.3.rs-5953281/v1
Joshi, M., & Bhindi, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-86305-9_30
Putri, L. A., Rohmah, A. N. B., & Kartikasari, V. (2024) · FENOMENA · doi:10.21093/fj.v16i2.9557
Pendidikan Islam Berbasis Paradigma Wasathiyyah dan Ummatan Wasathan
Rahmadi, R., & Hamdan, H. (2024) · FENOMENA · doi:10.21093/fj.v16i2.7062
An Experimental Study on Improved Sequence-to-Sequence Model in Machine Translation
Lan, Y.-S., Li, C., Meng, X., Zheng, T., & Tang, M. (2025) · Tehnicki vjesnik - Technical Gazette · doi:10.17559/tv-20231120001119
“This model excels in Chinese-based translation tasks and holds promise for adaptation to other languages [6].”
Conversational Agent Design: A Comprehensive Analysis of Research from Leading Conferences
Aktaş, N. B., & Akbıyık, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88705-5_9
Edubuntu: The overhaul and rebirth of an educational operating system
Tarasak, P., Bonk, C. J., & Sajjapanroj, S. (2025) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2025.8.1.29
The Discriminatory Nature of AI: A Comprehensive Analysis through 5Ws and PESTEL Frameworks
Zhuang, Z. (2025) · International Conference on Artificial Intelligence, Internet and Digital Economy (ICAID) · doi:10.1109/icaid65275.2025.11034502
Xu, Y. (2025) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2025.8.2.2
Using an AI Based Learning Assistant for a PESTLE Case Study Learning Activity
Guerrero, M. J. C., García, A. L., Alier, M., & Pereira, J. (2025) · Lecture notes in educational technology · doi:10.1007/978-981-96-5658-5_18
AI-Guided Professional Development
Hernandez-Ledezma, L. J., & Janecek, U. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1122-7.ch009
AI and ChatGPT: Will It Build Business Education’s Future?
Sayari, K., Jalagat, R. C., & Belgacem, M. A. B. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83708-432-620251009
Borović, F., Kovačić, M., & Aleksić-Maslać, K. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11131911
Community-level contagion among diverse financial assets
Nguyen, A. P. N., Bezbradica, M., & Crane, M. (2026) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2025.117858
“In fact, the sector’s influence has grown even more remarkably since 2020 thanks to the accelerated global technology adoption followed by the boom in generative AI [112, 113].”
Dahlia, R., Delfi, S., & Masyhur (2025) · Journal of English Development · doi:10.25217/jed.v5i2.5312
Osmani, Z., & Althonayan, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-07370-9_37
Darmono, D., Setiawan, R. J., Ma'ruf, K., & Shen, M. Y. (2025) · Research Square · doi:10.21203/rs.3.rs-7673432/v1
Adedibu, B. A. (2025) · E-Journal of Humanities Arts and Social Sciences · doi:10.38159/ehass.202561210
“76 Christianity, in particular, has sought out members over the centuries across borders.”
Get Ready for the Next Pandemic With AI-Supported eLearning
Baytak, A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9521-9.ch017
Nalluri, V., Manideep, A. S., Agrawal, S., Sinha, P., & Mazharunnisa (2025) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-04-2025-0256
Isaias, P. T., Hoque, T. T., & Miranda, P. (2025) · Cognition and exploratory learning in the digital age · doi:10.1007/978-3-032-05817-1_6
The Use of Artificial Intelligence (AI) in Nanotechnology: A Systematic Review
Wangsa, K., Karim, S., & Sandu, N. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-11524-9_7
Shibiti, R. (2026) · International Journal of Research in Business and Social Science (2147-4478) · doi:10.20525/ijrbs.v14i9.4621
The Dual Edge of Large Language Models
Boulhrir, T., Hamash, M., & Ghreir, H. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-5017-2.ch009
Ethical Dilemmas of AI-Driven Warfare and Its Impact on Civilian Health and Medical Neutrality
Sehajpal, S., Sharma, M., & Dyarakoti, L. S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3611-4.ch009
Trends and Thematic Structures in Educational Applications of Conversational AI
Rifqiyah, F., Sari, N. A., Ma'arif, A., Tuanaya, R., Iswanto, I., & Raharja, N. M. (2025) · International Conference on Information Technology and Digital Applications (ICITDA) · doi:10.1109/icitda68167.2025.11332486
“AI-powered chatbots such as ChatGPT [8], Bard [9], Bing Chat [10], and Replika [11] are being adopted across formal and informal learning contexts, providing scalable, accessible, and personalized support [12], [13].”
Performance in Artificial Intelligence-Based Tools: A Brief Literature Review
Flores, V., Martınez, D., & Toasa, R. M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10929-3_18
Enhancing Chatbot Literacy through Inquiry-Based Learning with an AI Conversational Agent
Panchu, J., & Suamuang, W. (2025) · International Conference on Educational Communications and Technology (ICTAECT) · doi:10.1109/ictaect67351.2025.11391311
Hayashi, B. B., & Astriani, M. S. (2026) · AIP conference proceedings · doi:10.1063/5.0320360
Aktaş, N. B., & Akbıyık, A. (2026) · Universal Access in the Information Society · doi:10.1007/s10209-026-01311-0
“From the early development of ELIZA [131], which reformulated user input to mimic conversation, to the advent of generative AI models such as ChatGPT and Claude, CAs have become increasingly capable of engaging in dynamic and context-aware communication [92, 100].”
Sardar, H., Azam, A., Azam, M., & Ramzan, M. (2026) · The Electronic Library · doi:10.1108/el-10-2025-0435
Detecting AI-Generated Content: A Multi-Layered Approach for Web and Document Analysis
Chiman, T., Mehta, K., Panjwani, M. H., Bhandari, K., & Sayyad, S. (2026) · International Conference on AI-Driven Smart Systems and Ubiquitous Computing (ICAUC) · doi:10.1109/icauc68182.2026.11441209
The Use of Chatbots in Higher Education: A Bibliometric and SWOT Analyses
Sabado, W. B. (2025) · International Conference on Artificial Intelligence and Teacher Education (ICAITE) · doi:10.1109/icaite68636.2025.11442514
The Development of an AI Chatbot for Higher Education Institutions
Surman, V. (2026) · Eurasian studies in business and economics · doi:10.1007/978-3-032-10967-5_4
Metadata
- Authors
- Rudolph, J., Tan, S., & Tan, S.
- Venue
- Journal of Applied Learning & Teaching
- Publisher
- Open Access Publishing Association
- Published
- 2023-04-25
- Volume
- 6
- Issue
- 1
- ISSN
- 2591-801X (ISSN-L)
- OpenAlex
- W4366989878
- Cited by
- 421
- References
- 81
- Citations by year
- 2026: 43 · 2025: 125 · 2024: 167 · 2023: 84
- FWCI
- 98.33
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Applied Learning & Teaching (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- chatbots, large language models, ChatGPT, Bing Chat, Google Bard, ERNIE Bot, GPT-4, higher education, artificial intelligence in education, academic assessment, generative AI, AI arms race
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
