The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; none of the 25 sources its claims cite is checked for support yet.
Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: The case of five AI content detection tools
Open accessAbstract
This paper set out to test the accuracy of five AI content tools, GPTZero, OpenAI Text Classifier, Writer.com’s AI Content Detector, Copyleaks AI Content Detector, and Giant Language model Test Room, to detect AI-generated content in the responses generated by ChatGPT, YouChat, and Chatsonic. The responses were generated from these three AI chatbots using English prompts related to applied English language studies. Then, the ChatGPT-generated responses were Google-translated into German, French, Spanish, Southern Sotho, and isiZulu, and inputted into GPTZero for it to detect the AI-generated content in them. Additionally, the ChatGPT-generated responses Google-translated into German, French and Spanish were inputted into Copyleaks AI Content Detector for it to detect the AI-generated content in them. For the ChatGPT-, YouChat-, and Chatsonic-generated responses, Copyleaks AI Content Detector emerged as the top-most performing AI content detector among the five AI content detectors. It was followed by OpenAI’s AI Text Classifier. Concerning the ChatGPT-generated responses that were Google-translated into five languages, GPTZero misidentified all of them as human-produced. For the ChatGPT-generated responses that were Google-translated into German, French and Spanish, Copyleaks AI Content Detector correctly identified three of the German-translated texts, five of the French-translated texts, and all the Spanish-translated texts as AI-generated. Thus, it is evident from this paper that all five AI content detectors seem not yet fully ready to accurately and convincingly detect AI-generated content from machine-generated texts in different contexts. This has dire consequences for AI-generated plagiarism in academic essay writing.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This exploratory study tested five free AI-content detectors on English responses generated by ChatGPT, YouChat, and Chatsonic, plus Google-translated versions of ChatGPT responses. Copyleaks AI Content Detector ranked first for correct English-text identification, while GPTZero misclassified all translated ChatGPT responses in five languages as human-written.
Key findings
- Across the three English response sets, the five detectors ranked by overall correct identification as: Copyleaks AI Content Detector (1), OpenAI AI Text Classifier (2), GPTZero (3), Writer.com’s AI Content Detector (4), and GLTR (5).
- For ChatGPT-generated English responses, Copyleaks correctly classified 5 texts and misclassified 1; GPTZero correctly classified 4 and was indecisive about 2; Writer.com’s detector classified 5 inaccurately as human-generated and 1 accurately; GLTR correctly classified 1 and misclassified 5.
- For YouChat-generated responses, Copyleaks correctly classified 5 and incorrectly classified 2; OpenAI AI Text Classifier correctly classified 3, misclassified 2, and was indecisive about 2; GPTZero correctly classified 1 and misclassified 6; GLTR misidentified all 7 as human-written.
- For Chatsonic-generated responses, Copyleaks correctly classified 5 and misclassified 3; GPTZero correctly classified 3, misclassified 4, and was indecisive about 1; OpenAI AI Text Classifier correctly identified 2, detected 1 incorrectly, and was indecisive about 5; GLTR misclassified 6 and was indecisive about 2.
- GPTZero incorrectly classified all ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and isiZulu as human-written; its highest perplexity scores were 2,478 (German), 221 (French), 361 (Spanish), 1,715 (Southern Sotho), and 938 (isiZulu). Copyleaks correctly classified 3 German-translated texts, 5 French-translated texts, and all Spanish-translated texts as AI-generated.
Methods
- Exploratory study design. Three sets of English prompts in applied English language studies were inputted into ChatGPT, YouChat, and Chatsonic; ChatGPT prompts were queried on 31 January 2023, and YouChat and Chatsonic prompts on 07 March 2023.
- Responses were inputted into five AI content detectors—GPTZero, OpenAI AI Text Classifier, Writer.com’s AI Content Detector, Copyleaks AI Content Detector, and Giant Language Model Test Room (GLTR)—in three phases from 30 March to 02 April 2023.
- Phase 1 fed English-only responses into all five detectors. Phase 2 fed ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and isiZulu into GPTZero. Phase 3 fed German, French, and Spanish translations into Copyleaks AI Content Detector. Analysis compared correct classifications, misclassifications, indecisive results, and reported perplexity or probability percentages.
Limitations
- The paper states a major limitation: it used free-to-use or non-premium versions of the AI content detectors; when data were collected, all five detectors had only free public versions.
- Evident from the text: the dataset is small and limited to prompts from applied English language studies, and only one detector (GPTZero) was used for the five translated languages while Copyleaks was used for three because it did not support Southern Sotho and isiZulu.
- Evident from the text: no texts were deliberately manipulated through editing, paraphrasing, or adding extra spaces, so the study did not test detector robustness against such evasion methods.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 25 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The same kind of classification pattern was yielded by OpenAI AI Text Classifier.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest percentages for human-generated content for these texts were 0% and 100%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest percentages for humangenerated content were 1% and 99%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only one AI detector (OpenAI AI Text Classifier) had two undecided texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“OpenAI AI Text Classifier had the most undecided texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Additionally, the five AI content detectors were not able to distinguish, in clear-cut terms, between AI-generated texts and human-produced texts.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPTZero incorrectly classified all the translated texts in all five languages as human-written.”
- Also stated in: § Discussion · plain
“As mentioned earlier, as regards the ChatGPT responses that were Google-translated into the five aforementioned languages, GPTZero misidentified all of them as humanproduced.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
- Also stated in: § Discussion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It was indecisive about one text.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Writer.com'sAI Content Detector recorded 1% and 99% as its lowest and highest percentages for human-generated content for two texts apiece.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“AI detectors with the joint-most indecisive texts were GPTZero and OpenAI AI Text Classifier.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this context, Writer.com'sAI Content Detector had 0% and 100% as its lowest and highest percentages for human-generated content: 0% and 100%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is particularly the case with the German, Southern Sotho and (isi)Zulu texts, whose highest perplexity scores were 2,478, 1,715 and 938, respectively.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast to the other four AI detectors, GLTR misidentified all seven texts as human-written.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPTZero had the highest perplexity score of 167”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, Copyleaks AI Content Detector trumped all the other four AI content detectors for the most correctly classified texts.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“GPTZero had a perplexity score of 90 for one of its AI-generated texts”
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Copyleaks AI Content Detector, its probability percentage for a humangenerated text was 19.4%”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in the current paper, GPTZero was the third-best performing AI content detector (see Table 1).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It recorded the lowest and highest perplexity scores for these eight texts as 32 and 118.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest percentages for human-generated content were 2% and 100%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the five AI content detectors were judged together in terms of their overall correct identification of the three sets of responses generated by the three AI tools, they rank as follows: Copyleaks AI Content Detector (1); OpenAI AI Text Classifier (2); GPTZero (3); and Writer.com'sAI Content Detector (4), and GLTR (5) (see Table 1).”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Lastly, Copyleaks AI Content Detector correctly classified all the Spanish-translated texts as AI-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It was indecisive about five more texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The highest perplexity score that GPTZero had for this set of texts is 167”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, its lowest and highest probability percentages for human-generated texts were 20% and 98.6%, respectively.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All they could do was to make estimates in percentages (e.g., Writer.com'sAI Content Detector and Copyleaks AI Content Detector) or in probabilistic terms such as likely (e.g., GPTZero) and probability (e.g., Copyleaks AI Content Detector), or in combined percentages and probabilistic terms (e.g., Copyleaks AI Content Detector).Others, such as GLTR, used estimating histograms.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Currently, Copyleaks AI Content Detector does not support Southern Sotho and (isi)Zulu.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“The distinction between the content generated by an AI tool and the one produced by a human, or what Uzun (2023) calls the author factor, becomes trickier to determine as manipulating any form of content tends to elude most of the currently available AI detection tools (see Aremu, 2023;Cai & Cui, 2023;Guo et al., 2023;Uzun, 2023;Ventayen, 2023;Weber-Wulff et al., 2023).”
What the sources say
- Not checked yetcites Aremu, 2023
- Not checked yetcites Cai & Cui, 2023
- Not checked yetcites Guo et al., 2023
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Since most of these AI content detectors are new, not much research has been conducted to evaluate their efficacy, accuracy, and reliability in terms of distinguishing between the content generated by current AI-powered LLM chatbots and the content written by humans.”
- Also stated in: § Methodology · plain
“Evaluating the efficacy, accuracy and reliability of AI content detection tools in differentiating between AI-generated content and human-written content is still a less researched area.”
What the sources say
- Not checked yetcites Aremu (2023)
- Not checked yetcites Cai and Cui (2023)
- Not checked yetcites Guo et al. (2023)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § Methodology · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The reason for choosing GPTZero for the translated responses is that it was the only AI detector that recognised all five languages at the time of conducting the study.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“All these tools make bold claims (mostly undercut by concomitant disclaimers) about their accuracy rate and their reliability in detecting AI-generated content (see Chaka, 2023a, b;Chrome, 2023;Copyleaks, 2023;Kirchner et al., 2023;Outlook Spotlight, 2023;Tech Desk, 2023;Tyrrell, 2023;Weber-Wulff et al., 2023;Wiggers, 2023).”
What the sources say
- Not checked yetcites Chaka, 2023a, b
- Not checked yetcites Chrome, 2023
- Not checked yetcites Copyleaks, 2023
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesWhat the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, when the data for this study were collected, all five AI content detectors had only free versions that were available to the public.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Copyleaks AI Content Detector tended to be the most consistent of the five AI content detectors if the correct identification of texts is used as a factor.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesWhat the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“A major limitation of this paper is that it used free-to-use AI content detectors or the non-premium versions of some of these AI content detectors.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“In their paper, Weber-Wulff et al. (2023) also found that fourteen of the AI detection tools they evaluated were prone to false negatives: they mistook AI-generated texts for human-produced texts.”
What the sources say
- Not checked yetcites Weber-Wulff et al., 2023
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“there are studies that have discovered that text manipulation reduces the detection efficacy of AI detection tools.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, with their instant launch, these AI content detectors seem not yet fully ready to accurately and convincingly detect AI-generated content from machine-generated texts in different contexts.”
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One major reason it misidentified all these translated texts is the higher perplexity scores it assigned to them.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“which is a high score given that AI-generated texts are supposed to have a lower perplexity index compared to human-written texts (Chaka, 2023a;Ofgang, 2023;Tech Desk, 2023;Tyrrell, 2023).”
What the sources say
- Not checked yetcites Chaka, 2023a
- Not checked yetcites Ofgang, 2023
- Not checked yetcites Tech Desk, 2023
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“This, in a way, highlights an element of shakiness related to equating a high perplexity with human-only-written texts in an instance where machines, too, can generate texts with a high perplexity index (Heel, 2023;Lim, 2023;Wiggers, 2023).”
What the sources say
- Not checked yetcites Heel, 2023
- Not checked yetcites Lim, 2023
- Not checked yetcites Wiggers, 2023
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most of them seem to be beset by the algorithmic shortcomings mentioned above.”
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Part of this problem might have to do with the algorithmic configuration of these AI detection tools, which assumes that distributional gaps exist between AI-generated and human-written content.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, of the five AI content detectors tested across the three sets of texts, OpenAI AI Text Classifier and GLTR appeared to be most consistent in their detection rates if the indecisiveness of texts and the misclassification of texts are, respectively, used factors.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“Once these distributional gaps are destabilised by, for example, intentionally adding single space characters before commas in AI-generated content, these tools tend to misrecognise the content output (see Cai & Cui, 2023).”
What the sources say
- Not checked yetcites Cai & Cui, 2023
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 35
- Probably not a substantial claim. 17
- The quote only partly states this claim. 2
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Writer.com's AI Content Detector classified five texts inaccurately as human-generated, while its classification of one text was accurate.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, GPTZero correctly classified four texts as AI-generated, while it was indecisive about two texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest perplexity scores for all these texts were 40 and 167.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest perplexity scores were 38 and 90.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pertaining to YouChat-generated responses, GPTZero classified one text correctly as AI-generated, while it misclassified the six other texts as human-written.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for AI-generated texts were 94% and 99.8%, while its probability percentage for humangenerated text was 19.5%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Copyleaks AI Content Detector classified five texts accurately, but classified one text inaccurately.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For its part, Copyleaks AI Content Detector correctly classified five texts as AI-generated and incorrectly classified the other two texts as humangenerated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for AI generated texts were 99.6% and 99.9%, respectively, with three texts having a 99.9% tie.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Writer.com'sAI Content Detector correctly classified four texts as AI-generated, but misclassified three texts as human-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For GLTR, it correctly classified one text as machine-generated, but misclassified five texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“OpenAI AI Text Classifier classified three texts accurately as AI-generated but misclassified two texts.It was indecisive about two texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, its lowest and highest probability percentages for AI-generated texts were 94.9% and 99.9%, with three texts having a tie at 99.9%.Its probability percentage for the human-generated text was 6.9%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for these texts were 99. 5% and 99.9%, with two texts and four texts tied at 99.5% and 99.9%, respectively.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It, then, correctly identified five French-translated texts as AI-generated but misidentified one text as humanwritten.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With reference to Chatsonic-generated responses, GPTZero correctly classified three texts as AI-generated, but misclassified four texts as human-generated.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Copyleaks AI Content Detector's highest probability percentage of 98.6% for human-generated text was the highest ever for these machine-generated texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GLTR had the highest text misclassification rate and could classify only one text correctly.As such, it ranked last among the five AI content detectors.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for AI-generated texts were 92.3% and 99.9%, with three texts having a 99.9% tie.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Again, in contrast to the other four AI detectors, GLTR misclassified six texts, while it was indecisive about two texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Copyleaks AI Content Detector correctly classified five texts as AI-generated but misclassified three texts as human-written.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its high and lowest perplexity scores for the texts translated into each of these languages were as follows: 110 and 2,478 (German); 57 and 221 (French); 108 and 361 (Spanish); 602 and 1,715 (Southern Sotho); and 651 and 938 ((isi)Zulu).”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both Writer.com'sAI Content Detector and GLTR had the most joint misidentified texts, followed by GPTZero.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This AI content detector correctly classified three German-translated texts as AI-generated but misclassified three texts as human-written.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, Copyleaks AI Content Detector still had the most correct identifications of the texts in this set, but with GPTZero following it.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“OpenAI AI Text Classifier identified two texts correctly as AI-generated but detected one text incorrectly.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Writer.com'sAI Content Detector correctly identified two texts as AI-generated but misidentified six texts as human-written.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In relation to YouChat-generated responses, again, Copyleaks AI Content Detector had more correct identifications of the texts in this set than the other four AI detectors.It was followed by OpenAI AI Text Classifier.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Copyleaks AI Content Detector recorded 19.4% and 19.8% as its lowest and highest probability percentages for human-generated texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for AI-generated texts were 83.7% and 99.9%, while its lowest and highest probability percentages for human-generated texts were 14.8% and 53.4%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Writer.com'sAI Content Detector recorded the contrasting lowest and highest percentages of 2% and 100% for humangenerated content.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For ChatGPT-generated responses, Copyleaks AI Content Detector had more correct classifications of the texts than the other four AI detectors.It was followed by GPTZero and OpenAI AI Text Classifier, which were joint second.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its lowest and highest probability percentages for human-generated texts were 19.4% and 19.8%.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
- Rests on an assumption: The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
- Rests on an assumption: The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In terms of misclassifications of texts (incorrect classifications of texts), GLTR topped the other four AI detectors with eighteen misclassifications; it was followed by Writer.com'sAI Content Detector with fourteen misclassifications.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The other AI detectors had more misclassifications of texts than the correct classifications, with GLTR racking up the most misclassified texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Some of these AI content detectors, such as GPTZero and OpenAI AI Text Classifier, were specifically intended to detect AI-generated content from ChatGPT (Eliaçik, 2023a, b;Ismael, 2023;Iyer, 2023: Kirchner et al., 2023;Lim, 2023;Ofgang, 2033).”
What the sources say
- Not checked yetcites Eliaçik, 2023a, b
- Not checked yetcites Ismael, 2023
- Not checked yetcites Iyer, 2023
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The fact that the five AI content detectors recognised some of the AI-generated texts inputted to them as human-written points to their propensity to false negativity.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This, more than what has been said earlier, highlights the shakiness and, at times, the unreliability of a higher perplexity as an indicator of human-only-written texts.This is more so given that machine-translated texts can have inordinately higher perplexity scores, such as the ones for the five languages in this paper.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“In an instance in which seven AI text detectors, which included OpenAI AI Text Classifier, GPTZero and Copyleaks, were tested to detect AI-generated content created by Claude (a generative AI tool similar to ChatGPT), GPTZero was the top consistent performer.It was followed by ChatGPT and OpenAI AI Text Classifier.”
What the sources say
- Not checked yetcites Wiggers, 2023
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means it identified the first text as 99% AI-generated, while it recognised the second text to be 1% AI-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Again, here, Copyleaks AI Content Detector outperformed GPTZero, an outcome that contrasts with that of Wiggers' testing (2023) of seven AI content detectors in which GPTZero was a top performer.”
What the sources say
- Not checked yetcites Wiggers, 2023
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If this is the case, this points to one of the inherent problems with current AI detection tools: their lack of reliability and consistency in accurately differentiating between AI-generated texts and human-produced texts.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which, again, is a high score for texts that were machine-generated.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“As mentioned earlier, Aremu's (2023) paper that tested the detection capabilities of six AI detection tools found both ZeroGPT and GPTZero to have a higher level of deception robustness and resistance than the other four AI detection tools.”
What the sources say
- Not checked yetcites Aremu, 2023
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These are two extremely contrasting detection rates for these texts when considering that all the texts in this set were ChatGPT-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is the case with the previous instance, these are two diametrically opposed detection rates for texts that were ChatGPT-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Again, these are high probability percentages for machine-generated texts.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, Cai and Cui (2023) found that effecting a mere single space, what they call an extra space, results in text detection evasion.”
What the sources say
- Not checked yetcites Cai & Cui, 2023
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which is a bit high for texts that were exclusively machine-generated.”
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which is a very high score for texts that were machine-generated.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, Guo et al. (2023) point out that by removing indicating words such as Nope, My take is, and Hmm, from human-written content and by removing I regret to hear that, I'm an AI assistant, and Here are steps to follow, from an AI-generated content, most AI detection tools are likely to be tricked in their detection capability.”
What the sources say
- Not checked yetcites Guo et al., 2023
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These inherent algorithmic shortcomings are likely to be there in the premium versions of these AI detection tools as well.”
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But it seems implausible that the premiums versions of these AI detection tools are free of the two algorithmic shortcomings mentioned above.”
Strength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
The AI-generated responses from ChatGPT, YouChat, and Chatsonic, elicited by the specific prompts used, are representative of AI-generated academic content in general.
“The prompts for these three AI chatbots were based on some of the aspects of applied English language studies (AELS).”
“The three sets of AI-generated English responses and the ChatGPT-generated English responses that were Google-translated into the five languages specified above constituted the datasets for this study.”
The paper relies on this without support.
The Google Translate translations of the ChatGPT-generated English responses into German, French, Spanish, Southern Sotho, and isiZulu accurately preserve the content such that detection results on the translated texts reflect the tools' ability to detect AI content in those languages.
“In the second phase, the ChatGPT-generated responses were machinetranslated into five languages using Google Translate and inputted into GPTZero.”
The paper relies on this without support.
The free-to-use or non-premium versions of the five AI content detectors are equivalent to their premium versions for the purpose of evaluating their accuracy.
“All of the five AI content detectors reviewed above were employed in this paper in their free-to-use or non-premium versions.”
The paper relies on this without support.
The classification outputs of the AI content detection tools (e.g., AI-generated vs human-generated, perplexity scores, probability percentages) are valid indicators of their accuracy in detecting AI-generated content.
“The idea of classified is a proxy for predicted as these tools attempt to predict whether a given text response is AI-or human-generated more than just classifying a given text.”
The paper relies on this without support.
The ChatGPT-generated responses Google-translated into German, French, Spanish, Southern Sotho, and (isi)Zulu remain AI-generated texts for the purpose of detection.
“All the ChatGPTgenerated responses were subjected to GPTZero for it to detect if they were AI-generated or human-written (see Table 2).”
“all the ChatGPT-generated responses, which were Google-translated into the three languages as mentioned above, were subjected to Copyleaks AI Content Detector for it to detect whether they were AI-generated or not (see Table 3).”
The paper relies on this without support.
The accuracy and correctness of the Google-translated texts is not relevant to testing the detectors' ability to classify them as machine-generated.
“In this section, what is at issue is not the accuracy and correctness of the Google-translated texts for all five languages but rather GPTZero's ability to classify them correctly as machine-generated texts.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite advancements in AI detection tools, the effectiveness of conventional anti-plagiarism software, such as iThenticate and Turnitin, remains questionable against AI-generated content (Perkins, 2023;Chaka, 2023Chaka, , 2024;;Hassoulas et al., 2023;Ifelebuegu, 2023;Mohammadkarimi et al., 2023;Sullivan et al., 2023).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Chaka, 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The detectors were chosen based on a review of existing studies (Chaka, 2023a; Gao etal., 2022; Krishna etal., 2023; Orenstrakh etal., 2023; Perkins, 2023; Weber-Wulff etal., 2023) as follows:”
From Simple techniques to bypass GenAI text detectors: implications for inclusive education (2024) · cites this paper as Chaka, 2023a
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Predominantly, contemporary detectors designed for LLM-generated text primarily target monolingual applications, often neglecting to evaluate and optimize performance across multiple languages.”
From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Chaka (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Wang et al. (2023b) and Chaka (2023) noted the lack of control observed in multilingual LLM-generated text detectors across various languages, despite the existence of certain language migration capabilities.”
From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Chaka (2023)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, claims of accuracy are contradicted by studies which demonstrate the varied levels of the detectors' ability to distinguish accurately between AI and human-generated content. (Chaka, 2023a; Gao etal., 2022; Krishna etal., 2023; Orenstrakh etal., 2023; Perkins etal., 2023; Walters, 2023; Weber-Wulff etal., 2023).”
From Simple techniques to bypass GenAI text detectors: implications for inclusive education (2024) · cites this paper as Chaka, 2023a
In the citing paper
“Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [10, 51]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Chatsonic perplexity maximum changes from 118 to 167
MinorThe results report 118 as GPTZero's highest perplexity score for Chatsonic texts, whereas the discussion reports 167. The results assign 167 to the YouChat texts instead.
“GPTZero had the highest perplexity score of 167, which, again, is a high score for texts that were machine-generated.”
“It recorded the lowest and highest perplexity scores for these eight texts as 32 and 118.”
Conflicting Copyleaks percentage for a ChatGPT-generated text
MinorThe results give 19.5% for Copyleaks' human-generated classification in the ChatGPT set, but the discussion gives 19.4%.
“For Copyleaks AI Content Detector, its probability percentage for a humangenerated text was 19.4%, which is a bit high for texts that were exclusively machine-generated.”
“Its lowest and highest probability percentages for AI-generated texts were 94% and 99.8%, while its probability percentage for humangenerated text was 19.5%.”
Translated-text perplexity limits are reversed
MinorThe results present 2,478 as the German texts' lowest score by listing it second after "high and lowest," but the discussion explicitly identifies 2,478 as their highest score. The same reversal applies to the Southern Sotho and (isi)Zulu figures.
“Its high and lowest perplexity scores for the texts translated into each of these languages were as follows: 110 and 2,478 (German); 57 and 221 (French); 108 and 361 (Spanish); 602 and 1,715 (Southern Sotho); and 651 and 938 ((isi)Zulu).”
“This is particularly the case with the German, Southern Sotho and (isi)Zulu texts, whose highest perplexity scores were 2,478, 1,715 and 938, respectively.”
Citations
Cites
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806
ChatGPT in higher education: considerations for academic integrity and student learning
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.17 · Cited by 853
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839
The role of ChatGPT in higher education: Benefits, challenges, and future research directions
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.29 · Cited by 561
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.23 · Cited by 421
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., & Wu, Y. (2023) · arXiv · doi:10.48550/arxiv.2301.07597 · Cited by 318
Will ChatGPT get you caught? Rethinking of Plagiarism Detection
Khalil, M., & Er, E. (2023) · doi:10.35542/osf.io/fnh48 · Cited by 259
The death of the short-form physics essay in the coming AI revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023) · Physics Education · doi:10.1088/1361-6552/acc5cf · Cited by 199
Flagged first, then by citations
Research works printed but not linked: 2
- Gerring, J., Mahomey, J., & Elman, C. (2020). Introduction. In C. Elman, J. Gerring & J. Mahoney (Eds.), The production of knowledge: Enhancing progress in social science (pp. 1-14). Cambridge University Press.
- Chaka Chaka. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. org/10.1038/d41586-022-04397-7. · DOI · Paper page
Cited by
Sorted by citations
Generative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
“In recent years, there has been a surge in software and AI models to detect AI-generated content, like text ( ZeroGPT , 9 GPTZero , 10 PlagiarismCheck , 11 GPT Radar , 12 and Writer.com , 13 etc.) [18, 66, 97], images ( AI or Not 14 or Is it AI?”
Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/q3azde36 · Cited by 291
A systematic literature review of empirical research on ChatGPT in education
Albadarin, Y., Saqr, M., Pope, N., & Tukiainen, M. (2024) · Discover Education · doi:10.1007/s44217-024-00138-2 · Cited by 191
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania
Rudolph, J., Mohamed Ismail, F. M., & Popenici, S. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/54fs5e77 · Cited by 189
“…in AI detection tools, the effectiveness of conventional anti-plagiarism software, such as iThenticate and Turnitin, remains questionable against AI-generated content (Perkins, 2023; Chaka, 2023, 2024; Hassoulas et al., 2023; Ifelebuegu, 2023; Mohammadkarimi et al., 2023; Sullivan et al., 2023).”
Method
Metadata
- Authors
- Chaka, C.
- Affiliations
- University of South Africa (ZA)
- Corresponding author
- Chaka Chaka
- ORCID
- Chaka Chaka
- Venue
- Journal of Applied Learning & Teaching
- Publisher
- Kaplan Business School, Australia
- Published
- 2023-07-24
- Volume
- 6
- Issue
- 2
- ISSN
- 2591-801X (ISSN-L)
- OpenAlex
- W4385156842
- Semantic Scholar
- 3c5f627391edb49a0a24e83f27eebb5a63a64d04
- Cited by
- 132
- References
- 22
- Citations by year
- 2026: 20 · 2025: 43 · 2024: 60 · 2023: 9
- FWCI
- 33.10
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Applied Learning & Teaching (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- AI-generated text detection, ChatGPT, GPTZero, multilingual translation, machine translation, academic plagiarism, English language teaching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
