Detection of AI-generated Text: An Experimental Study
Abstract
Artificial Intelligence is an important area in the computer science domain with a wide range of capabilities. With many people saying that they have encountered AI-generated content in the past years very few respondents feel confident in their ability to distinguish between real and fake content. This work aims to explore methods that effectively detect the content generated by AI. The investigation of the available online tools led to the determination of their limitations. Consequently, two separate algorithms were employed for detection and proper analysis: one is a classification-based approach that uses the Support Vector Machine (SVM) model for binary text classification, and the other is a statistical method that analyzes linguistic aspects to distinguish between writing generated by AI and humans. Both normal data, as well as rephrased data, were employed during the stress testing of these algorithms. The accuracy obtained on the raw dataset was very promising. The detection was 99.87% accurate. The accuracy rate of detecting rephrased texts reached 87.58% for the classification model and 84.91% for the statistical model. Additional analysis and observations were made regarding the ability of AI to generate text similar to humans.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Specific attention was paid to datasets that disaggregated results by region (Global North vs. South) and language (Native vs. Non-Native English speakers) to address the equity dimensions of the study.
“Specific attention was paid to datasets that disaggregated results by region (Global North vs. South) and language (Native vs. Non-Native English speakers) to address the equity dimensions of the study[20].”
From Leveraging Artificial Intelligence in Scholarly Publishing (2025) · cites this paper as [20]
Cites
Testing of detection tools for AI-generated text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00146-z · Cited by 582 · cited 1× · see citing works
“Looking into the discrepancy between scientific writing produced by AI and that written by humans, Weber-Wulff et al. [7] observed that: 1) the distributions of text produced by AI and humans differ markedly; 2) the scientific text produced by AI, particularly the scientific abstract, is devoid of…”
BackgroundParaphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1195 · Cited by 150 · see citing works
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · arXiv · doi:10.48550/arxiv.2303.13408 · Cited by 150 · cited 1× · see citing works
“Also, we use paraphrasing to humanize A.I.-generated text [11].”
MethodRADAR: Robust AI-Text Detection via Adversarial Learning
Hu, X., Chen, P.-Y., & Ho, T.-Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-0662 · Cited by 62 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Leveraging Artificial Intelligence in Scholarly Publishing
Fatima Zahra, O., Khouloud EL, M., & Soufiane, O. (2025) · Journal of Applied Social Sciences and Humanities · doi:10.71426/jassh.v1.i1.pp1-8 · Cited by 2 · see citing works
- Authors
- Kalra, M. P., Mathur, A., & Patvardhan, C.
- Affiliations
- Dayalbagh Educational Institute (IN)
- ORCID
- Patvardhan, C
- Venue
- 2024 IEEE 3rd World Conference on Applied Intelligence and Computing (AIC)
- Publisher
- IEEE
- Published
- 2024-07-27
- Pages
- 552-557
- OpenAlex
- W4403864612
- Cited by
- 3
- References
- 4
- Citations by year
- 2026: 1 · 2025: 2
- FWCI
- 0.87
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE World Conference on Applied Intelligence and Computing (AIC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- AI-generated text detection, AI-generated text, support vector machine, linguistic analysis, text classification accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
