CoChat
Sign Up
PreprintCited by 3Authorship Attribution and ProfilingMisinformation and Its ImpactsTopic Modeling

Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement

Open access
Cheng, Zihao, Zhou, Li, Jiang, Feng, Wang, Benyou, Li, Haizhou·2025·Proceedings of the ACM on Web Conference 2025
DOI 10.1145/3696410.3714770
Open this paper in CoChat — free accountOpen PDF

Abstract

The rapid development of large language models (LLMs), like ChatGPT, has resulted in the widespread presence of LLM-generated content on social media platforms, raising concerns about misinformation, data biases, and privacy violations, which can undermine trust in online discourse. While detecting LLM-generated content is crucial for mitigating these risks, current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-LLM collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content. This approach introduces two novel tasks: LLM Role Recognition (LLM-RR), a multi-class classification task that identifies specific roles of an LLM in content generation, and LLM Involvement Measurement (LLM-IM), a regression task that quantifies the extent of LLM involvement in content creation. To support these tasks, we propose LLMDetect, a benchmark designed to evaluate detectors' performance on these new tasks. LLMDetect includes the Hybrid News Detection Corpus (HNDC) for training detectors, as well as DetectEval, a comprehensive evaluation suite that considers five distinct cross-context variations and two multi-intensity variations within the same LLM role. This allows for a thorough assessment of detectors' generalization and robustness across diverse contexts. Our empirical validation of 10 baseline detection methods demonstrates that fine-tuned Pre-trained Language Model (PLM)-based models consistently outperform others on both tasks, while advanced LLMs face challenges in accurately detecting their own generated content. Our experimental results and analysis offer insights for developing more effective detection models for LLM-generated content. This research enhances the understanding of LLM-generated content and establishes a foundation for more nuanced detection methodologies.

Integrity signals

Registry

Crossref · 10.1145/3696410.3714770

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

19 of 23 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    When machine-generated content spreads quickly on social media, it increases the danger of misinformation.

    “The rapid spread of such content across social media heightens the risk of misinformation [42,46], challenging the accuracy and credibility of digital information.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [42] 42. Giovanni Puccetti, Anna Rogers, Chiara Alzetta, Felice Dell'Orletta, and Andrea Esuli. 2024. AI" News" Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian. arXiv preprint arXiv:2406.12128 (2024). source

      The Introduction (p. 1) says AI-generated news-like sites can deceive readers, and Broader Impacts (p. 10) identifies misinformation as a possible harm. The paper does not establish that this content spreads rapidly across social media or that such spread heightens the risk; its discussion concerns content-farm websites. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [46] 46. Jinyan Su, Claire Cardie, and Preslav Nakov. 2024. Adapting Fake News De- tection to the Era of Large Language Models. In Findings of the Association for Computational Linguistics: NAACL 2024, Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 1473-1490. https://doi.org/10.18653/v1/2024.findings-naacl.95 source

      The Ethics and Broader Impact section (page unavailable) says AI enables disinformation production at an unprecedented scale and that sensational information spreads faster than truthful news. It does not say that machine-generated content itself spreads rapidly across social media, so the claim's specific account of how the risk increases is unsupported. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Findingnone given

    The pattern of cross-cultural difficulty is consistent when evaluating intersectional aspects.

    “This pattern is consistent when evaluating intersectional aspects.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The cross-context variations in DetectEval are valid proxies for real-world distribution shifts.
  • Findingown data

    State-of-the-art LLMs struggle to reliably identify text that they themselves produced.

    “while advanced LLMs face challenges in accurately detecting their own generated content.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The five cross-context variations (time, prompt, source, cultural, domain) adequately represent real-world distribution shifts for LLM-generated content detection.
    • Rests on an assumption: The multi-intensity variations (variable-length extension and multi-staged polish) adequately capture within-role differences in LLM involvement.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.
  • Findingnone given

    Fine-tuned PLM-based detectors achieve outstanding detection performance across all LLM roles.

    “In contrast, fine-tuned PLM-based detectors exhibit outstanding detection performance across all LLM roles”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
  • Findingreasoning

    Detectors using only linguistic features, without language models, still achieve objective results.

    “Notably, detectors using only linguistic features, without language models involved, still achieve objective results”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
  • May combine two claims. 23
  • Classified as probably not a substantial claim. 12

  • May combine two claims.

    Premisecitation

    LLMs serve diverse roles in assisting human authors, with varying degrees of AI involvement across writing stages.

    “In real-world applications, LLMs play diverse roles, adapting to various user needs [9,10].”
    • Also stated in: § 1 INTRODUCTION · plain
      “These models assist in different stages of the writing process-from organizing ideas and drafting to refining text-resulting in varying degrees of AI involvement across contexts.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [9] 9. Zamzam Bibi and Aqsa Atta. 2024. The role of ChatGPT as AI English writing assistant: A study of student's perceptions, experiences, and satisfaction. Annals of Human and Social Sciences 5, 1 (2024), 433-443. source

      The Conclusion reports that ChatGPT helps students with different parts of English content creation and adapts to different writing choices and styles. The section on revision and editing also describes its use for generating ideas, drafting, rewriting, and editing. These passages support varied writing assistance, but the paper studies ChatGPT in student English writing, not LLMs across real-world applications and user needs generally. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [10] 10. Tigran Boynagryan and Anna Tshngryan. 2024. AI Writing Assistant: A Com- prehensive Study. (2024).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Text produced entirely by an LLM is usually simpler to spot and often has less quality control, whereas drafts written by a person and then improved by an LLM tend to be higher quality and much harder to flag as machine-generated.

    “Fully LLM-generated content is generally easier to detect and often lacks quality assurance, whereas human-authored drafts refined by LLM tend to achieve higher quality standards and are significantly harder to identify as LLM-generated [59].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [59] 59. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F Wong, and Lidia S Chao. 2023. A survey on llm-gernerated text detection: Necessity, methods, and future directions. arXiv preprint arXiv:2310.14724 (2023). source

      Section 7.3 (pp. 44–45) says mixed human/LLM text and human-edited LLM output pose detection challenges. It does not establish the claim's comparison with human drafts refined by an LLM, or say those drafts meet higher quality standards. Section 2.1 (p. 3) also cautions that fully LLM-generated text can itself be difficult to distinguish from human writing. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Role-playing prompts improve LLM performance consistently.

    “This approach leverages social role assignment, which has been shown to improve LLM performance consistently [62].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [62] 62. Xiao Yu, Kejiang Chen, Qi Yang, Weiming Zhang, and Nenghai Yu. 2024. Text Fluoroscopy: Detecting LLM-Generated Text through Intrinsic Features. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). As- sociation for Computational Linguistics, Miami, Florida, USA, 15838-15846. https://doi.org/10.18653/v1/2024.emnlp-main.885 source

      The Introduction (p. 1) discusses LLM capabilities, while Methods (p. 2) and Ablation studies (p. 5) address detecting LLM-generated text. Searches for social role assignment, role-playing prompts, and persona prompting found no passage about their effect on LLM performance. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Existing detection methods are mostly binary classifiers and cannot distinguish complex LLM-generated content such as mixed human-LLM input.

    “current methods often focus on binary classification, failing to address the complexities of realworld scenarios like human-LLM collaboration.”
    • Also stated in: § 1 INTRODUCTION · plain
      “current detection methods, which are largely limited to binary classification [14,18,35,50,52,54,55,60], fail to distinguish the complexity of LLM-generated content like mixed human-LLM input.”
    Quote in the paperSubstantial claim8 of 8 sources checked
    • cites [14] 14. Joseph Cornelius, Oscar Lithgow-Serrano, Sandra Mitrovic, Ljiljana Dolamic, and Fabio Rinaldi. 2024. BUST: Benchmark for the evaluation of detectors of LLM-Generated Text. In Proceedings of the 2024 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 8029-8057. https://doi.org/10.18653/v1/2024.naacl-long.444 source

      Section 2 (Related Work; page not available) describes classification of text as human-written or machine-generated as a prevalent detection strategy. Section 4.2.1 (Detectors evaluation; page not available) also describes binary decisions by two evaluated detectors. The paper does not assess mixed human–LLM writing or establish that current methods fail to handle collaboration, so that part of the claim is unsupported. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [18] 18. Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 12463-12492. https://doi.org/10.18653/v1/2024.acl-long.674 source

      Section 5.2, Detector Evaluation (page unavailable), evaluates detectors by converting their scores into binary human-written versus machine-generated predictions. The Conclusion reports that detectors lack robustness in several settings, but the paper does not establish that they fail specifically on mixed human–LLM writing or collaboration. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [35] 35. Shixuan Ma and Quan Wang. 2024. Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 17538-17553. https://doi.org/10.18653/v1/2024.emnlp-main.971 source
    • cites [50] 50. Zhen Tao, Zhiyu Li, Dinghao Xi, and Wei Xu. 2024. CUDRT: Benchmarking the Detection of Human vs. Large Language Models Generated Texts. arXiv preprint arXiv:2406.09056 (2024). source

      Section 2.2 describes model-based detectors classifying texts using human- and LLM-generated examples. Section 3.4.1 says LLM text is often interspersed with human text and that this raises detection requirements. The paper supports the concern about mixed content, but does not establish that current methods generally fail to address human–LLM collaboration. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [52] 52. Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 13706-13724. https://doi.org/10.18653/v1/2023.findings- emnlp.916 source

      The Introduction (p. 2) says substantial research focuses on distinguishing human from AI-generated text, and Related Work (p. 3) describes detectors trained on human and AI texts. This supports the point that detection commonly addresses a human-versus-AI distinction, but the paper does not say that most methods are binary classifiers or that they fail on mixed human–LLM content or collaboration. (LLM, conf=0.79)

      Experimental — accuracy not yet measured
    • cites [54] 54. Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. 2024. Ghostbuster: De- tecting Text Ghostwritten by Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, He- lena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 1702-1717. https://doi.org/10.18653/v1/2024.naacl-long.95 source

      The Additional information section says the paper’s main modeling work focuses on document-level detection and that paragraph-level detection is useful when AI assisted with only portions of an assignment. The Conclusion proposes detection within mixed human- and AI-authored documents as future work. These passages support the concern about mixed authorship, but do not establish that existing methods generally cannot distinguish it. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [55] 55. Ivan Vykopal, Matúš Pikuliak, Ivan Srba, Robert Moro, Dominik Macko, and Maria Bielikova. 2023. Disinformation capabilities of large language models. arXiv preprint arXiv:2311.08838 (2023). source

      The Detection methodology (page unavailable in our copy) says that all detectors evaluated in this study are binary classifiers. Section 7 describes testing them on fully LLM-generated and human-written articles, but the paper does not establish that current methods generally fail on mixed human–LLM content or human–LLM collaboration. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [60] 60. Kai-Cheng Yang and Filippo Menczer. 2024. Anatomy of an AI-powered malicious social botnet. Journal of Quantitative Description: Digital Media 4 (2024). source

      Section 2.2 says black-box detection methods are often framed as binary classification problems and describes challenges in detecting LLM-generated text. It also describes white-box methods, so it does not establish that detection methods overall are mostly binary. Neither section 2.2 nor the discussion of detector limitations in section 6 addresses whether methods fail on mixed human–LLM content or human–LLM collaboration; page numbers are unavailable in our copy. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    LLM-generated content inevitably contains hallucinations.

    “While LLMgenerated content can match the fluency of professional writing, it inevitably contains hallucinations [1,4]-misleading information that appears credible but lacks factual accuracy.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [1] 1. Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, et al. 2024. Factuality challenges in the era of large language models and opportunities for fact-checking. Nature Machine Intelligence 6, 8 (2024), 852-863. source
    • cites [4] 4. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi source

      Section 4, “Factuality and Hallucination,” says LLMs are susceptible to generating untruthful information and reports hallucinations in several ChatGPT tasks. It does not say hallucinations are inevitable in all LLM-generated content; it also notes that some extrinsic hallucinations are factual. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    LLM-generated content often exhibits inherent biases and can be misused for malicious purposes.

    “Additionally, LLM-generated content often exhibits inherent biases [20] and can be misused for malicious purposes [37,53], further complicating efforts to maintain information integrity.”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [20] 20. Xiao Fang, Shangkun Che, Minjia Mao, Hongzhe Zhang, Ming Zhao, and Xiao- hang Zhao. 2024. Bias of AI-generated content: an examination of news produced by large language models. Scientific Reports 14, 1 (2024), 5224. source
    • cites [37] 37. Filippo Menczer, David Crandall, Yong-Yeol Ahn, and Apu Kapadia. 2023. Address- ing the harms of AI-generated inauthentic content. Nature Machine Intelligence 5, 7 (2023), 679-680. source
    • cites [53] 53. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. Authorship Attribu- tion for Neural Text Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 8384-8395. https://doi.org/10.18653/v1/2020.emnlp-main.673 source

      This source is cited for the malicious-use clause; the bias clause is cited to [20]. The available passages contain only title and bibliographic metadata, with no substantive text addressing misuse. Searches for malicious purposes, misinformation, and propaganda found no relevant passage, so the attribution cannot be verified from our copy. (LLM, conf=0.72)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Existing supervised classifiers frequently overfit to particular domains, perform poorly on out-of-distribution data, and lose capability on unseen datasets from other domains.

    “However, these classifiers often overfit to specific domains, leading to poor performance on out-of-distribution data [12,51], and their capabilities significantly degrade when applied to unseen datasets from different domains [32].”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [12] 12. Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. 2023. On the possibilities of ai-generated text detection. arXiv preprint arXiv:2304.04736 (2023). source

      The citation to this paper is attached to the claim that classifier performance degrades on unseen datasets from different domains. Section 4.1 reports results on several datasets and a performance drop under paraphrasing, but does not report testing a classifier trained in one domain on an unseen dataset from another. Searches for cross-domain generalization, out-of-distribution testing, and unseen-dataset performance found no passage establishing that claim. (LLM, conf=0.73)

      Experimental — accuracy not yet measured
    • cites [51] 51. Nafis Tripto, Adaku Uchendu, Thai Le, Mattia Setzu, Fosca Giannotti, and Dong- won Lee. 2023. HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis. In Findings of the Association for Computational Linguistics: EMNLP 2023, source
    • cites [32] 32. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. MAGE: Machine-generated Text Detection in the Wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 36-53. https://doi.org/10.18653/v1/2024.acl-long.3 source

  • May combine two claims.

    Findingnone given

    In cross-cultural settings, news from higher-visibility countries such as Germany and China is easier to identify, while news from Qatar is more challenging.

    “Interestingly, in cross-cultural settings, news from countries with higher visibility, like Germany and China, is easier to identify, while news from Qatar poses more challenges.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The cross-context variations in DetectEval are valid proxies for real-world distribution shifts.

  • May combine two claims.

    Findingown data

    Among LLM-assisted writing categories, LLM-creator texts are easiest to distinguish, while distinguishing LLM-Polisher from Human-Author texts remains challenging.

    “Among categories of LLM-assisted writings, LLM-creator texts are easiest to distinguish, whereas distinguishing between LLM-Polisher and Human-Author texts remains challenging.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Findingown data

    On the HNDC test set, supervised detectors beat zero-shot LLM detectors, and fine-tuned PLM-based models perform best on both tasks.

    “The results show that supervised methods outperform the zero-shot LLM detector, with fine-tuned PLM-based models consistently achieving superior performance across both tasks.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Findingown data

    Text written by LLMs is more standardized, contains fewer grammatical mistakes, and uses a less informal style than human writing.

    “LLM writing is more standardized, with fewer grammatical errors and less informal style.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • May combine two claims.

    Findingnone given

    Feature-based detection models show intermediate performance, with detection ability varying across different types of LLM-generated content.

    “Feature-based models show intermediate performance, with varying detection capabilities across different LLM-generated content types.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Findingown data

    When giving rationales, GPT-4o tends to label content as human-authored if it contains specific details such as citations or data, and as LLM-generated if it is fluent and structured.

    “When providing detection rationales, GPT-4o tends to classify content as human-authored if it includes specific details, such as citations or data, while fluent and structured content is more likely to be identified as LLM-generated.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Findingreasoning

    For short texts, LIR predictions from the three PLM-based detectors generally align with true labels, but discrepancies increase as more original text is retained.

    “For short texts, the LIR predictions from the three PLM-based detectors generally align with the true labels, but as more original text is retained, the prediction discrepancies increase.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The LIR metric accurately quantifies the extent of LLM involvement in text.

  • May combine two claims.

    Findingreasoning

    GPT-4o has limited ability to separate human-authored from LLM-generated content and struggles to detect human-LLM collaboration.

    “GPT-4o, while demonstrating limited ability to differentiate between human-authored and LLM-generated content, struggles to detect human-LLM collaboration.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Findingown data

    Human writing has greater lexical and syntactic variation and lower reading difficulty than LLM writing.

    “while human writing has greater lexical and syntactic variation with lower reading difficulty.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • May combine two claims.

    Findingown data

    The HNDC contains 16,076 human-written articles and totals 64,304 articles.

    “The HNDC consists of 16,076 human-written articles, leading to a total dataset size of 64,304 articles.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • May combine two claims.

    Findingreasoning

    For long texts, RoBERTa and DeBERTa predict LIR values significantly lower than actual values due to input length limits, while Longformer closely matches true labels.

    “For long texts, due to the input length limitations of RoBERTa and DeBERTa, their predicted LIR values are significantly lower than the actual values, while Longformer continues to closely match the true labels.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The LIR metric accurately quantifies the extent of LLM involvement in text.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The fine-tuned model's t-SNE visualization shows improved distinguishability.

    “Appendix Figure6shows the t-SNE visualization of the fine-tuned version, highlighting improved distinguishability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Exposure to news during training misleads zero-shot detectors' judgment, which may explain their poor performance reported in prior work.

    “This suggests that exposure to news during training misleads the judgment of zero-shot detectors, which could also explain the poor performance reported in the literature [8] when using LLMs to distinguish LLM-generated news from human-authored news.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] 8. Amrita Bhattacharjee and Huan Liu. 2024. Fighting fire with fire: can ChatGPT detect AI-generated text? ACM SIGKDD Explorations Newsletter 25, 2 (2024), 14-21. source
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The HNDC was divided at random into training, validation, and test portions in a 7:2:1 proportion, with balanced data across the three sets.

    “For training supervised detection methods, we randomly split the HNDC into training, validation, and test sets in a 7:2:1 ratio, ensuring balanced data distribution across all sets.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • May combine two claims.

    Interpretationown data

    There are discernible linguistic differences between human-authored and LLM-assisted content, which provide an interpretable basis for detection.

    “indicating discernible linguistic differences between human-authored and LLM-assisted content, providing an interpretable basis for detection.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The degree of LLM involvement can vary even when the LLM plays the same assisted-writing role.

    “Multi-intensity variations are introduced to address the fact that, even within the same LLM-assisted writing role, the level of LLM involvement can differ.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The multi-intensity variations (variable-length extension and multi-staged polish) adequately capture within-role differences in LLM involvement.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Human-written news articles in HNDC were limited to those published before 2019 to ensure they are purely human-authored.

    “To ensure that the articles are purely human-authored, we limit our selection to articles published before 2019, prior to the emergence of ChatGPT.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Using pre-2019 articles in HNDC lowers the chance of LLM involvement, yet data leakage remains a possibility.

    “(i) Cross-time: HNDC's pre-2019 articles reduce LLM involvement, but data leakage is still possible.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The five cross-context variations (time, prompt, source, cultural, domain) adequately represent real-world distribution shifts for LLM-generated content detection.

  • May combine two claims.

    Interpretationreasoning

    The decrease in misclassifications in the post-release dataset is likely due to GPT-4o's prior exposure to human-authored news, showing that data leakage affects LLM judgment.

    “In the post-release dataset, the proportion decreases, likely due to GPT-4o's prior exposure to human-authored news, proving that data leakage affects LLM judgment.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

  • May combine two claims.

    Interpretationown data

    Using detectors trained on weaker LLM corpora to detect stronger LLM content can reduce computational resources while maintaining accuracy.

    “Specifically, in the cross-source scenario, detectors trained on corpora generated by weaker LLMs can effectively detect content from stronger LLMs, reducing computational resources while maintaining accuracy.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The cross-context variations in DetectEval are valid proxies for real-world distribution shifts.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The varying-intensity performance advantage of Longformer-based models is due to their ability to process longer input sequences.

    “while Longformer-based models perform best on datasets with varying intensity levels, benefiting from their ability to process longer input sequences.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The five cross-context variations (time, prompt, source, cultural, domain) adequately represent real-world distribution shifts for LLM-generated content detection.
    • Rests on an assumption: The multi-intensity variations (variable-length extension and multi-staged polish) adequately capture within-role differences in LLM involvement.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The cross-context generalization advantage of DeBERTa-based detectors is due to their advanced contextual representation capabilities.

    “Specifically, DeBERTa-based detectors excel in cross-context generalization due to their advanced contextual representation capabilities”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.
    • Rests on an assumption: The five cross-context variations (time, prompt, source, cultural, domain) adequately represent real-world distribution shifts for LLM-generated content detection.
    • Rests on an assumption: The multi-intensity variations (variable-length extension and multi-staged polish) adequately capture within-role differences in LLM involvement.
    • Rests on an assumption: The human-written news from the New York Times and the Guardian is representative of human-authored news.

  • May combine two claims.

    Interpretationown data

    DeBERTa captures relevant features more effectively, which explains its superior generalization in cross-context evaluations.

    “This suggests that DeBERTa captures relevant features more effectively, explaining its superior generalization in the cross-context evaluations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.
    • Rests on an assumption: The cross-context variations in DetectEval are valid proxies for real-world distribution shifts.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Using 2024 news data with GPT-4o, whose knowledge cutoff is October 2023, helps avoid data leakage.

    “GPT-4o has a knowledge cutoff date of October 2023, and since the selected news data comes from 2024, it helps to avoid the data leakage issue.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: GPT-4o has a knowledge cutoff date of October 2023.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Some LLM-generated texts may be truncated because RoBERTa and DeBERTa have a 512-token input length limit.

    “Due to the 512-token input length limit of RoBERTa and DeBERTa, some LLMgenerated texts may be truncated.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: RoBERTa and DeBERTa have a 512-token input length limit.
32 of 32

  • The paper relies on this without support.

    The four roles (Human-Author, LLM-Creator, LLM-Polisher, LLM-Extender) capture the meaningful and exhaustive range of LLM involvement in real-world content creation.

    “This benchmark encompasses four distinct roles in content creation: Human-Author, LLM-Creator, LLM-Polisher, and LLM-Extender.”
    “Each of the four roles represents a distinct level of LLM participation”

  • The paper relies on this without support.

    The five cross-context variations (time, prompt, source, cultural, domain) adequately represent real-world distribution shifts for LLM-generated content detection.

    “Cross-context variations examine data diversity across five dimensions: publication time, generation prompts, LLM sources, cultural differences, and content domain, resulting in five out-of-distribution settings: cross-time, cross-prompt, cross-source, cross-cultural, and cross-domain.”

  • The paper relies on this without support.

    The multi-intensity variations (variable-length extension and multi-staged polish) adequately capture within-role differences in LLM involvement.

    “Multi-intensity variations are introduced to address the fact that, even within the same LLM-assisted writing role, the level of LLM involvement can differ.”

  • The paper relies on this without support.

    The human-written news from the New York Times and the Guardian is representative of human-authored news.

    “The human-written news articles, categorized as Human-Author, are sourced from two reputable newspapers, the New York Times and the Guardian, both known for their commitment to high-quality journalism.”

  • The paper relies on this without support.

    The HNDC and DetectEval datasets adequately represent real-world news and human-LLM collaboration.

    “To support these tasks, we develop LLMDetect, a benchmark combining the Hybrid News Detection Corpus (HNDC) and DetectEval, designed to assess model robustness and generalization across diverse contexts.”

  • The paper relies on this without support.

    The LIR metric accurately quantifies the extent of LLM involvement in text.

    “Your task is to evaluate the extent of AI-assisted writing in the provided article. The evaluation scores range from 0 to 1, where 0 indicates the article is completely human-written, and 1 indicates it is entirely AI-created.”

  • The paper relies on this without support.

    The cross-context variations in DetectEval are valid proxies for real-world distribution shifts.

    “We apply the three best-performing PLM-based detectors trained on HNDC to five distinct types of cross-context variations in De-tectEval to assess their generalization capability.”

  • The paper relies on this without support.

    LLM-generated content is becoming more prevalent, particularly on social media, creating a need for more effective and fine-grained detection models.

    “As LLM-generated content becomes more prevalent, particularly on social media, these findings highlight the importance of developing more effective and fine-grained detection models.”

  • The paper relies on this without support.

    GPT-4o has a knowledge cutoff date of October 2023.

    “GPT-4o has a knowledge cutoff date of October 2023, and since the selected news data comes from 2024, it helps to avoid the data leakage issue.”

  • The paper relies on this without support.

    RoBERTa and DeBERTa have a 512-token input length limit.

    “Due to the 512-token input length limit of RoBERTa and DeBERTa, some LLMgenerated texts may be truncated.”
10 of 10

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • HNDC baseline MSE values change between tables

    Major

    Table 2 reports MSEs of 0.0019, 0.0027, and 0.0013 for RoBERTa, DeBERTa, and Longformer, respectively. Table 3 labels its base row as the HNDC test set but assigns those models 0.0027, 0.0013, and 0.0019 instead.

  • N24News citation points to another dataset

    Minor

    The data-source description cites [56] for N24News, but reference 56 describes SeqXGPT. N24News is listed as reference 58.

  • HNDC baseline results differ between Tables 2 and 3

    Minor

    For the HNDC test set, Table 2 reports DeBERTa's LLM-RR F1 as 99.82 and the RoBERTa, DeBERTa, and Longformer MSEs as 0.0019, 0.0027, and 0.0013. Table 3 instead gives DeBERTa an F1 of 99.84 and assigns the three MSE values in a different order.

  • Polish-ratio citation points to a different reference

    Minor

    The method attributes the polish-ratio approach to Yang et al. [59], but reference 59 is by Wu et al. The listed Yang paper on measuring the polish ratio is reference 61.

  • Polish-ratio attribution points to the wrong authors

    Minor

    The method is attributed to Yang et al. at reference 59, but that entry lists Wu et al. A Yang paper on measuring the polish ratio is listed at reference 61.

  • DeBERTa's HNDC F1 differs between tables

    Minor

    The HNDC test-set overall F1 for DeBERTa is 99.82 in Table 2 but 99.84 in Table 3's HNDC base row.

6 of 6

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. Sameer Badaskar, Sachin Agarwal, and Shilpa Arora. 2008. Identifying Real or Fake Articles: Towards better Language Modeling. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II. https://aclanthology.org/I08--2115/
  2. 10.18653/v1 · DOI
  3. 10.18653/v1 · DOI
  4. 10.18653/v1 · DOI
  5. 10.18653/v1 · DOI
  6. 10.18653/v1 · DOI
  7. 10.18653/v1 · DOI
  8. Qi Pang, Shengyuan Hu, Wenting Zheng, and Virginia Smith. [n.,d.]. No free lunch in llm watermarking: Trade-offs in watermarking design choices. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  9. 10.18653/v1 · DOI
  10. Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI blog Vol. 1 8 (2019) 9.
  11. 10.18653/v1 · DOI
  12. Zhen Tao, Zhiyu Li, Dinghao Xi, and Wei Xu. 2024. CUDRT: Benchmarking the Detection of Human vs. Large Language Models Generated Texts. arXiv preprint arXiv:2406.09056 (2024). · arXiv
  13. 10.18653/v1 · DOI
  14. 10.18653/v1 · DOI
  15. 10.18653/v1 · DOI
  16. Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. 2021. A Robustly Optimized BERT Pre-training Approach with Post-training. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Kang Liu, Wanxiang Che, Shizhu He, and Gaoqi Rao (Eds.). Chinese Information Processing Society of China, Huhhot, China, 1218--1227. https://aclanthology.org/2021.ccl-1.108

Cited by

Sorted by citations

Related

Authors
Cheng, Z., Zhou, L., Jiang, F., Wang, B., & Li, H.
Affiliations
Chinese University of Hong Kong, Shenzhen (CN)
ORCID
Venue
Proceedings of the ACM on Web Conference 2025
Publisher
ACM
Published
2025-04-22
Pages
2677-2688
DOI
10.1145/3696410.3714770
OpenAlex
W4409657361
Cited by
3
References
19
Citations by year
2025: 3
FWCI
2.84
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • International World Wide Web Conference (WWW) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Shenzhen Science and Technology Program (ZDSYS20230626091302006)
  • Shenzhen Science and Technology Research Fund (JCYJ20220818103001002)
  • Key Project of Shenzhen Higher Education Stability Support Program (2024SC0009)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
LLM-generated text detection, large language models, fine-grained detection, human-LLM collaboration, misinformation, multi-class classification, regression tasks, pre-trained language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review