CoChat
Sign Up
PreprintCited by 4Authorship Attribution and ProfilingTopic ModelingDigital Media Forensic Detection

ConDA: Contrastive Domain Adaptation for AI-generated Text Detection

Open access
Bhattacharjee, Amrita, Kumarage, Tharindu, Moraffah, Raha, Liu, Huan·2023·arXiv
DOI 10.48550/arxiv.2309.03992arXiv 2309.03992
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) are increasingly being used for generating text in a variety of use cases, including journalistic news articles. Given the potential malicious nature in which these LLMs can be used to generate disinformation at scale, it is important to build effective detectors for such AI-generated text. Given the surge in development of new LLMs, acquiring labeled training data for supervised detectors is a bottleneck. However, there might be plenty of unlabeled text data available, without information on which generator it came from. In this work we tackle this data problem, in detecting AI-generated news text, and frame the problem as an unsupervised domain adaptation task. Here the domains are the different text generators, i.e. LLMs, and we assume we have access to only the labeled source data and unlabeled target data. We develop a Contrastive Domain Adaptation framework, called ConDA, that blends standard domain adaptation techniques with the representation power of contrastive learning to learn domain invariant representations that are effective for the final unsupervised detection task. Our experiments demonstrate the effectiveness of our framework, resulting in average performance gains of 31.7% from the best performing baselines, and within 0.8% margin of a fully supervised detector. All our code and data is available at https://github.com/AmritaBh/ConDA-gen-text-detection.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2309.03992

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Bhattacharjee et al. (2023) proposed ConDA, a contrastive domain adaptation framework, which greatly improves the model's defense capabilities against unknown models.

    “Bhattacharjee et al. (2023) proposed ConDA, a contrastive domain adaptation framework, which combines standard domain adaptation technology with the representation capabilities of contrastive learning, greatly improving the model's defense capabilities against unknown models.”

    From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Bhattacharjee et al. 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 51

  1. Amrita Bhattacharjee, Mansooreh Karami, and Huan Liu. 2022. Text transformations in contrastive self-supervised learning: a review. arXiv preprint arXiv:2203.12000. · arXiv
  2. Amrita Bhattacharjee and Huan Liu. 2023. Fighting fire with fire: Can chatgpt detect ai-generated text? arXiv preprint arXiv:2308.01284. · arXiv
  3. Jérémie Bogaert, Marie-Catherine de Marneffe, Antonin Descampe, and Francois-Xavier Standaert. 2022. Au- tomatic and manual detection of generated news: Case study, limitations and challenges. In Proceed- ings of the 1st International Workshop on Multimedia AI against Disinformation, pages 18-26.
  4. Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. 1993. Signature verifi- cation using a" siamese" time delay neural network. Advances in neural information processing systems, 6.
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
  6. Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. · arXiv
  7. Junfan Chen, Richong Zhang, Yongyi Mao, and Jie Xu. 2022a. Contrastnet: A contrastive learning frame- work for few-shot text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10492-10500.
  8. Qianben Chen, Richong Zhang, Yaowei Zheng, and Yongyi Mao. 2022b. Dual contrastive learning: Text classification via label-aware data augmentation. arXiv preprint arXiv:2201.08702. · arXiv
  9. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In In- ternational conference on machine learning, pages 1597-1607. PMLR.
  10. Holly Else. 2023. Abstracts written by chatgpt fool scientists. Nature, 613(7944):423-423.
  11. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Lavi- olette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096- 2030.
  12. Sebastian Gehrmann, Hendrik Strobelt, and Alexan- der M Rush. 2019. Gltr: Statistical detection and visualization of generated text. arXiv preprint arXiv:1906.04043. · arXiv
  13. Aritra Ghosh and Andrew Lan. 2021. Contrastive learn- ing improves model robustness under label noise. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 2703- 2708.
  14. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceed- ings of the 28th international conference on machine learning (ICML-11), pages 513-520.
  15. Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773.
  16. Xiaochuang Han and Jacob Eisenstein. 2019. Unsu- pervised domain adaptation of contextualized em- beddings for sequence labeling. arXiv preprint arXiv:1904.02817. · arXiv
  17. Daphne Ippolito, Daniel Duckworth, Chris Callison- Burch, and Douglas Eck. 2019. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650. · arXiv
  18. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks VS Lakshmanan. 2020. Automatic detection of machine generated text: A critical survey. arXiv preprint arXiv:2011.01314. · arXiv
  19. Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2022. Contrastive learning for prompt-based few-shot language learners. arXiv preprint arXiv:2205.01308. · arXiv
  20. Keenan Jones, Jason RC Nurse, and Shujun Li. 2022. Are you robert or roberta? deceiving online author- ship attribution models using neural text generators. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 429- 440.
  21. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for control- lable generation. arXiv preprint arXiv:1909.05858. · arXiv
  22. Youngwook Kim, Shinwoo Park, and Yo-Sub Han. 2022. Generalizable implicit hate speech detection using contrastive learning. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, pages 6667-6679.
  23. Tharindu Kumarage, Joshua Garland, Amrita Bhat- tacharjee, Kirill Trapeznikov, Scott Ruston, and Huan Liu. 2023. Stylometric detection of ai- generated text in twitter timelines. arXiv preprint arXiv:2303.03697. · arXiv
  24. Guillaume Lample and Alexis Conneau. 2019. Cross- lingual language model pretraining. arXiv preprint arXiv:1901.07291. · arXiv
  25. Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In International confer- ence on machine learning, pages 97-105.
  26. PMLR.
  27. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detec- tion using probability curvature. arXiv preprint arXiv:2301.11305. · arXiv
  28. Shaoor Munir, Brishna Batool, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. 2021. Through the looking glass: Learning to attribute synthetic text generated by language models. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics: Main Volume, pages 1811-1822.
  29. Paul Neculoiu, Maarten Versteegh, and Mihai Rotaru. 2016. Learning text similarity with siamese recurrent networks. In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 148-157.
  30. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair's wmt19 news translation task submission. arXiv preprint arXiv:1907.06616. · arXiv
  31. OpenAI. 2023. Gpt-4 technical report. arXiv.
  32. Lin Pan, Chung-Wei Hang, Avirup Sil, and Saloni Pot- dar. 2022. Improved text classification via contrastive adversarial training. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 11130-11138.
  33. Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. 2010. Domain adaptation via transfer component analysis. IEEE transactions on neural networks, 22(2):199-210.
  34. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  35. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-5551.
  36. Alan Ramponi and Barbara Plank. 2020. Neural unsu- pervised domain adaptation in nlp-a survey. arXiv preprint arXiv:2006.00632. · arXiv
  37. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. · arXiv
  38. Ozan Sener, Hyun Oh Song, Ashutosh Saxena, and Silvio Savarese. 2016. Learning transferrable repre- sentations for unsupervised domain adaptation. Ad- vances in neural information processing systems, 29.
  39. Ingo Steinwart. 2001. On the influence of the kernel on the consistency of support vector machines. Journal of machine learning research, 2(Nov):67-93.
  40. Reuben Tan, Bryan A Plummer, and Kate Saenko. 2020. Detecting cross-modal inconsistency to de- fend against neural fake news. arXiv preprint arXiv:2009.07698. · arXiv
  41. Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. 2020. What makes for good views for contrastive learning? Ad- vances in neural information processing systems, 33:6827-6839.
  42. Ahmed Tlili, Boulus Shehata, Michael Agyemang Adarkwah, Aras Bozkurt, Daniel T Hickey, Ronghuai Huang, and Brighter Agyemang. 2023. What if the devil is my guardian angel: Chatgpt as a case study of using chatbots in education. Smart Learning Envi- ronments, 10(1):15.
  43. Nghia Ngo Trung, Linh Ngo Van, and Thien Huu Nguyen. 2022. Unsupervised domain adaptation for text classification via meta self-paced learning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4741-4752.
  44. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. Authorship attribution for neural text gener- ation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8384-8395.
  45. Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. Turingbench: A benchmark environment for turing test in the age of neural text generation. arXiv preprint arXiv:2109.13296. · arXiv
  46. Onur Varol, Emilio Ferrara, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2017. On- line human-bot interactions: Detection, estimation, and characterization. In Proceedings of the interna- tional AAAI conference on web and social media, volume 11, pages 280-289.
  47. Zhenrui Yue, Bernhard Kratzwald, and Stefan Feuer- riegel. 2021. Contrastive domain adaptation for ques- tion answering using limited text corpora. arXiv preprint arXiv:2108.13854. · arXiv
  48. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems, 32.
  49. Jianguo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, and Philip Yu. 2021. Few-shot intent detection via contrastive pre-training and fine-tuning. arXiv preprint arXiv:2109.06349. · arXiv
  50. Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Ré. 2022. Correct- n-contrast: A contrastive approach for improving robustness to spurious correlations. arXiv preprint arXiv:2203.01517. · arXiv
  51. Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. 2023. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Pro- ceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-20.

Cited by

Sorted by citations

Authors
Bhattacharjee, A., Kumarage, T., Moraffah, R., & Liu, H.
Affiliations
Arizona State University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2309.03992
arXiv
2309.03992
OpenAlex
W4386614352
Cited by
3
References
0
Citations by year
2025: 3
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Defense Advanced Research Projects Agency (HR001120C0123)
  • Office of Naval Research (N00014-21-1-4002, N00014)
  • Army Research Office (W911NF2020124, W911NF2110030)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
AI-generated text detection, unsupervised domain adaptation, large language models, disinformation, news articles, domain-invariant features, contrastive learning, unlabeled target data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review