CoChat
Sign Up
PreprintCited by 20Topic ModelingNatural Language Processing TechniquesAcademic integrity and plagiarism

Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs

Open access
Balloccu, Simone, Schmidtová, Patrícia, Lango, Mateusz, Dušek, Ondřej·2024·arXiv
DOI 10.48550/arxiv.2402.03927arXiv 2402.03927
Open this paper in CoChat — free accountOpen PDF

Abstract

Natural Language Processing (NLP) research is increasingly focusing on the use of Large Language Models (LLMs), with some of the most popular ones being either fully or partially closed-source. The lack of access to model details, especially regarding training data, has repeatedly raised concerns about data contamination among researchers. Several attempts have been made to address this issue, but they are limited to anecdotal evidence and trial and error. Additionally, they overlook the problem of \emph{indirect} data leaking, where models are iteratively improved by using data coming from users. In this work, we conduct the first systematic analysis of work using OpenAI's GPT-3.5 and GPT-4, the most prominently used LLMs today, in the context of data contamination. By analysing 255 papers and considering OpenAI's data usage policy, we extensively document the amount of data leaked to these models during the first year after the model's release. We report that these models have been globally exposed to $\sim$4.7M samples from 263 benchmarks. At the same time, we document a number of evaluation malpractices emerging in the reviewed papers, such as unfair or missing baseline comparisons and reproducibility issues. We release our results as a collaborative project on https://leak-llm.github.io/, where other researchers can contribute to our efforts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.03927

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • If data leakage issues are present, renaming the methods should give very different results.

    “If data leakage issues are present, renaming the methods should give very different results [4].”

    From Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localization (2024) · cites this paper as [4]

    Sections 3 and 5.1 discuss data leakage and its possible effects on evaluation, but do not say that renaming methods should produce very different results when leakage is present. Searches for renaming methods and for name changes as a contamination test found no passage addressing that prediction. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 101

  1. Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong- yeol Ahn. 2023. Can we trust the evaluation on Chat- GPT? In Proceedings of the 3rd Workshop on Trust- worthy Natural Language Processing (TrustNLP 2023), pages 47-54, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  2. Anya Belz, Craig Thomson, Ehud Reiter, and Simon Mille. 2023. Non-repeatable experiments and non- reproducible results: The reproducibility crisis in GitHub. 2022. About github copilot. https://github. com/features/copilot. · DOI · Paper page
  3. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Alek- sander Wawer. 2019. SAMSum corpus: A human- annotated dialogue dataset for abstractive summa- rization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70-79, Hong Kong, China. Association for Computational Linguis- tics. · DOI · Paper page
  4. Shahriar Golchin and Mihai Surdeanu. 2023. Time travel in LLMs: Tracing data contamination in large language models. arXiv preprint arXiv:2308.08493. · arXiv
  5. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online. Association for Computational Linguistics. · DOI · Paper page
  6. Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christo- pher Choquette Choo, and Nicholas Carlini. 2023. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Lan- guage Generation Conference, pages 28-53, Prague, Czechia. Association for Computational Linguistics.
  7. Khalid S Khan, Regina Kunz, Jos Kleijnen, and Gerd Antes. 2003. Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96(3):118-121. · DOI · Paper page
  8. Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, Anna Kocoń, Bartłomiej Koptyra, Wik- toria Mieleszczenko-Kowszewicz, Piotr Miłkowski, Marcin Oleksy, Maciej Piasecki, Łukasz Radliński, Konrad Wojtasik, Stanisław Woźniak, and Prze- mysław Kazienko. 2023. ChatGPT: Jack of all trades, master of none. Information Fusion, 99:101861. · DOI · Paper page
  9. TH Kung, M Cheatham, A Medenilla, C Sillos, L De Leon, C Elepaño, M Madriaga, R Aggabao, G Diaz-Candido, J Maningo, et al. 2023. Perfor- mance of ChatGPT on USMLE: Potential for AI- assisted medical education using large language mod- els. Plos Digit Health, 2:000198. · DOI · Paper page
  10. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically ordered prompts and where to find them: Overcoming few- shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 8086-8098, Dublin, Ireland. Association for Compu- tational Linguistics. · DOI · Paper page
  11. Inbal Magar and Roy Schwartz. 2022. Data contamina- tion: From memorization to exploitation. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers), pages 157-165, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
  12. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sob- hani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31- 41, San Diego, California. Association for Computa- tional Linguistics. · DOI
  13. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730-27744.
  14. Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Sasha Luccioni, Yacine Jernite, and Anna Rogers. 2023. The ROOTS search tool: Data transparency for LLMs. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 304-314, Toronto, Canada. Association for Computational Linguistics. · DOI
  15. Zhengxaing Shi and Aldo Lipani. 2023. Don't stop pretraining? make prompt-based fine-tuning power- ful learner. In Thirty-seventh Conference on Neural Information Processing Systems.
  16. Jakub Szefer and Sanjay Deshpande. 2023. Analyzing chatgpt's aptitude in an introductory computer engi- neering course. arXiv preprint arXiv:2304.06122. · arXiv
  17. Craig Thomson, Ehud Reiter, and Anya Belz. 2024. Common Flaws in Running Human Evaluation Ex- periments in NLP. Computational Linguistics, pages 1-10. · DOI · Paper page
  18. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung- Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. LaMDA: Language models for dialog ap- plications. arXiv preprint arXiv:2201.08239. · arXiv
  19. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. · arXiv
  20. Fanghua Ye, Jarana Manotumruksa, and Emine Yilmaz. 2022. MultiWOZ 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 351-360, Edinburgh, UK. Association for Computational Lin- guistics.
  21. Bowen Zhang, Daijun Ding, and Liwen Jing. 2022. How would stance detection techniques evolve after the launch of ChatGPT? arXiv preprint arXiv:2212.14548. · arXiv
  22. Afra Feyza Akyurek, Ekin Akyurek, Ashwin Kalyan, Peter Clark, Derry Tanti Wijaya, and Niket Tandon. 2023. RL4F: Generating natural language feedback with reinforcement learning for repairing model out- puts. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 7716-7733, Toronto, Canada. Association for Computational Linguistics. · DOI
  23. Fares Antaki, Samir Touma, Daniel Milad, Jonathan El-Khoury, and Renaud Duval. 2023. Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings. Ophthal- mology Science, 3(4):100324.
  24. Ritwik Bose, Ian Perera, and Bonnie Dorr. 2023. Detox- ifying online discourse: A guided response genera- tion approach for reducing toxicity in user-generated text. In Proceedings of the First Workshop on Social Influence in Conversations (SICon 2023), pages 9- 14, Toronto, Canada. Association for Computational Linguistics. · DOI
  25. Zhenguang G Cai, David A Haslett, Xufeng Duan, Shuqi Wang, and Martin J Pickering. 2023. Does ChatGPT resemble humans in language use? arXiv preprint arXiv:2303.08014. · arXiv
  26. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between ChatGPT and hu- man societies: An empirical study. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 53-67, Dubrovnik, Croatia. Association for Computational Linguistics. · DOI · Paper page
  27. Mohna Chakraborty, Adithya Kulkarni, and Qi Li. 2023. Zero-shot approach to overcome perturbation sensi- tivity of prompts. In Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 5698-5711, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  28. Shreya Chandrasekhar, Chieh-Yang Huang, and Ting- Hao Huang. 2023. Good data, large data, or no data? comparing three approaches in developing re- search aspect classifiers for biomedical papers. In The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 103- 113, Toronto, Canada. Association for Computational Linguistics. · DOI
  29. Lingjiao Chen, Matei Zaharia, and James Zou. 2023b. How is ChatGPT's behavior changing over time? arXiv preprint arXiv:2307.09009. · arXiv
  30. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evalua- tions? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 15607-15631, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  31. Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, and Daniel B. Schwarcz. 2023. ChatGPT goes to law school. Journal of Legal Education. · DOI
  32. Mohita Chowdhury, Ernest Lim, Aisling Higham, Rory McKinnon, Nikoletta Ventoura, Yajie He, and Nick De Pennington. 2023. Can large language models safely address patient questions following cataract surgery? In Proceedings of the 5th Clinical Natu- ral Language Processing Workshop, pages 131-137, Toronto, Canada. Association for Computational Lin- guistics. · DOI · Paper page
  33. Haoran Chu and Sixiao Liu. 2023. Can AI tell good stories? narrative transportation and persuasion with ChatGPT. PsyArXiv. · DOI
  34. Ted M. Clark. 2023. Investigating the use of an ar- tificial intelligence chatbot with general chemistry exam questions. Journal of Chemical Education, 100(5):1905-1916. · DOI
  35. Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Boyang Li, Shafiq Joty, and Lidong Bing. 2023. Is GPT-3 a good data annotator? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11173-11195, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  36. George Duenas, Sergio Jimenez, and Geral Ma- teus Ferro. 2023. You've got a friend in ... a lan- guage model? a comparison of explanations of multiple-choice items of reading comprehension be- tween ChatGPT and humans. In Proceedings of the 18th Workshop on Innovative Use of NLP for Build- ing Educational Applications (BEA 2023), pages 372- 381, Toronto, Canada. Association for Computational Linguistics. · DOI
  37. Sarah E. Finch, Ellie S. Paek, and Jinho D. Choi. 2023. Leveraging large language models for automated dia- logue analysis. In Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dia- logue, pages 202-215, Prague, Czechia. Association for Computational Linguistics.
  38. Kathleen Fraser, Svetlana Kiritchenko, Isar Nejadgholi, and Anna Kerkhof. 2023. What makes a good counter-stereotype? Evaluating strategies for auto- mated responses to stereotypical text. In Proceedings of the First Workshop on Social Influence in Conver- sations (SICon 2023), pages 25-38, Toronto, Canada. Association for Computational Linguistics. · DOI
  39. Wayne Geerling, G. Dirk Mateer, Jadrian Wooten, and Nikhil Damodaran. 2023. ChatGPT has aced the test of understanding in college economics: Now what? The American Economist, 68(2):233-245. · DOI · Paper page
  40. Omid Ghahroodi, Seyed Arshan Dalili, Sahel Mes- foroush, and Ehsaneddin Asgari. 2023. SUT at SemEval-2023 task 1: Prompt generation for visual word sense disambiguation. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2160-2163, Toronto, Canada. Association for Computational Linguistics. · DOI
  41. Shahriar Golchin and Mihai Surdeanu. 2023. Time travel in LLMs: Tracing data contamination in large language models. arXiv preprint arXiv:2308.08493. · arXiv
  42. Srinivas Gowriraj, Soham Dinesh Tiwari, Mitali Potnis, Srijan Bansal, Teruko Mitamura, and Eric Nyberg. 2023. Language-agnostic transformers and assess- ing ChatGPT-based query rewriting for multilingual document-grounded QA. In Proceedings of the Third DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering, pages 101- 108, Toronto, Canada. Association for Computational Linguistics. · DOI
  43. Reto Gubelmann, Aikaterini-lida Kalouli, Christina Niklaus, and Siegfried Handschuh. 2023. When truth matters -addressing pragmatic categories in natural language inference (NLI) by large language models (LLMs). In Proceedings of the 12th Joint Conference on Lexical and Computational Seman- tics (*SEM 2023), pages 24-39, Toronto, Canada. Association for Computational Linguistics. · DOI
  44. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online. Association for Computational Linguistics. · DOI · Paper page
  45. Tahsina Hashem, Weiqing Wang, Derry Tanti Wijaya, Mohammed Eunus Ali, and Yuan-Fang Li. 2023. Generating faithful text from a knowledge graph with noisy reference text. In Proceedings of the 16th Inter- national Natural Language Generation Conference, pages 106-122, Prague, Czechia. Association for Computational Linguistics.
  46. Shreya Havaldar, Bhumika Singhal, Sunny Rai, Langchen Liu, Sharath Chandra Guntuku, and Lyle Ungar. 2023. Multilingual language models are not multicultural: A case study in emotion. In Proceed- ings of the 13th Workshop on Computational Ap- proaches to Subjectivity, Sentiment, & Social Media Analysis, pages 202-214, Toronto, Canada. Associa- tion for Computational Linguistics. · DOI
  47. Michael Heck, Nurul Lubis, Benjamin Ruppik, Renato Vukovic, Shutong Feng, Christian Geishauser, Hsien- chin Lin, Carel van Niekerk, and Milica Gasic. 2023. ChatGPT for zero-shot dialogue state tracking: A solution or an opportunity? In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 2: Short Papers), pages 936-950, Toronto, Canada. Association for Compu- tational Linguistics. · DOI
  48. Takanobu Hirosawa, Yukinori Harada, Masashi Yokose, Tetsu Sakamoto, Ren Kawamura, and Taro Shimizu. 2023. Diagnostic accuracy of differential-diagnosis lists generated by generative pretrained transformer 3 chatbot for clinical vignettes with common chief complaints: A pilot study. International Journal of Environmental Research and Public Health, 20(4).
  49. Ruixin Hong, Hongming Zhang, Hong Zhao, Dong Yu, and Changshui Zhang. 2023. Faithful question an- swering with Monte-Carlo planning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3944-3965, Toronto, Canada. Association for Computational Linguistics. · DOI
  50. Nannan Huang, Lin Tian, Haytham Fayek, and Xiuzhen Zhang. 2023c. Examining bias in opinion summarisa- tion through the perspective of opinion diversity. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Me- dia Analysis, pages 149-161, Toronto, Canada. Asso- ciation for Computational Linguistics. · DOI
  51. Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christo- pher Choquette Choo, and Nicholas Carlini. 2023. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Lan- guage Generation Conference, pages 28-53, Prague, Czechia. Association for Computational Linguistics.
  52. Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, and Jimmy Huang. 2023. Evaluation of ChatGPT on biomedical tasks: A zero-shot comparison with fine- tuned generative transformers. In The 22nd Work- shop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 326-336, Toronto, Canada. Association for Computational Linguistics. · DOI
  53. Sophie Jentzsch and Kristian Kersting. 2023. ChatGPT is fun, but it is not funny! humor is still challenging large language models. In Proceedings of the 13th Workshop on Computational Approaches to Subjec- tivity, Sentiment, & Social Media Analysis, pages 325-340, Toronto, Canada. Association for Compu- tational Linguistics. · DOI
  54. David Kartchner, Selvi Ramalingam, Irfan Al-Hussaini, Olivia Kronick, and Cassie Mitchell. 2023. Zero- shot information extraction for clinical meta-analysis using large language models. In The 22nd Work- shop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 396-405, Toronto, Canada. Association for Computational Linguistics. · DOI
  55. Yuheun Kim, Lu Guo, Bei Yu, and Yingya Li. 2023. Can ChatGPT understand causal language in science claims? In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 379-389, Toronto, Canada. Association for Computational Linguistics. · DOI
  56. Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, Anna Kocoń, Bartłomiej Koptyra, Wik- toria Mieleszczenko-Kowszewicz, Piotr Miłkowski, Marcin Oleksy, Maciej Piasecki, Łukasz Radliński, Konrad Wojtasik, Stanisław Woźniak, and Prze- mysław Kazienko. 2023. ChatGPT: Jack of all trades, master of none. Information Fusion, 99:101861. · DOI · Paper page
  57. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. 2023. Language generation models can cause harm: So what can we do about it? an actionable survey. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 3299-3321, Dubrovnik, Croatia. As- sociation for Computational Linguistics. · DOI · Paper page
  58. Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz- Candido, James Maningo, and Victor Tseng. 2023. Performance of ChatGPT on usmle: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2):1-12. · DOI · Paper page
  59. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christo- pher D Manning, Christopher Re, Diana Acosta- Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan An- drew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Ko- reeda. 2023a. Holistic evaluation of language models. Transactions on Machine Learning Research. Fea- tured Certification, Expert Certification.
  60. Yancheng Liang, Jiajie Zhang, Hui Li, Xiaochen Liu, Yi Hu, Yong Wu, Jiaoyao Zhang, Yongyan Liu, and Yi Wu. 2023b. Breaking the bank with ChatGPT: Few-shot text classification for finance. In Proceed- ings of the Fifth Workshop on Financial Technology and Natural Language Processing and the Second Multimodal AI For Financial Forecasting, pages 74- 80, Macao. -.
  61. Xin Liu, Yuan Tan, Zhenghang Xiao, Jianwei Zhuge, and Rui Zhou. 2023f. Not the end of story: An eval- uation of ChatGPT-driven vulnerability description mappings. In Findings of the Association for Com- putational Linguistics: ACL 2023, pages 3724-3731, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  62. Zequn Liu, Wei Zhang, Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, and Tie-Yan Liu. 2023i. MolXPT: Wrapping molecules with text for genera- tive pre-training. In Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 1606-1616, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  63. Zhexiong Liu, Diane Litman, Elaine Wang, Lindsay Matsumura, and Richard Correnti. 2023k. Predicting the quality of revisions in argumentative writing. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 275-287, Toronto, Canada. Association for Computational Linguistics. · DOI
  64. Mengsay Loem, Masahiro Kaneko, Sho Takase, and Naoaki Okazaki. 2023. Exploring effectiveness of GPT-3 in grammatical error correction: A study on performance and controllability in prompt-based methods. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 205-219, Toronto, Canada. Association for Computational Linguistics. · DOI
  65. Qingyu Lu, Liang Ding, Liping Xie, Kanjian Zhang, Derek F. Wong, and Dacheng Tao. 2023b. Toward human-like evaluation for natural language genera- tion with error analysis. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5892- 5907, Toronto, Canada. Association for Computa- tional Linguistics. · DOI
  66. Xin Lu, Zhuojun Li, Yanpeng Tong, Yanyan Zhao, and Bing Qin. 2023d. HIT-SCIR at WASSA 2023: Em- pathy and emotion analysis at the utterance-level and the essay-level. In Proceedings of the 13th Work- shop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 574-580, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  67. Lars Mehnen, Stefanie Gruarin, Mina Vasileva, and Bernhard Knapp. 2023. ChatGPT as a medical doc- tor? a diagnostic accuracy study on common and rare diseases. medRxiv. · DOI · Paper page
  68. Robert Morabito, Jad Kabbara, and Ali Emami. 2023. Debiasing should be good and bad: Measuring the consistency of debiasing techniques in language mod- els. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4581-4597, Toronto, Canada. Association for Computational Linguistics. · DOI
  69. Vishvak Murahari, Ameet Deshpande, Carlos Jimenez, Izhak Shafran, Mingqiu Wang, Yuan Cao, and Karthik Narasimhan. 2023. MUX-PLMs: Pre- training language models with data multiplexing. In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023), pages 196-211, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  70. Duan Nan. 2023. Frontier review of multimodal AI. In Proceedings of the 22nd Chinese National Confer- ence on Computational Linguistics (Volume 2: Fron- tier Forum), pages 110-118, Harbin, China. Chinese Information Processing Society of China.
  71. Andrew Nedilko. 2023. Generative pretrained trans- formers for emotion detection in a code-switching setting. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 616-620, Toronto, Canada. Association for Computational Linguistics. · DOI
  72. Shinhyeok Oh, Hyojun Go, Hyeongdon Moon, Yunsung Lee, Myeongho Jeong, Hyun Seung Lee, and Seung- taek Choi. 2023. Evaluation of question generation needs more references. In Findings of the Associa- tion for Computational Linguistics: ACL 2023, pages 6358-6367, Toronto, Canada. Association for Com- putational Linguistics. · DOI
  73. Amin Omidvar and Aijun An. 2023. Empowering con- versational agents using semantic in-context learning. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 766-771, Toronto, Canada. Asso- ciation for Computational Linguistics. · DOI
  74. Lidiia Ostyakova, Veronika Smilga, Kseniia Petukhova, Maria Molchanova, and Daniel Kornev. 2023. Chat- GPT vs. crowdsourcing vs. experts: Annotating open- domain conversations with speech functions. In Pro- ceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue, pages 242-254, Prague, Czechia. Association for Computational Lin- guistics.
  75. Naoki Otani, Jun Araki, HyeongSik Kim, and Eduard Hovy. 2023. On the underspecification of situa- tions in open-domain conversational datasets. In Proceedings of the 5th Workshop on NLP for Con- versational AI (NLP4ConvAI 2023), pages 12-28, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  76. Denis Peskoff and Brandon Stewart. 2023. Credible without credit: Domain experts assess generative lan- guage models. In Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 427-438, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  77. Andrei Popescu-Belis, Àlex R. Atrio, Bastien Bernath, Etienne Boisson, Teo Ferrari, Xavier Theimer- lienhard, and Giorgos Vernikos. 2023. GPoeT: a language model trained for rhyme generation on syn- thetic data. In Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Litera- ture, pages 10-20, Dubrovnik, Croatia. Association for Computational Linguistics. · DOI
  78. Dongqi Pu and Vera Demberg. 2023. ChatGPT vs human-authored text: Insights into controllable text summarization and sentence style transfer. In Pro- ceedings of the 61st Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 4: Student Research Workshop), pages 1-18, Toronto, Canada. Association for Computational Linguistics. · DOI
  79. Ali Quidwai, Chunhui Li, and Parijat Dube. 2023. Be- yond black box AI generated plagiarism detection: From sentence to document level. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 727-735, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  80. Arya Rao, Michael Pang, John Kim, Meghana Kami- neni, Winston Lie, Anoop K. Prasad, Adam Land- man, Keith J Dreyer, and Marc D. Succi. 2023a. As- sessing the utility of ChatGPT throughout the entire clinical workflow. medRxiv. · DOI · Paper page
  81. Mathieu Ravaut, Shafiq Joty, and Nancy Chen. 2023. Unsupervised summarization re-ranking. In Find- ings of the Association for Computational Linguis- tics: ACL 2023, pages 8341-8376, Toronto, Canada. Association for Computational Linguistics. · DOI
  82. Anton Razzhigaev, Mikhail Salnikov, Valentin Malykh, Pavel Braslavski, and Alexander Panchenko. 2023. A system for answering simple questions in multiple languages. In Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 524-537, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  83. Jakob Schuster and Katja Markert. 2023. Nut- cracking sledgehammers: Prioritizing target language data over bigger language models for cross-lingual metaphor detection. In Proceedings of the 2023 CLASP Conference on Learning with Small Data (LSD), pages 98-106, Gothenburg, Sweden. Associa- tion for Computational Linguistics.
  84. Natalie Shapira, Guy Zwirn, and Yoav Goldberg. 2023. How well do large language models perform on faux pas tests? In Findings of the Association for Compu- tational Linguistics: ACL 2023, pages 10438-10451, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  85. Zhengxaing Shi and Aldo Lipani. 2023. Don't stop pretraining? make prompt-based fine-tuning power- ful learner. In Thirty-seventh Conference on Neural Information Processing Systems.
  86. David Stap and Ali Araabi. 2023. ChatGPT is not a good indigenous translator. In Proceedings of the Workshop on Natural Language Processing for In- digenous Languages of the Americas (AmericasNLP), pages 163-167, Toronto, Canada. Association for Computational Linguistics. · DOI
  87. Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023a. Towards benchmarking and improving the temporal reasoning capability of large language models. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 14820-14835, Toronto, Canada. Association for Computational Linguistics. · DOI
  88. Zeqi Tan, Shen Huang, Zixia Jia, Jiong Cai, Yinghui Li, Weiming Lu, Yueting Zhuang, Kewei Tu, Pengjun Xie, Fei Huang, and Yong Jiang. 2023c. DAMO- NLP at SemEval-2023 task 2: A unified retrieval- augmented system for multilingual named entity recognition. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2014-2028, Toronto, Canada. Association for Computational Linguistics. · DOI
  89. Liyan Tang, Tanya Goyal, Alex Fabbri, Philippe La- ban, Jiacheng Xu, Semih Yavuz, Wojciech Kryscin- ski, Justin Rousseau, and Greg Durrett. 2023a. Un- derstanding factual errors in summarization: Errors, summarizers, datasets, error detectors. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11626-11644, Toronto, Canada. Association for Computational Linguistics. · DOI
  90. Jens Van Nooten and Walter Daelemans. 2023. Improv- ing Dutch vaccine hesitancy monitoring via multi- label data augmentation with GPT-3.5. In Proceed- ings of the 13th Workshop on Computational Ap- proaches to Subjectivity, Sentiment, & Social Media Analysis, pages 251-270, Toronto, Canada. Associa- tion for Computational Linguistics. · DOI
  91. Bhaskara Hanuma Vedula, Prashant Kodali, Manish Shrivastava, and Ponnurangam Kumaraguru. 2023. PrecogIIITH@WASSA2023: Emotion detection for Urdu-English code-mixed text. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 601-605, Toronto, Canada. Association for Computational Linguistics. · DOI
  92. Junda Wang, Zonghai Yao, Avijit Mitra, Samuel Osebe, Zhichao Yang, and Hong Yu. 2023f. UMASS_BioNLP at MEDIQA-chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations? In Proceedings of the 5th Clinical Natural Language Processing Workshop, pages 460-471, Toronto, Canada. Association for Computational Linguistics. · DOI
  93. Rose Wang and Dorottya Demszky. 2023. Is ChatGPT a good teacher coach? measuring zero-shot perfor- mance for scoring and providing actionable insights on classroom instruction. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Ed- ucational Applications (BEA 2023), pages 626-667, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  94. Qianhui Wu, Huiqiang Jiang, Haonan Yin, Börje Karls- son, and Chin-Yew Lin. 2023b. Multi-level knowl- edge distillation for out-of-distribution detection in text. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 7317-7332, Toronto, Canada. Association for Computational Linguistics. · DOI
  95. Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie, and Kewei Tu. 2023c. Do PLMs know and under- stand ontological knowledge? In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 3080-3101, Toronto, Canada. Association for Com- putational Linguistics. · DOI
  96. Changrong Xiao, Sean Xin Xu, Kunpeng Zhang, Yu- fang Wang, and Lei Xia. 2023. Evaluating read- ing comprehension exercises generated by LLMs: A showcase of ChatGPT in education applications. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 610-625, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  97. Zihang Xu, Ziqing Yang, Yiming Cui, and Shijin Wang. 2023c. IDOL: Indicator-oriented logic pre-training for logical reasoning. In Findings of the Associa- tion for Computational Linguistics: ACL 2023, pages 8099-8111, Toronto, Canada. Association for Com- putational Linguistics. · DOI
  98. Bowen Zhang, Daijun Ding, and Liwen Jing. 2022. How would stance detection techniques evolve after the launch of ChatGPT? arXiv preprint arXiv:2212.14548. · arXiv
  99. Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, and Bing Qin. 2023a. Is ChatGPT equipped with emotional dialogue capabilities? Xiaofeng Zhao, Min Zhang, Miaomiao Ma, Chang Su, Yilun Liu, Minghan Wang, Xiaosong Qiao, Jiaxin Guo, Yinglu Li, and Wenbing Ma. 2023b. HW-TSC at SemEval-2023 task 7: Exploring the natural lan- guage inference capabilities of ChatGPT and pre- trained language model for clinical trial. In Proceed- ings of the 17th International Workshop on Seman- tic Evaluation (SemEval-2023), pages 1603-1608, Toronto, Canada. Association for Computational Lin- guistics. · DOI
  100. Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu, and Lei Li. 2023c. Pre-trained language models can be fully zero-shot learners. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15590- 15606, Toronto, Canada. Association for Computa- tional Linguistics. · DOI
  101. Yi Zheng, Björn Ross, and Walid Magdy. 2023c. What makes good counterspeech? a comparison of genera- tion approaches and evaluation metrics. In Proceed- ings of the 1st Workshop on CounterSpeech for On- line Abuse (CS4OA), pages 62-71, Prague, Czechia. Association for Computational Linguistics.

Cited by

Sorted by citations

Authors
Balloccu, S., Schmidtová, P., Lango, M., & Dušek, O.
Affiliations
Charles University (CZ)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.03927
arXiv
2402.03927
OpenAlex
W4391667444
Cited by
20
References
0
Citations by year
2026: 4 · 2025: 9 · 2024: 6 · 2013: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Univerzita Karlova v Praze
  • HORIZON EUROPE European Research Council (101039303)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
contaminated data, GPT-3.5, GPT-4, reproducibility, baseline comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review