CoChat
Sign Up
PreprintCited by 110Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Access Control and Trust

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

Open access
Wang, Boxin, Chen, Weixin, Pei, Hengzhi, Xie, Chulin, Kang, Mintong, Zhang, Chenhui, Xu, Chejian, Xiong, Zidi +11 more·2023·arXiv
DOI 10.48550/arxiv.2306.11698arXiv 2306.11698
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative Pre-trained Transformer (GPT) models have exhibited exciting progress in their capabilities, capturing the interest of practitioners and the public alike. Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly. To this end, this work proposes a comprehensive trustworthiness evaluation for large language models with a focus on GPT-4 and GPT-3.5, considering diverse perspectives -- including toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness on adversarial demonstrations, privacy, machine ethics, and fairness. Based on our evaluations, we discover previously unpublished vulnerabilities to trustworthiness threats. For instance, we find that GPT models can be easily misled to generate toxic and biased outputs and leak private information in both training data and conversation history. We also find that although GPT-4 is usually more trustworthy than GPT-3.5 on standard benchmarks, GPT-4 is more vulnerable given jailbreaking system or user prompts, potentially because GPT-4 follows (misleading) instructions more precisely. Our work illustrates a comprehensive trustworthiness evaluation of GPT models and sheds light on the trustworthiness gaps. Our benchmark is publicly available at https://decodingtrust.github.io/ ; our dataset can be previewed at https://huggingface.co/datasets/AI-Secure/DecodingTrust ; a concise version of this work is at https://openreview.net/pdf?id=kaHpo8OZw2 .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.11698

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs are not trustworthy.

    “Despite their demonstrated power and social influence, these models are not trustworthy [21].”

    From ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering (2024) · cites this paper as [21]

    The Conclusion reports trustworthiness vulnerabilities in GPT-3.5 and GPT-4, particularly under misleading prompts, supporting concern about their reliability. But the paper primarily evaluates specific versions of those models and does not establish that LLMs generally are categorically untrustworthy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • DecodingTrust evaluated the trustworthiness of GPT models in terms of adversarial robustness, OOD knowledge, vulnerability of jail-breaking, misleading instructions, etc.

    “Similarly, the DecodingTrust [265] evaluated the trustworthiness of GPT models in terms of adversarial robustness, OODknowledge, vulnerability of jail-breaking, misleading instructions, etc.”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [265]

  • LLMs may manifest credibility deficits, potentially giving rise to fabricated information or erroneous facts within dialogues.

    “LLMs may manifest credibility deficits [201], potentially giving rise to fabricated information or erroneous facts within dialogues [163, 253].”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [201]

  • GPT-4 often showcases improved trustworthiness over GPT-3.5 in standard evaluations, but it is simultaneously more susceptible to attacks.

    “They revealed that while GPT-4 often showcases improved trustworthiness over GPT-3.5 in standard evaluations, it is simultaneously more susceptible to attacks.”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [201]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 177

  1. [3][3] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016. 62 https://www.jailbreakchat.com/ https://lingojam.com/shakespearean
  2. [4][4] R. Abebe, S. Barocas, J. Kleinberg, K. Levy, M. Raghavan, and D. G. Robinson. Roles for computing in social change. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2019. doi: 10.1145/3351095.3372871. · DOI
  3. [6][6] A. Acharya, K. Talamadupula, and M. A. Finlayson. An atlas of cultural commonsense for machine reasoning. CoRR, abs/2009.05664, 2020. · arXiv
  4. [7][7] O. Agarwal and A. Nenkova. Temporal effects on pre-trained models for language processing tasks. Transactions of the Association for Computational Linguistics, 10:904–921, 2022.
  5. [8][8] A. F. Akyürek, S. Paik, M. Kocyigit, S. Akbiyik, S. L. Runyun, and D. Wijaya. On mea- suring social biases in prompt-based multi-task learning. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 551–564, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-naacl.42. URL https://aclanthology.org/2022.findings-naacl.42. · DOI
  6. [13][13] U. Arora, W. Huang, and H. He. Types of out-of-distribution texts and how to detect them. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10687–10701, Online and Punta Cana, Dominican Republic, Nov. 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.835. URL https:// aclanthology.org/2021.emnlp-main.835. · DOI
  7. [16][16] S. Barocas and A. D. Selbst. Big data's disparate impact. California Law Review, 104:671, 2016.
  8. [18][18] J. A. Berg. Opposition to pro-immigrant public policy: Symbolic racism and group threat. Sociological Inquiry, 83(1):1–31, 2013. doi: https://doi.org/10.1111/j.1475-682x.2012.00437. x. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-682x.2012. 00437.x. · DOI
  9. [19][19] S. Bird, E. Klein, and E. Loper. Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.", 2009.
  10. [20][20] S. L. Blodgett, S. Barocas, H. Daumé III, and H. Wallach. Language (technology) is power: A critical survey of "bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.485. URL https://aclanthology.org/2020.acl-main.485. · DOI · Paper page
  11. [21][21] S. L. Blodgett, G. Lopez, A. Olteanu, R. Sim, and H. Wallach. Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1004– 1015, Online, Aug. 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021. acl-long.81. URL https://aclanthology.org/2021.acl-long.81. 63 https://aclanthology.org/2022.findings-naacl.42 https://www.aauw.org/resources/research/barrier-bias/ https://antisemitism.adl.org/greed/ https://antisemitism.adl.org/greed/ https://www.adl.org/resources/tools-and-strategies/myths-and-facts-about-muslim-people-and-islam https://www.adl.org/resources/tools-and-strategies/myths-and-facts-about-muslim-people-and-islam https://aclanthology.org/2021.emnlp-main.835 https://aclanthology.org/2021.emnlp-main.835 https://www.psychologicalscience.org/news/motr/bad-drivers-no-just-bad-stereotypes.html https://www.psychologicalscience.org/news/motr/bad-drivers-no-just-bad-stereotypes.html https://digitalcommons.law.seattleu.edu/faculty/296 https://digitalcommons.law.seattleu.edu/faculty/296 https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-682x.2012.00437.x https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-682x.2012.00437.x https://aclanthology.org/2020.acl-main.485 https://aclanthology.org/2021.acl-long.81 · DOI
  12. [24][24] S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, Sept. 2015. Association for Computational Linguistics. doi: 10.18653/v1/D15-1075. URL https: //aclanthology.org/D15-1075. · DOI · Paper page
  13. [25][25] S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. A large annotated corpus for learning natural language inference. In L. Màrquez, C. Callison-Burch, J. Su, D. Pighin, and Y. Marton, editors, EMNLP, 2015.
  14. [27][27] H. Brown, K. Lee, F. Mireshghallah, R. Shokri, and F. Tramèr. What does it mean for a language model to preserve privacy? In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2280–2292, 2022.
  15. [29][29] S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. · arXiv
  16. [30][30] N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium, USENIX Security 2019, 2019.
  17. [31][31] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. B. Brown, D. Song, U. Erlingsson, et al. Extracting training data from large language models. In USENIX Security Symposium, volume 6, 2021.
  18. [32][32] N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramer, B. Balle, D. Ippolito, and E. Wallace. Extracting training data from diffusion models. In arXiv:2301.13188v1, 2023. · arXiv
  19. [33][33] N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang. Quantifying memoriza- tion across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=TatRHT_1cK.
  20. [34][34] B. J. Casad, P. Hale, and F. L. Wachs. Stereotype threat among girls: Differences by gender identity and math education context. Psychology of Women Quarterly, 41(4):513–529, 2017. doi: 10.1177/0361684317711412. URL https://doi.org/10.1177/0361684317711412. · DOI
  21. [35][35] S. Caton and C. Haas. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053, 2020. · arXiv
  22. [38][38] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. M. Dai, H. Yu, S. Petrov, E. Chi, J. Dean, 64 https://crfm.stanford.edu/2023/06/15/eu-ai-act.html https://crfm.stanford.edu/2023/06/15/eu-ai-act.html https://aclanthology.org/D15-1075 https://aclanthology.org/D15-1075 https://www.brookings.edu/blog/brookings-now/2017/08/24/do-immigrants-steal-jobs-from-american-workers/ https://www.brookings.edu/blog/brookings-now/2017/08/24/do-immigrants-steal-jobs-from-american-workers/ https://openreview.net/forum?id=TatRHT_1cK https://doi.org/10.1177/0361684317711412 https://lmsys.org/blog/2023-03-30-vicuna/ https://lmsys.org/blog/2023-03-30-vicuna/ J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei. Scaling instruction-finetuned language models. ARXIV.ORG, 2022. doi: 10.48550/arXiv.2210.11416. · DOI
  23. [42][42] M. Côté, Á. Kádár, X. Yuan, B. Kybartas, T. Barnes, E. Fine, J. Moore, M. J. Hausknecht, L. E. Asri, M. Adada, W. Tay, and A. Trischler. Textworld: A learning environment for text- based games. In Computer Games - 7th Workshop, CGW, Held in Conjunction with the 27th International Conference on Artificial Intelligence, IJCAI, volume 1017 of Communications in Computer and Information Science, pages 41–75. Springer, 2018.
  24. [43][43] G. Cui, L. Yuan, B. He, Y. Chen, Z. Liu, and M. Sun. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. arXiv preprint arXiv:2206.08514, 2022. · arXiv
  25. [45][45] J. Dai, C. Chen, and Y. Li. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872–138878, 2019.
  26. [47][47] J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, and T. Solorio, editors, NAACL-HLT, 2019.
  27. [48][48] J. Dhamala, T. Sun, V. Kumar, S. Krishna, Y. Pruksachatkun, K.-W. Chang, and R. Gupta. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 862–872, 2021.
  28. [50][50] D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. · arXiv
  29. [51][51] H. Duan, A. Dziedzic, N. Papernot, and F. Boenisch. Flocks of stochastic parrots: Differentially private prompt learning for large language models. arXiv preprint arXiv:2305.15594, 2023. · arXiv
  30. [52][52] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214– 226, 2012. 65 https://www.cnn.com/2023/03/16/tech/openai-gpt-microsoft-365/index.html https://www.cnn.com/2023/03/16/tech/openai-gpt-microsoft-365/index.html https://eur-lex.europa.eu/resource.html?uri=cellar:e0649735-a372-11eb-9585-01aa75ed71a1.0001.02/DOC_1&format=PDF https://eur-lex.europa.eu/resource.html?uri=cellar:e0649735-a372-11eb-9585-01aa75ed71a1.0001.02/DOC_1&format=PDF https://eur-lex.europa.eu/resource.html?uri=cellar:e0649735-a372-11eb-9585-01aa75ed71a1.0001.02/DOC_1&format=PDF https://github.com/togethercomputer/RedPajama-Data https://cybernews.com/security/chatgpt-samsung-leak-explained-lessons/ https://cybernews.com/security/chatgpt-samsung-leak-explained-lessons/ https://watcher.guru/news/how-to-jailbreak-chatgpt https://watcher.guru/news/how-to-jailbreak-chatgpt
  31. [53][53] C. Dwork, A. Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014.
  32. [54][54] D. Emelin, R. L. Bras, J. D. Hwang, M. Forbes, and Y. Choi. Moral stories: Situated reasoning about norms, intents, actions, and their consequences. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP, pages 698–718. Association for Computational Linguistics, 2021.
  33. [55][55] A. Fan, M. Lewis, and Y. Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18-1082. URL https://aclanthology.org/P18-1082. · DOI · Paper page
  34. [56][56] A. Fisch, A. Talmor, R. Jia, M. Seo, E. Choi, and D. Chen. MRQA 2019 shared task: Evaluating generalization in reading comprehension. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pages 1–13, Hong Kong, China, Nov. 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-5801. URL https: //aclanthology.org/D19-5801. · DOI
  35. [57][57] L. Floridi, M. Holweg, M. Taddeo, J. Amaya Silva, J. Mökander, and Y. Wen. Capai-a procedure for conducting conformity assessment of ai systems in line with the eu artificial intelligence act. Available at SSRN 4064091, 2022.
  36. [58][58] M. Forbes, J. D. Hwang, V. Shwartz, M. Sap, and Y. Choi. Social chemistry 101: Learn- ing to reason about social and moral norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP, pages 653–670. Association for Computational Linguistics, 2020.
  37. [60][60] L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
  38. [61][61] T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. Daumé III, and K. Crawford. Datasheets for datasets. arXiv preprint arXiv:1803.09010, 2018. · arXiv
  39. [62][62] S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith. RealToxicityPrompts: Evaluat- ing neural toxic degeneration in language models. In Findings in EMNLP, 2020.
  40. [63][63] A. Gentile, S. Boca, and I. Giammusso. 'you play like a woman!' effects of gender stereotype threat on women's performance in physical and sport activities: A meta-analysis. Psychology of Sport and Exercise, 39:95–103, 2018. ISSN 1469-0292. doi: https://doi.org/10.1016/j. psychsport.2018.07.013. URL https://www.sciencedirect.com/science/article/ pii/S1469029217305083. · DOI
  41. [64][64] K. Goel, N. Rajani, J. Vig, S. Tan, J. Wu, S. Zheng, C. Xiong, M. Bansal, and C. Ré. Robustness gym: Unifying the nlp evaluation landscape. arXiv preprint arXiv:2101.04840, 2021. · arXiv
  42. [67][67] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz. More than you've asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models. CoRR, abs/2302.12173, 2023. · arXiv
  43. [68][68] T. Gui, X. Wang, Q. Zhang, Q. Liu, Y. Zou, X. Zhou, R. Zheng, C. Zhang, Q. Wu, J. Ye, et al. Textflint: Unified multilingual robustness evaluation toolkit for natural language processing. arXiv preprint arXiv:2103.11441, 2021. 66 https://aclanthology.org/P18-1082 https://aclanthology.org/D19-5801 https://aclanthology.org/D19-5801 https://www.sciencedirect.com/science/article/pii/S1469029217305083 https://www.sciencedirect.com/science/article/pii/S1469029217305083 http://Skylion007.github.io/OpenWebTextCorpus http://Skylion007.github.io/OpenWebTextCorpus https://web.archive.org/web/20220919192024/https://twitter.com/goodside/status/1569128808308957185 https://web.archive.org/web/20220919192024/https://twitter.com/goodside/status/1569128808308957185 · arXiv
  44. [69][69] M. Hardt, E. Price, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/ 9d2682367c3935defcb1f9e247a97c0d-Paper.pdf.
  45. [70][70] W. Hariri. Unlocking the potential of chatgpt: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing. arXiv preprint arXiv:2304.02017, 2023. · arXiv
  46. [71][71] M. J. Hausknecht, P. Ammanabrolu, M. Côté, and X. Yuan. Interactive fiction games: A colossal adventure. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI, pages 7903–7910. AAAI Press, 2020.
  47. [72][72] D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, and D. Song. Pretrained transformers improve out-of-distribution robustness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2744–2751, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.244. URL https://aclanthology.org/2020.acl-main.244. · DOI
  48. [73][73] D. Hendrycks, C. Burns, S. Basart, A. Critch, J. Li, D. Song, and J. Steinhardt. Aligning AI with shared human values. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  49. [74][74] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Mea- suring massive multitask language understanding. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=d7KBjmI3GmQ.
  50. [75][75] D. Hendrycks, M. Mazeika, A. Zou, S. Patel, C. Zhu, J. Navarro, D. Song, B. Li, and J. Stein- hardt. What would jiminy cricket do? towards agents that behave morally. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021.
  51. [76][76] A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi. The curious case of neural text degeneration. In ICLR, 2019.
  52. [77][77] S. Horton, J. Baker, W. Pearce, and J. M. Deakin. Immunity to popular stereotypes of aging? seniors and stereotype threat. Educational Gerontology, 36(5):353–371, 2010. doi: 10.1080/03601270903323976. URL https://doi.org/10.1080/03601270903323976. · DOI
  53. [78][78] J. Huang, H. Shao, and K. C.-C. Chang. Are large pre-trained language models leaking your personal information? EMNLP Findings, 2022.
  54. [79][79] M. Iyyer, J. Wieting, K. Gimpel, and L. Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. In M. A. Walker, H. Ji, and A. Stent, edi- tors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 1875–1885. Association for Computational Linguistics, 2018. doi: 10.18653/v1/n18-1170. URL https://doi.org/10.18653/v1/n18-1170. · DOI
  55. [80][80] R. Jia and P. Liang. Adversarial examples for evaluating reading comprehension systems. In M. Palmer, R. Hwa, and S. Riedel, editors, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2021–2031. Association for Computational Linguistics, 2017. doi: 10. 18653/v1/d17-1215. URL https://doi.org/10.18653/v1/d17-1215. · DOI · Paper page
  56. [81][81] D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In AAAI, 2020.
  57. [82][82] Z. Jin, S. Levine, F. G. Adauto, O. Kamal, M. Sap, M. Sachan, R. Mihalcea, J. Tenenbaum, and B. Schölkopf. When to make exceptions: Exploring language models as accounts of human moral judgment. In NeurIPS, 2022.
  58. [83][83] N. Kandpal, E. Wallace, and C. Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pages 10697–10707. PMLR, 2022. 67 https://proceedings.neurips.cc/paper_files/paper/2016/file/9d2682367c3935defcb1f9e247a97c0d-Paper.pdf https://proceedings.neurips.cc/paper_files/paper/2016/file/9d2682367c3935defcb1f9e247a97c0d-Paper.pdf https://aclanthology.org/2020.acl-main.244 https://openreview.net/forum?id=d7KBjmI3GmQ https://doi.org/10.1080/03601270903323976 https://doi.org/10.18653/v1/n18-1170 https://doi.org/10.18653/v1/d17-1215 · DOI
  59. [84][84] D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. CoRR, abs/2302.05733, 2023. · arXiv
  60. [85][85] M. Kang, L. Li, M. Weber, Y. Liu, C. Zhang, and B. Li. Certifying some distributional fairness with subpopulation decomposition. Advances in Neural Information Processing Systems, 35: 31045–31058, 2022.
  61. [86][86] J. Kasai, K. Sakaguchi, Y. Takahashi, R. L. Bras, A. Asai, X. Yu, D. Radev, N. A. Smith, Y. Choi, and K. Inui. Realtime qa: What's the answer right now? arXiv preprint arXiv:2207.13332, 2022. · arXiv
  62. [87][87] D. Kaushik, E. Hovy, and Z. Lipton. Learning the difference that makes a difference with counterfactually-augmented data. In International Conference on Learning Representations, 2019.
  63. [88][88] M. Keevak. 204How Did East Asians Become Yellow? In Reconsidering Race: Social Science Perspectives on Racial Categories in the Age of Genomics. Oxford University Press, 06 2018. ISBN 9780190465285. doi: 10.1093/oso/9780190465285. 003.0011. URL https://doi.org/10.1093/oso/9780190465285.003.0011. · DOI
  64. [89][89] F. Khani and P. Liang. Feature noise induces loss discrepancy across groups. International Conference On Machine Learning, 2019.
  65. [90][90] J. Kim, H. J. Kim, H. Cho, H. Jo, S.-W. Lee, S.-g. Lee, K. M. Yoo, and T. Kim. Ground-truth labels matter: A deeper look into input-label demonstrations. arXiv preprint arXiv:2205.12685, 2022. · arXiv
  66. [91][91] B. Klimt and Y. Yang. The enron corpus: A new dataset for email classification research. In Machine Learning: ECML 2004: 15th European Conference on Machine Learning, Pisa, Italy, September 20-24, 2004. Proceedings 15, pages 217–226. Springer, 2004.
  67. [92][92] P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Ya- sunaga, R. L. Phillips, I. Gao, T. Lee, E. David, I. Stavness, W. Guo, B. Earnshaw, I. S. Haque, S. M. Beery, J. Leskovec, A. Kundaje, E. Pierson, S. Levine, C. Finn, and P. Liang. WILDS: A benchmark of in-the-wild distribution shifts. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 5637–5664. PMLR, 2021. URL http://proceedings.mlr.press/v139/koh21a.html.
  68. [93][93] T. Kojima, S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners. Neural Information Processing Systems, 2022.
  69. [94][94] K. Krishna, J. Wieting, and M. Iyyer. Reformulating unsupervised style transfer as para- phrase generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 737–762, Online, Nov. 2020. Association for Compu- tational Linguistics. doi: 10.18653/v1/2020.emnlp-main.55. URL https://aclanthology. org/2020.emnlp-main.55. · DOI
  70. [95][95] M. J. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. Advances in neural information processing systems, 30, 2017.
  71. [96][96] H. Kwon. Dual-targeted textfooler attack on text classification systems. IEEE Access, 11: 15164–15173, 2023. doi: 10.1109/ACCESS.2021.3121366. URL https://doi.org/10. 1109/ACCESS.2021.3121366. · DOI
  72. [98][98] K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, and N. Carlini. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8424–8445, 2022.
  73. [99][99] A. Lees, V. Q. Tran, Y. Tay, J. S. Sorensen, J. Gupta, D. Metzler, and L. Vasserman. A new generation of perspective api: Efficient multilingual character-level transformers. Knowledge Discovery And Data Mining, 2022. doi: 10.1145/3534678.3539147. · DOI · Paper page
  74. [100][100] H. Li, D. Guo, W. Fan, M. Xu, and Y. Song. Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197, 2023. 68 https://doi.org/10.1093/oso/9780190465285.003.0011 http://proceedings.mlr.press/v139/koh21a.html https://aclanthology.org/2020.emnlp-main.55 https://aclanthology.org/2020.emnlp-main.55 https://doi.org/10.1109/ACCESS.2021.3121366 https://doi.org/10.1109/ACCESS.2021.3121366 https://learnprompting.org/docs/prompt_hacking/intro https://learnprompting.org/docs/prompt_hacking/intro · DOI
  75. [101][101] J. Li, S. Ji, T. Du, B. Li, and T. Wang. Textbugger: Generating adversarial text against real-world applications. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February 24-27, 2019. The Internet Society, 2019. URL https://www.ndss-symposium.org/ndss-paper/ textbugger-generating-adversarial-text-against-real-world-applications/.
  76. [102][102] L. Li, R. Ma, Q. Guo, X. Xue, and X. Qiu. BERT-ATTACK: adversarial attack against BERT using BERT. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6193–6202. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.emnlp-main.500. URL https://doi.org/10.18653/v1/ 2020.emnlp-main.500. · DOI
  77. [103][103] T. Li, D. Khashabi, T. Khot, A. Sabharwal, and V. Srikumar. UNQOVERing stereotyping biases via underspecified questions. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3475–3489, Online, Nov. 2020. Association for Computational Linguis- tics. doi: 10.18653/v1/2020.findings-emnlp.311. URL https://aclanthology.org/2020. findings-emnlp.311. · DOI
  78. [104][104] X. Li, F. Tramer, P. Liang, and T. Hashimoto. Large language models can be strong differen- tially private learners. arXiv preprint arXiv:2110.05679, 2021. · arXiv
  79. [105][105] Y. Li and Y. Zhang. Fairness of chatgpt. arXiv preprint arXiv:2305.18569, 2023. · arXiv
  80. [106][106] P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022. · arXiv
  81. [107][107] J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen. What makes good in-context examples for gpt-3? arXiv preprint arXiv:2101.06804, 2021. · arXiv
  82. [108][108] Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu, et al. Summary of chatgpt/gpt-4 research and perspective towards the future of large language models. arXiv preprint arXiv:2304.01852, 2023. · arXiv
  83. [109][109] Y. Liu, Y. Yao, J.-F. Ton, X. Zhang, R. G. H. Cheng, Y. Klochkov, M. F. Taufiq, and H. Li. Trustworthy llms: a survey and guideline for evaluating large language models' alignment. 2023. URL https://api.semanticscholar.org/CorpusID:260775522.
  84. [110][110] S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).
  85. [111][111] Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8086–8098, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.556. URL https://aclanthology.org/2022.acl-long. · DOI · Paper page
  86. [556]556. [112] N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, and S. Zanella-Béguelin. Ana- lyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539, 2023. · arXiv
  87. [113][113] J. Mattern, Z. Jin, B. Weggenmann, B. Schoelkopf, and M. Sachan. Differentially pri- vate language models for secure data sharing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4860–4873, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. URL https: //aclanthology.org/2022.emnlp-main.323.
  88. [114][114] N. Maus, P. Chao, E. Wong, and J. Gardner. Adversarial prompting for black box foundation models. arXiv preprint arXiv:2302.04237, 2023. · arXiv
  89. [115][115] T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1334. URL https:// aclanthology.org/P19-1334. 69 https://www.ndss-symposium.org/ndss-paper/textbugger-generating-adversarial-text-against-real-world-applications/ https://www.ndss-symposium.org/ndss-paper/textbugger-generating-adversarial-text-against-real-world-applications/ https://doi.org/10.18653/v1/2020.emnlp-main.500 https://doi.org/10.18653/v1/2020.emnlp-main.500 https://aclanthology.org/2020.findings-emnlp.311 https://aclanthology.org/2020.findings-emnlp.311 https://api.semanticscholar.org/CorpusID:260775522 https://aclanthology.org/2022.acl-long.556 https://aclanthology.org/2022.acl-long.556 https://aclanthology.org/2022.emnlp-main.323 https://aclanthology.org/2022.emnlp-main.323 https://aclanthology.org/P19-1334 https://aclanthology.org/P19-1334 · DOI · Paper page
  90. [116][116] K. McGuffie and A. Newhouse. The radicalization risks of GPT-3 and advanced neural language models. arXiv, 2020.
  91. [117][117] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1–35, 2021.
  92. [118][118] J. P. Miller, R. Taori, A. Raghunathan, S. Sagawa, P. W. Koh, V. Shankar, P. Liang, Y. Carmon, and L. Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International Conference on Machine Learning, pages 7721–7735. PMLR, 2021.
  93. [119][119] S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Rethink- ing the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.emnlp-main.759.
  94. [120][120] F. Mireshghallah, A. Uniyal, T. Wang, D. K. Evans, and T. Berg-Kirkpatrick. An empirical analysis of memorization in fine-tuned autoregressive language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1816–1826, 2022.
  95. [121][121] S. Mishra, D. Khashabi, C. Baral, and H. Hajishirzi. Cross-task generalization via natu- ral language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3470–3487, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/ 2022.acl-long.244. URL https://aclanthology.org/2022.acl-long.244. · DOI
  96. [122][122] J. X. Morris, J. T. Chiu, R. Zabih, and A. M. Rush. Unsupervised text deidentification. arXiv:2210.11528v1, 2022. · arXiv
  97. [123][123] M. Nadeem, A. Bethke, and S. Reddy. StereoSet: Measuring stereotypical bias in pre- trained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5356–5371, Online, Aug. 2021. Associa- tion for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.416. URL https: //aclanthology.org/2021.acl-long.416. · DOI · Paper page
  98. [124][124] A. Naik, A. Ravichander, N. M. Sadeh, C. P. Rosé, and G. Neubig. Stress test evaluation for natural language inference. In E. M. Bender, L. Derczynski, and P. Isabelle, editors, Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 2340–2353. Association for Computational Linguistics, 2018. URL https://aclanthology.org/C18-1198/.
  99. [125][125] N. Nangia, C. Vania, R. Bhalerao, and S. R. Bowman. CrowS-pairs: A challenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953–1967, Online, Nov. 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main. · DOI
  100. [154]154. URL https://aclanthology.org/2020.emnlp-main.154.
  101. [126][126] Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, and D. Kiela. Adversarial nli: A new benchmark for natural language understanding. In ACL, 2020.
  102. [127][127] H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375, 2023. · arXiv
  103. [130][130] OpenAI. GPT-4 technical report. arXiv, 2023.
  104. [131][131] Y. Oren, S. Sagawa, T. B. Hashimoto, and P. Liang. Distributionally robust language mod- eling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4227–4237, Hong Kong, China, Nov. 2019. Association for Com- putational Linguistics. doi: 10.18653/v1/D19-1432. URL https://aclanthology.org/ D19-1432. 70 https://aclanthology.org/2022.emnlp-main.759 https://aclanthology.org/2022.acl-long.244 https://aclanthology.org/2021.acl-long.416 https://aclanthology.org/2021.acl-long.416 https://aclanthology.org/C18-1198/ https://aclanthology.org/2020.emnlp-main.154 https://chat.openai.com https://platform.openai.com/docs/guides/chat/introduction https://platform.openai.com/docs/guides/chat/introduction https://aclanthology.org/D19-1432 https://aclanthology.org/D19-1432 · DOI
  105. [132][132] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  106. [133][133] A. Pan, J. S. Chan, A. Zou, N. Li, S. Basart, T. Woodside, J. Ng, H. Zhang, S. Emmons, and D. Hendrycks. Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the MACHIAVELLI benchmark. CoRR, abs/2304.03279, 2023. · arXiv
  107. [134][134] A. Panda, T. Wu, J. T. Wang, and P. Mittal. Differentially private in-context learning. arXiv preprint arXiv:2305.01639, 2023. · arXiv
  108. [137][137] F. Perez and I. Ribeiro. Ignore previous prompt: Attack techniques for language models. CoRR, abs/2211.09527, 2022. · arXiv
  109. [139][139] F. Qi, Y. Chen, X. Zhang, M. Li, Z. Liu, and M. Sun. Mind the style of text! adversarial and backdoor attacks based on text style transfer. In EMNLP, 2021.
  110. [140][140] F. Qi, M. Li, Y. Chen, Z. Zhang, Z. Liu, Y. Wang, and M. Sun. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In ACL-IJCNLP, 2021.
  111. [141][141] H. Qiu, S. Zhang, A. Li, H. He, and Z. Lan. Latent jailbreak: A benchmark for evaluating text safety and output robustness of large language models. ArXiv, abs/2307.08487, 2023. URL https://api.semanticscholar.org/CorpusID:259937347. · arXiv
  112. [142][142] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://jmlr.org/papers/v21/ 20-074.html.
  113. [143][143] B. Ray Chaudhury, L. Li, M. Kang, B. Li, and R. Mehta. Fairness in federated learning via core-stability. Advances in neural information processing systems, 35:5738–5750, 2022.
  114. [144][144] L. Reynolds and K. McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. In In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021.
  115. [145][145] M. T. Ribeiro, T. Wu, C. Guestrin, and S. Singh. Beyond accuracy: Behavioral testing of NLP models with checklist (extended abstract). In Z. Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pages 4824–4828. ijcai.org, 2021. doi: 10.24963/ ijcai.2021/659. URL https://doi.org/10.24963/ijcai.2021/659. · DOI
  116. [147][147] S. Santurkar, D. Tsipras, and A. Madry. Breeds: Benchmarks for subpopulation shift. International Conference On Learning Representations, 2020.
  117. [148][148] R. Schaeffer, B. Miranda, and S. Koyejo. Are emergent abilities of large language models a mirage? arXiv preprint arXiv:2304.15004, 2023. · arXiv
  118. [149][149] H. Shao, J. Huang, S. Zheng, and K. C.-C. Chang. Quantifying association capabilities of large language models and its implications on privacy leakage. arXiv preprint arXiv:2305.12707, 2023. 71 https://www.europarl.europa.eu/doceo/document/TA-9-2023-0236_EN.pdf https://www.europarl.europa.eu/doceo/document/TA-9-2023-0236_EN.pdf https://www.pewresearch.org/hispanic/2021/11/04/majority-of-latinos-say-skin-color-impacts-opportunity-in-america-and-shapes-daily-life/ https://www.pewresearch.org/hispanic/2021/11/04/majority-of-latinos-say-skin-color-impacts-opportunity-in-america-and-shapes-daily-life/ https://api.semanticscholar.org/CorpusID:259937347 http://jmlr.org/papers/v21/20-074.html http://jmlr.org/papers/v21/20-074.html https://doi.org/10.24963/ijcai.2021/659 https://www.salon.com/2016/04/06/this_racist_stereotype_is_shattered_study_finds_white_youth_are_more_likely_to_abuse_hard_drugs_than_black_youth_partner/ https://www.salon.com/2016/04/06/this_racist_stereotype_is_shattered_study_finds_white_youth_are_more_likely_to_abuse_hard_drugs_than_black_youth_partner/ https://www.salon.com/2016/04/06/this_racist_stereotype_is_shattered_study_finds_white_youth_are_more_likely_to_abuse_hard_drugs_than_black_youth_partner/ · DOI
  119. [150][150] F. Shi, M. Suzgun, M. Freitag, X. Wang, S. Srivats, S. Vosoughi, H. W. Chung, Y. Tay, S. Ruder, D. Zhou, et al. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057, 2022. · arXiv
  120. [151][151] W. Shi, R. Shea, S. Chen, C. Zhang, R. Jia, and Z. Yu. Just fine-tune twice: Selective differential privacy for large language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6327–6340, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. URL https:// aclanthology.org/2022.emnlp-main.425.
  121. [152][152] T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv, 2020.
  122. [153][153] N. Shinn, B. Labash, and A. Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv: Arxiv-2303.11366, 2023.
  123. [154][154] M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. J. Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. In 9th International Conference on Learning Representations, ICLR, 2021.
  124. [155][155] C. Si, Z. Gan, Z. Yang, S. Wang, J. Wang, J. L. Boyd-Graber, and L. Wang. Prompting GPT-3 to be reliable. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=98p5x51L5af.
  125. [156][156] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Seattle, Washington, USA, Oct. 2013. Association for Computational Linguistics. URL https://aclanthology.org/D13-1170.
  126. [157][157] I. Solaiman and C. Dennison. Process for adapting language models to society (palms) with values-targeted datasets. Advances in Neural Information Processing Systems, 34:5861–5873, 2021.
  127. [158][158] A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615, 2022. · arXiv
  128. [159][159] StabilityAI. StableVicuna: An RLHF Fine-Tune of Vicuna-13B v0. Available at https: //github.com/StabilityAI/StableVicuna, 4 2023. URL https://stability.ai/ blog/stablevicuna-open-source-rlhf-chatbot. DOI:10.57967/hf/0588. · DOI
  129. [160][160] M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. · arXiv
  130. [165][165] J. Thorne and A. Vlachos. Adversarial attacks against fact extraction and verification. CoRR, abs/1903.05543, 2019. URL http://arxiv.org/abs/1903.05543. · arXiv
  131. [166][166] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. · arXiv
  132. [167][167] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. Canton-Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. 72 https://aclanthology.org/2022.emnlp-main.425 https://aclanthology.org/2022.emnlp-main.425 https://openreview.net/forum?id=98p5x51L5af https://aclanthology.org/D13-1170 https://github.com/StabilityAI/StableVicuna https://github.com/StabilityAI/StableVicuna https://stability.ai/blog/stablevicuna-open-source-rlhf-chatbot https://stability.ai/blog/stablevicuna-open-source-rlhf-chatbot https://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca www.mosaicml.com/blog/mpt-7b https://www.teenvogue.com/story/fox-eye-trend-cultural-appropriation-asian-features https://www.teenvogue.com/story/fox-eye-trend-cultural-appropriation-asian-features https://www.hrc.org/resources/debunking-common-myths-about-hiv https://www.hrc.org/resources/debunking-common-myths-about-hiv http://arxiv.org/abs/1903.05543 Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.09288, 2023. doi: 10.48550/arXiv.2307.09288. URL https: //doi.org/10.48550/arXiv.2307.09288. · DOI · Paper page
  133. [168][168] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv: 2307.09288, 2023. · arXiv
  134. [169][169] F. Tram'er, K. Gautam, and N. C. Carlini. Considerations for differentially private learning with large-scale public pretraining. arXiv:2212.06470, 2022. · arXiv
  135. [170][170] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In NIPS, 2017.
  136. [171][171] S. D. Visco. Yellow peril, red scare: race and communism in national review. Ethnic and Racial Studies, 42(4):626–644, 2019. doi: 10.1080/01419870.2017.1409900. URL https://doi.org/10.1080/01419870.2017.1409900. · DOI
  137. [172][172] E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh. Universal adversarial triggers for attacking and analyzing nlp. In EMNLP, 2019.
  138. [173][173] A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. In NeurIPS, 2019.
  139. [174][174] A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In ICLR, 2019.
  140. [175][175] B. Wang, H. Pei, B. Pan, Q. Chen, S. Wang, and B. Li. T3: tree-autoencoder constrained adversarial text generation for targeted attack. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6134–6150. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.emnlp-main.495. URL https: //doi.org/10.18653/v1/2020.emnlp-main.495. · DOI
  141. [176][176] B. Wang, C. Xu, S. Wang, Z. Gan, Y. Cheng, J. Gao, A. H. Awadallah, and B. Li. Adversarial GLUE: A multi-task benchmark for robustness evaluation of language models. In J. Van- schoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. URL https://datasets-benchmarks-proceedings.neurips.cc/ paper/2021/hash/335f5352088d7d9bf74191e006d8e24c-Abstract-round2.html.
  142. [177][177] B. Wang, W. Ping, C. Xiao, P. Xu, M. Patwary, M. Shoeybi, B. Li, A. Anandkumar, and B. Catanzaro. Exploring the limits of domain-adaptive training for detoxifying large-scale language models. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum? id=v_0F4IZJZw.
  143. [178][178] B. Wang, C. Xu, X. Liu, Y. Cheng, and B. Li. SemAttack: Natural textual attacks via different semantic spaces. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022.
  144. [179][179] B. Wang, W. Ping, L. McAfee, P. Xu, B. Li, M. Shoeybi, and B. Catanzaro. Instructretro: Instruction tuning post retrieval-augmented pretraining. arXiv preprint arXiv: 2310.07713, 2023. 73 https://doi.org/10.48550/arXiv.2307.09288 https://doi.org/10.48550/arXiv.2307.09288 https://doi.org/10.1080/01419870.2017.1409900 https://doi.org/10.18653/v1/2020.emnlp-main.495 https://doi.org/10.18653/v1/2020.emnlp-main.495 https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/335f5352088d7d9bf74191e006d8e24c-Abstract-round2.html https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/335f5352088d7d9bf74191e006d8e24c-Abstract-round2.html https://openreview.net/forum?id=v_0F4IZJZw https://openreview.net/forum?id=v_0F4IZJZw · DOI · Paper page
  145. [180][180] B. Wang, W. Ping, P. Xu, L. McAfee, Z. Liu, M. Shoeybi, Y. Dong, O. Kuchaiev, B. Li, C. Xiao, A. Anandkumar, and B. Catanzaro. Shall we pretrain autoregressive language models with retrieval? a comprehensive study. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
  146. [181][181] J. Wang, X. Hu, W. Hou, H. Chen, R. Zheng, Y. Wang, L. Yang, H. Huang, W. Ye, X. Geng, et al. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302.12095, 2023. · arXiv
  147. [182][182] J. Wang, Z. Liu, K. H. Park, M. Chen, and C. Xiao. Adversarial demonstration attacks on large language models. arXiv preprint arXiv:2305.14950, 2023. · arXiv
  148. [183][183] S. Wang, Z. Zhao, X. Ouyang, Q. Wang, and D. Shen. Chatcad: Interactive computer-aided diagnosis on medical image using large language models. arXiv preprint arXiv:2302.07257, 2023. · arXiv
  149. [184][184] Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022. · arXiv
  150. [185][185] Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Naik, A. Ashok, A. S. Dhanasekaran, A. Arunkumar, D. Stap, E. Pathak, G. Karamanolakis, H. Lai, I. Purohit, I. Mondal, J. Anderson, K. Kuznia, K. Doshi, K. K. Pal, M. Patel, M. Moradshahi, M. Par- mar, M. Purohit, N. Varshney, P. R. Kaza, P. Verma, R. S. Puri, R. Karia, S. Doshi, S. K. Sampat, S. Mishra, S. Reddy A, S. Patro, T. Dixit, and X. Shen. Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5085–5109, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.emnlp-main.340.
  151. [186][186] A. Warstadt, Y. Zhang, X. Li, H. Liu, and S. R. Bowman. Learning which features matter: RoBERTa acquires a preference for linguistic generalizations (eventually). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 217–235, Online, Nov. 2020. Association for Computational Linguistics. doi: 10.18653/ v1/2020.emnlp-main.16. URL https://aclanthology.org/2020.emnlp-main.16.
  152. [188][188] M. Weber, L. Li, B. Wang, Z. Zhao, B. Li, and C. Zhang. Certifying out-of-domain generaliza- tion for blackbox functions. International Conference on Machine Learning, 2022.
  153. [189][189] J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https://openreview.net/forum?id=gEZrGCozdqR.
  154. [190][190] J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022. · arXiv
  155. [191][191] J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, et al. Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846, 2023. · arXiv
  156. [192][192] J. Welbl, A. Glaese, J. Uesato, S. Dathathri, J. Mellor, L. A. Hendricks, K. Anderson, P. Kohli, B. Coppin, and P.-S. Huang. Challenges in detoxifying language models. Findings of EMNLP, 2021.
  157. [193][193] K. Welch. Black criminal stereotypes and racial profiling. Journal of Contemporary Criminal Justice, 23(3):276–288, 2007. doi: 10.1177/1043986207306870. URL https://doi.org/ 10.1177/1043986207306870. · DOI
  158. [194][194] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston. Neural text generation with unlikelihood training. In International Conference on Learning Representations, 2020.
  159. [195][195] White House Office of Science and Technology Policy. Blueprint for an ai bill of rights. 2022. 74 https://aclanthology.org/2022.emnlp-main.340 https://aclanthology.org/2020.emnlp-main.16 https://www.washingtonpost.com/news/answer-sheet/wp/2013/10/28/five-stereotypes-about-poor-families-and-education/ https://www.washingtonpost.com/news/answer-sheet/wp/2013/10/28/five-stereotypes-about-poor-families-and-education/ https://openreview.net/forum?id=gEZrGCozdqR https://doi.org/10.1177/1043986207306870 https://doi.org/10.1177/1043986207306870 · DOI
  160. [198][198] A. Xu, E. Pathak, E. Wallace, S. Gururangan, M. Sap, and D. Klein. Detoxifying language models risks marginalizing minority voices. In NAACL, 2021.
  161. [199][199] L. Yang, S. Zhang, L. Qin, Y. Li, Y. Wang, H. Liu, J. Wang, X. Xie, and Y. Zhang. Glue-x: Evaluating natural language understanding models from an out-of-distribution generalization perspective. arXiv preprint arXiv:2211.08073, 2022. · arXiv
  162. [200][200] Z. Yang, Z. Zhao, B. Wang, J. Zhang, L. Li, H. Pei, B. Karlaš, J. Liu, H. Guo, C. Zhang, et al. Improving certified robustness via statistical learning with logical reasoning. Advances in Neural Information Processing Systems, 35:34859–34873, 2022.
  163. [201][201] S. Yao, R. Rao, M. Hausknecht, and K. Narasimhan. Keep calm and explore: Language models for action generation in text-based games. In Empirical Methods in Natural Language Processing (EMNLP), 2020.
  164. [202][202] K. M. Yoo, J. Kim, H. J. Kim, H. Cho, H. Jo, S.-W. Lee, S.-g. Lee, and T. Kim. Ground-truth labels matter: A deeper look into input-label demonstrations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2422–2437, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.emnlp-main.155.
  165. [203][203] D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y. T. Lee, A. Manoel, L. Wutschitz, et al. Differentially private fine-tuning of language models. In International Conference on Learning Representations, 2022.
  166. [204][204] L. Yuan, Y. Chen, G. Cui, H. Gao, F. Zou, X. Cheng, H. Ji, Z. Liu, and M. Sun. Revisiting out- of-distribution robustness in nlp: Benchmark, analysis, and llms evaluations. arXiv preprint arXiv:2306.04618, 2023. · arXiv
  167. [205][205] X. Yue, H. A. Inan, X. Li, G. Kumar, J. McAnallen, H. Sun, D. Levitan, and R. Sim. Synthetic text generation with differential privacy: A simple and practical recipe. ACL, 2023.
  168. [206][206] Y. Zang, F. Qi, C. Yang, Z. Liu, M. Zhang, Q. Liu, and M. Sun. Word-level textual ad- versarial attacking as combinatorial optimization. In D. Jurafsky, J. Chai, N. Schluter, and J. R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 6066–6080. Asso- ciation for Computational Linguistics, 2020. doi: 10.18653/v1/2020.acl-main.540. URL https://doi.org/10.18653/v1/2020.acl-main.540. · DOI
  169. [207][207] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In S. Dasgupta and D. McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 325–333, Atlanta, Georgia, USA, 17–19 Jun 2013. PMLR. URL https://proceedings.mlr.press/ v28/zemel13.html.
  170. [208][208] C. Zhang, D. Ippolito, K. Lee, M. Jagielski, F. Tramèr, and N. Carlini. Counterfactual memorization in neural language models. arXiv preprint arXiv:2112.12938, 2021. · arXiv
  171. [209][209] H. Zhao and G. Gordon. Inherent tradeoffs in learning fair representations. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ b4189d9de0fb2b9cce090bd1a15e3420-Paper.pdf.
  172. [210][210] X. Zhao, L. Li, and Y.-X. Wang. Provably confidential language modelling. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 943–955, 2022.
  173. [211][211] Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198, 2023. 75 http://web.archive.org/web/20220928004736/https://simonwillison.net/2022/Sep/12/prompt-injection/ http://web.archive.org/web/20220928004736/https://simonwillison.net/2022/Sep/12/prompt-injection/ https://web.archive.org/web/20220924105826/https://twitter.com/simonw/status/1570933190289924096 https://web.archive.org/web/20220924105826/https://twitter.com/simonw/status/1570933190289924096 https://aclanthology.org/2022.emnlp-main.155 https://doi.org/10.18653/v1/2020.acl-main.540 https://proceedings.mlr.press/v28/zemel13.html https://proceedings.mlr.press/v28/zemel13.html https://proceedings.neurips.cc/paper_files/paper/2019/file/b4189d9de0fb2b9cce090bd1a15e3420-Paper.pdf https://proceedings.neurips.cc/paper_files/paper/2019/file/b4189d9de0fb2b9cce090bd1a15e3420-Paper.pdf · DOI
  174. [212][212] J. Zhou, H. Müller, A. Holzinger, and F. Chen. Ethical chatgpt: Concerns, challenges, and commandments. arXiv preprint arXiv:2305.10646, 2023. · arXiv
  175. [213][213] K. Zhou, D. Jurafsky, and T. Hashimoto. Navigating the grey area: Expressions of overconfi- dence and uncertainty in language models. arXiv:2302.13439v1, 2023. · arXiv
  176. [214][214] K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y. Wang, L. Yang, W. Ye, N. Z. Gong, Y. Zhang, et al. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.04528, 2023. · arXiv
  177. [215][215] T. Y. Zhuo, Y. Huang, C. Chen, and Z. Xing. Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv preprint arXiv:2301.12867, 2023. 76 · arXiv

Cited by

Sorted by citations

Authors
Wang, B., Chen, W., Pei, H., Xie, C., Kang, M., Zhang, C., Xu, C., Xiong, Z., Dutta, R., Schaeffer, R., Truong, S. T., Arora, S., Mazeika, M., Hendrycks, D., Lin, Z., Cheng, Y., Koyejo, S., Song, D., & Li, B.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
31232-31339
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2306.11698
arXiv
2306.11698
OpenAlex
W7133235460
Cited by
110
References
0
Citations by year
2026: 34 · 2025: 37 · 2024: 35 · 2023: 4
FWCI
27.41
Type
conference paper
Language
en
Open access
Licence
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Sociology and Political Science › Access Control and Trust
Keywords
trustworthiness, large language models, GPT, trust evaluation, AI evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review