A Survey on Evaluation of Large Language Models

Chang, Yupeng, Wang, Xu, Wang, Jindong, Wu, Yuan, Yang, Linyi, Zhu, Kaijie, Chen, Hao, Yi, Xiaoyuan +8 more·2024·ACM Transactions on Intelligent Systems and Technology
DOI 10.1145/3641289

Cites (293)

Items 201–327 of 327

  1. CMB: A Comprehensive Medical Benchmark in Chinese
    Xidong Wang, Guiming Chen, Song Dingjie, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Junying Chen, Feng Jiang, Jianquan Li, Xiang Wan et al. · 2024 · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
  2. The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation
    Hartmann, Jochen, Schwenzow, Jasper, Witte, Maximilian · 2023 · arXiv
  3. INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
    Chia, Yew Ken, Hong, Pengfei, Bing, Lidong, Poria, Soujanya · 2023 · arXiv
  4. TALM: Tool Augmented Language Models
    Parisi, Aaron, Zhao, Yao, Fiedel, Noah · 2022 · arXiv
  5. PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization
    Wang, Yidong, Yu, Zhuohao, Zeng, Zhengran, Yang, Linyi, Wang, Cunxiang, Chen, Hao, Jiang, Chaoya, Xie, Rui, Wang, Jindong, Xie, Xing et al. · 2023 · arXiv
  6. Large Language Models are not Fair Evaluators
    Wang, Peiyi, Li, Lei, Chen, Liang, Cai, Zefan, Zhu, Dawei, Lin, Binghuai, Cao, Yunbo, Liu, Qi, Liu, Tianyu, Sui, Zhifang · 2023 · arXiv
  7. Introduction to the special issue on statistical language modeling
    Jianfeng Gao, Chin-Yew Lin · 2004 · ACM Transactions on Asian Language Information Processing
  8. On Evaluating Adversarial Robustness of Large Vision-Language Models
    Yunqing Zhao, Tianyu Pang, Chao‐Hai Du · 2023 · arXiv (Cornell University)
  9. Benchmarking Foundation Models with Language-Model-as-an-Examiner
    Bai, Yushi, Ying, Jiahao, Cao, Yixin, Lv, Xin, He, Yuze, Wang, Xiaozhi, Yu, Jifan, Zeng, Kaisheng, Xiao, Yijia, Lyu, Haozhe et al. · 2023 · arXiv
  10. Personality testing of large language models: limited temporal stability, but highlighted prosociality
    Bojana Bodroža, Bojana M. Dinić, Ljubiša Bojić · 2024 · Royal Society Open Science
  11. M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models
    Chuang Liu, Ren-Ren Jin, Yuqi Ren, Lin-Hao Yu, Tian-Yu Dong, Xia Peng, Shuting Zhang, Jianxiang Peng, Peiyi Zhang, Qing Lyu et al. · 2023 · arXiv.org
  12. TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
    Gekhman, Zorik, Herzig, Jonathan, Aharoni, Roee, Elkind, Chen, Szpektor, Idan · 2023 · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
  13. Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
    John J. Nay, David Karamardian, Sarah B. Lawsky, Wenting Tao, Meghana Bhat, Raghav Jain, Aaron Travis Lee, Jonathan H. Choi, Jungo Kasai · 2023 · SSRN Electronic Journal
  14. Leveraging Large Language Models for Topic Classification in the Domain of Public Affairs
    Alejandro Peña, Aythami Morales, Julián Fiérrez, Ignacio Serna, Javier Ortega-García, Íñigo Puente, Jorge Córdova, Gonzalo Córdova · 2023 · Lecture notes in computer science
  15. UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation
    Xun Liang, Shichao Song, Simin Niu, Zhiyu Li, Feiyu Xiong, Bo Tang, Yezhaohui Wang, Dawei He, Peng Cheng, Zhonghao Wang et al. · 2024 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  16. On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex
    Zhuo, Terry Yue, Li, Zhuang, Huang, Yujin, Shiri, Fatemeh, Wang, Weiqing, Haffari, Gholamreza, Li, Yuan-Fang · 2023 · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
  17. A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
    Chenyang Lyu, Du, Zefeng, Jitao Xu · 2023 · arXiv (Cornell University)
  18. How well do Large Language Models perform in Arithmetic tasks?
    Yuan, Zheng, Yuan, Hongyi, Tan, Chuanqi, Wang, Wei, Huang, Songfang · 2023 · arXiv
  19. Document-Level Machine Translation with Large Language Models
    Longyue Wang, Chenyang Lyu, Tianbo Ji · 2023 · arXiv (Cornell University)
  20. KoLA: Carefully Benchmarking World Knowledge of Large Language Models
    Yu, Jifan, Wang, Xiaozhi, Tu, Shangqing, Cao, Shulin, Zhang-Li, Daniel, Lv, Xin, Peng, Hao, Yao, Zijun, Zhang, Xiaohan, Li, Hanming et al. · 2023 · arXiv
  21. Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation
    Gu, Zhouhong, Zhu, Xiaoxuan, Ye, Haoning, Zhang, Lin, Wang, Jianchen, Zhu, Yixin, Jiang, Sihang, Xiong, Zhuozhi, Li, Zihan, Wu, Weijie et al. · 2024 · Proceedings of the AAAI Conference on Artificial Intelligence
  22. How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?
    Deroy, Aniket, Ghosh, Kripabandhu, Ghosh, Saptarshi · 2023 · arXiv
  23. LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
    Zheng, Lianmin, Chiang, Wei-Lin, Sheng, Ying, Li, Tianle, Zhuang, Siyuan, Wu, Zhanghao, Zhuang, Yonghao, Li, Zhuohan, Lin, Zi, Xing, Eric P. et al. · 2023 · arXiv
  24. What is intelligence?
    Nathan Brody · 1999 · International Review of Psychiatry
  25. Adaptive Testing of Computer Vision Models
    Irena Gao, Gabriel Ilharco, Scott Lundberg, Marco Túlio Ribeiro · 2023 · IEEE/CVF International Conference on Computer Vision (ICCV)
  26. Can Large Language Models Understand Real-World Complex Instructions?
    Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, Yanghua Xiao · 2024 · Proceedings of the AAAI Conference on Artificial Intelligence
  27. DDXPlus: A New Dataset for Automatic Medical Diagnosis
    Arsène Fansi Tchango, Rishab Goel, Zhi Wen, Julien Martel, Joumana Ghosn · 2022 · neural information processing systems
  28. Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking
    Zhiyi Ma, Kawin Ethayarajh, Tristan Thrush, Jain, Somya, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, Douwe Kiela · 2021 · arXiv (Cornell University)
  29. Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models
    Zhang, Sarah J., Florin, Samuel, Lee, Ariel N., Niknafs, Eamon, Marginean, Andrei, Wang, Annie, Tyser, Keith, Chin, Zad, Hicke, Yann, Singh, Nikhil et al. · 2023 · arXiv
  30. Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
    Wu, Zhaofeng, Qiu, Linlu, Ross, Alexis, Akyürek, Ekin, Chen, Boyuan, Wang, Bailin, Kim, Najoung, Andreas, Jacob, Kim, Yoon · 2023 · arXiv
  31. SafetyBench: Evaluating the Safety of Large Language Models
    Zhang, Zhexin, Lei, Leqi, Wu, Lindong, Sun, Rui, Huang, Yongkang, Long, Chong, Liu, Xiao, Lei, Xuanyu, Tang, Jie, Huang, Minlie · 2023 · arXiv
  32. Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today
    Zhuo Wang, Rongzhen Li, Dong, Bowen, Jie Wang, Xiuxing Li, Ning Liu, Chenhui Mao, Zhang, Wei, Liling Dong, Jing Gao et al. · 2023 · arXiv (Cornell University)
  33. MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
    Wang, Xingyao, Wang, Zihan, Liu, Jiateng, Chen, Yangyi, Yuan, Lifan, Peng, Hao, Ji, Heng · 2023 · arXiv
  34. Evaluation of AI Chatbots for Patient-Specific EHR Questions
    Alaleh Hamidi, Kirk Roberts · 2023 · arXiv.org
  35. Position: AI Evaluation Should Learn from How We Test Humans
    Zhuang, Yan, Liu, Qi, Pardos, Zachary A., Kyllonen, Patrick C., Zu, Jiyun, Huang, Zhenya, Wang, Shijin, Chen, Enhong · 2023 · arXiv
  36. MEGA: Multilingual Evaluation of Generative AI
    Ahuja, Kabir, Diddee, Harshita, Hada, Rishav, Ochieng, Millicent, Ramesh, Krithika, Jain, Prachi, Nambi, Akshay, Ganu, Tanuja, Segal, Sameer, Axmed, Maxamed et al. · 2023 · arXiv
  37. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
    Min, Sewon, Krishna, Kalpesh, Lyu, Xinxi, Lewis, Mike, Yih, Wen-tau, Koh, Pang Wei, Iyyer, Mohit, Zettlemoyer, Luke, Hajishirzi, Hannaneh · 2023 · arXiv
  38. LAraBench: Benchmarking Arabic AI with Large Language Models
    Ahmed Abdelalí, Hamdy Mubarak, Shammur Absar Chowdhury et al. · 2023 · arXiv (Cornell University)
  39. CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
    Zhao, Jiaxu, Fang, Meng, Shi, Zijing, Li, Yitong, Chen, Ling, Pechenizkiy, Mykola · 2023 · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  40. Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench
    Huang, Jen-tse, Lam, Man Ho, Li, Eric John, Ren, Shujie, Wang, Wenxuan, Jiao, Wenxiang, Tu, Zhaopeng, Lyu, Michael R. · 2023 · arXiv
  41. MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization
    Manakul, Potsawee, Liusie, Adian, Gales, Mark · 2023 · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
  42. LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models
    Yen‐Ting Lin, Yun-Nung Chen · 2023 · arXiv (Cornell University)
  43. Ask Again, Then Fail: Large Language Models' Vacillations in Judgement
    Qiming Xie, Zengzhi Wang, Yi Feng, R. Xia · 2023 · Annual Meeting of the Association for Computational Linguistics
  44. Bring Your Own Data! Self-Supervised Evaluation for Large Language Models
    Neel Jain, Saifullah, Khalid, Yuxin Wen · 2023 · arXiv (Cornell University)
  45. Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
    Fangzhi Xu, Qika Lin, Jiawei Han · 2023 · arXiv (Cornell University)
  46. Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair
    Jason Phang, Angelica Chen, William Huang, Samuel R. Bowman · 2022
  47. DDXPlus: A New Dataset For Automatic Medical Diagnosis
    Arsène Fansi Tchango, Rishab Goel, Zhi Wen · 2022 · arXiv (Cornell University)
  48. Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
    Yuan, Lifan, Chen, Yangyi, Cui, Ganqu, Gao, Hongcheng, Zou, Fangyuan, Cheng, Xingyi, Ji, Heng, Liu, Zhiyuan, Sun, Maosong · 2023 · arXiv
  49. Emotional Intelligence of Large Language Models
    Wang, Xuena, Li, Xueting, Yin, Zi, Wu, Yue, Jia, Liu · 2023 · arXiv
  50. Towards a Unified Multi-Dimensional Evaluator for Text Generation
    Zhong, Ming, Liu, Yang, Yin, Da, Mao, Yuning, Jiao, Yizhu, Liu, Pengfei, Zhu, Chenguang, Ji, Heng, Han, Jiawei · 2022 · arXiv
  51. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation
    Vu, Tu, Iyyer, Mohit, Wang, Xuezhi, Constant, Noah, Wei, Jerry, Wei, Jason, Tar, Chris, Sung, Yun-Hsuan, Zhou, Denny, Le, Quoc et al. · 2023 · arXiv
  52. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
    Tian, Katherine, Mitchell, Eric, Zhou, Allan, Sharma, Archit, Rafailov, Rafael, Yao, Huaxiu, Finn, Chelsea, Manning, Christopher D. · 2023 · arXiv
  53. ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer
    Dongqi Pu, Vera Demberg · 2023 · arXiv (Cornell University)
  54. Evaluating Language Models for Mathematics through Interactions
    Katherine M. Collins, Albert Qiaochu Jiang, Simon Frieder et al. · 2023 · arXiv (Cornell University)
  55. Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs
    Hongru Wang, Rui Wang, Fei Mi · 2023 · arXiv (Cornell University)
  56. On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex
    Terry Yue Zhuo, Zhuang Li, Yujin Huang · 2023 · arXiv (Cornell University)
  57. Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks
    Tristan Thrush, Kushal Tirumala, Anmol Gupta · 2022 · arXiv (Cornell University)
  58. Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair
    Jason Phang, Angelica Chen, William C. W. Huang · 2021 · arXiv (Cornell University)
  59. covLLM: Large Language Models for COVID-19 Biomedical Literature
    Yousuf A. Khan, Clarisse Hokia, Jennifer Xu · 2023 · arXiv (Cornell University)
  60. Adaptive Testing of Computer Vision Models
    Irena Gao, Gabriel Ilharco, Scott Lundberg · 2022 · arXiv (Cornell University)
  61. Log Parsing: How Far Can ChatGPT Go?
    Van-Hoang Le, Hongyu Zhang · 2023 · arXiv (Cornell University)
  62. Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views
    Katerina Margatina, Shuai Wang, Yogarshi Vyas · 2023 · arXiv (Cornell University)
  63. Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models
    Liang Tian, Zhiwei He, Huang, Jen-tse · 2023 · arXiv (Cornell University)
  64. Aligning AI With Shared Human Values
    Hendrycks, Dan, Burns, Collin, Basart, Steven, Critch, Andrew, Li, Jerry, Song, Dawn, Steinhardt, Jacob · 2020 · arXiv
  65. CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
    Hendrycks, Dan, Burns, Collin, Chen, Anya, Ball, Spencer · 2021 · arXiv
  66. TRUE: Re-evaluating Factual Consistency Evaluation
    Honovich, Or, Aharoni, Roee, Herzig, Jonathan, Taitelbaum, Hagai, Kukliansy, Doron, Cohen, Vered, Scialom, Thomas, Szpektor, Idan, Hassidim, Avinatan, Matias, Yossi · 2022 · arXiv
  67. GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective
    Yang, Linyi, Zhang, Shuibai, Qin, Libo, Li, Yafu, Wang, Yidong, Liu, Hanmeng, Wang, Jindong, Xie, Xing, Zhang, Yue · 2022 · arXiv
  68. ThoughtSource: A central hub for large language model reasoning data
    Ott, Simon, Hebenstreit, Konstantin, Liévin, Valentin, Hother, Christoffer Egeberg, Moradi, Milad, Mayrhauser, Maximilian, Praas, Robert, Winther, Ole, Samwald, Matthias · 2023 · arXiv
  69. Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
    Qin, Chengwei, Zhang, Aston, Zhang, Zhuosheng, Chen, Jiaao, Yasunaga, Michihiro, Yang, Diyi · 2023 · arXiv
  70. Exploring Vision-Language Models for Imbalanced Learning
    Wang, Yidong, Yu, Zhuohao, Wang, Jindong, Heng, Qiang, Chen, Hao, Ye, Wei, Xie, Rui, Xie, Xing, Zhang, Shikun · 2023 · arXiv
  71. Can Large Language Models Transform Computational Social Science?
    Ziems, Caleb, Held, William, Shaikh, Omar, Chen, Jiaao, Zhang, Zhehao, Yang, Diyi · 2023 · arXiv
  72. Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation
    Zhang, Jizhi, Bao, Keqin, Zhang, Yang, Wang, Wenjie, Feng, Fuli, He, Xiangnan · 2023 · arXiv
  73. C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models
    Huang, Yuzhen, Bai, Yuzhuo, Zhu, Zhihao, Zhang, Junlei, Zhang, Jinghan, Su, Tangjun, Liu, Junteng, Lv, Chuancheng, Zhang, Yikai, Lei, Jiayi et al. · 2023 · arXiv
  74. Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs
    Song, Xiaoyang, Gupta, Akshat, Mohebbizadeh, Kiyan, Hu, Shujie, Singh, Anant · 2023 · arXiv
  75. EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models
    Tao, Zhengwei, Jin, Zhi, Bai, Xiaoying, Zhao, Haiyan, Feng, Yanlin, Li, Jia, Hu, Wenpeng · 2023 · arXiv
  76. Understanding the Capabilities of Large Language Models for Automated Planning
    Pallagani, Vishal, Muppasani, Bharath, Murugesan, Keerthiram, Rossi, Francesca, Srivastava, Biplav, Horesh, Lior, Fabiano, Francesco, Loreggia, Andrea · 2023 · arXiv
  77. Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
    Arora, Daman, Singh, Himanshu Gaurav, Mausam · 2023 · arXiv
  78. ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Tasks
    Sridhara, Giriprasad, G., Ranjani H., Mazumdar, Sourav · 2023 · arXiv
  79. Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
    Fu, Yao, Ou, Litu, Chen, Mingyu, Wan, Yuhao, Peng, Hao, Khot, Tushar · 2023 · arXiv
  80. MathChat: Converse to Tackle Challenging Math Problems with LLM Agents
    Wu, Yiran, Jia, Feiran, Zhang, Shaokun, Li, Hangyu, Zhu, Erkang, Wang, Yue, Lee, Yin Tat, Peng, Richard, Wu, Qingyun, Wang, Chi · 2023 · arXiv
  81. Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning
    Zhang, Beichen, Zhou, Kun, Wei, Xilin, Zhao, Wayne Xin, Sha, Jing, Wang, Shijin, Wen, Ji-Rong · 2023 · arXiv
  82. Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction
    Wang, Rose E., Demszky, Dorottya · 2023 · arXiv
  83. Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers
    Jahan, Israt, Laskar, Md Tahmid Rahman, Peng, Chun, Huang, Jimmy · 2023 · arXiv
  84. ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
    Jentzsch, Sophie, Kersting, Kristian · 2023 · arXiv
  85. The Two Word Test: A Semantic Benchmark for Large Language Models
    Riccardi, Nicholas, Desai, Rutvik H. · 2023 · arXiv
  86. Measuring and Modifying Factual Knowledge in Large Language Models
    Pezeshkpour, Pouya · 2023 · arXiv
  87. LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
    Yin, Zhenfei, Wang, Jiong, Cao, Jianjian, Shi, Zhelun, Liu, Dingning, Li, Mukai, Sheng, Lu, Bai, Lei, Huang, Xiaoshui, Wang, Zhiyong et al. · 2023 · arXiv
  88. Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
    Nay, John J., Karamardian, David, Lawsky, Sarah B., Tao, Wenting, Bhat, Meghana, Jain, Raghav, Lee, Aaron Travis, Choi, Jonathan H., Kasai, Jungo · 2023 · arXiv
  89. CMMLU: Measuring massive multitask language understanding in Chinese
    Li, Haonan, Zhang, Yixuan, Koto, Fajri, Yang, Yifei, Zhao, Hai, Gong, Yeyun, Duan, Nan, Baldwin, Timothy · 2023 · arXiv
  90. Performance of ChatGPT on USMLE: Unlocking the Potential of Large Language Models for AI-Assisted Medical Education
    Sharma, Prabin, Thapa, Kisan, Thapa, Dikshya, Dhakal, Prastab, Upadhaya, Mala Deep, Adhikari, Santosh, Khanal, Salik Ram · 2023 · arXiv
  91. ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience
    Xu, Ruiyun, Feng, Yue, Chen, Hailiang · 2023 · arXiv
  92. SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
    Li, Bohao, Wang, Rui, Wang, Guangzhi, Ge, Yuying, Ge, Yixiao, Shan, Ying · 2023 · arXiv
  93. MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning
    Zhao, Haozhe, Cai, Zefan, Si, Shuzheng, Ma, Xiaojian, An, Kaikai, Chen, Liang, Liu, Zixuan, Wang, Sheng, Han, Wenjuan, Chang, Baobao · 2023 · arXiv
  94. Can Large Language Models Understand Real-World Complex Instructions?
    He, Qianyu, Zeng, Jie, Huang, Wenhao, Chen, Lina, Xiao, Jin, He, Qianxi, Zhou, Xunzhe, Chen, Lida, Wang, Xintao, Huang, Yuncheng et al. · 2023 · arXiv
  95. Choice-75: A Dataset on Decision Branching in Script Learning
    Hou, Zhaoyi Joey, Zhang, Li, Callison-Burch, Chris · 2023 · arXiv
  96. GPT-4 Technical Report
    OpenAI, Achiam, Josh, Adler, Steven, Agarwal, Sandhini, Ahmad, Lama, Akkaya, Ilge, Aleman, Florencia Leoni, Almeida, Diogo, Altenschmidt, Janko, Altman, Sam et al. · 2023 · arXiv
  97. OPT: Open Pre-trained Transformer Language Models
    Zhang, Susan, Roller, Stephen, Goyal, Naman, Artetxe, Mikel, Chen, Moya, Chen, Shuohui, Dewan, Christopher, Diab, Mona, Li, Xian, Lin, Xi Victoria et al. · 2022 · arXiv
  98. Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
    Liu, Yiheng, Han, Tianle, Ma, Siyuan, Zhang, Jiayue, Yang, Yuanyuan, Tian, Jiaming, He, Hao, Li, Antong, He, Mengshen, Liu, Zhengliang et al. · 2023 · arXiv
  99. Large Language Models Can Be Used to Estimate the Latent Positions of Politicians
    Wu, Patrick Y., Nagler, Jonathan, Tucker, Joshua A., Messing, Solomon · 2023 · arXiv
  100. The Self-Perception and Political Biases of ChatGPT
    Rutinowski, Jérôme, Franke, Sven, Endendyk, Jan, Dormuth, Ina, Pauly, Markus · 2023 · arXiv
  101. CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility
    Xu, Guohai, Liu, Jiayi, Yan, Ming, Xu, Haotian, Si, Jinghui, Zhou, Zhuoran, Yi, Peng, Gao, Xing, Sang, Jitao, Zhang, Rong et al. · 2023 · arXiv
  102. Human-Like Intuitive Behavior and Reasoning Biases Emerged in Language Models -- and Disappeared in GPT-4
    Hagendorff, Thilo, Fabi, Sarah · 2023 · arXiv
  103. ARB: Advanced Reasoning Benchmark for Large Language Models
    Sawada, Tomohiro, Paleka, Daniel, Havrilla, Alexander, Tadepalli, Pranav, Vidas, Paula, Kranias, Alexander, Nay, John J., Gupta, Kshitij, Komatsuzaki, Aran · 2023 · arXiv
  104. Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study
    Chen, Yi, Wang, Rui, Jiang, Haiyun, Shi, Shuming, Xu, Ruifeng · 2023 · arXiv
  105. CMB: A Comprehensive Medical Benchmark in Chinese
    Wang, Xidong, Chen, Guiming Hardy, Song, Dingjie, Zhang, Zhiyi, Chen, Zhihong, Xiao, Qingying, Jiang, Feng, Li, Jianquan, Wan, Xiang, Wang, Benyou et al. · 2023 · arXiv
  106. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    Devlin, Jacob, Chang, Ming-Wei, Lee, Kenton, Toutanova, Kristina · 2018 · arXiv
  107. Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited
    Yuan, Zheng, Yuan, Fajie, Song, Yu, Li, Youhua, Fu, Junchen, Yang, Fei, Pan, Yunzhu, Ni, Yongxin · 2023 · arXiv
  108. Large Language Models Are Not Strong Abstract Reasoners
    Gendron, Gaël, Bao, Qiming, Witbrock, Michael, Dobbie, Gillian · 2023 · arXiv
  109. LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
    Xu, Peng, Shao, Wenqi, Zhang, Kaipeng, Gao, Peng, Liu, Shuo, Lei, Meng, Meng, Fanqing, Huang, Siyuan, Qiao, Yu, Luo, Ping · 2023 · arXiv
  110. Evaluating Open-QA Evaluation
    Wang, Cunxiang, Cheng, Sirui, Guo, Qipeng, Yue, Yuanhao, Ding, Bowen, Xu, Zhikun, Wang, Yidong, Hu, Xiangkun, Zhang, Zheng, Zhang, Yue · 2023 · arXiv
  111. CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
    Zhao, Jiaxu, Fang, Meng, Shi, Zijing, Li, Yitong, Chen, Ling, Pechenizkiy, Mykola · 2023 · arXiv
  112. Can Large Language Models Capture Dissenting Human Voices?
    Lee, Noah, An, Na Min, Thorne, James · 2023 · arXiv
  113. Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights
    Li, Ruyu, Deng, Wenhao, Cheng, Yu, Yuan, Zheng, Zhang, Jiaqi, Yuan, Fajie · 2023 · arXiv
  114. Recommender Systems in the Era of Large Language Models (LLMs)
    Zihuai Zhao, Wenqi Fan, Jiatong Li et al. · 2023 · arXiv
  115. Uncovering ChatGPT's Capabilities in Recommender Systems
    Dai, Sunhao, Shao, Ninglu, Zhao, Haiyuan, Yu, Weijie, Si, Zihua, Xu, Chen, Sun, Zhongxiang, Zhang, Xiao, Xu, Jun · 2023 · arXiv
  116. Dynabench: Rethinking Benchmarking in NLP
    Douwe Kiela, Max Bartolo, Yixin Nie et al. · 2021 · arXiv
  117. MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization
    Potsawee Manakul, Adian Liusie, Mark J. F. Gales · 2023 · arXiv
  118. ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design
    Pier Luca Lanzi, Daniele Loiacono · 2023 · arXiv
  119. Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark
    Choi, Minje, Pei, Jiaxin, Kumar, Sagar, Shu, Chang, Jurgens, David · 2023 · arXiv
  120. Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests
    Hellas, Arto, Leinonen, Juho, Sarsa, Sami, Koutcheme, Charles, Kujanpää, Lilja, Sorva, Juha · 2023 · arXiv
  121. Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics
    Holmes, Jason, Liu, Zhengliang, Zhang, Lian, Ding, Yuzhen, Sio, Terence T., McGee, Lisa A., Ashman, Jonathan B., Li, Xiang, Liu, Tianming, Shen, Jiajian et al. · 2023 · arXiv
  122. UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation
    Liang, Xun, Song, Shichao, Niu, Simin, Li, Zhiyu, Xiong, Feiyu, Tang, Bo, Wang, Yezhaohui, He, Dawei, Cheng, Peng, Wang, Zhonghao et al. · 2023 · arXiv
  123. Translating Radiology Reports into Plain Language using ChatGPT and GPT-4 with Prompt Learning: Promising Results, Limitations, and Potential
    Lyu, Qing, Tan, Josh, Zapadka, Michael E., Ponnatapura, Janardhana, Niu, Chuang, Myers, Kyle J., Wang, Ge, Whitlow, Christopher T. · 2023 · arXiv
  124. Leveraging Large Language Models for Topic Classification in the Domain of Public Affairs
    Alejandro Peña, Aythami Morales, Julian Fierrez, Ignacio Serna, Javier Ortega-Garcia, Iñigo Puente, Jorge Cordova, Gonzalo Cordova · 2023 · arXiv
  125. Personality testing of Large Language Models: Limited temporal stability, but highlighted prosociality
    Bodroza, Bojana, Dinic, Bojana M., Bojic, Ljubisa · 2023 · arXiv
  126. A Survey on Out-of-Distribution Evaluation of Neural NLP Models
    Xinzhe Li, Ming Liu, Shang Gao, Wray Buntine · 2023 · arXiv
  127. Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models
    Yang, Kai-Cheng, Menczer, Filippo · 2023 · arXiv