ChatBench: From Static Benchmarks to Human-AI Evaluation
Open accessAbstract
With the rapid adoption of LLM-based chatbots, there is a pressing need to evaluate what humans and LLMs can achieve together.However, standard benchmarks, such as MMLU, measure LLM capabilities in isolation (i.e., "AIalone").Here, we design and conduct a user study to convert MMLU questions into user-AI conversations, by seeding the user with the question and having them carry out a conversation with the LLM to answer their question.We release ChatBench, a new dataset with AIalone, user-alone, and user-AI data for 396 questions and two LLMs, including 144K answers and 7,336 user-AI conversations.We find that AI-alone accuracy fails to predict user-AI accuracy, with significant differences across multiple subjects (math, physics, and moral reasoning), and we analyze the user-AI conversations to provide insight into how they diverge from AI-alone benchmarks.Finally, we show that fine-tuning a user simulator on a subset of ChatBench improves its ability to estimate user-AI accuracies, increasing correlation on heldout questions by more than 20 points, creating possibilities for scaling interactive evaluation. 1 A number is rounded to the nearest hundred.The result is 500.Which number could not be the number before it was rounded to the nearest hundred? A. 458 B. 463 C. 547 D. 559 Elementary MathematicsTo say that a rose petal is red is to say that it A. absorbs red B. reflects red C. emits red D. transmits red Conceptual PhysicsFor which of these two scenarios does the main character do something clearly morally wrong?Scenario 1 | I embraced my sister's newly widowed husband to show him sympathy.Scenario 2 | I came home late one night to find that I had lost my house keys, so I had to break into my neighbor's house.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The outcome of a multi-turn interaction can be upper bounded by both human and AI participants.
“This is due to the outcome of a multi-turn interaction can be upper bounded by both human and AI participants (Chang et al., 2025).”
From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Chang et al., 2025
Section 4.2 reports that user-AI accuracy on the math datasets was significantly better than user-alone accuracy, so the human's unaided performance was not an upper bound. Section 4.3 also reports questions where user-AI accuracy exceeded AI-alone accuracy. The paper describes interaction outcomes as varying with the combination, not as bounded by both participants' standalone performance; page numbers are unavailable in our copy. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredChang et al. found that users leave some detail underspecified a majority of the time.
“Chang et al. [7] hired annotators to manually reproduce fully-specified instructions through a chat interface, and found that the users reveal the entirety of the instruction in 34% of the time, leaving some detail underspecified a majority of the time.”
From LLMs Get Lost In Multi-Turn Conversation (2025) · cites this paper as [7]
Section 4.3 reports that the first substantive prompt was not a near-exact rephrasing of the benchmark question in a majority of interactions: 66.2% with GPT-4o and 59.8% with Llama-3.1-8b. It says such prompts were often ambiguous, sometimes omitting critical information, but does not establish that every non-rephrasing left a detail unspecified. The same section describes classification by a separate LLM, not manual annotation, and does not report a single 34% rate for revealing the entire question. (LLM, conf=0.88)
Experimental — accuracy not yet measuredChang et al. found that users reveal the entirety of the instruction in 34% of the time.
“Chang et al. [7] hired annotators to manually reproduce fully-specified instructions through a chat interface, and found that the users reveal the entirety of the instruction in 34% of the time, leaving some detail underspecified a majority of the time.”
From LLMs Get Lost In Multi-Turn Conversation (2025) · cites this paper as [7]
Section 4.3 reports that 66.2% of GPT-4o interactions began with a prompt that was not a near-exact rephrasing of the benchmark question, implying 33.8% were—approximately the cited 34%. But it measures near-exact rephrasing, not whether users revealed the entirety of an instruction; it also gives a different rate for Llama-3.1-8b. The section says omitted critical information is one source of divergence, not that every non-rephrasing prompt was underspecified. (LLM, conf=0.86)
Experimental — accuracy not yet measuredPrior works have demonstrated the 'interactiveness' of how general users collaborate with language assistants, where ambiguous user queries are usually given at first followed by a series of clarifying actions.
“Rather than single-pass instruction following, prior works (Lee et al., 2022; Chang et al., 2025; Laban et al., 2025) have demonstrated the”
From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Chang et al., 2025
The Introduction and §4.3 describe early user prompts that omit information and are often ambiguous, supporting the point that interaction differs from single-pass instruction following. But Appendix B.3 says the study’s conversations tended to be short and cites real-world data in which around 60% of conversations had only one user–AI turn; it does not support a usual series of clarifying actions. (LLM, conf=0.87)
Experimental — accuracy not yet measuredLLM performance on multi-turn tasks is worse than on single-turn tasks.
“(Chang et al., 2025; Laban et al., 2025) shows that LLM performance on multi-turn tasks is worse than on single-turn tasks.”
From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Chang et al., 2025
Section 4.2 reports that GPT-4o's accuracy on the math datasets was significantly lower in user–AI interactions than when the AI answered alone. But the comparison is between user–AI conversations and AI-alone benchmark prompts, not a general comparison of multi-turn and single-turn tasks. The same section says the gap varies by model and dataset and finds no further drop for Llama-3.1-8b on the math datasets. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Still being checked.
Research works printed but not linked: 35
- Gagan Bansal, Tongshuang Wu, Joyce Zhou, Ray- mond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In CHI '21: Pro- ceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 81, pages 1-16.
- Serina Chang, Alicja Chaszczewicz, Emma Wang, Maya Josifovska, Emma Pierson, and Jure Leskovec. 2024. LLMs generate structurally realistic social net- works but overestimate political homophily. arXiv preprint arXiv:2408.16629. · arXiv
- Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023a. Marked personas: Using natural language prompts to measure stereotypes in language models. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL'23).
- Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023b. Compost: Characterizing and evaluating caricature in LLM simulations. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing (EMNLP'23).
- Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anasta- sios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot Arena: An open platform for evaluating LLMs by human prefer- ence. arXiv preprint arXiv:2403.04132. · arXiv
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems. arXiv preprint arXiv:2110.14168. · arXiv
- Katherine M. Collins, Albert Q. Jiang, Simon Frieder, Lionel Wong, Miri Zilka, Umang Bhatt, Thomas Lukasiewicz, Yuhuai Wu, Joshua B. Tenenbaum, William Hart, Timothy Gowers, Wenda Li, Adrian Weller, and Mateja Jamnik. 2024. Evaluating lan- guage models for mathematics through interactions. Proceedings of the National Academy of Sciences (PNAS).
- Paramveer S. Dhillon, Somayeh Molaei, Jiaqi Li, Max- imilian Golub, Shaochun Zheng, and Lionel Peter Robert. 2024. Shaping human-ai collaboration: Var- ied scaffolding levels in co-writing with language models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI'24).
- Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpaca- farm: A simulation framework for methods that learn from human feedback. In Proceedings of the 37th International Conference on Neural Information Pro- cessing Systems (NeurIPS'23).
- Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Man- cino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Bar- ale, Robert McHardy, Joshua Harris, Jean Kad- dour, Emile van Krieken, and Pasquale Minervini. 2024. Are we done with MMLU? arXiv preprint arXiv:2406.04127. · arXiv
- Gemini Team Google. 2023. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. · arXiv
- Amelia Hardy, Anka Reuel, Kiana Jafari Meimandi, Lisa Soder, Allie Griffith, Dylan M. Asmar, Sanmi Koyejo, Michael S. Bernstein, and Mykel J. Kochen- derfer. 2024. More than marketing? on the informa- tion value of ai benchmarks for practitioners. arXiv preprint arXiv:2412.05520. · arXiv
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language under- standing. In The Ninth International Conference on Learning Representations (ICLR'21).
- John J. Horton. 2023. Large language models as sim- ulated economic agents: What can we learn from homo silicus? arXiv preprint arXiv:2301.07543. · arXiv
- EunJeong Hwang, Bodhisattwa Majumder, and Niket Tandon. 2023. Aligning language models to user opinions. In Findings of the Association for Compu- tational Linguistics: EMNLP 2023.
- Irina Jurenka, Markus Kunesch amd Kevin R. McKee, Daniel Gillick, Shaojian Zhu, Sara Wiltberger, Shub- ham Milind Phal, Katherine Hermann, Daniel Kasen- berg, Avishkar Bhoopchand, et al. 2024. Towards responsible development of generative ai for educa- tion: An evaluation-driven approach. arXiv preprint arXiv:2407.12687. · arXiv
- Chuyi Kong, Yaxin Fan, Xiang Wan, Feng Jiang, and Benyou Wang. 2024. PlatoLM: Teaching LLMs in multi-round dialogue via a user simulator. In Pro- ceedings of the 62nd Annual Meeting of the Associa- tion for Computational Linguistics (ACL'24).
- Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, Rose E. Wang, Minae Kwon, Joon Sung Park, Hancheng Cao, Tony Lee, Rishi Bommasani, Michael Bernstein, and Percy Liang. 2023. Evaluat- ing human-language model interaction. Transactions on Machine Learning Research.
- Ruosen Li, Ruochen Li, Barry Wang, and Xinya Du. 2024a. IQA-EVAL: Automatic evaluation of human- model interactive question answering. In Proceed- ings of the 38th International Conference on Neural Information Processing Systems (NeurIPS'24).
- Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. 2024b. Mediq: Question- asking LLMs and a benchmark for reliable interac- tive clinical reasoning. In Proceedings of the 38th International Conference on Neural Information Pro- cessing Systems (NeurIPS'24).
- Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, et al. 2023. Holistic evaluation of language mod- els. Transactions of Machine Learning Research.
- Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. 2024. WildBench: Benchmarking LLMs with challenging tasks from real users in the wild. In arXiv preprint arXiv:2406.04770. · arXiv
- Llama Team, AI@Meta. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. · arXiv
- Evan Miller. 2024. Adding error bars to evals: A statis- tical approach to language model evaluations. arXiv preprint arXiv:2411.00640. · arXiv
- OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
- Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simu- lacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST'23).
- Lisa Posch, Arnim Bleier, Fabian Flöck, Clemens M. Lechner, Katharina Kinder-Kurlanda, Denis Helic, and Markus Strohmaier. 2022. Characterizing the global crowd workforce:a cross-country comparison of crowdworker demographics. Human Computation, 9(1):22-57.
- Ruiyang Ren, Peng Qiu, Yingqi Qu, Jing Liu, Xin Zhao, Hua Wu, Ji-Rong Wen, and Haifeng Wang. 2024. BASES: Large-scale web search user simu- lation with large language model based agents. In Findings of the Association for Computational Lin- guistics: EMNLP 2024.
- Yijia Shao, Vinay Samuel, Yucheng Jiang, John Yang, and Diyi Yang. 2024. Collaborative gym: A frame- work for enabling and evaluating human-agent col- laboration. arXiv preprint arXiv:2412.15701. · arXiv
- Joseph Suh, Erfan Jahanparast, Suhong Moon, Min- woo Kang, and Serina Chang. 2025. Language model fine-tuning on scaled survey data for predict- ing distributions of public opinions. arXiv preprint arXiv:2502.16761. · arXiv
- Michelle Vaccaro, Abdullah Almaatouq, and Thomas Malone. 2024. When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8:2293-2303.
- Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answer- ing. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing (EMNLP'18).
- Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. WildChat: 1m chatgpt interaction logs in the wild. In Proceedings of the 12th International Conference on Learning Representations (ICLR'24).
- Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2024. LMSYS-Chat-1M: A large-scale real-world LLM conversation dataset. In Proceedings of the 12th International Conference on Learning Representations (ICLR'24).
- Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with mt-bench and chatbot arena. In Proceedings of the 37th International Confer- ence on Neural Information Processing Systems (NeurIPS'23).
Cited by
Sorted by citations
LLMs Get Lost In Multi-Turn Conversation
Laban, P., Hayashi, H., Zhou, Y., & Neville, J. (2025) · arXiv · Cited by 10 · see citing works
“Existing studies implemented such user simulation in different ways: relying on templates [12, 68, 39, 16], using an LLM [63, 46, 7, 48], involving human annotators [21, 7], or real users as part of a study [67, 38, 11].”
User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal
Liu, Y., Zhang, M. J. Q., & Choi, E. (2025) · arXiv · doi:10.48550/arxiv.2507.23158 · Cited by 3 · see citing works
“This is due to the outcome of a multi-turn interaction can be upper bounded by both human and AI participants (Chang et al., 2025).”
InfluentialGu, F., & Boyd-Graber, J. L. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1018041
- Authors
- Chang, S., Anderson, A., & Hofman, J. M.
- Affiliations
- University of California, Berkeley (US)
- University of Toronto (CA)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2025
- Pages
- 26009-26038
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4412889966
- Semantic Scholar
- 810c1d51e992f7755ace1cc479879a13eb4cabc9
- Cited by
- 5
- References
- 0
- Citations by year
- 2026: 3 · 2025: 2
- FWCI
- 3.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- human-AI interaction, LLM chatbots, MMLU, interactive assessment, moral reasoning, conversational AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
