CoChat
Sign Up
PreprintCited by 1Natural Language Processing TechniquesTopic ModelingEthics and Social Impacts of AI

LLMs Corrupt Your Documents When You Delegate

Open access
Laban, Philippe, Schnabel, Tobias, Neville, Jennifer·2026·arXiv
DOI 10.48550/arxiv.2604.15597arXiv 2604.15597
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2604.15597

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Improving citation recall in deep research systems is challenging because their complex multi-agent architectures pass information through multiple agents, and both content and citations can get corrupted along the way.

    “improving citation recall is challenging because DR systems are complex multi-agent architectures where information passes through agents like a telephone game, and both content and citations can get corrupted along the way.”

    From Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research (2026) · cites this paper as Laban et al., 2026

    The Benchmarking AI Systems for Document Editing section says errors can compound across long delegated workflows, and the Deletion vs. Corruption section describes distortion of document content. The paper does not establish that deep research systems pass information between multiple agents or that citations become corrupted, so it does not support the claimed explanation for difficulty improving citation recall. Page numbers were not available in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 1

Research works printed but not linked: 76

  1. Miltiadis Allamanis, Sheena Panthaplackel, and Pengcheng Yin. Unsupervised evaluation of code llms with round-trip correctness. pp. 1050-1066, 2024.
  2. Alexander Bick, A. Blandin, and David J. Deming. The rapid adoption of generative ai. SSRN Electronic Journal, 2024.
  3. Michelle Brachman, Amina El-Ashry, Casey Dugan, and Werner Geyer. How knowledge workers use and want to use llms in an enterprise context. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2024.
  4. Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18392-18402, 2022.
  5. Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Anton Lozhkov, C. Anderson, and Arjun Guha. Can it edit? evaluating the ability of large language models to follow code editing instructions. ArXiv, abs/2312.12450, 2023. · arXiv
  6. Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. Can ai writing be salvaged? mitigating idiosyn- crasies and improving human-ai alignment in the writing process through edits. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2024.
  7. Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. Ai-slop to ai-polish? aligning language models through edit-based writing rewards and test-time computation. ArXiv, abs/2504.07532, 2025. · arXiv
  8. Aaron Chatterji, T. Cunningham, David Deming, Zoë Hitzig, Christopher Ong, Carl Shan, and Kevin Wadman. How people use chatgpt. SSRN Electronic Journal, 2025.
  9. Kaiyuan Chen, Yixin Ren, Yang Liu, X. Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, Chen Sun, Han Hou, Hui Yang, James Pan, Jian-Guang Lou, Jiayi Mao, Jizheng Liu, Jinpeng Li, Kangyi Liu, Ke Liu, Rui Wang, Runhao Li, Tong Niu, Wenlong Zhang, Wenqi Yan, Xuanzheng Wang, Yuchen Zhang, Yi-Hsin Hung, Yuan Jiang, Zexuan Liu, Zihang Yin, Zi min Ma, and Zhi-Wei Mo. xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations. ArXiv, abs/2506.13651, 2025a. · arXiv
  10. Gheorghe Comanici et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. ArXiv, abs/2507.06261, 2025. · arXiv
  11. Fabrizio Dell'Acqua, Edward McFowland, Ethan Mollick, Hila Lifshitz-Assaf, Katherine C. Kellogg, Saran Rajendran, Lisa A. Krayer, F. Candelon, and K. Lakhani. Navigating the jagged technological frontier: Field experimental evidence of the effects of ai on knowledge worker productivity and quality. SSRN Electronic Journal, 2023.
  12. Alexandre Drouin, Maxime Gasse, Massimo Caccia, I. Laradji, Manuel Del Verme, Tom Marty, L'eo Boisvert, Megh Thakkar, Quentin Cappart, David Vázquez, Nicolas Chapados, and Alexandre Lacoste. Workarena: How capable are web agents at solving common knowledge work tasks? ArXiv, abs/2403.07718, 2024. · arXiv
  13. Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang, M. Lomeli, Patrick Lewis, Gautier Izacard, Edouard Grave, Sebastian Riedel, and F. Petroni. Editeval: An instruction-based benchmark for text improvements. ArXiv, abs/2209.13331, 2022. · arXiv
  14. Tyna Eloundou, Sam Manning, Pamela Mishkin, and Daniel Rock. Gpts are gpts: An early look at the labor market impact potential of large language models. ArXiv, abs/2303.10130, 2023. · arXiv
  15. Jiawei Guo, Ziming Li, Xueling Liu, Kaijing Ma, Tianyu Zheng, Zhouliang Yu, Ding Pan, Yizhi Li, Ruibo Liu, Yue Wang, Shuyue Guo, Xingwei Qu, Xiang Yue, Ge Zhang, Wenhu Chen, and Jie Fu. Codeeditorbench: Evaluating code editing capability of large language models. ArXiv, abs/2404.03543, 2024. · arXiv
  16. Kunal Handa, Alex Tamkin, Miles McCain, Saffron Huang, Esin Durmus, Sarah Heck, J. Mueller, Jerry Hong, Stuart Ritchie, Tim Belonax, Kevin K. Troy, Dario Amodei, Jared Kaplan, Jack Clark, and Deep Ganguli. Which economic tasks are performed with ai? evidence from millions of claude conversations. ArXiv, abs/2503.04761, 2025. · arXiv
  17. Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. Dual learning for machine translation. pp. 820-828, 2016.
  18. Christine Herlihy, Jennifer Neville, Tobias Schnabel, and Adith Swaminathan. On overcoming miscalibrated conversational priors in llm-based chatbots. ArXiv, abs/2406.01633, 2024. · arXiv
  19. Cong Duy Vu Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. Iterative back-translation for neural machine translation. pp. 18-24, 2018.
  20. Zhaochen Hong, Haofei Yu, and Jiaxuan You. Consistencychecker: Tree-based evaluation of llm generaliza- tion capabilities. pp. 33039-33075, 2025.
  21. Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan, Yixin Mao, Huan Wang, Silvio Savarese, Caiming Xiong, Philippe Laban, and Chien-Sheng Wu. Crmarena: Understanding the capacity of llm agents to perform professional crm tasks in realistic environments. pp. 3830-3850, 2024.
  22. Jihyoung Jang, Minseong Boo, and Hyounghun Kim. Conversation chronicles: Towards diverse temporal and relational dynamics in multi-session conversations. ArXiv, abs/2310.13420, 2023. · arXiv
  23. Saurabh Jha, Rohan R. Arora, Yuji Watanabe, Takumi Yanagawa, Yinfang Chen, Jackson Clark, Bhavya Bhavya, Mudit Verma, Harshit Kumar, Hirokuni Kitahara, Noah Zheutlin, Saki Takano, Divya Pathak, Felix George, Xinbo Wu, B. Turkkan, Gerard Vanloo, M. Nidd, Ting Dai, Oishik Chatterjee, Pranjal Gupta, Suranjana Samanta, Pooja Aggarwal, Rong Lee, Pavankumar Murali, Jae wook Ahn, Debanjana Kar, Ameet Rahane, Carlos Fonseca, Amit M. Paradkar, Yu Deng, Pratibha Moogi, P. Mohapatra, Naoki Abe, Chandra Narayanaswami, Tianyin Xu, Lav R. Varshney, R. Mahindru, A. Sailer, Laura Shwartz, Daby M. Sow, Nicholas C. Fuller, Ruchir Puri Ibm, and U. I. Urbana-Champaign. Itbench: Evaluating ai agents across diverse real-world it automation tasks. ArXiv, abs/2502.05352, 2025. · arXiv
  24. Albert Qiaochu Jiang et al. Mistral 7b. ArXiv, abs/2310.06825, 2023. · arXiv
  25. Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, C. J. Taylor, and Dan Roth. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale. ArXiv, abs/2504.14225, 2025. · arXiv
  26. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? ArXiv, abs/2310.06770, 2023. · arXiv
  27. Tae Soo Kim, Yoonjoo Lee, Jaesang Yu, John Joon Young Chung, and Juho Kim. Discoverllm: From executing intents to discovering them. arXiv preprint arXiv:2602.03429, 2026. · arXiv
  28. Philippe Laban, Jesse Vig, Wojciech Kryscinski, Shafiq R. Joty, Caiming Xiong, and Chien-Sheng Wu. Swipe: A dataset for document-level simplification of wikipedia pages. pp. 10674-10695, 2023.
  29. Philippe Laban, A. R. Fabbri, Caiming Xiong, and Chien-Sheng Wu. Summary of a haystack: A challenge to long-context llms and rag systems. pp. 9885-9903, 2024.
  30. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. Llms get lost in multi-turn conversation. ArXiv, abs/2505.06120, 2025. · arXiv
  31. Black Forest Labs, Stephen Batifol, A. Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Muller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space. ArXiv, abs/2506.15742, 2025. · arXiv
  32. M. Lachaux, Baptiste Rozière, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of programming languages. ArXiv, abs/2006.03511, 2020. · arXiv
  33. Guillaume Lample, Ludovic Denoyer, and Marc'Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. ArXiv, abs/1711.00043, 2017. · arXiv
  34. Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Di Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. Charte3: A comprehensive benchmark for end-to-end chart editing. ArXiv, abs/2601.21694, 2026. · arXiv
  35. Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, J. Weston, and M. Lewis. Self-alignment with instruction backtranslation. ArXiv, abs/2308.06259, 2023. · arXiv
  36. Xintong Li, Jalend Bantupalli, Ria Dharmani, Yuwei Zhang, and Jingbo Shang. Toward multi-session personalized conversation: A large-scale dataset and hierarchical tree framework for implicit reasoning. pp. 11493-11506, 2025.
  37. Zheng Li, Xiang Chen, and Xiaojun Wan. Wikitableedit: A benchmark for table editing by natural language instruction. ArXiv, abs/2403.02962, 2024. · arXiv
  38. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Annual Meeting of the Association for Computational Linguistics, pp. 74-81, 2004.
  39. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, F. Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2023.
  40. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019. · arXiv
  41. Adyasha Maharana, Dong-Ho Lee, S. Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluat- ing very long-term conversational memory of llm agents. ArXiv, abs/2402.17753, 2024. · arXiv
  42. Nickil Maveli, Antonio Vergari, and Shay B. Cohen. Can llms compress (and decompress)? evaluating code understanding and execution via invertibility. ArXiv, abs/2601.13398, 2026. · arXiv
  43. Mantas Mazeika, Alice Gatti, Cristina Menghini, Udari Madhushani Sehwag, Shivam Singhal, Yury Orlovskiy, Steven Basart, Manasi Sharma, Denis Peskoff, Elaine Lau, Jaehyuk Lim, Lachlan Carroll, Alice Blair, V. Sivakumar, Sumana Basu, Brad Kenstler, Yuntao Ma, Julian Michael, Xiaoke Li, Oliver Ingebretsen, Aditya Mehta, Jean Mottola, John Teichmann, Kevin Yu, Zaina Shaik, Adam Khoja, Richard Ren, J. Hausen- loy, Long Phan, Ye Htet, Ankit Aich, Tahseen Rabbani, Vivswan Shah, Andriy Novykov, F. Binder, K. Chugunov, L. Ramírez, Matias Geralnik, Hern'an Mesura, Dean Lee, E. Cardona, A. Diamond, Summer Yue, Alexandr Wang, Bing Liu, Ernesto Hernandez, and Dan Hendrycks. Remote labor index: Measuring ai automation of remote work. ArXiv, abs/2510.26787, 2025. · arXiv
  44. Marcus J. Min, Yangruibo Ding, Luca Buratti, Saurabh Pujar, Gail E. Kaiser, Suman Jana, and Baishakhi Ray. Beyond accuracy: Evaluating self-consistency of code large language models with identitychain. ArXiv, abs/2310.14053, 2023. · arXiv
  45. Tarek Naous, Philippe Laban, Wei Xu, and Jennifer Neville. Flipping the dialogue: Training and evaluating user language models. arXiv preprint arXiv:2510.06552, 2025. · arXiv
  46. Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, N. Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, G. Sastry, Gretchen Krueger, D. Schnurr, F. Such, K. Hsu, Madeleine Thompson, Tabarak Khan, T. Sherbakov, Joanne Jang, Peter Welinder, and Lilian Weng. Text and code embeddings by contrastive pre-training. ArXiv, abs/2201.10005, 2022. · arXiv
  47. Thao Nguyen, Jeffrey Li, Sewoong Oh, Ludwig Schmidt, Jason Weston, Luke S. Zettlemoyer, and Xian Li. Better alignment with instruction back-and-forth translation. pp. 13289-13308, 2024.
  48. Kunato Nishina and Yusuke Matsui. Svgeditbench: A benchmark dataset for quantitative assessment of llm's svg editing capabilities. ArXiv, abs/2404.13710, 2024. · arXiv
  49. Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. Nomic embed: Training a reproducible long context text embedder. ArXiv, abs/2402.01613, 2024. · arXiv
  50. Michael Ofengenden, Yunze Man, Ziqi Pang, and Yu-Xiong Wang. Pptarena: A benchmark for agentic powerpoint editing. ArXiv, abs/2512.03042, 2025. · arXiv
  51. Tejal Patwardhan, Rachel Dias, Elizabeth Proehl, Grace Kim, Michele Wang, Olivia Watkins, S. Fishman, Marwan Aljubeh, Phoebe Thacker, Laurance Fauconnet, Natalie S. Kim, Patrick Chao, Samuel Miserendino, Gildas Chabot, David Li, Michael Sharman, Alexandra Barr, Amelia Glaese, and Jerry Tworek. Gdpval: Evaluating ai model performance on real-world economically valuable tasks. ArXiv, abs/2510.04374, 2025. · arXiv
  52. V. Pimenova, Sarah Fakhoury, Christian Bird, M. Storey, and Madeline Endres. Good vibrations? a qualitative study of co-creation, communication, flow, and trust in vibe coding. ArXiv, abs/2509.12491, 2025. · arXiv
  53. Vipul Raheja, Dhruv Kumar, Ryan Koo, and Dongyeop Kang. Coedit: Text editing by task-specific instruction tuning. ArXiv, abs/2305.09857, 2023. · arXiv
  54. Baptiste Rozière, J Zhang, François Charton, M. Harman, Gabriel Synnaeve, and Guillaume Lample. Lever- aging automated unit tests for unsupervised code translation. ArXiv, abs/2110.06773, 2021. · arXiv
  55. Rico Sennrich, B. Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. ArXiv, abs/1511.06709, 2015. · arXiv
  56. Yijia Shao, Humishka Zope, Yucheng Jiang, Jiaxin Pei, D. Nguyen, Erik Brynjolfsson, and Diyi Yang. Future of work with ai agents: Auditing automation and augmentation potential across the u.s. workforce. ArXiv, abs/2506.06576, 2025. · arXiv
  57. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Scharli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. pp. 31210-31227, 2023.
  58. Alexa Siu and Raymond Fok. Augmenting expert cognition in the age of generative ai: Insights from document-centric knowledge work. ArXiv, abs/2503.24334, 2025. · arXiv
  59. J. Skalse, Nikolaus H. R. Howe, D. Krasheninnikov, and David Krueger. Defining and characterizing reward hacking. ArXiv, abs/2209.13085, 2022. · arXiv
  60. H. Somers. Round-trip translation: What is it good for? pp. 127-133, 2005.
  61. Alexander Spangher, Xiang Ren, Jonathan May, and Nanyun Peng. Newsedits: A news article revision dataset and a novel document-level reasoning challenge. ArXiv, abs/2206.07106, 2022. · arXiv
  62. Adam Suma and Sam Dauncey. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. ArXiv, abs/2501.12948, 2025. · arXiv
  63. Kimi Team et al. Kimi k1.5: Scaling reinforcement learning with llms. ArXiv, abs/2501.12599, 2025. · arXiv
  64. Mara Ulloa, Jenna L. Butler, Sankeerti Haniyur, Courtney Miller, Barrett Amos, Advait Sarkar, and M. Storey. Product manager practices for delegating work to generative ai: "accountability must not be delegated to non-human actors". ArXiv, abs/2510.02504, 2025. · arXiv
  65. Bolin Wei, Ge Li, Xin Xia, Zhiyi Fu, and Zhi Jin. Code generation as a dual task of code summarization. pp. 6559-6569, 2019.
  66. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. Longmemeval: Bench- marking chat assistants on long-term interactive memory. ArXiv, abs/2410.10813, 2024. · arXiv
  67. Frank F. Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Meng Bao, Z. Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Ming-Hsuan Yang, Hao Lu, Amaad Martin, Zhe Su, L. Maben, Raj Mehta, Wayne Chi, Lawrence Jang, Yiqing Xie, Shuyan Zhou, and Graham Neubig. Theagentcompany: Benchmarking llm agents on consequential real world tasks. ArXiv, abs/2412.14161, 2024. · arXiv
  68. Jing Xu, Arthur Szlam, and J. Weston. Beyond goldfish memory: Long-term open-domain conversation. ArXiv, abs/2107.07567, 2021. · arXiv
  69. Jialin Yang, Dongfu Jiang, Lipeng He, Sherman Siu, Yuxuan Zhang, Disen Liao, Zhuofeng Li, Huaye Zeng, Yiming Jia, Haozhe Wang, Benjamin Schneider, Chi Ruan, Wentao Ma, Z. Lyu, Yifei Wang, Yi Lu, Quy Duc Do, Ziyan Jiang, Ping Nie, and Wenhu Chen. Structeval: Benchmarking llms' capabilities to generate structural outputs. Trans. Mach. Learn. Res., 2026, 2025.
  70. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. ArXiv, abs/2210.03629, 2022. · arXiv
  71. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent- user interaction in real-world domains. ArXiv, abs/2406.12045, 2024. · arXiv
  72. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. ArXiv, abs/1904.09675, 2019. · arXiv
  73. Junhao Zheng, Xidi Cai, Qiuke Li, Duzhen Zhang, Zhongzhi Li, Yingying Zhang, Le Song, and Qianli Ma. Lifelongagentbench: Evaluating llm agents as lifelong learners. ArXiv, abs/2505.11942, 2025. · arXiv
  74. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, E. Xing, Haotong Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a- judge with mt-bench and chatbot arena. ArXiv, abs/2306.05685, 2023. · arXiv
  75. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242-2251, 2017.
  76. Terry Yue Zhuo, Qiongkai Xu, Xuanli He, and Trevor Cohn. Rethinking round-trip translation for machine translation evaluation. pp. 319-337, 2022.

Authors
Laban, P., Schnabel, T., & Neville, J.
Affiliations
Microsoft (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2604.15597
arXiv
2604.15597
OpenAlex
W7154980156
Cited by
1
References
0
Citations by year
2026: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
large language models, document editing, LLM reliability, knowledge work

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review