CoChat
Sign Up
PreprintCited by 1Speech and dialogue systemsNatural Language Processing TechniquesSentiment Analysis and Opinion Mining

Naturally Occurring Feedback is Common, Extractable and Useful

Open access
Don-Yehiya, Shachar, Choshen, Leshem, Abend, Omri·2024·arXiv
DOI 10.48550/arxiv.2407.10944arXiv 2407.10944
Open this paper in CoChat — free accountOpen PDF

Abstract

Human feedback data is a critical component in developing language models. However, collecting this feedback is costly and ultimately not scalable. Inspired by the way human interlocutors provide spontaneous unsolicited feedback to each other, we propose to extract feedback that users naturally include when interacting with chat models. We manually annotated conversations to confirm the presence of naturally occurring feedback in a standard corpus, finding that as much as 30% of the chats include explicit feedback. Comparing to older datasets, we find that naturally occurring feedback is more prevalent in recent conversation datasets, suggesting that more than ever, naturally occurring feedback can serve as a valuable resource for feedback data. We propose a method for automatically extracting this feedback, and apply it to over 1M conversations to obtain hundreds of thousands of feedback samples. The extracted feedback shows promise: training with it improves over baseline models and enhances model alignment to human preferences.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2407.10944

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Don-Yehiya et al. (2024) classifies implicit feedback into two categories: positive and negative.

    “Don-Yehiya et al. (2024) classifies implicit feedback into two categories: (1) positive feedback which praises the model's response (i.e., 'Great job!') and (2) negative feedback which signals the model's previous response was not satisfactory.”

    From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Don-Yehiya et al., 2024

    Section 3.1, “Feedback Taxonomy” (page not provided), distinguishes positive from negative feedback and describes praise or thanks as positive feedback. But it defines five categories—four negative and one positive—not just two categories. The citing sentence captures the broad positive–negative distinction while oversimplifying the paper’s taxonomy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Don-Yehiya et al. (2024) also studied naturally occurring, implicit feedback in large-scale human-LLM interactions datasets.

    “Most closely relevant to our work, Don-Yehiya et al. (2024) also studied naturally occurring, implicit feedback in large-scale human-LLM interactions datasets.”

    From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Don-Yehiya et al., 2024

    Section 4.1 says the paper studies feedback in LMSYS-Chat-1M, a large dataset of real-world human–LLM conversations; Appendix B also reports an analysis of WildChat. But the Introduction distinguishes implicit feedback from the explicit cues the authors say they focus on, so describing their study as one of implicit feedback overstates its scope. Page numbers are unavailable in our copy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Existing annotations are not comprehensive (not every turn labeled).

    “These existing annotations are not comprehensive (i.e., not every turn in the conversation is labeled).”

    From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Don-Yehiya et al., 2024

  • Shaikh et al. (2025) introduces seven categories of user responses, and five of these categories could be mapped back to the five feedback types from Don-Yehiya et al. (2024).

    “Shaikh et al. (2025) introduces seven categories of user responses, and five of these categories could be mapped back to the five feedback types from Don-Yehiya et al. (2024).”

    From User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025) · cites this paper as Don-Yehiya et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “The more data at hand, the better the model (Kaplan et al., 2020; Roberts et al., 2023).”

Flagged first, then by citations

Research works printed but not linked: 54

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. · arXiv
  2. Mohammad Amin Bassiri. 2011. Interactional feedback and the impact of attitude and motivation on noticing l2 form. English Language and Literature Studies, 1(2):61.
  3. Janet Beavin Bavelas and Jennifer Gerwing. 2011. The listener as addressee in face-to-face dialogue. Inter- national Journal of Listening, 25(3):178-198. · DOI
  4. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023. Pythia: A suite for analyzing large language mod- els across training and scaling. In International Conference on Machine Learning, pages 2397-2430.
  5. PMLR.
  6. Hendrik Buschmeier and Stefan Kopp. 2018. Commu- nicative listener feedback in human-agent interaction: Artificial speakers need to be attentive and adaptive. In Proceedings of the 17th international conference on autonomous agents and multiagent systems, pages 1213-1221.
  7. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anasta- sios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, et al. 2024. Chatbot arena: An open plat- form for evaluating llms by human preference. arXiv preprint arXiv:2403.04132. · arXiv
  8. Paul F Christiano, Jan Leike, Tom Brown, Miljan Mar- tic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Ad- vances in neural information processing systems, 30.
  9. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. Ultrafeedback: Boosting lan- guage models with high-quality feedback. arXiv preprint arXiv:2310.01377. · arXiv
  10. Shachar Don-Yehiya, Ben Burtenshaw, Ramon Fernan- dez Astudillo, Cailean Osborne, Mimansa Jaiswal, Tzu-Sheng Kuo, Wenting Zhao, Idan Shenfeld, Andi Peng, Mikhail Yurochkin, Atoosa Kasirzadeh, Yangsibo Huang, Tatsunori Hashimoto, Yacine Jer- nite, Daniel Vila-Suero, Omri Abend, Jennifer Ding, Sara Hooker, Hannah Rose Kirk, and Leshem Choshen. 2024a. The future of open human feed- back. Preprint, arXiv:2408.16961. · arXiv
  11. Shachar Don-Yehiya, Leshem Choshen, and Omri Abend. 2023. Human learning by model feedback: The dynamics of iterative prompting with midjour- ney. In Proceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, pages 4146-4161, Singapore. Association for Com- putational Linguistics. · DOI
  12. Shachar Don-Yehiya, Leshem Choshen, and Omri Abend. 2024b. The sharelm collection and plugin: Contributing human-model chats for the benefit of the community. Preprint, arXiv:2408.08291. · arXiv
  13. Yann Dubois, Balázs Galambosi, Percy Liang, and Tat- sunori B Hashimoto. 2024. Length-controlled al- pacaeval: A simple way to debias automatic evalua- tors. arXiv preprint arXiv:2404.04475. · arXiv
  14. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. Understanding dataset difficulty with V-usable information. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988-6008.
  15. PMLR.
  16. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. · arXiv
  17. Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston. 2019. Learning from dialogue after deployment: Feed yourself, chatbot! In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 3667- 3684, Florence, Italy. Association for Computational Linguistics. · DOI · Paper page
  18. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. Unnatural instructions: Tuning lan- guage models with (almost) no human labor. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 14409-14428, Toronto, Canada. Association for Computational Linguistics. · DOI
  19. Hemadri Jayalath and Lakshmish Ramaswamy. 2022. Enhancing performance of operationalized machine learning models by analyzing user feedback. In Pro- ceedings of the 2022 4th International Conference on Image, Video and Signal Processing, IVSP '22, page 197-203, New York, NY, USA. Association for Computing Machinery. · DOI
  20. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. · arXiv
  21. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gi- anna Lengyel, Guillaume Bour, Guillaume Lam- ple, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024. Mix- tral of experts. Preprint, arXiv:2401.04088. · arXiv
  22. Di Jin, Shikib Mehri, Devamanyu Hazarika, Aishwarya Padmakumar, Sungjin Lee, Yang Liu, and Mahdi Namazifar. 2023. Data-efficient alignment of large language models with human feedback through natu- ral language. arXiv preprint arXiv:2311.14543. · arXiv
  23. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale. 2024. The prism alignment project: What participatory, representa- tive and individualised human feedback reveals about the subjective and multicultural alignment of large language models. Preprint, arXiv:2404.16019. · arXiv
  24. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stan- ley, Richárd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neural Information Process- ing Systems, 36.
  25. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. 2024. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787. · arXiv
  26. Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Car- bune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267. · arXiv
  27. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledge- intensive nlp tasks. Preprint, arXiv:2005.11401. · arXiv
  28. Ying-Chun Lin, Jennifer Neville, Jack W Stokes, Longqi Yang, Tara Safavi, Mengting Wan, Scott Counts, Siddharth Suri, Reid Andersen, Xiaofeng Xu, et al. 2024. Interpretable user satisfaction estima- tion for conversational systems with large language models. arXiv preprint arXiv:2403.12388. · arXiv
  29. Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. 2023a. Chain of hindsight aligns language models with feed- back. arXiv preprint arXiv:2302.02676. · arXiv
  30. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023b. G-eval: NLG evaluation using gpt-4 with better human align- ment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511-2522, Singapore. Association for Com- putational Linguistics. · DOI · Paper page
  31. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730-27744.
  32. Arjun Panickssery, Samuel R. Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. Preprint, arXiv:2404.13076. · arXiv
  33. Andi Peng, Yuying Sun, Tianmin Shu, and David Abel. 2024. Pragmatic feature preferences: Learn- ing reward-relevant preferences from human input. Preprint, arXiv:2405.14769. · arXiv
  34. Dominic Petrak, Nafise Moosavi, Ye Tian, Nikolai Rozanov, and Iryna Gurevych. 2023. Learning from free-text human feedback -collect new datasets or extend existing ones? In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pages 16259-16279, Singapore. Association for Computational Linguistics. · DOI
  35. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  36. Adam Roberts, Hyung Won Chung, Gaurav Mishra, Anselm Levskaya, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, et al. 2023. Scaling up models and data with t5x and seqio. Journal of Machine Learning Research, 24(377):1-8.
  37. Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. 2023. Verbosity bias in preference la- beling by large language models. arXiv preprint arXiv:2310.10076. · arXiv
  38. Jérémy Scheurer, Jon Ander Campos, Jun Shern Chan, Angelica Chen, Kyunghyun Cho, and Ethan Perez. 2022. Training language models with language feed- back. Preprint, arXiv:2204.14146. · arXiv
  39. Weiyan Shi, Emily Dinan, Kurt Shuster, Jason We- ston, and Jing Xu. 2022. When life gives you lemons, make cherryade: Converting feedback from bad responses into good labels. arXiv preprint arXiv:2210.15893. · arXiv
  40. Makesh Narsimhan Sreedhar, Kun Ni, and Siva Reddy. 2020. Learning improvised chatbots from adversar- ial modifications of natural language feedback. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 2445-2453, Online. Association for Computational Linguistics. · DOI
  41. Shivchander Sudalairaj, Abhishek Bhandwaldar, Aldo Pareja, Kai Xu, David D Cox, and Akash Srivas- tava. 2024. Lab: Large-scale alignment for chatbots. arXiv preprint arXiv:2403.01081. · arXiv
  42. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Di- ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine- tuned chat models. Preprint, arXiv:2307.09288. · arXiv
  43. Jelena Vranjes, Geert Brône, and Kurt Feyaerts. 2018. Dual feedback in interpreter-mediated interactions: On the role of gaze in the production of listener re- sponses. Journal of Pragmatics, 134:15-30. · DOI
  44. Margaret G. Werts, Mark Wolery, Ariane Holcombe, and David L. Gast. 1995. Instructive feedback: Re- view of parameters and effects. Journal of Behav- ioral Education, 5(1):55-75.
  45. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2024. Fine- grained human feedback gives better rewards for lan- guage model training. Advances in Neural Informa- tion Processing Systems, 36.
  46. Hao Yan, Saurabh Srivastava, Yintao Tai, Sida I. Wang, Wen-tau Yih, and Ziyu Yao. 2023. Learning to simu- late natural language feedback for interactive seman- tic parsing. In Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3149-3170, Toronto, Canada. Association for Computational Linguistics. · DOI
  47. Asaf Yehudai, Boaz Carmeli, Yosi Mass, Ofir Arviv, Nathaniel Mills, Assaf Toledo, Eyal Shnarch, and Leshem Choshen. 2024. Genie: Achieving human parity in content-grounded datasets generation. arXiv preprint arXiv:2401.14367. · arXiv
  48. Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shim- ing Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiao- hui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, and Zonghong Dai. 2024. Yi: Open foundation models by 01.ai. Preprint, arXiv:2403.04652. · arXiv
  49. Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu, and Maosong Sun. 2024. Advancing llm reasoning generalists with pref- erence trees. Preprint, arXiv:2404.02078. · arXiv
  50. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470. · arXiv
  51. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. Preprint, arXiv:2309.03882. · arXiv
  52. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. P Xing, Joseph E. Gon- zalez, Ion Stoica, and Hao Zhang. 2023a. Lmsys- chat-1m: A large-scale real-world llm conversation dataset. Preprint, arXiv:2309.11998. · arXiv
  53. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023b. Judg- ing llm-as-a-judge with mt-bench and chatbot arena. Preprint, arXiv:2306.05685. · arXiv
  54. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiv:2310.17631. · arXiv

Cited by

Sorted by citations

Authors
Don-Yehiya, S., Choshen, L., & Abend, O.
Affiliations
  • Hebrew University of Jerusalem (IL)
  • IBM (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2407.10944
arXiv
2407.10944
OpenAlex
W4400720181
Cited by
1
References
1
Citations by year
2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
chatbots, conversational datasets, language model training, model alignment, human preferences

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review