Towards a Human-like Open-Domain Chatbot
Open accessAbstract
We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of a human-like multi-turn conversation. Our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The broad range of capabilities in generative conversational AI and its human-likeness in conversation style substantially exceeds that of previous conversational technologies.
“The broad range of capabilities in generative conversational AI and its human-likeness in conversation style substantially exceeds that of previous conversational technologies.”
From Why do people use ChatGPT? Exploring user motivations for generative conversational AI (2024) · cites this paper as Adiwardana, et al., 2020
The Introduction (pp. 1–2) says open-domain chatbots can converse on any topic and reports that Meena scored closer to human performance on sensibleness and specificity than the other chatbots tested. It does not establish that generative conversational AI broadly exceeds previous conversational technologies across a broad range of capabilities; the Discussion (p. 14) notes that capabilities such as humor, empathy, deep reasoning, and question answering were not measured. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Self-citationDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
(2004) · Cited by 9,718 · see citing works
Computing Machinery and Intelligence (1950)
Turing, A. M. (2004) · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,120 · see citing works
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1014 · Cited by 1,981 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · see citing works
Self-citationLarge-Scale Evolution of Image ClassifiersPreprint
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 1,372 · see citing works
Self-citationThe Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
Neural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · doi:10.3115/v1/p15-1152 · Cited by 1,009 · see citing works
A Persona-Based Neural Conversation Model
Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., & Dolan, B. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1094 · Cited by 898 · see citing works
Adversarial Learning for Neural Dialogue Generation
Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., & Jurafsky, D. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1230 · Cited by 880 · see citing works
Levesque, H. J., Davis, E., & Morgenstern, L. (2011) · Cited by 879 · see citing works
Hill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856 · see citing works
Computing Krippendorff's Alpha-Reliability
Krippendorff, K. (2011) · ScholarlyCommons (University of Pennsylvania) · Cited by 809 · see citing works
The design and implementation of Xiaoice, an empathetic social chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2020) · Computational Linguistics · doi:10.1162/coli_a_00368 · Cited by 777 · see citing works
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J., Gao, J., & Dolan, B. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1020 · Cited by 768 · see citing works
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works
Zhao, T., Zhao, R., & Eskenazi, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1061 · Cited by 736 · see citing works
Addressing the Rare Word Problem in Neural Machine Translation
Luong, M.-T., Sutskever, I., Le, Q. V., Vinyals, O., & Zaremba, W. (2015) · doi:10.3115/v1/p15-1002 · Cited by 626 · see citing works
Self-citationTopic Aware Neural Response Generation
Chen, X., Wu, W., Wu, Y., Liu, J., Huang, Y., Zhou, M., & Ma, W. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10981 · Cited by 448 · see citing works
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational AgentsPreprint
Wolf, T., Sanh, V., Chaumond, J., & Delangue, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08149 · Cited by 435 · see citing works
Why We Need New Evaluation Metrics for NLG
Novikova, J., Dušek, O., Cercas Curry, A., & Rieser, V. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1238 · Cited by 422 · see citing works
Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses
Lowe, R., Noseworthy, M., Serban, I. V., Angelard-Gontier, N., Bengio, Y., & Pineau, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1103 · Cited by 341 · see citing works
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328 · see citing works
Self-citationSo, D. R., Liang, C., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1901.11117 · Cited by 277 · see citing works
Self-citationGenerating Informative and Diverse Conversational Responses via Adversarial Information MaximizationPreprint
Zhang, Y., Galley, M., Gao, J., Gan, Z., Li, X., Brockett, C., & Dolan, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05972 · Cited by 250 · see citing works
What makes a good conversation? How controllable attributes affect human judgments
See, A., Roller, S., Kiela, D., & Weston, J. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1170 · Cited by 247 · see citing works
RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems
Tao, C., Mou, L., Zhao, D., & Yan, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11321 · Cited by 213 · see citing works
A Deep Reinforcement Learning ChatbotPreprint
Serban, I. V., Sankar, C., Germain, M., Zhang, S., Lin, Z., Subramanian, S., Kim, T., Pieper, M., Chandar, S., Ke, N. R., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02349 · Cited by 195 · see citing works
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models
Shao, Y., Gouws, S., Britz, D., Goldie, A., Strope, B. P., & Kurzweil, R. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1235 · Cited by 179 · see citing works
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · arXiv · doi:10.48550/arxiv.1801.07243 · Cited by 165 · see citing works
Unifying Human and Statistical Evaluation for Natural Language Generation
Hashimoto, T. B., Zhang, H., & Liang, P. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1169 · Cited by 161 · see citing works
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 124 · see citing works
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-EncoderPreprint
Gu, X., Cho, K., Ha, J.-W., & Kim, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12352 · Cited by 116 · see citing works
The price of debiasing automatic metrics in natural language evalaution
Chaganty, A., Mussmann, S., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1060 · Cited by 96 · see citing works
Comparison of Diverse Decoding Methods from Conditional Language Models
Ippolito, D., Kriz, R., Sedoc, J., Kustikova, M., & Callison-Burch, C. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1365 · Cited by 95 · see citing works
Jointly Optimizing Diversity and Relevance in Neural Response Generation
Gao, X., Lee, S., Zhang, Y., Brockett, C., Galley, M., Gao, J., & Dolan, B. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1125 · Cited by 85 · see citing works
Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings
Ghazarian, S., Wei, J. T.-Z., Galstyan, A., & Peng, N. (2019) · doi:10.18653/v1/w19-2310 · Cited by 84 · see citing works
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog SystemsPreprint
Ghandeharioun, A., Shen, J. H., Jaques, N., Ferguson, C., Jones, N. C., Lapedriza, À., & Picard, R. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09308 · Cited by 77 · see citing works
Learning to Control the Specificity in Neural Response Generation
Zhang, R., Guo, J., Fan, Y., Lan, Y., Xu, J., & Cheng, X. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1102 · Cited by 74 · see citing works
Steering Output Style and Topic in Neural Response Generation
Wang, D., Jojić, N., Brockett, C., & Nyberg, E. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1228 · Cited by 64 · see citing works
The price of debiasing automatic metrics in natural language evaluationPreprint
Chaganty, A. T., Mussmann, S., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02202 · Cited by 36 · see citing works
I Know the Feeling: Learning to Converse with EmpathyPreprint
Rashkin, H., Smith, E. M., Li, M., & Boureau, Y.-L. (2018) · arXiv (Cornell University) · Cited by 30 · see citing works
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation
Ko, W.-J., Durrett, G., & Li, J. J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1349 · Cited by 21 · see citing works
Xu, Z. X., Jiang, N., Liu, B., Rong, W., Wu, B., Wang, B., Wang, Z., & Wang, X. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1188 · Cited by 11 · see citing works
Human and Automatic Detection of Generated Text.Preprint
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2019) · arXiv (Cornell University) · Cited by 6 · see citing works
Flagged first, then by citations
4 of the 56 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR .
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog , 1(8):9.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. ACL .
- Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive Learning Rates with Sublinear Memory Cost. ICML .
- Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit. 2018. Tensor2tensor for neural machine translation. CoRR , abs/1803.07416. · arXiv
- Anu Venkatesh, Chandra Khatri, Ashwin Ram, Fenfei Guo, Raefer Gabriel, Ashish Nagar, Rohit Prasad, Ming Cheng, Behnam Hedayatnia, Angeliki Metallinou, Rahul Goel, Shaohua Yang, and Anirudh Raju. 2018. On evaluating and comparing conversational agents. CoRR , abs/1801.03625. · arXiv
- Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019. Dialogpt: Largescale generative pre-training for conversational response generation. CoRR , abs/1911.00536. · arXiv
Cited by
Sorted by citations
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Self-citationPre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
“In addition to sampling, re-ranking is also a common approach to improving generation quality in language models (Adiwardana et al., 2020; Shen et al., 2021).”
BackgroundMethodInfluentialSelf-citationFrom ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
Gupta, M., Akiri, C., Aryal, K., Parker, E., & Praharaj, L. (2023) · IEEE Access · doi:10.1109/access.2023.3300381 · Cited by 714 · see citing works
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784 · see citing works
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783 · see citing works
AraBERT: Transformer-based Model for Arabic Language UnderstandingPreprint
Antoun, W., Baly, F., & Hajj, H. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00104 · Cited by 1,032 · see citing works
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · see citing works
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
“Such research can draw on methods from human-computer-interaction research [5].”
BackgroundMethodRaschka, S., Patterson, J. M., & Nolet, C. (2020) · Information · doi:10.3390/info11040193 · Cited by 777 · see citing works
Bengio, Y., LeCun, Y., & Hinton, G. E. (2021) · Communications of the ACM · doi:10.1145/3448250 · Cited by 676 · see citing works
Yenduri, G., Ramalingam, M., Selvi, G. C., Supriya, Y., Srivastava, G., Maddikunta, P. K. R., Raj, G. D., Jhaveri, R. H., Prabadevi, B., Wang, W., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3389497 · Cited by 759 · see citing works
Retrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575 · see citing works
Green Algorithms: Quantifying the Carbon Footprint of Computation
Lannelongue, L., Grealey, J., & Inouye, M. (2021) · Advanced Science · doi:10.1002/advs.202100707 · Cited by 604 · see citing works
Wu, T. S., Terry, M., & Cai, C. (2021) · International Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 757 · see citing works
PTR: Prompt Tuning with Rules for Text Classification
Han, X., Zhao, W., Ding, N., Liu, Z., & Sun, M. (2022) · AI Open · doi:10.1016/j.aiopen.2022.11.003 · Cited by 503 · see citing works
PTR: Prompt Tuning with Rules for Text Classification
Han, X., Zhao, W., Ding, N., Liu, Z., & Sun, M. (2021) · arXiv · doi:10.48550/arxiv.2105.11259 · Cited by 478 · see citing works
“, 2019; Liu and Lapata, 2019), question answering (Adiwardana et al., 2020), to”
BackgroundA Literature Survey of Recent Advances in Chatbots
Caldarini, G., Jaf, S., & McGarry, K. (2022) · Information · doi:10.3390/info13010041 · Cited by 470 · see citing works
Pre-Trained Language Models and Their Applications
Wang, H., Li, J., Wu, H. R., Hovy, E. H., & Sun, Y. (2022) · Engineering · doi:10.1016/j.eng.2022.04.024 · Cited by 404 · see citing works
A Simple Language Model for Task-Oriented DialoguePreprint
Hosseini-Asl, E., McCann, B., Wu, C.-S., Yavuz, S., & Socher, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00796 · Cited by 368 · see citing works
Challenges in building intelligent open-domain dialog systems
Huang, M., Zhu, X., & Gao, J. (2020) · ACM Transactions on Information Systems · doi:10.1145/3383123 · Cited by 288 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2210.03350 · Cited by 294 · see citing works
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J. Y., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 325 · see citing works
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289 · see citing works
Blockchain and Artificial Intelligence Technology in E-Health
Tagde, P., Tagde, S., Bhattacharya, T., Tagde, P., Chopra, H., Akter, R., Kaushik, D., & Rahman, M. H. (2021) · Environmental Science and Pollution Research · doi:10.1007/s11356-021-16223-0 · Cited by 281 · see citing works
Ross, S., Martinez, F., Houde, S., Muller, M. J., & Weisz, J. D. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584037 · Cited by 272 · see citing works
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N., & Lewis, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.378 · Cited by 303 · see citing works
Chatbots: Security, privacy, data protection, and social aspects
Hasal, M., Nowaková, J., Ahmed Saghair, K., Abdulla, H., Snášel, V., & Ogiela, L. (2021) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.6426 · Cited by 249 · see citing works
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · see citing works
Trust me, I'm a bot – repercussions of chatbot disclosure in different service frontline settings
Mozafari, N., Weiger, W. H., & Hammerschmidt, M. (2021) · Journal of service management · doi:10.1108/josm-10-2020-0380 · Cited by 246 · see citing works
Haugeland, I. K. F., Følstad, A., Taylor, C., & Bjørkli, C. A. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102788 · Cited by 235 · see citing works
Dalalah, D., & Dalalah, O. M. A. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100822 · Cited by 218 · see citing works
Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI
Abbasian, M., Khatibi, E., Azimi, I., Oniani, D., Shakeri Hossein Abad, Z., Thieme, A., Sriram, R., Yang, Z., Wang, Y., Lin, B., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01074-z · Cited by 213 · see citing works
Self-citationInternet-Augmented Dialogue Generation
Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192 · see citing works
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.356 · Cited by 188 · see citing works
Asatiani, A., Malo, P., Nagbøl, P. R., Penttinen, E., Rinta-Kahila, T., & Salovaara, A. (2021) · Journal of the AIS · doi:10.17705/1jais.00664 · Cited by 172 · see citing works
Few-shot Natural Language Generation for Task-Oriented Dialog
Peng, B., Zhu, C., Li, C., Li, X., Li, J., Zeng, M., & Gao, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.17 · Cited by 162 · see citing works
Sparks: Inspiration for Science Writing using Language Models
Gero, K. I., Liu, V., & Chilton, L. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533533 · Cited by 161 · see citing works
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models
Zhao, X., Wu, W., Xu, C., Tao, C., Zhao, D., & Yan, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.272 · Cited by 168 · see citing works
Time-Aware Language Models as Temporal Knowledge Bases
Dhingra, B., Cole, J. R., Eisenschlos, J. M., Gillick, D., Eisenstein, J., & Cohen, W. W. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00459 · Cited by 169 · see citing works
Hagos, D. H., Battle, R., & Rawat, D. B. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3444742 · Cited by 192 · see citing works
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
Lin, Z., Madotto, A., Winata, G. I., & Fung, P. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.273 · Cited by 158 · see citing works
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07567 · Cited by 148 · see citing works
Prompt-learning for Fine-grained Entity Typing
Ding, N., Chen, Y., Han, X., Xu, G., Wang, X., Xie, P., Zheng, H., Liu, Z., Li, J., & Kim, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.512 · Cited by 152 · see citing works
Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases
Weikum, G., Dong, X. L., Razniewski, S., & Suchanek, F. M. (2021) · Foundations and Trends in Databases · doi:10.1561/1900000064 · Cited by 140 · see citing works
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
Xu, C., Guo, D., Duan, N., & McAuley, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.385 · Cited by 191 · see citing works
Video Probabilistic Diffusion Models in Projected Latent Space
Yu, S., Sohn, K., Kim, S., & Shin, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01770 · Cited by 128 · see citing works
Wang, X., & Qiu, X. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2024.103777 · Cited by 118 · see citing works
He, W., Dai, Y., Zheng, Y., Wu, Y., Cao, Z., Liu, D., Jiang, P., Yang, M., Huang, F., Si, L., et al. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21320 · Cited by 115 · see citing works
Honovich, O., Choshen, L., Aharoni, R., Neeman, E., Szpektor, I., & Abend, O. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.619 · Cited by 113 · see citing works
“Generative conversational agents show remarkable progress lately (Shuster et al., 2020; Adiwardana et al., 2020).”
BackgroundForecasting extreme labor displacement: A survey of AI practitioners
Gruetzemacher, R., Paradice, D., & Lee, K. B. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120323 · Cited by 105 · see citing works
RealFormer: Transformer Likes Residual Attention
He, R., Ravula, A., Kanagal, B., & Ainslie, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.81 · Cited by 114 · see citing works
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R. J. L., & Choi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.82 · Cited by 117 · see citing works
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Laskar, M. T. R., Bari, M. S., Rahman, M., Bhuiyan, M. A. H., Joty, S., & Huang, J. X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.29 · Cited by 117 · see citing works
Recipes for Safety in Open-domain ChatbotsPreprint
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07079 · Cited by 130 · see citing works
SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine TeachingPreprint
Peng, B., Li, C., Li, J., Shayandeh, S., Lidén, L., & Gao, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05298 · Cited by 96 · see citing works
S oloist : BuildingTask Bots at Scale with Transfer Learning and Machine Teaching
Peng, B., Li, C., Li, J., Shayandeh, S., Lidén, L., & Gao, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00399 · Cited by 264 · see citing works
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
Bao, S., Huang, H., Wang, F., Wu, H. R., Wang, H., Wu, W., Guo, Z., Liu, Z., & Xu, X. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.222 · Cited by 105 · see citing works
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented DialoguePreprint
Mehri, S., Eric, M., & Hakkani-Tür, D. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13570 · Cited by 118 · see citing works
Pellert, M., Lechner, C. M., Wagner, C., Rammstedt, B., & Strohmaier, M. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231214460 · Cited by 97 · see citing works
Dialogue State Tracking with a Language Model using Schema-Driven Prompting
Lee, C., Cheng, H., & Mari Ostendorf, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.404 · Cited by 100 · see citing works
Why do people use ChatGPT? Exploring user motivations for generative conversational AI
Skjuve, M., Brandtzaeg, P. B., & Følstad, A. (2024) · First Monday · doi:10.5210/fm.v29i1.13541 · Cited by 95 · see citing works
SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive ModelPreprint
Peng, B., Li, C., Li, J., Shayandeh, S., Lidén, L., & Gao, J. (2020) · arXiv (Cornell University) · Cited by 86 · see citing works
Khennouche, F., Elmir, Y., Himeur, Y., Djebari, N., & Amira, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123224 · Cited by 91 · see citing works
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep NetworksPreprint
Ginsburg, B., Castonguay, P., Hrinchuk, O., Kuchaiev, O., Lavrukhin, V., Leary, R., Li, J., Nguyen, H., Zhang, Y., & Cohen, J. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11286 · Cited by 83 · see citing works
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84 · see citing works
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training
Zhang, Y., Wang, G., Li, C., Gan, Z., Brockett, C., & Dolan, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.698 · Cited by 94 · see citing works
Do Transformer Modifications Transfer Across Implementations and Applications?
Narang, S., Chung, H. W., Tay, Y., Fedus, L., Févry, T., Matena, M. S., Malkan, K., Fiedel, N., Shazeer, N., Lan, Z., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.465 · Cited by 79 · see citing works
Self-citationKim, B., Kim, H., Lee, S., Lee, G., Kwak, D., Hyeon, J. D., Park, S., Kim, S., Kim, S., Seo, D., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.274 · Cited by 97 · see citing works
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
Huang, L., Ye, Z., Qin, J., Lin, L., & Liang, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.742 · Cited by 81 · see citing works
Prompt-Learning for Short Text Classification
Zhu, Y. Q., Wang, Y., Qiang, J., & Wu, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3332787 · Cited by 83 · see citing works
“I’m sorry to hear that”: Finding new biases in language models with a holistic descriptor dataset
Smith, E. M., Hall, M., Kambadur, M., Presani, E., & Williams, A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.625 · Cited by 97 · see citing works
Dialogue Response Ranking Training with Large-Scale Human Feedback Data
Gao, X., Zhang, Y., Galley, M., Brockett, C., & Dolan, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.28 · Cited by 84 · see citing works
Chen, X., Meng, F., Li, P., Chen, F., Xu, S., Xu, B., & Zhou, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.275 · Cited by 75 · see citing works
Irfan, B., Kuoppamäki, S., Hosseini, A., & Skantze, G. (2025) · Autonomous Robots · doi:10.1007/s10514-025-10190-y · Cited by 75 · see citing works
Meta-learning via Language Model In-context Tuning
Chen, Y., Zhong, R., Zha, S., Karypis, G., & He, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.53 · Cited by 84 · see citing works
The Power of Personalization: A Systematic Review of Personality-Adaptive Chatbots
Baha, T. A., Hajji, M. E., Es-Saady, Y., & Fadili, H. (2023) · SN Computer Science · doi:10.1007/s42979-023-02092-6 · Cited by 75 · see citing works
Reducing Conversational Agents’ Overconfidence Through Linguistic Calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00494 · Cited by 72 · see citing works
Shuster, K., Komeili, M., Adolphs, L., Roller, S., Szlam, A. D., & Weston, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.27 · Cited by 78 · see citing works
Reducing conversational agents' overconfidence through linguistic calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2020) · arXiv · doi:10.48550/arxiv.2012.14983 · Cited by 72 · see citing works
Generative Artificial Intelligence for Software Engineering - a Research Agenda
NguyenDuc, A., Cabrero-Daniel, B., Arora, C., Przybylek, A., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K.-K., et al. (2023) · doi:10.2139/ssrn.4622517 · Cited by 71 · see citing works
DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances
Gu, X., Yoo, K. M., & Ha, J.-W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17527 · Cited by 70 · see citing works
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions
Aliannejadi, M., Kiseleva, J., Chuklin, A., Dalton, J., & Burtsev, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.367 · Cited by 77 · see citing works
Contrastive Learning Reduces Hallucination in Conversations
Sun, W., Shi, Z., Gao, S., Ren, P., De Rijke, M., & Ren, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26596 · Cited by 69 · see citing works
Bot-Adversarial Dialogue for Safe Conversational Agents
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.235 · Cited by 64 · see citing works
User reactions to COVID-19 screening chatbots from reputable providers
Dennis, A. R., Kim, A., Raimi, R., & Ayabakan, S. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa167 · Cited by 59 · see citing works
A Comprehensive Assessment of Dialog Evaluation Metrics
Yeh, Y.-T., Eskénazi, M., & Mehri, S. (2021) · doi:10.18653/v1/2021.eancs-1.3 · Cited by 71 · see citing works
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00486 · Cited by 71 · see citing works
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
Shakeri, S., dos Santos, C. N., Zhu, H., Ng, P., Nan, F., Wang, Z., Nallapati, R. M., & Xiang, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.439 · Cited by 74 · see citing works
TIMEDIAL: Temporal Commonsense Reasoning in Dialog
Qin, L., Gupta, A., Upadhyay, S., He, L., Choi, Y., & Faruqui, M. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.549 · Cited by 59 · see citing works
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
Ziems, C., Yu, J., Wang, Y., Halevy, A. Y., & Yang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.261 · Cited by 73 · see citing works
The Emerging Science of Interacting Minds
Wheatley, T. P., Thornton, M. A., Stolk, A., & Chang, L. J. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231200177 · Cited by 59 · see citing works
EmoSen: Generating Sentiment and Emotion Controlled Responses in a Multimodal Dialogue System
Firdaus, M., Chauhan, H., Ekbal, A., & Bhattacharyya, P. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3015491 · Cited by 59 · see citing works
JAICOB: A Data Science Chatbot
Carlander-Reuterfelt, D., Carrera, Á., Iglesias, C. Á., Araque, Ó., Sánchez-Rada, J. F., & Muñoz, S. (2020) · IEEE Access · doi:10.1109/access.2020.3024795 · Cited by 58 · see citing works
Generative Spoken Dialogue Language Modeling
Nguyen, T. A., Kharitonov, E., Copet, J., Adi, Y., Hsu, W.-N., Elkahky, A. M., Tomasello, P., Algayres, R., Sagot, B., Mohamed, A., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00545 · Cited by 65 · see citing works
A survey on near-human conversational agents
Singh, S., & Beniwal, H. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.10.013 · Cited by 63 · see citing works
Deng, Y., Liao, L., Chen, L., Wang, H., Lei, W., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.711 · Cited by 77 · see citing works
Fusing Task-Oriented and Open-Domain Dialogues in Conversational Agents
Young, T., Xing, F., Pandelea, V., Ni, J., & Cambria, E. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21416 · Cited by 54 · see citing works
- Authors
- Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., & Le, Q. V.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2001.09977
- OpenAlex
- W3000779003
- Cited by
- 614
- References
- 56
- Citations by year
- 2026: 6 · 2025: 21 · 2024: 66 · 2023: 137 · 2022: 145 · 2021: 154 · 2020: 82 · 2019: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- open-domain chatbots, multi-turn dialogue, end-to-end learning, social media discourse, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review