CoChat
Sign Up
PreprintCited by 7

The DOI matches its DataCite record.

ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models

Bian, Ning, Han, Xianpei, Sun, Le, Lin, Hongyu, Lu, Yaojie, He, Ben, Jiang, Shanshan, Dong, Bin·2023·arXiv
DOI 10.48550/arxiv.2303.16421arXiv 2303.16421
Open this paper in CoChat — free account

Abstract

Large language models (LLMs) have made significant progress in NLP. However, their ability to memorize, represent, and leverage commonsense knowledge has been a well-known pain point. In this paper, we specifically focus on ChatGPT, a widely used and easily accessible LLM, and ask the following questions: (1) Can ChatGPT effectively answer commonsense questions? (2) Is ChatGPT aware of the underlying commonsense knowledge for answering a specific question? (3) Is ChatGPT knowledgeable in commonsense? (4) Can ChatGPT effectively leverage commonsense for answering questions? We conduct a series of experiments on 11 datasets to evaluate ChatGPT's commonsense abilities, including answering commonsense questions, identifying necessary knowledge, generating knowledge descriptions, and using knowledge descriptions to answer questions again. Experimental results show that: (1) ChatGPT can achieve good QA accuracies in commonsense tasks, while still struggling with certain domains of datasets. (2) ChatGPT is knowledgeable, and can accurately generate most of the commonsense knowledge using knowledge prompts. (3) Despite its knowledge, ChatGPT is an inexperienced commonsense problem solver, which cannot precisely identify the needed commonsense for answering a specific question. These findings raise the need to explore improved mechanisms for effectively incorporating commonsense into LLMs like ChatGPT, such as better instruction following and commonsense guidance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2303.16421

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 58

  1. Shourya Aggarwal, Divyanshu Mandowara, Vishwa- jeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. 2021. Explanations for Common- senseQA: New Dataset and Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3050-3065, Online. Association for Computational Linguistics. · DOI
  2. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. ArXiv preprint, abs/2302.04023. · arXiv
  3. Lisa Bauer, Yicheng Wang, and Mohit Bansal. 2018. Commonsense for generative multi-hop question an- swering tasks. In Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Processing, pages 4220-4230, Brussels, Belgium. Association for Computational Linguistics. · DOI
  4. David Bender. 2015. Establishing a human baseline for the winograd schema challenge. In MAICS, pages 39-45.
  5. Prajjwal Bhargava and Vincent Ng. 2022. Common- sense knowledge reasoning and generation with pre- trained language models: A survey. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Ar- tificial Intelligence, EAAI 2022 Virtual Event, Febru- ary 22 -March 1, 2022, pages 12317-12325. AAAI Press.
  6. Yonatan Bisk, Rowan Zellers, Ronan LeBras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: reasoning about physical commonsense in natural language. In The Thirty-Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7432- 7439. AAAI Press.
  7. Michael Boratko, Xiang Li, Tim O'Gorman, Rajarshi Das, Dan Le, and Andrew McCallum. 2020. Pro- toQA: A question answering dataset for prototypi- cal common-sense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1122-1136, Online. Association for Computational Linguistics. · DOI
  8. Ronald J. Brachman and Hector J. Levesque. 2022. To- ward a new science of common sense. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Ar- tificial Intelligence, EAAI 2022 Virtual Event, Febru- ary 22 -March 1, 2022, pages 12245-12249. AAAI Press.
  9. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
  10. Tyler A Chang and Benjamin K Bergen. 2023. Lan- guage model behavior: A comprehensive survey. ArXiv preprint, abs/2303.11504. · arXiv
  11. Qianglong Chen, Guohai Xu, Ming Yan, Ji Zhang, Fei Huang, Luo Si, and Yin Zhang. 2023. Distinguish before answer: Generating contrastive explanation as knowledge for commonsense question answering. ArXiv preprint, abs/2305.08135. · arXiv
  12. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question an- swering? try arc, the ai2 reasoning challenge. ArXiv preprint, abs/1803.05457. · arXiv
  13. Anthony G Cohn and Jose Hernandez-Orallo. 2023. Di- alectical language model evaluation: An initial ap- praisal of the commonsense spatial reasoning abilities of llms. ArXiv preprint, abs/2304.11164. · arXiv
  14. Leyang Cui, Sijie Cheng, Yu Wu, and Yue Zhang. 2021. On commonsense cues in BERT for solving common- sense tasks. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 683-693, Online. Association for Computational Lin- guistics. · DOI
  15. Wanyun Cui and Xingran Chen. 2023. Free lunch for ef- ficient textual commonsense integration in language models. ArXiv preprint, abs/2305.15516. · arXiv
  16. Ernest Davis. 2023. Benchmarks for automated com- monsense reasoning: A survey. ArXiv preprint, abs/2302.04752. · arXiv
  17. Joe Davison, Joshua Feldman, and Alexander Rush. 2019. Commonsense knowledge mining from pre- trained models. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1173-1178, Hong Kong, China. Association for Computational Linguistics. · DOI · Paper page
  18. Jonathan Gordon and Benjamin Van Durme. 2013. Re- porting bias and knowledge acquisition. In Proceed- ings of the 2013 workshop on Automated knowledge base construction, pages 25-30.
  19. Yuling Gu, Bhavana Dalvi Mishra, and Peter Clark. 2022. Do language models have coherent men- tal models of everyday things? ArXiv preprint, abs/2212.10029. · arXiv
  20. Jie He, Víctor Gutiérrez-Basulto, Jeff Z Pan, et al. 2023. Buca: A binary classification approach to un- supervised commonsense question answering. ArXiv preprint, abs/2305.15932. · arXiv
  21. Liwei Jiang, Antoine Bosselut, Chandra Bhagavatula, and Yejin Choi. 2021. "I'm not mad": Commonsense implications of negation and contradiction. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4380-4397, Online. Association for Computational Linguistics. · DOI
  22. Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. QASC: A dataset for question answering via sentence compo- sition. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Con- ference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 8082-8090. AAAI Press.
  23. Tassilo Klein and Moin Nabi. 2021. Towards zero- shot commonsense reasoning with self-supervised refinement of language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8737-8743, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI
  24. Kazushi Kondo, Saku Sugawara, and Akiko Aizawa. 2023. Commonsense knowledge transfer for pre-trained language models. ArXiv preprint, abs/2306.02258. · arXiv
  25. Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, and Jimmy Xiangji Huang. 2023. A systematic study and comprehensive evaluation of chatgpt on bench- mark datasets. ArXiv preprint, abs/2305.18486. · arXiv
  26. Hector J Levesque, Ernest Davis, and Leora Morgen- stern. 2012. The winograd schema challenge. In Proceedings of the Thirteenth International Confer- ence on Principles of Knowledge Representation and Reasoning, pages 552-561.
  27. Xiang Lorraine Li, Adhiguna Kuncoro, Cyprien de Mas- son d'Autume, Phil Blunsom, and Aida Nematzadeh. 2021. Do language models learn commonsense knowledge? ArXiv preprint, abs/2111.00607. · arXiv
  28. Xiang Lorraine Li, Adhiguna Kuncoro, Jordan Hoff- mann, Cyprien de Masson d'Autume, Phil Blunsom, and Aida Nematzadeh. 2022. A systematic investiga- tion of commonsense knowledge in large language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11838-11855, Abu Dhabi, United Arab Emi- rates. Association for Computational Linguistics.
  29. Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. 2020. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre- Trained Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6862-6868, Online. Association for Computational Linguistics. · DOI
  30. Hugo Liu and Push Singh. 2004. Conceptnet-a practi- cal commonsense reasoning tool-kit. BT technology journal, 22:211-226.
  31. Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Pe- ter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022. Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 3154-3169, Dublin, Ireland. Association for Compu- tational Linguistics. · DOI
  32. Ye Liu, Yao Wan, Lifang He, Hao Peng, and Philip S. Yu. 2021. KG-BART: knowledge graph-augmented BART for generative commonsense reasoning. In Thirty-Fifth AAAI Conference on Artificial Intelli- gence, AAAI 2021, Thirty-Third Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 6418-6425. AAAI Press.
  33. Ye Liu, Tao Yang, Zeyu You, Wei Fan, and Philip S. Yu. 2020. Commonsense evidence generation and injec- tion in reading comprehension. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 61-73, 1st virtual meeting. Association for Computational Linguistics.
  34. Kaixin Ma, Filip Ilievski, Jonathan Francis, Satoru Ozaki, Eric Nyberg, and Alessandro Oltramari. 2021. Exploring strategies for generalizable commonsense reasoning with pre-trained models. In Proceedings of the 2021 Conference on Empirical Methods in Nat- ural Language Processing, pages 5474-5483, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI
  35. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct elec- tricity? a new dataset for open book question an- swering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381-2391, Brussels, Belgium. Association for Computational Linguistics. · DOI
  36. Yilin Niu, Fei Huang, Jiaming Liang, Wenkai Chen, Xiaoyan Zhu, and Minlie Huang. 2021. A semantic- based method for unsupervised commonsense ques- tion answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3037-3049, Online. Association for Computational Linguistics. · DOI
  37. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow in- structions with human feedback. ArXiv preprint, abs/2203.02155. · arXiv
  38. Ian Porada, Alessandro Sordoni, and Jackie Cheung. 2022. Does pre-training induce systematic inference? how masked language models acquire commonsense knowledge. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 4550-4557, Seattle, United States. Association for Computational Linguistics. · DOI
  39. Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics. · DOI
  40. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Com- monsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 4463- 4473, Hong Kong, China. Association for Computa- tional Linguistics. · DOI
  41. Vered Shwartz and Yejin Choi. 2020. Do neural lan- guage models overcome reporting bias? In Proceed- ings of the 28th International Conference on Com- putational Linguistics, pages 6863-6870, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics. · DOI
  42. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4615-4629, Online. Association for Computa- tional Linguistics. · DOI
  43. Yueqing Sun, Yu Zhang, Le Qi, and Qi Shi. 2022. TSGP: Two-stage generative prompting for unsuper- vised commonsense question answering. In Find- ings of the Association for Computational Linguistics: EMNLP 2022, pages 968-980, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
  44. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4149-4158, Minneapolis, Minnesota. Association for Computational Linguistics. · DOI
  45. Alexandre Tamborrino, Nicola Pellicanò, Baptiste Pan- nier, Pascal Voitot, and Louise Naudin. 2020. Pre- training is (almost) all you need: An application to commonsense reasoning. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3878-3887, Online. Association for Computational Linguistics. · DOI
  46. Xiaojuan Tang, Zilong Zheng, Jiaqi Li, Fanxu Meng, Song-Chun Zhu, Yitao Liang, and Muhan Zhang. 2023. Large language models are in-context seman- tic reasoners rather than symbolic reasoners. ArXiv preprint, abs/2305.14825. · arXiv
  47. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applica- tions. ArXiv preprint, abs/2201.08239. · arXiv
  48. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. ArXiv preprint, abs/2302.13971. · arXiv
  49. Peifeng Wang, Filip Ilievski, Muhao Chen, and Xiang Ren. 2021. Do language models perform gener- alizable commonsense inference? In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 3681-3688, Online. Association for Computational Linguistics. · DOI
  50. Peifeng Wang, Nanyun Peng, Filip Ilievski, Pedro Szekely, and Xiang Ren. 2020. Connecting the dots: A knowledgeable path generator for commonsense question answering. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4129-4140, Online. Association for Computational Linguistics. · DOI
  51. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.11903. · arXiv
  52. Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language mod- els to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 4602-4625, Seat- tle, United States. Association for Computational Linguistics. · DOI
  53. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a ma- chine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 4791-4800, Florence, Italy. Association for Computational Linguistics. · DOI · Paper page
  54. Yi Zhang, Lei Li, Yunfang Wu, Qi Su, and Xu Sun. 2022. Alleviating the knowledge-language inconsis- tency: A study for deep commonsense knowledge. IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, 30:594-604.
  55. Zirui Zhao, Wee Sun Lee, and David Hsu. 2023. Large language models as commonsense knowl- edge for large-scale task planning. ArXiv preprint, abs/2305.14078. · arXiv
  56. Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019. "going on a vacation" takes longer than "go- ing for a walk": A study of temporal commonsense understanding. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3363-3369, Hong Kong, China. Association for Computational Linguistics. · DOI · Paper page
  57. Wangchunshu Zhou, Ronan Le Bras, and Yejin Choi. 2023. Commonsense knowledge transfer for pre-trained language models. ArXiv preprint, abs/2306.02388. · arXiv
  58. Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang. 2020. Evaluating commonsense in pre- trained language models. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial In- telligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9733-9740. AAAI Press.

Cited by

Sorted by citations

Metadata

Authors
Bian, N., Han, X., Sun, L., Lin, H., Lu, Y., He, B., Jiang, S., & Dong, B.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2303.16421
arXiv
2303.16421
References
0
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review