CoChat
Sign Up
PreprintCited by 2Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

The DOI matches its DataCite record; 3 of the 3 references it cites were found in a registry or an index.

An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Models

Open access
Zhang, Yizhou, Du, Lun, Cao, Defu, Fu, Qiang, Liu, Yan·2024·arXiv
DOI 10.48550/arxiv.2402.05359arXiv 2402.05359
Open this paper in CoChat — free accountOpen PDF

Abstract

Foundation models, such as Large language Models (LLMs), have attracted significant amount of interest due to their large number of applications. However, when handling tasks involving repetitive sub-tasks and/or deceptive contents, such as arithmetic calculation and article-level fake news detection, simple instructional prompts suffer from inaccurate responses. Existing works show that more complicated prompting strategies, such as Chain-of-Thoughts and Least-to-Most, can unlock LLM's powerful capacity in diverse areas. Recent researches reveal that simple divide-and-conquer prompting strategy, i.e. simply dividing the input sequence to multiple sub-inputs, can also substantially improve LLM's performance in some specific tasks such as misinformation detection. In this paper, we aim at examining the utility of divide-and-conquer prompting strategy and answer on which kind of tasks this strategy gets advantages. Specifically, we provide a theoretic analysis to divide-and-conquer prompting strategy and help us identify the specific tasks where DaC prompting can bring performance boost with theoretic guarantee. We then present two cases (large integer arithmetic and fact verification) where experimental results aligns with our theoretic analysis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.05359

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ChatGPT performs better with 'chain of thoughts' and 'divide-and-conquer' prompting, which enhance LLMs' ability to handle complex tasks.

    “Additionally, ChatGPT performs better with ''chain of thoughts'' and ''divide-and-conquer'' prompting, techniques used in natural language processing and machine learning, which involve a series of intermediate reasoning steps, which has been observed to enhance LLMs' ability to handle complex tasks [30,31].”

    From Using artificial intelligence to semi-automate trustworthiness assessment of randomized controlled trials: a case study (2025) · cites this paper as [31]

    The Introduction (page unavailable) describes chain-of-thought prompting as using intermediate reasoning steps and says such strategies can improve LLM reasoning. Section 5.2.1 reports better performance with divide-and-conquer prompting on the tasks tested. But the paper does not support the unqualified claim that ChatGPT performs better with both methods: its integer-addition results say neither chain-of-thought nor divide-and-conquer significantly outperformed standard prompting. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
  2. Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. · arXiv
  3. Canyu Chen and Kai Shu. 2023. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788. · arXiv
  4. Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. 2023a. Hallucination detection: Robustly discerning reliable answers in large lan- guage models. In Proceedings of the 32nd ACM International Conference on Information and Knowl- edge Management, pages 245-255.
  5. Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al. 2023b. Exploring the potential of large language models (llms) in learning on graphs. arXiv preprint arXiv:2307.03393. · arXiv
  6. Vincent Cheng and Yu Zhang. 2023. Analyzing Chat- GPT's mathematical deficiencies: Insights and con- tributions. In Proceedings of the 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023), pages 188-193, Taipei City, Tai- wan. The Association for Computational Linguistics and Chinese Language Processing (ACLCLP).
  7. Corinna Cortes and Neil D Lawrence. 2021. In- consistency in conference peer review: revisit- ing the 2014 neurips experiment. arXiv preprint arXiv:2109.09774. · arXiv
  8. Antonia Creswell and Murray Shanahan. 2022. Faith- ful reasoning using large language models. arXiv preprint arXiv:2208.14271. · arXiv
  9. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lor- raine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, et al. 2024. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36.
  10. Guhao Feng, Yuntian Gu, Bohang Zhang, Haotian Ye, Di He, and Liwei Wang. 2023. Towards revealing the mystery behind chain of thought: a theoretical perspective. arXiv preprint arXiv:2305.15408. · arXiv
  11. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Gra- ham Neubig. 2023. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764-10799.
  12. PMLR.
  13. Jose A Garcia, Rosa Rodriguez-Sánchez, and J Fdez- Valdivia. 2021. Quality censoring in peer review. Scientometrics, 126:825-830.
  14. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at ma- chine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210. · arXiv
  15. Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2023. Bad ac- tor, good advisor: Exploring the role of large lan- guage models in fake news detection. arXiv preprint arXiv:2309.12247. · arXiv
  16. Birgit Jenner, Johannes Köbler, Pierre McKenzie, and Jacobo Torán. 2003. Completeness results for graph isomorphism. Journal of Computer and System Sci- ences, 66(3):549-566.
  17. Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brah- man, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. arXiv preprint arXiv:2205.11822. · arXiv
  18. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sab- harwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406. · arXiv
  19. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023a. Halueval: A large- scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 6449-6464.
  20. Siyu Li, Jin Yang, and Kui Zhao. 2023b. Are you in a masquerade? exploring the behavior and impact of large language model driven social bots in online social networks. arXiv preprint arXiv:2307.10337. · arXiv
  21. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023. Trust- worthy llms: a survey and guideline for evaluating large language models' alignment. arXiv preprint arXiv:2308.05374. · arXiv
  22. Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria. 2023. Gpteval: A survey on assessments of chatgpt and gpt-4. arXiv preprint arXiv:2308.12488. · arXiv
  23. William Merrill and Ashish Sabharwal. 2023a. The expresssive power of transformers with chain of thought. arXiv preprint arXiv:2310.07923. · arXiv
  24. William Merrill and Ashish Sabharwal. 2023b. The parallelism tradeoff: Limitations of log-precision transformers. Transactions of the Association for Computational Linguistics, 11:531-545. · DOI · Paper page
  25. Karishma Sharma, Sungyong Seo, Chuizheng Meng, Sirisha Rambhatla, and Yan Liu. 2020. Coronavirus on social media: Analyzing misinformation in twitter conversations. arXiv preprint arXiv:2003.12309. · arXiv
  26. Karishma Sharma, Yizhou Zhang, and Yan Liu. 2022. Covid-19 vaccine misinformation campaigns and so- cial media narratives. In Proceedings of the Interna- tional AAAI Conference on Web and Social Media, volume 16, pages 920-931.
  27. Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen, and Guilin Qi. 2023. Evalu- ation of chatgpt as a question answering system for answering complex questions. arXiv preprint arXiv:2303.07992. · arXiv
  28. Jonathan P Tennant and Tony Ross-Hellauer. 2020. The limitations to our understanding of peer review. Re- search integrity and peer review, 5(1):6.
  29. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534-7550, Online. As- sociation for Computational Linguistics. · DOI · Paper page
  30. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. · arXiv
  31. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837.
  32. Chenxi Whitehouse, Tillman Weyde, Pranava Mad- hyastha, and Nikos Komninos. 2022. Evaluation of fake news detection with knowledge-enhanced language models. In Proceedings of the Interna- tional AAAI Conference on Web and Social Media, volume 16, pages 1425-1429.
  33. Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Cheng Niu, Randy Zhong, Juntong Song, and Tong Zhang. 2023. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. arXiv preprint arXiv:2401.00396. · arXiv
  34. Chenghao Yang and Allyson Ettinger. 2023. Can you follow me? testing situational understanding for chat- gpt. In Proceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, pages 6385-6398.
  35. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. · arXiv
  36. Xuan Zhang and Wei Gao. 2023. Towards llm-based fact verification on news claims with a hierarchi- cal step-by-step prompting method. arXiv preprint arXiv:2310.00305. · arXiv
  37. Yizhou Zhang, Defu Cao, and Yan Liu. 2022. Counter- factual neural temporal point process for estimating causal influence of misinformation on social media. Advances in Neural Information Processing Systems, 35:10643-10655.
  38. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren's song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. · arXiv
  39. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. 2022. Least-to-most prompting enables complex reason- ing in large language models. arXiv preprint arXiv:2205.10625. · arXiv
  40. Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang. 2022. Solving math word problem via cooperative rea- soning induced language models. arXiv preprint arXiv:2210.16257. · arXiv
  41. Mingyu Zong and Bhaskar Krishnamachari. 2023. Solv- ing math word problems concerning systems of equa- tions with gpt-3. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 37, pages 15972-15979.

Cited by

Sorted by citations

Metadata

Authors
Zhang, Y., Du, L., Cao, D., Fu, Q., & Liu, Y.
Affiliations
  • University of Southern California (US)
  • Coupang (South Korea) (KR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.05359
arXiv
2402.05359
OpenAlex
W4391709455
Cited by
2
References
3
Citations by year
2025: 1 · 2024: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
large language models, prompt engineering, chain-of-thought prompting, least-to-most prompting, arithmetic calculation, fact-checking, misinformation detection, fake news detection, deceptive content

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review