CoChat
Sign Up
PreprintCited by 4Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models

Open access
Shang, Yu, Li, Yu, Xu, Fengli, Li, Yong·2024·arXiv
DOI 10.48550/arxiv.2402.02563arXiv 2402.02563
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have shown impressive emergent abilities in a wide range of tasks, but the associated expensive API cost greatly limits the real application. Previous works like chain-of-thought (CoT) and tree-of-thoughts (ToT) have predominately focused on enhancing accuracy, but overlook the rapidly increasing API cost, which could be particularly problematic for open-ended real-world tasks with huge solution spaces. Motivated by the dual process theory of human cognition, we propose "Synergy of Thoughts"(SoT) to unleash the synergistic potential of hybrid LLMs with different scales for efficient reasoning. By default, SoT uses smaller-scale language models to generate multiple low-cost intuitive thoughts, which resembles the parallel intuitions produced by System 1. We then design a confidence evaluator where the intuitive thoughts are cross-evaluated and introduce a controllable threshold mechanism to decide their mutual conflict. If these intuitive thoughts exhibit conflicts, SoT will invoke the reflective reasoning of scaled-up language models to emulate the intervention of System 2, which will override the intuitive thoughts and rectify the reasoning results. This framework is model-agnostic and training-free, which can be flexibly implemented with various off-the-shelf LLMs. Experiments on six representative reasoning tasks show that SoT substantially reduces the API cost by 38.3%-75.1%, and simultaneously achieves state-of-the-art reasoning accuracy and solution diversity. Notably, the average token cost reduction on open-ended tasks reaches up to 69.1%.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.02563

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs have exhibited remarkable reasoning abilities under advanced prompting approaches such as CoT, ToT, and SoT, shaping the foundation of the intelligence of LLM agents.

    “LLMs have exhibited remarkable reasoning abilities under advanced prompting approaches such as CoT (Wei et al., 2022), ToT (Yao et al., 2024), and SoT (Shang et al., 2024), shaping the foundation of the intelligence of LLM agents.”

    From AgentSquare: Automatic LLM Agent Search in Modular Design Space (2024) · cites this paper as Shang et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “• TFLOPS : It reflects the computational complexity, which is estimated according to the number of parameters following [33].”
    Background

Flagged first, then by citations

Research works printed but not linked: 40

  1. Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  2. Anil, R.; Dai, A. M.; Firat, O.; Johnson, M.; Lepikhin, D.; Passos, A.; Shakeri, S.; Taropa, E.; Bailey, P.; Chen, Z.; et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403. · arXiv
  3. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Ad- vances in neural information processing systems, 33: 1877- 1901.
  4. Chan, C.-M.; Chen, W.; Su, Y.; Yu, J.; Xue, W.; Zhang, S.; Fu, J.; and Liu, Z. 2023. Chateval: Towards better llm- based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201. · arXiv
  5. Chen, G.; Dong, S.; Shu, Y.; Zhang, G.; Sesay, J.; Karls- son, B. F.; Fu, J.; and Shi, Y. 2023. AutoAgents: A Frame- work for Automatic Agent Generation. arXiv preprint arXiv:2309.17288. · arXiv
  6. Chen, J. C.-Y.; Saha, S.; and Bansal, M. 2023. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. arXiv preprint arXiv:2309.13007. · arXiv
  7. Chen, L.; Zaharia, M.; and Zou, J. 2023. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv preprint arXiv:2305.05176. · arXiv
  8. Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2023. Palm: Scaling language model- ing with pathways. Journal of Machine Learning Research, 24(240): 1-113.
  9. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. · arXiv
  10. Du, Y.; Li, S.; Torralba, A.; Tenenbaum, J. B.; and Mor- datch, I. 2023. Improving Factuality and Reasoning in Lan- guage Models through Multiagent Debate. arXiv preprint arXiv:2305.14325. · arXiv
  11. Evans, J. S. B. 2006. The heuristic-analytic theory of rea- soning: Extension and evaluation. Psychonomic bulletin & review, 13(3): 378-395.
  12. Evans, J. S. B. 2010. Intuition and reasoning: A dual-process perspective. Psychological Inquiry, 21(4): 313-326.
  13. Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. · arXiv
  14. Kahneman, D.; Frederick, S.; et al. 2002. Representative- ness revisited: Attribute substitution in intuitive judgment. Heuristics and biases: The psychology of intuitive judgment, 49(49-81): 74.
  15. Kirk, R.; Mediratta, I.; Nalmpantis, C.; Luketina, J.; Ham- bro, E.; Grefenstette, E.; and Raileanu, R. 2023. Under- standing the Effects of RLHF on LLM Generalisation and Diversity. arXiv preprint arXiv:2310.06452. · arXiv
  16. Liang, T.; He, Z.; Jiao, W.; Wang, X.; Wang, Y.; Wang, R.; Yang, Y.; Tu, Z.; and Shi, S. 2023. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. arXiv preprint arXiv:2305.19118. · arXiv
  17. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. 2023. Self-refine: Iterative refinement with self- feedback. arXiv preprint arXiv:2303.17651. · arXiv
  18. Padmakumar, V.; and He, H. 2023. Does Writing with Lan- guage Models Reduce Content Diversity? arXiv preprint arXiv:2309.05196. · arXiv
  19. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8): 9.
  20. Šakota, M.; Peyrard, M.; and West, R. 2023. Fly-swat or cannon? cost-effective language model choice via meta- modeling. arXiv preprint arXiv:2308.06077. · arXiv
  21. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K. R.; and Yao, S. 2023. Reflexion: Language agents with ver- bal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  22. Srivastava, A.; Rastogi, A.; Rao, A.; Shoeb, A. A. M.; Abid, A.; Fisch, A.; Brown, A. R.; Santoro, A.; Gupta, A.; Garriga- Alonso, A.; et al. 2022. Beyond the imitation game: Quanti- fying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. · arXiv
  23. Stanovich, K. 2011. Rationality and the reflective mind. Ox- ford University Press, USA.
  24. Sun, M.; Liu, Z.; Bair, A.; and Kolter, J. Z. 2023a. A Simple and Effective Pruning Approach for Large Language Mod- els. arXiv preprint arXiv:2306.11695. · arXiv
  25. Sun, Q.; Yin, Z.; Li, X.; Wu, Z.; Qiu, X.; and Kong, L. 2023b. Corex: Pushing the Boundaries of Complex Rea- soning through Multi-Model Collaboration. arXiv preprint arXiv:2310.00280. · arXiv
  26. Tao, C.; Hou, L.; Zhang, W.; Shang, L.; Jiang, X.; Liu, Q.; Luo, P.; and Wong, N. 2022. Compression of generative pre- trained language models via quantization. arXiv preprint arXiv:2203.10705. · arXiv
  27. Team, G.; Anil, R.; Borgeaud, S.; Wu, Y.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. · arXiv
  28. Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient founda- tion language models. arXiv preprint arXiv:2302.13971. · arXiv
  29. Wang, P.; Li, L.; Chen, L.; Zhu, D.; Lin, B.; Cao, Y.; Liu, Q.; Liu, T.; and Sui, Z. 2023a. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926. · arXiv
  30. Wang, X.; Wei, J.; Schuurmans, D.; Le, Q.; Chi, E.; Narang, S.; Chowdhery, A.; and Zhou, D. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. · arXiv
  31. Wang, Z.; Mao, S.; Wu, W.; Ge, T.; Wei, F.; and Ji, H. 2023b. Unleashing the Emergent Cognitive Syn- ergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration. arXiv preprint arXiv:2307.05300. · arXiv
  32. Wei, J.; Tay, Y.; Bommasani, R.; Raffel, C.; Zoph, B.; Borgeaud, S.; Yogatama, D.; Bosma, M.; Zhou, D.; Metzler, D.; et al. 2022a. Emergent abilities of large language mod- els. arXiv preprint arXiv:2206.07682. · arXiv
  33. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022b. Chain-of- thought prompting elicits reasoning in large language mod- els. Advances in Neural Information Processing Systems, 35: 24824-24837.
  34. Yao, S.; Yu, D.; Zhao, J.; Shafran, I.; Griffiths, T. L.; Cao, Y.; and Narasimhan, K. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. · arXiv
  35. Yin, Z.; Sun, Q.; Chang, C.; Guo, Q.; Dai, J.; Huang, X.-J.; and Qiu, X. 2023. Exchange-of-thought: Enhancing large language model capabilities through cross-model communi- cation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 15135-15153.
  36. Young, A.; Chen, B.; Li, C.; Huang, C.; Zhang, G.; Zhang, G.; Li, H.; Zhu, J.; Chen, J.; Chang, J.; et al. 2024. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652. · arXiv
  37. Yue, M.; Zhao, J.; Zhang, M.; Du, L.; and Yao, Z. 2023. Large language model cascades with mixture of thoughts representations for cost-efficient reasoning. arXiv preprint arXiv:2310.03094. · arXiv
  38. Zhang, Y.; Yang, J.; Yuan, Y.; and Yao, A. C.-C. 2023. Cumulative reasoning with large language models. arXiv preprint arXiv:2308.04371. · arXiv
  39. Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2023. Judg- ing LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685. · arXiv
  40. Zhou, D.; Schärli, N.; Hou, L.; Wei, J.; Scales, N.; Wang, X.; Schuurmans, D.; Cui, C.; Bousquet, O.; Le, Q. V.; et al. 2022. Least-to-Most Prompting Enables Complex Reason- ing in Large Language Models. In The Eleventh Interna- tional Conference on Learning Representations.

Cited by

Sorted by citations

Authors
Shang, Y., Li, Y., Xu, F., & Li, Y.
Affiliations
  • Tsinghua University (CN)
  • Harbin Institute of Technology (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.02563
arXiv
2402.02563
OpenAlex
W4391591205
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
efficient reasoning, dual-process theory, chain-of-thought, tree-of-thought, API cost reduction, confidence estimation, solution diversity, token cost

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review