Language Model Cascades
Open accessAbstract
Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs can be viewed as probabilistic production systems that sample a possible completion each time they are called.
“LLMs can thus be viewed as probabilistic production systems that sample a possible completion each time they are called, e.g., X ∼ ∼ ▸ XY .”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Dohan et al., 2022
Chaining multiple calls to an LLM allows for increasingly complicated algorithms.
“Chaining multiple calls to an LLM (Wu et al., 2022a;b; Dohan et al., 2022) allows for increasingly complicated algorithms (Fig. 3).”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Dohan et al., 2022
LLMs integrate naturally with the causal graph to solve STRICTA problems reconstructing human reasoning from partial information by turning the causal graph into an LLM program.
“In Section 5 we demonstrate that LLMs integrate naturally with the causal graph to solve STRICTA problems reconstructing human reasoning from partial information by turning the causal graph into an LLM program (Dohan et al., 2022).”
From STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond (2025) · cites this paper as Dohan et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
Cited by 22
Research works printed but not linked: 13
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., and Others. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165 , 2020.
- Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. 2021. doi: 10.48550/ARXIV. 2110.14168. URL https://arxiv.org/abs/2110.14168. · DOI
- Cranmer, K., Brehmer, J., and Louppe, G. The frontier of simulation-based inference. Proc. Natl. Acad. Sci. U. S. A. , 117(48):30055-30062, December 2020.
- ARC's first technical report: Eliciting latent knowledge -AI alignment forum, 2022. URL https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s-first-technical-report-eliciting-latent-knowledge. BIG-bench collaboration. Beyond the imitation game: Kochurov, M., Carroll, C., Wiecki, T., and Lao, J. PyMC4: Exploiting coroutines for implementing a probabilistic programming framework. In NeurIPS 2019 Workshop Program Transformations , 2019.
- Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. Human-level concept learning through probabilistic program induction. Science , 350(6266):1332-1338, December 2015.
- Levine, S. Reinforcement learning and control as probabilistic inference: Tutorial and review. May 2018.
- Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., Jiang, X., Cobbe, K., Eloundou, T., Krueger, G., Button, K., Knight, M., Chess, B., and Schulman, J. WebGPT: Browser-assisted question-answering with human feedback. December 2021.
- Ortega, P. A., Kunesch, M., Del´ etang, G., Genewein, T., Grau-Moya, J., Veness, J., Buchli, J., Degrave, J., Piot, B., Perolat, J., et al. Shaking the foundations: delusions in sequence models for interaction and control. arXiv preprint arXiv:2110.10819 , 2021. · arXiv
- Saunders, W., Yeh, C., Wu, J., Bills, S., Ouyang, L., Ward, J., and Leike, J. Self-critiquing models for assisting human evaluators, 2022. URL https://arxiv.org/abs/2206.05802. · arXiv
- Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., Li, Y., Lee, H., Zheng, H. S., Ghafouri, A., Menegali, M., Huang, Y., Krikun, M., Lepikhin, D., Qin, J., Chen, D., Xu, Y., Chen, Z., Roberts, A., Bosma, M., Zhao, V., Zhou, Y., Chang, C.-C., Krivokon, I., Rusch, W., Pickett, M., Srinivasan, P., Man, L., MeierHellstern, K., Morris, M. R., Doshi, T., Santos, R. D., Duke, T., Soraker, J., Zevenbergen, B., Prabhakaran, V., Diaz, M., Hutchinson, B., Olson, K., Molina, A., Hoffman-John, E., Lee, J., Aroyo, L., Rajakumar, R., Butryna, A., Lamm, M., Kuzmina, V., Fenton, J., Cohen, A., Bernstein, R., Kurzweil, R., Aguera-Arcas, B., Cui, C., Croak, M., Chi, E., and Le, Q. LaMDA: Language models for dialog applications. January 2022. URL https://arxiv.org/abs/2201.08239. · arXiv
- Tolpin, D., van de Meent, J., Yang, H., and Wood, F. D. Design and implementation of probabilistic programming language anglican. CoRR , abs/1608.05263, 2016. URL http://arxiv.org/abs/1608.05263. · arXiv
- Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., and Cai, C. J. Promptchainer: Chaining large language model prompts through visual programming, 2022. URL https://arxiv.org/abs/2203.06566. · arXiv
- Zelikman, E., Wu, Y., and Goodman, N. D. Star: Bootstrapping reasoning with reasoning. arXiv preprint arXiv:2203.14465 , 2022. · arXiv
- Authors
- Dohan, D., Xu, W., Lewkowycz, A., Austin, J., Bieber, D., Lopes, R. G., Wu, Y., Michalewski, H., Saurous, R. A., Sohl-dickstein, J., Murphy, K., & Sutton, C.
- Affiliations
- Google (United States) (US)
- Alphabet (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2207.10342
- OpenAlex
- W4320858367
- Cited by
- 22
- References
- 0
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 4 · 2023: 14 · 2022: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- probabilistic programming, probabilistic graphical models, chain-of-thought, Scratchpads, verifier, stars, selection inference, tool use, few-shot learning, control flow, inference methods
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
