Think before you speak: Training Language Models With Pause Tokens
Open accessAbstract
Language models generate responses by producing a series of tokens in immediate succession: the $(K+1)^{th}$ token is an outcome of manipulating $K$ hidden vectors per layer, one vector per preceding token. What if instead we were to let the model manipulate say, $K+10$ hidden vectors, before it outputs the $(K+1)^{th}$ token? We operationalize this idea by performing training and inference on language models with a (learnable) $\textit{pause}$ token, a sequence of which is appended to the input prefix. We then delay extracting the model's outputs until the last pause token is seen, thereby allowing the model to process extra computation before committing to an answer. We empirically evaluate $\textit{pause-training}$ on decoder-only models of 1B and 130M parameters with causal pretraining on C4, and on downstream tasks covering reasoning, question-answering, general understanding and fact recall. Our main finding is that inference-time delays show gains when the model is both pre-trained and finetuned with delays. For the 1B model, we witness gains on 8 of 9 tasks, most prominently, a gain of $18\%$ EM score on the QA task of SQuAD, $8\%$ on CommonSenseQA and $1\%$ accuracy on the reasoning task of GSM8k. Our work raises a range of conceptual and practical future research questions on making delayed next-token prediction a widely applicable new paradigm.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pause tokens mainly introduce parallel but not sequential compute, which is less powerful than adaptive depth.
“As discussed in Goyal et al. (2023), pause tokens mainly introduce parallel but not sequential compute, which is less powerful than adaptive depth.”
From Looped Transformers for Length Generalization (2024) · cites this paper as Goyal et al. (2023)
Section I.2 says pause tokens are processed in parallel and do not add sequential computation. The discussion of adaptive compute says some methods increase serial computation, but it does not establish that pause tokens are generally less powerful than adaptive depth; the paper also identifies tasks where parallel computation is advantageous. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2× · see citing works
“Consider a Transformer (Vaswani et al., 2017) block T ( · ) : R K × D → R K × D that operates over a sequence of K intermediate vectors.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · cited 6× · see citing works
“, 2019), <sep> or <mask> in BERT (Devlin et al., 2019) and in a line of work on adding memory to transformers (Burtsev et al.”
BackgroundInfluentialExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2× · see citing works
“For this, we study pause-training on a 1B and 130M parameter decoder-only model, trained on C4 (Raffel et al., 2020) and finetuned on nine downstream tasks spanning extractive question answering, reasoning, general understanding and fact recall.”
MethodSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 4× · see citing works
“We consider nine varied downstream tasks: (a) reasoning (GSM8k (Cobbe et al., 2021)), (b) ex-tractive question answering (SQuAD (Rajpurkar et al., 2016), CoQA (Reddy et al., 2019)), (c) general understanding (CommonSenseQA (Talmor et al., 2019), PhysicalIQA (Bisk et al., 2020)), (d) (Berant et al.,…”
BackgroundMethodInfluentialChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · cited 3× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 1× · see citing works
“Interestingly, on the face of it, pause-finetuning bears some resemblance to an orthogonal line of work on lightweight finetuning/ensembling techniques (Liu et al., 2022; Li & Liang, 2021; Lester et al., 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Logeswaran et al., 2020; Liu et al., 2021; Zhong et al., 2021…”
BackgroundCommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1421 · Cited by 2,805 · cited 2× · see citing works
“CommonSenseQA: Requires different of commonsense knowledge to choose the correct answer (Talmor et al., 2019).”
BackgroundPrefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · cited 1× · see citing works
“CoQA: Question-answering task based on a context (Reddy et al., 2019).”
BackgroundSemantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 2,318 · cited 2× · see citing works
“Web Questions: A fact-recall dataset of commonly-asked questions on the web (Berant et al., 2013).”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 3× · see citing works
“We consider nine varied downstream tasks: (a) reasoning (GSM8k (Cobbe et al., 2021)), (b) ex-tractive question answering (SQuAD (Rajpurkar et al., 2016), CoQA (Reddy et al., 2019)), (c) general understanding (CommonSenseQA (Talmor et al., 2019), PhysicalIQA (Bisk et al., 2020)), (d) (Berant et al., 2013), Natural Ques…”
BackgroundTree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158 · cited 1× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
Background(2020) · Journal of Failure Analysis and Prevention · doi:10.1007/s11668-020-00853-6 · Cited by 1,071
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · cited 4× · see citing works
“We consider nine varied downstream tasks: (a) reasoning (GSM8k (Cobbe et al., 2021)), (b) ex-tractive question answering (SQuAD (Rajpurkar et al., 2016), CoQA (Reddy et al., 2019)), (c) general understanding (CommonSenseQA (Talmor et al., 2019), PhysicalIQA (Bisk et al., 2020)), (d) (Berant et al., 2013), Natural Ques…”
BackgroundInfluentialHellaSwag: Can a Machine Really Finish Your Sentence?
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1472 · Cited by 830 · cited 1× · see citing works
“HellaSwag: Next-sentence prediction task based on common-sense inference (Zellers et al., 2019).”
BackgroundSelf-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · cited 3× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundIt’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
Schick, T., & Schütze, H. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.185 · Cited by 747 · cited 1× · see citing works
“CoQA: Question-answering task based on a context (Reddy et al., 2019).”
BackgroundTraining Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · cited 3× · see citing works
“We consider nine varied downstream tasks: (a) reasoning (GSM8k (Cobbe et al., 2021)), (b) ex-tractive question answering (SQuAD (Rajpurkar et al., 2016), CoQA (Reddy et al., 2019)), (c) general understanding (CommonSenseQA (Talmor et al., 2019), PhysicalIQA (Bisk et al., 2020)), (d) (Berant et al.,…”
BackgroundFine-tune BERT for Extractive Summarization
Liu, Y. (2019) · arXiv.org · doi:10.48550/arxiv.1903.10318 · Cited by 544 · cited 2×
“The idea of using tokens that occur only as an input has found its use in various forms, most commonly as <cls> (Chang et al., 2023; Liu, 2019; Devlin et al., 2019), <sep> or <mask> in BERT (Devlin et al.”
BackgroundTowards Understanding Chain-of-Thought Prompting: An Empirical Study of What MattersPreprint
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., & Sun, H. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2212.10001 · Cited by 428 · cited 3× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundP-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and TasksPreprint
Liu, X., Ji, K., Fu, Y., Tam, W. L., Du, Z., Yang, Z., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07602 · Cited by 386 · see citing works
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · cited 2× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundLearning How to Ask: Querying LMs with Mixtures of Soft Prompts
Qin, G., & Eisner, J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.410 · Cited by 342 · see citing works
Adaptive Computation Time for Recurrent Neural Networks
Graves, A. (2016) · arXiv · doi:10.48550/arxiv.1603.08983 · Cited by 335 · cited 1× · see citing works
“Graves (2017) explored a similar recurrence over hidden units in the context of RNNs.”
BackgroundImproving Neural Language Models with a Continuous CachePreprint
Grave, E., Joulin, A., & Usunier, N. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1612.04426 · Cited by 316 · cited 1× · see citing works
“Grave et al. (2016) use a memory cache as a way to attend to past hidden vectors.”
MethodFactual Probing Is [MASK]: Learning vs. Learning to Recall
Zhong, Z., Friedman, D., & Chen, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.398 · Cited by 274 · cited 1× · see citing works
“Interestingly, on the face of it, pause-finetuning bears some resemblance to an orthogonal line of work on lightweight finetuning/ensembling techniques (Liu et al., 2022; Li & Liang, 2021; Lester et al., 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Logeswaran et al., 2020; Liu et al., 2021; Zhong et al., 2021…”
BackgroundTurpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · cited 1× · see citing works
“Other empirical works (Turpin et al., 2023; Wang et al., 2023a; Madaan & Yazdanbakhsh, 2022) have shown that the generated intermediate reasoning steps can be unfaithful, not representing the model’s true reasoning process.”
BackgroundWARP: Word-level Adversarial ReProgramming
Hambardzumyan, K., Khachatrian, H., & May, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.381 · Cited by 243 · cited 1× · see citing works
“Interestingly, on the face of it, pause-finetuning bears some resemblance to an orthogonal line of work on lightweight finetuning/ensembling techniques (Liu et al., 2022; Li & Liang, 2021; Lester et al., 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Logeswaran et al., 2020; Liu et al., 2021; Zhong et al., 2021…”
BackgroundSelf-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 219 · see citing works
STaR: Bootstrapping Reasoning With Reasoning
Zelikman, E., Wu, Y., Mu, J., & Goodman, N. D. (2022) · arXiv · doi:10.48550/arxiv.2203.14465 · Cited by 182 · cited 1× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundShow Your Work: Scratchpads for Intermediate Computation with Language Models
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00114 · Cited by 144 · cited 2× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundThe Right Tool for the Job: Matching Model and Instance Complexities
Schwartz, R., Stanovsky, G., Swayamdipta, S., Dodge, J., & Smith, N. A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.593 · Cited by 136 · see citing works
Vision Transformers Need Registers
Darcet, T., Oquab, M., Mairal, J., & Bojanowski, P. (2023) · arXiv · doi:10.48550/arxiv.2309.16588 · Cited by 90 · cited 4× · see citing works
“…only as an input has found its use in various forms, most commonly as <cls> (Chang et al., 2023; Liu, 2019; Devlin et al., 2019), <sep> or <mask> in BERT (Devlin et al., 2019) and in a line of work on adding memory to transformers (Burtsev et al., 2020; Bulatov et al., 2022; Darcet et al., 2023).”
BackgroundInfluentialThe LAMBADA dataset: Word prediction requiring a broad discourse context
Paperno, D., Kruszewski, G., Lazaridou, A., Pham, N. Q., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., & Fernandez, R. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1144 · Cited by 87 · cited 2× · see citing works
“, 2020)), (d) long term context recall (LAMBADA (Paperno et al., 2016)), (e) natural language inference (HellaSwag (Zellers et al.”
BackgroundMethodWhen Does Confidence-Based Cascade Deferral Suffice?
Jitkrittum, W., Gupta, N., Menon, A., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2307.02764 · Cited by 79
Self-citationAugmenting Self-attention with Persistent Memory
Sukhbaatar, S., Grave, E., Lample, G., Jegou, H., & Joulin, A. (2019) · arXiv · doi:10.48550/arxiv.1907.01470 · Cited by 75 · cited 1× · see citing works
“Orthogonally, Sukhbaatar et al. (2019) introduce a large set of “global” memory tokens (of the order of ten million new parameters) across all layers.”
BackgroundTowards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.15408 · Cited by 69 · cited 2× · see citing works
“Feng et al. (2023); Merrill & Sabharwal (2023) theoretically argue that CoT aids by increasing the computational expressivity of the Trans-former.”
BackgroundInfluentialCRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 57 · see citing works
Bulatov, A., Kuratov, Y., & Burtsev, M. S. (2022) · arXiv · doi:10.48550/arxiv.2207.06881 · Cited by 55 · cited 1× · see citing works
“…only as an input has found its use in various forms, most commonly as <cls> (Chang et al., 2023; Liu, 2019; Devlin et al., 2019), <sep> or <mask> in BERT (Devlin et al., 2019) and in a line of work on adding memory to transformers (Burtsev et al., 2020; Bulatov et al., 2022; Darcet et al., 2023).”
BackgroundLearning to Reason and Memorize with Self-Notes
Lanchantin, J., Toshniwal, S., Weston, J., Szlam, A., & Sukhbaatar, S. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.00833 · Cited by 47 · cited 2×
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundChallenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.09261 · Cited by 44 · see citing works
PIQA: Reasoning about Physical Commonsense in Natural Language
Bisk, Y., Zellers, R., Le bras, R., Gao, J., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6239 · Cited by 37 · cited 6× · see citing works
“HellaSwag and PhysicalIQA are scoring tasks.”
BackgroundInfluentialMeasuring Faithfulness in Chain-of-Thought Reasoning
Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13702 · Cited by 33 · see citing works
Adaptive Computation with Elastic Input SequencePreprint
Xue, F., Likhosherstov, V., Arnab, A., et al. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2301.13195 · Cited by 31 · cited 3× · see citing works
“For vision transformers (ViTs), AdaTape (Xue et al., 2023) appends tokens from a learnt memory bank.”
BackgroundAn Embarrassingly Simple Model for Dialogue Relation Extraction
Xue, F., Sun, A., Zhang, H., Ni, J., & Chng, E. S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747486 · Cited by 25 · cited 1× · see citing works
“Interestingly, on the face of it, pause-finetuning bears some resemblance to an orthogonal line of work on lightweight finetuning/ensembling techniques (Liu et al., 2022; Li & Liang, 2021; Lester et al., 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Logeswaran et al., 2020; Liu et al., 2021; Zhong et al., 2021…”
BackgroundBanino, A., Balaguer, J., & Blundell, C. (2021) · arXiv · doi:10.48550/arxiv.2107.05407 · Cited by 24 · see citing works
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango
Madaan, A., & Yazdanbakhsh, A. (2022) · arXiv · doi:10.48550/arxiv.2209.07686 · Cited by 24 · cited 1× · see citing works
“Other empirical works (Turpin et al., 2023; Wang et al., 2023a; Madaan & Yazdanbakhsh, 2022) have shown that the generated intermediate reasoning steps can be unfaithful, not representing the model’s true reasoning process.”
BackgroundMulti-CLS BERT: An Efficient Alternative to Traditional Ensembling
Chang, H.-S., Sun, R.-Y., Ricci, K., et al. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2210.05043 · Cited by 21 · cited 3×
“The idea of using tokens that occur only as an input has found its use in various forms, most commonly as <cls> (Chang et al., 2023; Liu, 2019; Devlin et al., 2019), <sep> or <mask> in BERT (Devlin et al.”
BackgroundBurtsev, M. S., Kuratov, Y., Peganov, A., & Sapunov, G. V. (2020) · arXiv · doi:10.48550/arxiv.2006.11527 · Cited by 19 · cited 3× · see citing works
“…only as an input has found its use in various forms, most commonly as <cls> (Chang et al., 2023; Liu, 2019; Devlin et al., 2019), <sep> or <mask> in BERT (Devlin et al., 2019) and in a line of work on adding memory to transformers (Burtsev et al., 2020; Bulatov et al., 2022; Darcet et al., 2023).”
BackgroundRetroformer: Retrospective Large Language Agents with Policy Gradient Optimization
Yao, W., Heinecke, S., Niebles, J. C., Liu, Z., Feng, Y., Xue, L., Murthy, R., Chen, Z., Zhang, J., Arpit, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.02151 · Cited by 10 · cited 1× · see citing works
“CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
BackgroundFew-shot Sequence Learning with TransformersPreprint
Logeswaran, L., Lee, A., Ott, M., Lee, H., Ranzato, M., & Szlam, A. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09543 · Cited by 10 · cited 1× · see citing works
“Interestingly, on the face of it, pause-finetuning bears some resemblance to an orthogonal line of work on lightweight finetuning/ensembling techniques (Liu et al., 2022; Li & Liang, 2021; Lester et al., 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Logeswaran et al., 2020; Liu et al., 2021; Zhong et al., 2021…”
BackgroundDACT-BERT: Differentiable Adaptive Computation Time for an Efficient BERT Inference
Eyzaguirre, C., del Rio, F., Araujo, V., & Soto, Á. (2022) · NLPPOWER · doi:10.18653/v1/2022.nlppower-1.10 · Cited by 8 · see citing works
The expressive power of transformers with chain of thought
Merrill, W., & Sabharwal, A. (2023) · arXiv · doi:10.48550/arxiv.2310.07923 · Cited by 4 · cited 1× · see citing works
“Feng et al. (2023); Merrill & Sabharwal (2023) theoretically argue that CoT aids by increasing the computational expressivity of the Trans-former.”
BackgroundRL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs
Akyürek, A. F., Akyürek, E., Madaan, A., Kalyan, A., Clark, P., Wijaya, D., & Tandon, N. (2023) · arXiv · doi:10.48550/arxiv.2305.08844 · Cited by 4 · cited 1× · see citing works
“CoQA: Question-answering task based on a context (Reddy et al., 2019).”
Background
Flagged first, then by citations
14 of the 67 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Addressing 6 challenges in generative AI for digital health: A scoping review
Templin, T., Perez, M. W., Sylvia, S., Leek, J. T., & Sinnott-Armstrong, N. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000503 · Cited by 115 · see citing works
Shojaee, P., Mirzadeh, I., Alizadeh vahid, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-3603 · Cited by 47 · see citing works
Training Large Language Models to Reason in a Continuous Latent Space
Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.06769 · Cited by 18 · see citing works
“(4) Pause token (Goyal et al., 2023): The model is trained using only the question and answer, without a reasoning chain.”
Beyond the Frontier: Predicting Unseen Walls From Occupancy Grids by Learning From Floor Plans
Ericson, L., & Jensfelt, P. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3410164 · Cited by 10 · see citing works
“Furthermore, increasing the sequence by padding tokens improves model performance [34], [35], suggesting that shorter segments should improve performance, ceteris paribus.”
BackgroundDo NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D., Song, L., Liu, Q., Zhou, M., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.21187 · Cited by 3 · see citing works
Looped Transformers for Length Generalization
Fan, Y., Du, Y., Ramchandran, K., & Lee, K. (2024) · arXiv · doi:10.48550/arxiv.2409.15647
FlowReasoner: Reinforcing Query-Level Meta-Agents
Gao, H., Liu, Y., He, Y., Dou, L., Du, C., Deng, Z., Hooi, B., Lin, M., & Pang, T. (2025) · arXiv · doi:10.48550/arxiv.2504.15257
- Authors
- Goyal, S., Ji, Z., Rawat, A. S., Menon, A. K., Kumar, S., & Nagarajan, V.
- Affiliations
- Carnegie Mellon University (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2310.02226
- OpenAlex
- W4387356407
- Cited by
- 2
- References
- 67
- Citations by year
- 2024: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language model training, inference latency, decoder-only models, question answering, reasoning, fact recall, SQuAD, GSM8K, CommonsenseQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
