Theoretical Limitations of Self-Attention in Neural Sequence Models
Open accessAbstract
Transformers are emerging as the new workhorse of NLP, showing great success across tasks. Unlike LSTMs, transformers process input sequences entirely through self-attention. Previous work has suggested that the computational capabilities of self-attention to process hierarchical structures are limited. In this work, we mathematically investigate the computational power of self-attention to model formal languages. Across both soft and hard attention, we show strong theoretical limitations of the computational abilities of self-attention, finding that it cannot model periodic finite-state languages, nor hierarchical structure, unless the number of layers or heads increases with input length. These limitations seem surprising given the practical success of self-attention and the prominent role assigned to hierarchical structure in linguistics, suggesting that natural language can be approximated well with models that are too weak for the formal languages typically assumed in theoretical linguistics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A Transformer cannot even fit the training set of parity sequences over some minimal length.
“Many works have shown that a Transformer cannot even fit the training set of parity sequences over some minimal length (Hahn, 2020;Delétang et al., 2023).”
From What Algorithms can Transformers Learn? A Study in Length Generalization (2023) · cites this paper as Hahn, 2020
The Discussion says that any fixed transformer will make mistakes on PARITY when inputs are sufficiently long. It does not establish that a Transformer cannot fit a finite training set beyond a particular minimum length; the same section says a transformer can be constructed to achieve perfect accuracy on all examples up to any chosen finite length bound. (LLM, conf=0.88)
Experimental — accuracy not yet measuredLLMs can occasionally exhibit unpredictable reasoning hallucinations spanning both long-range and short-range dependencies, which potentially arise from the limitations of soft attention, where attention becomes diluted across positions as sequence length increases.
“Moreover, recent research has exposed that LLMs can occasionally exhibit unpredictable reasoning hallucinations spanning both long-range and short-range dependencies, which potentially arise from the limitations of soft attention [52, 111], where attention becomes diluted across positions as sequence length increases.”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [111]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe approach bypasses the limitations presented in Hahn (2020), which exploit non-saturating softmaxes.
“This bypasses the limitations presented in Hahn (2020), which exploit non-saturating softmaxes.”
From What Algorithms can Transformers Learn? A Study in Length Generalization (2023) · cites this paper as Hahn (2020)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · see citing works
Chomsky, N. (1957) · doi:10.1515/9783112316009 · Cited by 7,885 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Computational Complexity: A Modern Approach
Arora, S., & Barak, B. (2009) · Cited by 2,375 · see citing works
Handbook Of Mathematical Psychology
Galanter, & Eugène (1963) · Cited by 1,950 · see citing works
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · see citing works
Mitzenmacher, M., & Upfal, E. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511813603 · Cited by 1,739 · see citing works
What Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,514 · see citing works
A Decomposable Attention Model for Natural Language Inference
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,453 · see citing works
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307 · see citing works
Long Short-Term Memory-Networks for Machine Reading
Cheng, J.-P., Dong, L., & Lapata, M. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1053 · Cited by 1,200 · see citing works
A Deep Reinforced Model for Abstractive Summarization
Paulus, R., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1705.04304 · Cited by 1,182 · see citing works
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1580 · Cited by 1,146 · see citing works
Generating Long Sequences with Sparse Transformers
Child, R., Gray, S., Radford, A., & Sutskever, I. (2019) · arXiv · doi:10.48550/arxiv.1904.10509 · Cited by 1,063 · see citing works
An Activation‐Based Model of Sentence Processing as Skilled Memory Retrieval
Lewis, R. L., & Vasishth, S. (2005) · Cognitive Science · doi:10.1207/s15516709cog0000_25 · Cited by 1,038 · see citing works
Parity, circuits, and the polynomial-time hierarchy
Furst, M., Saxe, J. B., & Sipser, M. (1984) · Theory of Computing Systems · doi:10.1007/bf01744431 · Cited by 942 · see citing works
(1963) · Cited by 855 · see citing works
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Shen, T., Zhou, T., Long, G., Jiang, J., Pan, S., & Zhang, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11941 · Cited by 779 · see citing works
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Linzen, T., Emmanuel Dupoux, A., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 776 · see citing works
LSTM recurrent networks learn simple context-free and context-sensitive languages
Gers, F. A., & Schmidhuber, E. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963769 · Cited by 724 · see citing works
The Algebraic Theory of Context-Free Languages
Chomsky, N. A. D. T., & Schützenberger, M. P. (1963) · Studies in logic and the foundations of mathematics · doi:10.1016/s0049-237x(08)72023-8 · Cited by 721 · see citing works
On the Computational Power of Neural Nets
Siegelmann, H. T., & Sontag, E. D. (1995) · Journal of Computer and System Sciences · doi:10.1006/jcss.1995.1013 · Cited by 703 · see citing works
Evidence against the context-freeness of natural language
Shieber, S. M. (1985) · Linguistics and Philosophy · doi:10.1007/bf00630917 · Cited by 657 · see citing works
Separating the polynomial-time hierarchy by oracles
Yao, A. C.-C. (1985) · Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.1985.49 · Cited by 556 · see citing works
Colorless Green Recurrent Networks Dream Hierarchically
Gulordava, K., Bojanowski, P., Grave, É., Linzen, T., & Baroni, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1108 · Cited by 489 · see citing works
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
Chen, M., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G. F., Jones, L., Schuster, M., Shazeer, N., Parmar, N., et al. (2018) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p18-1008 · Cited by 484 · see citing works
Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., & Kaiser, Ł. (2018) · arXiv · doi:10.48550/arxiv.1807.03819 · Cited by 467 · see citing works
Targeted Syntactic Evaluation of Language Models
Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334 · see citing works
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Lapata, M., & Barzilay, R. (2005) · Cited by 333 · see citing works
Studies in Logic and the Foundations of Mathematics
(2008) · Studies in logic and the foundations of mathematics · doi:10.1016/s0049-237x(13)72046-9 · Cited by 284 · see citing works
Structures, Not Strings: Linguistics as Part of the Cognitive Sciences
Everaert, M. B. H., Huybregts, M. A. C., Chomsky, N. A. D. T., Berwick, R. C., & Bolhuis, J. J. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.09.008 · Cited by 247 · see citing works
Open Sesame: Getting inside BERT’s Linguistic Knowledge
Lin, Y., Tan, Y. C., & Frank, R. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4825 · Cited by 233 · see citing works
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
Weiss, G., Goldberg, Y., & Yahav, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2117 · Cited by 226 · see citing works
Counter-Free Automata (M.I.T. research monograph no. 65)
McNaughton, R., & Papert, S. A. (1971) · The MIT Press eBooks · Cited by 206 · see citing works
Gibson, E. A. F., & Thomas, J. (1999) · Language and Cognitive Processes · doi:10.1080/016909699386293 · Cited by 194 · see citing works
BERT Rediscovers the Classical NLP PipelinePreprint
Tenney, I., Das, D., & Pavlick, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05950 · Cited by 192 · see citing works
On the Turing Completeness of Modern Neural Network ArchitecturesPreprint
Pérez, J., Marinkovic, J., & Barceló, P. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1901.03429 · Cited by 187 · see citing works
The Importance of Being Recurrent for Modeling Hierarchical Structure
Tran, K. M., Bisazza, A., & Monz, C. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1503 · Cited by 161 · see citing works
Evidence Against the Context-Freeness of Natural Language
Shieber, S. M. (1985) · Studies in cognitive systems · doi:10.1007/978-94-009-2727-8_4 · Cited by 154 · see citing works
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
Shen, T., Zhou, T., Long, G., Jiang, J. B., Wang, S., & Zhang, C. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/604 · Cited by 145 · see citing works
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better
Kuncoro, A., Dyer, C., Hale, J., Yogatama, D., Clark, S. C., & Blunsom, P. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1132 · Cited by 131 · see citing works
The average sensitivity of bounded-depth circuits
Boppana, R. B. (1997) · Information Processing Letters · doi:10.1016/s0020-0190(97)00131-2 · Cited by 106 · see citing works
McCoy, R. T., Min, J., & Linzen, T. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.21 · Cited by 105 · see citing works
Barrington, D. A. M., Compton, K. J., Howard Straubing, C., & Thérien, D. (1992) · Journal of Computer and System Sciences · doi:10.1016/0022-0000(92)90014-a · Cited by 104 · see citing works
On the Robustness of Self-Attentive Models
Hsieh, Y.-L., Cheng, M., Juan, D.-C., Wei, W., Hsu, W.-L., & Hsieh, C.-J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1147 · Cited by 96 · see citing works
Modeling Recurrence for Transformer
Hao, J., Wang, X., Yang, B., Wang, L., Zhang, J., & Tu, Z. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1122 · Cited by 85 · see citing works
Sequential Neural Networks as Automata
Merrill, W. (2019) · Proceedings of the Workshop on Deep Learning and Formal Languages: Building Bridges · doi:10.18653/v1/w19-3901 · Cited by 54 · see citing works
Evaluating the Ability of LSTMs to Learn Context-Free Grammars
Sennhauser, L., & Berwick, R. C. (2018) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w18-5414 · Cited by 53 · see citing works
Fractal encoding of context‐free grammars in connectionist networks
Tabor, W. (2000) · Expert Systems · doi:10.1111/1468-0394.00126 · Cited by 48 · see citing works
When Recurrent Models Don't Need To Be RecurrentPreprint
Miller, J., & Hardt, M. (2018) · arXiv.org · doi:10.48550/arxiv.1805.10369 · Cited by 38 · see citing works
Srinivasan, A. (2018) · ACM SIGACT News · doi:10.1145/3289137.3289142 · Cited by 36 · see citing works
Assessing the Ability of Self-Attention Networks to Learn Word Order
Yang, B., Wang, L., Wong, D. F., Chao, L. S., & Tu, Z. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1354 · Cited by 35 · see citing works
Random deep neural networks are biased towards simple functionsPreprint
Palma, G. D., Kiani, B. T., & Lloyd, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.10156 · Cited by 35 · see citing works
On Evaluating the Generalization of LSTM Models in Formal LanguagesPreprint
Süzgün, M., Belinkov, Y., & Shieber, S. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01001 · Cited by 30 · see citing works
Evidence Against the Context-Freeness of Natural Language
Shieber, S. M. (1985) · Studies in linguistics and philosophy · doi:10.1007/978-94-009-3401-6_12 · Cited by 28 · see citing works
Proceedings of the Society for Computation in Linguistics (SCiL) 2019
Jarosz, G. E., Nelson, M., O’Connor, B., & Pater, J. (2019) · Cited by 25 · see citing works
Assessing the Ability of LSTMs to Learn Syntax-Sensitive DependenciesPreprint
Linzen, T., Dupoux, E., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01368 · Cited by 25 · see citing works
Closing Brackets with Recurrent Neural Networks
Skachkova, N., Trost, T. A., & Klakow, D. (2018) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w18-5425 · Cited by 24 · see citing works
On the Computational Power of RNNsPreprint
Korsky, S., & Berwick, R. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06349 · Cited by 24 · see citing works
Can Recurrent Neural Networks Learn Nested Recursion?
Bernardy, J.-P. (2018) · Linguistic Issues in Language Technology · doi:10.33011/lilt.v16i.1417 · Cited by 23 · see citing works
On the implicit acquisition of a context-free grammar by a simple recurrent neural network
Cartling, B. G. (2007) · Neurocomputing · doi:10.1016/j.neucom.2007.05.006 · Cited by 22 · see citing works
A New Approach to the Sensitivity Conjecture
Gilmer, J., Koucký, M., & Saks, M. (2015) · Conference on Innovations in Theoretical Computer Science · doi:10.1145/2688073.2688096 · Cited by 19 · see citing works
McCoy, R. T., Min, J., & Linzen, T. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4110592 · Cited by 19 · see citing works
Processing of nested and cross-serial dependencies: an automaton perspective on SRN behaviour
Kirov, C., & Frank, R. (2011) · Connection Science · doi:10.1080/09540091.2011.641939 · Cited by 15 · see citing works
Computation in recurrent neural networks: From counters to iterated function systems
Kalinke, Y., & Lehmann, H. (1998) · Lecture notes in computer science · doi:10.1007/bfb0095051 · Cited by 14 · see citing works
Optimal Depth, Very Small Size Circuits for Symmetrical Functions in AC0
Håstad, J., Wegener, I., Wurm, N., & Yi, S. Z. (1994) · Information and Computation · doi:10.1006/inco.1994.1008 · Cited by 12 · see citing works
Smooth Boolean Functions are Easy
Gopalan, P., Nisan, N., Servedio, R. A., Talwar, K., & Wigderson, A. (2016) · Conference on Innovations in Theoretical Computer Science · doi:10.1145/2840728.2840738 · Cited by 9 · see citing works
Stack-like and queue-like dynamics in recurrent neural networks
Grüning, A. (2006) · Connection Science · doi:10.1080/09540090500317291 · Cited by 9 · see citing works
Deep neural networks are biased towards simple functions.Preprint
Palma, G. D., Kiani, B. T., & Lloyd, S. (2018) · arXiv (Cornell University) · Cited by 8 · see citing works
Smooth Boolean functions are easy: efficient algorithms for low-sensitivity functionsPreprint
Gopalan, P., Nisan, N., Servedio, R. A., Talwar, K., & Wigderson, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.02420 · Cited by 5 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 78 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Austin, Texas. Association for Computational Linguistics.
- Robert McNaughton and Seymour A. Papert. 1971. Counter-Free Automata (MIT Research Monograph No. 65), The MIT Press.
- John Miller and Moritz Hardt. 2019. Stable recurrent models. In International Conference on Learning Representations.
- Dan Parker, Michael Shvartsman, and Julie A. Van Dyke. 2017. The cue-based retrieval theory of sentence comprehension: New findings and new challenges. Language Processing and Disorders, pages 121-144.
Cited by
Sorted by citations
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
“A potential cause is the limitations of soft attention (Hahn, 2020; Chi-ang and Cholak, 2022), where attention becomes diluted across positions as sequence length increases.”
TransNeXt: Robust Foveal Visual Perception for Vision Transformers
Shi, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01683 · Cited by 452 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · see citing works
“A potential cause is the limitations of soft attention (Hahn, 2020; Chi-ang and Cholak, 2022), where attention becomes diluted across positions as sequence length increases.”
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202 · see citing works
Transformer-based deep learning for predicting protein properties in the life sciences
Chandra, A. A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023) · eLife · doi:10.7554/elife.82819 · Cited by 192 · see citing works
Hallucination is Inevitable: An Innate Limitation of Large Language Models
Xu, Z., Jain, S., & Kankanhalli, M. (2024) · arXiv · doi:10.48550/arxiv.2401.11817 · Cited by 180 · see citing works
“Attention mechanism, a key component of transformer-based LLMs, can also contribute to hallucination [36], possibly because attention across tokens is diluted as the sequence length grows [10, 22].”
BackgroundFaith and Fate: Limits of Transformers on Compositionality
Dziri, N., Lu, X., Sclar, M., Li, X. (., Jiang, L., Lin, B. Y., Welleck, S., West, P., Bhagavatula, C., Bras, R. L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3081 · Cited by 124 · see citing works
“We observe that all analyzed models match the Relative Information Gain prediction that o 1 (whether the first element goes into the output sequence or not) should be the easiest value to predict (see Figures 30, 31, and 32).”
BackgroundMethodLanguage Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works
Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.15408 · Cited by 69 · see citing works
“Besides affirmative results, other works characterized the expressivity limitation of Transformers via the perspective of modeling formal languages [23, 6, 57] or simulating circuits [24, 37, 36].”
ResultScreen Parsing: Towards Reverse Engineering of UI Models from Screenshots
Wu, J., Zhang, X., Nichols, J., & Bigham, J. P. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474763 · Cited by 65 · see citing works
Using Computational Models to Test Syntactic Learnability
Wilcox, E., Futrell, R., & Lévy, R. (2022) · Linguistic Inquiry · doi:10.1162/ling_a_00491 · Cited by 66 · see citing works
PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS
Ye, J., Zen, H., Shen, J., Zhang, Y., & Wu, Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1757 · Cited by 72 · see citing works
A Redhead Walks into a Bar: Experiences of Writing Fiction with Artificial Intelligence
Ghajargar, M., Bardzell, J., & Lagerkvist, L. (2022) · Proceedings of the 25th International Academic Mindtrek Conference · doi:10.1145/3569219.3569418 · Cited by 54 · see citing works
On the Ability and Limitations of Transformers to Recognize Formal Languages
Bhattamishra, S., Ahuja, K., & Goyal, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.576 · Cited by 48 · see citing works
Overcoming a Theoretical Limitation of Self-Attention
Chiang, D., & Cholak, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.527 · Cited by 39 · see citing works
Towards Trustworthy AI: A Review of Ethical and Robust Large Language Models
Ferdaus, M. M., Abdelguerfi, M., Ioup, E., Niles, K. N., Pathak, K., & Sloan, S. D. (2025) · ACM Computing Surveys · doi:10.1145/3777382 · Cited by 38 · see citing works
What Context Features Can Transformer Language Models Use?
O’Connor, J., & Andreas, J. (2021) · doi:10.18653/v1/2021.acl-long.70 · Cited by 59 · see citing works
Kim, J., Lee, S., Hwang, E., Ryu, K. S., Jeong, H., Lee, J. W., Hwangbo, Y., Choi, K. S., & Cha, H. S. (2020) · Journal of Medical Internet Research · doi:10.2196/18418 · Cited by 34 · see citing works
Autoregressive Structured Prediction with Language Models
Liu, T., Jiang, Y. E., Monath, N., Cotterell, R., & Sachan, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.70 · Cited by 33 · see citing works
$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse TransformersPreprint
Yun, C., Chang, Y.-W., Bhojanapalli, S., Rawat, A. S., Reddi, S. J., & Kumar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04862 · Cited by 35 · see citing works
Sartran, L., Barrett, S., Kuncoro, A., Stanojević, M., Blunsom, P., & Dyer, C. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00526 · Cited by 33 · see citing works
Transferring Inductive Biases through Knowledge Distillation
Abnar, S., Dehghani, M., & Zuidema, W. (2020) · arXiv · doi:10.48550/arxiv.2006.00555 · Cited by 30 · see citing works
Addressing Some Limitations of Transformers with Feedback MemoryPreprint
Fan, A., Lavril, T., Grave, É., Joulin, A., & Sukhbaatar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09402 · Cited by 41 · see citing works
Waqas, M., Tahir, M. A., Author, M. D., Al-Maadeed, S., Bouridane, A., & Wu, J. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09417-3 · Cited by 28 · see citing works
Adapting to Length Shift: FlexiLength Network for Trajectory Prediction
Xu, Y., & Fu, Y. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01442 · Cited by 51 · see citing works
Quantum mixed-state self-attention network
Fu, C., Zhao, Q., Feng, L., Chen, C., Lin, Y., & Lin, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107123 · Cited by 31 · see citing works
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck LanguagesPreprint
Süzgün, M., Gehrmann, S., Belinkov, Y., & Shieber, S. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03329 · Cited by 33 · see citing works
1D Convolutional LSTM-based wind power prediction integrated with PkNN data imputation technique
Shahid, F., Mehmood, A., Khan, R., Smadi, A. A., Yaqub, M., Alsmadi, M. K., & Zheng, Z. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101816 · Cited by 27 · see citing works
DeepRSSI: Generative Model for Fingerprint-Based Localization
Yoon, N., Jung, W., & Kim, H. (2024) · IEEE Access · doi:10.1109/access.2024.3398734 · Cited by 27 · see citing works
Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity
Hao, Y., Angluin, D., & Frank, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00490 · Cited by 27 · see citing works
What Formal Languages Can Transformers Express? A Survey
Strobl, L., Merrill, W., Weiss, G., Chiang, D., & Angluin, D. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00663 · Cited by 31 · see citing works
Du, K.-L., Zhang, R., Jiang, B., Zeng, J., & Lu, J. (2025) · Mathematics · doi:10.3390/math13030451 · Cited by 28 · see citing works
Saturated Transformers are Constant-Depth Threshold Circuits
Merrill, W., Sabharwal, A., & Smith, N. A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00493 · Cited by 25 · see citing works
Chen, F., Li, Y., Chen, Y.-L., Bian, Z.-X., Duo, L., Zhou, Q., & Zhang, L. (2025) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.70075 · Cited by 24 · see citing works
The Parallelism Tradeoff: Limitations of Log-Precision Transformers
Merrill, W., & Sabharwal, A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00562 · Cited by 22 · see citing works
Optimal Subarchitecture Extraction For BERTPreprint
de Wynter, A., & Perry, D. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10499 · Cited by 26 · see citing works
Self-Attention Networks Can Process Bounded Hierarchical Languages
Yao, S., Peng, B., Papadimitriou, C. H., & Narasimhan, K. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.292 · Cited by 24 · see citing works
How Powerful are Decoder-Only Transformer Neural Models?
Roberts, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651286 · Cited by 22 · see citing works
SEQ2SEQ++: A Multitasking-Based Seq2seq Model to Generate Meaningful and Relevant Answers
Palasundram, K., Sharef, N. M., Kasmiran, K. A., & Azman, A. (2021) · IEEE Access · doi:10.1109/access.2021.3133495 · Cited by 17 · see citing works
State-space modeling in long sequence processing: a survey on recurrence in the transformer era
Tiezzi, M., Casoni, M., Betti, A., Gori, M., & Melacci, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108039 · Cited by 21 · see citing works
Learning Bounded Context-Free-Grammar via LSTM and the Transformer: Difference and the Explanations
Shi, H., Gao, S., Tian, Y., Chen, X., & Zhao, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20801 · Cited by 16 · see citing works
Randomized Positional Encodings Boost Length Generalization of Transformers
Ruoss, A., Delétang, G., Genewein, T., Grau-Moya, J., Csordás, R., Bennani, M., Legg, S., & Veness, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.161 · Cited by 24 · see citing works
Xu, F., Zhou, T., Nguyen, T., Bao, H., Lin, C., & Du, J. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103402 · Cited by 16 · see citing works
Leivada, E., Marcus, G., Günther, F., & Murphy, E. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0008 · Cited by 20 · see citing works
Sensitivity as a Complexity Measure for Sequence Classification Tasks
Hahn, M., Jurafsky, D., & Futrell, R. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00403 · Cited by 14 · see citing works
Self-citationHow Can Self-Attention Networks Recognize Dyck-n Languages?
Ebrahimi, J., Gelda, D., & Zhang, W. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.384 · Cited by 13 · see citing works
(2023) · doi:10.18653/v1/2023.acl-short.38 · Cited by 12 · see citing works
Structural generalization is hard for sequence-to-sequence models
Yao, Y., & Koller, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.337 · Cited by 12 · see citing works
How BPE Affects Memorization in TransformersPreprint
Kharitonov, E., Baroni, M., & Hupkes, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02782 · Cited by 14 · see citing works
Characterizing In-Context Learning: When Can Transformers Match Standard Learning Algorithms?
Hawarey, M. (2026) · doi:10.65737/airmcs2026277 · Cited by 11 · see citing works
What Algorithms can Transformers Learn? A Study in Length Generalization
Zhou, H., Bradley, A., Littwin, E., Razin, N., Saremi, O., Susskind, J., Bengio, S., & Nakkiran, P. (2023) · arXiv · doi:10.48550/arxiv.2310.16028 · Cited by 10 · see citing works
“Many works have shown that a Transformer cannot even fit the training set of parity sequences over some minimal length (Hahn, 2020; Delétang et al., 2023).”
BackgroundTan, S., Shen, Y., Chen, Z., Courville, A. C., & Gan, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.12 · Cited by 11 · see citing works
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions
Bhattamishra, S., Patel, A., Kanade, V., & Blunsom, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.317 · Cited by 10 · see citing works
Explainable Multi-hop Verbal Reasoning Through Internal Monologue
Liang, Z., Bethard, S. J., & Surdeanu, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.97 · Cited by 11 · see citing works
On the Difficulty of Translating Free-Order Case-Marking Languages
Bisazza, A., Üstün, A., & Sportel, S. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00424 · Cited by 10 · see citing works
Back to recurrent processing at the crossroad of transformers and state-space models
Tiezzi, M., Casoni, M., Betti, A., Guidi, T., Gori, M., & Melacci, S. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01034-6 · Cited by 8 · see citing works
Hybrid ST-ResNet and LSTM approach for precise crime hotspot prediction
Shahmoradi, N., Alesheikh, A. A., Jafari, A., & Lotfata, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-24559-7 · Cited by 8 · see citing works
A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning
Vilaça, L., Yu, Y., & Viana, P. (2025) · ACM Computing Surveys · doi:10.1145/3696445 · Cited by 8 · see citing works
Finlayson, M., Richardson, K., Sabharwal, A., & Clark, P. A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.27 · Cited by 8 · see citing works
Formal Language Theory Meets Modern NLPPreprint
Merrill, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10094 · Cited by 8 · see citing works
On the Limits of Language Generation: Trade-Offs between Hallucination and Mode-Collapse
Kalavasis, A., Mehrotra, A., & Velegkas, G. (2025) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3717823.3718108 · Cited by 7 · see citing works
Memon, Y., Zeng, F., & Yao, L. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108270 · Cited by 7 · see citing works
Liu, Z., Kou, J., Zhang, W., Gu, C., Fang, X., Huang, Z., Hao, Y., Li, H., Lu, X., Yin, A., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189137 · Cited by 9 · see citing works
A provably stable neural network Turing Machine with finite precision and time
Stogin, J., Mali, A., & Giles, C. L. (2023) · Information Sciences · doi:10.1016/j.ins.2023.120034 · Cited by 7 · see citing works
Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation
Munro, R., & Morrison, A. (. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.157 · Cited by 7 · see citing works
Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units
Mali, A., Ororbia, A. G., Kifer, D., & Giles, C. L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i6.16634
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text
Shibata, C., Uchiumi, K., & Mochihashi, D. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.356 · Cited by 8 · see citing works
AI-driven rational design of promiscuous and selective plastic-binding peptides
Jain, V., Bergman, M. T., Hall, C. K., & You, F. (2025) · Chemical Science · doi:10.1039/d5sc04903b · Cited by 6 · see citing works
Chain-of-Thought ICL for GeoAI: Breaking the Hardness Barrier
Hawarey, M. (2026) · doi:10.65737/airmcs2026482 · Cited by 6 · see citing works
Provably Learning a Multi-head Attention Layer
Chen, S., & Li, Y. (2025) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3717823.3718174 · Cited by 6 · see citing works
Set-to-Sequence Methods in Machine Learning: A Review
Jurewicz, M., & Derczynski, L. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12839 · Cited by 6 · see citing works
Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement
Li, B., Wisniewski, G., & Crabbé, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.377 · Cited by 6 · see citing works
A machine learning method to variable classification in OpenMP
Shen, Y., Peng, M., Wu, Q., & Li, R. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.10.010 · Cited by 6 · see citing works
ActiFormer: Sign-Aware Linear Attention for Sensor-Based Human Activity Recognition
Sun, Q., Fu, Z., Zhang, L., Chen, G., Huang, W., Wu, H., & Song, A. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3633705 · Cited by 5 · see citing works
Weird-Net: Weighted Relative Distance Attention for Efficient and Robust Sequence Processing
Hu, L., Zhan, F., Huang, W., Gan, W., Hu, H., He, H., & Han, K. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3513912 · Cited by 5 · see citing works
Cui, H., Behrens, F., Krząkała, F., & Zdeborová, L. (2025) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ade137 · Cited by 5 · see citing works
Categorical representation learning and RG flow operators for algorithmic classifiers
Sheshmani, A., You, Y., Fu, W., & Azizi, A. (2023) · Machine Learning Science and Technology · doi:10.1088/2632-2153/acb488 · Cited by 5 · see citing works
Stogin, J., Mali, A., & Giles, C. L. (2020) · arXiv (Cornell University) · Cited by 5 · see citing works
PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTSPreprint
Ye, J., Zen, H., Shen, J., Zhang, Y., & Wu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.15060 · Cited by 5 · see citing works
Hallucination in Large Language Models: From Mechanistic Understanding to Novel Control Frameworks
Zhang, W., Zhang, C., Gu, C., Kou, J., Yuan, H., Fang, X., Duan, X., & Fang, Y. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189226 · Cited by 6 · see citing works
Memon, Y., Zeng, F., & Khawaja, F. R. (2025) · Cluster Computing · doi:10.1007/s10586-025-05707-0 · Cited by 4 · see citing works
MultiFix: Learning to Repair Multiple Errors by Optimal Alignment Learning
Seo, H., Han, Y.-S., & Ko, S. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.417 · Cited by 4 · see citing works
Semantic Web-Based Prediction Method for Goodwill Impairment in Listed Companies Using Mamba
Li, Y. (2024) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.356492 · Cited by 4 · see citing works
Real-time fine finger motion decoding for transradial amputees with surface electromyography
Weng, Z., Yang, X. F., Li, P., Yi, C., Bashivan, P., Ma, H., Yao, G., Lin, Y., Li, F., Yao, D., et al. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107605 · Cited by 6 · see citing works
Nguyen, P., Le, T., Nguyen, H. T., Tran, V., & Nguyen, L.-M. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04246-0 · Cited by 5 · see citing works
Kahardipraja, P., Madureira, B., & Schlangen, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.257 · Cited by 4 · see citing works
Evaluating Transformer’s Ability to Learn Mildly Context-Sensitive Languages
Wang, S., & Steinert‐Threlkeld, S. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.21 · Cited by 3 · see citing works
Zubek, J., Korbak, T., & Rączaszek‐Leonardi, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11048-y · Cited by 6 · see citing works
Parameter Norm Growth During Training of Transformers.Preprint
Merrill, W., Ramanujan, V., Goldberg, Y., Schwartz, R., & Smith, N. A. (2020) · arXiv (Cornell University) · Cited by 4 · see citing works
Improving neural machine translation of languages with little data and rich morphology
Dhar, P. (2024) · doi:10.33612/diss.859674171 · Cited by 3 · see citing works
Irie, K., Csordás, R., & Schmidhuber, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.588 · Cited by 3 · see citing works
MO-Transformer: Extract High-Level Relationship Between Words for Neural Machine Translation
Duan, S., & Zhao, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3507556 · Cited by 3 · see citing works
Impartial Games: A Challenge for Reinforcement Learning
Zhou, B., & Riis, S. (2026) · Machine Learning · doi:10.1007/s10994-026-06996-1 · Cited by 3 · see citing works
Deshpande, C., Gens, D., & Franz, M. (2021) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3474369.3486865 · Cited by 3 · see citing works
Baier-Reinio, A., & Sterck, H. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11358 · Cited by 3 · see citing works
Transformers, parallel computation, and logarithmic depth
Sanford, C., Hsu, D., & Telgarsky, M. (2024) · arXiv · doi:10.48550/arxiv.2402.09268 · Cited by 2
Clustered Bootstrap Similarity Learning Framework: Time Series Forecasting
Kubheka, S., van Zyl, T. L., & Wang, H. (2025) · IEEE Access · doi:10.1109/access.2025.3630067 · Cited by 2 · see citing works
Kou, J., Lu, X., Yin, A., Zhang, W., Fang, X., Li, Y., Pan, T., Zhang, L., Cheng, G., Yuan, H., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189154 · Cited by 2 · see citing works
Multi-Layer Self-Attention Residual Network for Code Search
Hu, H., Liu, J., & Zhang, X. (2022) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7650 · Cited by 2 · see citing works
- Authors
- Hahn, M.
- Affiliations
- Stanford University (US)
- Corresponding author
- Hahn, Michael
- ORCID
- Hahn, Michael
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 156-171
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2952744660
- Cited by
- 165
- References
- 78
- Citations by year
- 2026: 22 · 2025: 29 · 2024: 25 · 2023: 25 · 2022: 16 · 2021: 30 · 2020: 17 · 2019: 1
- FWCI
- 8.65
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computational Theory and Mathematics › semigroups and automata theory
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- self-attention, transformer models, computational power, formal languages, hierarchy, soft attention, hard attention, sequence models, theoretical limitations, natural language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
