Shortcut Learning of Large Language Models in Natural Language Understanding
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Shortcuts often hinder the robustness of large language models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent works show that even Large-Language Models (LLMs), pre-trained on text amounting to billions of words, suffer from shortcut learning, largely due to collection artifacts in training data [7], [21].”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [21]
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 12×
“Recently, large language models (LLMs), such as BERT [8], RoBERTa [20], T5 [30], GPT-3 [4], have been reported to achieve state-of-the-art performance in a series of high-level NLU tasks.”
BackgroundInfluentialAdvances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“Natural language understanding (NLU) is a subfield of artificial intelligence that requires computer software to comprehend input in the form of sentences.”
BackgroundChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1×
“Note that this type of method can only be used when prior knowledge is known in advance about shortcuts.”
MethodRight for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944 · cited 2×
“NLI, QA, and reading comprehension (Zhou and Bansal, 2020; McCoy et al., 2019).”
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927 · cited 3×
“This is distinct from the literature that works with the toy task (e.g., Colored MNIST [1]), which typically contains a single type of bias and the bias is fully known.”
BackgroundMethodInfluential
Research works printed but not linked: 14
- Branco, R., Branco, A., Silva, J., and Rodrigues, J. Shortcutted commonsense: Data spuriousness in deep learning of commonsense reasoning. In Proceedings of the 2021 Conf. Empirical Methods in Natural Language Processing.
- Devlin, J., Chang, M., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. North American Chapter of the Assoc. Computational Linguistics, 2019.
- Ethayarajh, K. How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conf. Empirical Methods in Natural Language Processing and the 9th Intern. Joint Conf. Natural Language Processing, 55--65.
- Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S.R., and Smith, N.A. Annotation artifacts in natural language inference data. North American Chapter of the Assoc. Computational Linguistics, 2018.
- Liu, Y. et al. RoBERTa: A robustly optimized BERT pretraining approach, 2019; arXiv:1907.11692. · arXiv
- Robinson, J., Sun, L., Yu, K., Batmanghelich, K., Jegelka, S., and Sra, S. Can contrastive learning avoid shortcut solutions? Advances in Neural Information Processing Systems, 2021.
- Sen, P. and Saffari, A. What do models learn from question answering datasets? In Proceedings of the 2020 Conf. Empirical Methods in Natural Language Processing.
- Si, C. et al. Prompting gpt-3 to be reliable, 2022; arXiv:2210.09150. · arXiv
- Stacey, J., Belinkov, Y., and Rei, M. Supervising Model Attention with Human Explanations for Robust Natural Language Inference. In Proceedings of the 2022 AAAI Conf. Artificial Intelligence.
- Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the 2017 Intern. Conf. Machine Learning.
- Tu, L., Lalwani, G., Gella, S., and He, H. An empirical study on robustness to spurious correlations using pre-trained language models. Trans. Assoc. Computational Linguistics (2020).
- Webson, A. and Pavlick, E. Do prompt-based models really understand the meaning of their prompts? In Proceedings of the 2022 Conf. North American Chapter of the Assoc. Computational Linguistics: Human Language Technologies.
- Zellers, R., Bisk, Y., Schwartz, R., and Choi, Y. Swag: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of the 2018 Conf. Empirical Methods in Natural Language Processing.
- Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. Calibrate before use: Improving few-shot performance of language models. In Proceedings of the 2021 Intern. Conf. Machine Learning, 12697--12706.
Cited by
Items 101–164 of 164
Reasoning Engine with Pre-Trained LLMs: An Operation GPT
Pany, P. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.68761 · Cited by 1
Khant, K. S., Lin, H. Y., & Thongtanunam, P. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00044 · Cited by 2
“Such a behavior is canonically known as shortcut learning [28].”
DBR: Divergence-Based Regularization for Debiasing Natural Language Understanding Models
Li, Z., Tang, R., Cheng, L., Wang, S., Yin, D., & Du, M. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748241 · Cited by 1
“However, recent studies suggest that these PLMs heavily rely on a phenomenon called “shortcut learning” [4; 34], where they capture shallow correlations between labels and shortcut features of examples instead of developing a deeper semantic understanding of language [1; 5].”
Self-citationMichaelov, J. A., Estacio, R., Zhang, Z., & Bergen, B. K. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.696 · Cited by 5
Liu, Z., Shao, W., Shaolong, E., & Cao, X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104418 · Cited by 2
Gradient-based Model Shortcut Detection for Time Series Classification
Ibarra, S. E., Cantu, F., Zhou, K., & Zhang, L. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00104 · Cited by 2
Supervised contrastive learning in few-shot soft prompt tuning
Edalat, A., & Yaghoobzadeh, Y. (2026) · Natural language processing. · doi:10.1017/nlp.2026.10013 · Cited by 1
Lee, H., Seo, S., Park, S., Tam, L., Kim, S., & Jang, Y. (2026) · Public Relations Review · doi:10.1016/j.pubrev.2026.102709 · Cited by 1
Data Foundations of Long-Context Language Models: A Survey
Sun, Z., Sun, Y., Su, Z., Tang, Z., Li, J., Zhou, A., Chen, W., & Zhang, M. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.775 · Cited by 2
On the Persistent Effects of Lexicality in Large Language ModelsPreprint
Rizwan, H., Haider, M. U., Subramani, N., Diab, M., Siddique, A. B., & Sajjad, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.02750 · Cited by 1
“In this regime, next-token prediction can make it statistically efficient for the model to rely on surface-level lexical cues (e.g., entity names, salient attributes) that correlate with the target [Du et al., 2024, Bandel et al., 2022].”
Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature LeakagePreprint
Namjoo, F., Ogasawara, R., Abdullah, A., Anderson, C., Oozeer, N. F., & Phillips, J. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.13003 · Cited by 1
“The general phenomenon is now known as shortcut learning [Geirhos et al., 2020, Du et al., 2024]; Niven and Kao [2019], Gardner et al. [2021] and Pacchiardi et al. [2024] formalize artifacts as features predictive of the label without being competency-relevant.”
Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model BehaviorPreprint
Jia, R., & Wu, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.04395 · Cited by 1
Jeyakodi, G., Balakrishnan, S., & Bala, P. S. (2024) · Salud Ciencia y Tecnología - Serie de Conferencias · doi:10.56294/sctconf2024758
Representation Learning of Human Diseases for Indication Expansion and Investment DecisionsPreprint
Ravandi, B., Mowrey, W. R., Chatterjee, A., Khanshan, F., Haddadi, P., Ughetto, M., Barrett, I. P., Ding, W., del Angel, G., Diethe, T., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.19.624381
“Consequently, they have become insufficient to distinguish superficial pattern matching from true mechanistic understanding [61].”
«The Trouble with Bias» in Neural Networks: Conflict and Ethical Challenges
Николаевич, С. А., & Идрисович, М. А. (2024) · Administrative Consulting · doi:10.22394/1726-1139-2024-5-150-158
Towards generalizable neural program understanding and synthesis
Li, Z. (2024) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/181413
Николаевич, С. А. (2024) · Vestnik of Saint Petersburg University Philosophy and Conflict Studies · doi:10.21638/spbu17.2024.308
AI Governance in Academia: Guidelines for Generative AI
Peterson, C., & Deschênes, M. (2025) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.38.1.138855 · Cited by 1
Research on Distributed Training Architecture for Large Scale Models for Natural Language Processing
Gong, J., & Wang, Y. (2025) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1145/3728725.3728812
“99.99% of the total system interfaces are available, and interface anomalies can be quickly detected and recovered through the monitoring and alarm mechanism [5].”
InfluentialHierarchical Mixture-of-Experts model for unified code search
Lin, Z., Zhang, X., Hu, X., Li, P., Zhou, D., & Lai, D. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103489 · Cited by 1
Self-citationExplaining and Interpreting Generative AI
Samek, W. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0005
Vandermeerschen, E., & de Lhoneux, M. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.499 · Cited by 1
Hasanov, I. J., Hakkala, A., Isoaho, J., & Virtanen, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-00071-2_13
Enabling large language models with explainability
Galitsky, B. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30046-2.00011-9
A Survey of Learning-based Method Name Prediction
Qian, H., Liu, W., Xu, T., Yin, J., Feng, X., Sun, W., Ding, Z., Chen, Y., Miao, Y., Li, J., et al. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3771999
An OCR-LLM Framework Towards Tabular Data Extraction
Pieterse, C. J., & Venter, P. V. Z. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-08384-5_15
Li, Z., Yang, J., & Liu, Y. (2025) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2025.blackboxnlp-1.24
Joint Learning for Efficient German Argument Mining
Wendlinger, L., Kühn, R., Mitrović, J., & Granitzer, M. (2025) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai66417.2025.00111
“Du et al. [8] cite various biases that can lead to shortcut learning and inadequate performance in the wild.”
Bae, S., Choi, Y., & Lee, J.-H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.19426
Language Model Pretraining with Lexical Semantic Relations
Romascanu, A. (2023) · eScholarship@McGill (McGill) · doi:10.82308/33958
Shuai, L., Fang, L., Yuxiang, P., Ying, C., Dengke, X., Liping, F., Jie, Z., & Yaru, G. (2025) · Earth Science-Journal of China University of Geosciences · doi:10.3799/dqkx.2025.287
Ding, Y., Han, X., Yang, J., Wang, T., Bi, Z., Song, X., Hao, J., Song, J., Ge, E., Peng, B., et al. (2026) · doi:10.36227/techrxiv.176784493.33333564/v1
WaterCopilot: An AI-Driven Virtual Assistant for Water ManagementPreprint
Vickneswaran, K., Andarcia, M. G., Retief, H., Dickens, C., & Silva, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.08559
Wendlinger, L. (2026) · OPUS (University of Passau)
Feng, X., Kang, Y., Yin, K., Zhang, T., & Wang, P. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2623224
“…parameters and complex computational structures (Kalota, 2024), Generative AI deeply integrates technologies such as large language models (Peng et al., 2023; Soltani et al., 2025), natural language understanding (Du et al., 2024; Koenigstein, 2025), and neural networks (Rajalakshmi et al., 2025).”
Han, M.-Q., Sun, X., Wang, X.-J., Zhan, W., Chen, X., & Quek, T. Q. S. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3665406
“In particular, we introduce the type of devices, i.e., vehicle or RSU, in the transformer-based Bert module as a grouped attention approach to further mitigate the performance degradation of BMQN in inexperienced scenarios [33], [34].”
Large Language Model Evaluation
坂井, 優. (2024) · Institutional Repositories DataBase (IRDB) · doi:10.34413/dr.02100
Doumanas, D., Soularidis, A., Zafeiropoulos, N., Chatzistamatis, S., Tsekouras, G. E., Saer, A. E., Nathanailidis, C., & Kotis, K. (2026) · Information · doi:10.3390/info17040342
Argument Mining with Customized and Feature-lnjected Large Language Models
Mushtaq, U. (2023) · theses.fr (ABES) · doi:10.70675/805e9f95z56afz4453zb477zd7f0103d5406
Large Learning Rates Simultaneously Achieve Robustness to Spurious Correlations and Compressibility
Barsbey, M., Prieto, L., Zafeiriou, S. P., & Birdal, T. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00199 · Cited by 4
“Similarly, [17, 37] show that AI systems and large language models (LLMs) can be susceptible to large performance drops if training data contains easily exploitable patterns not carrying over to new environments.”
Precedent Retrieval Based In-Context Learning for Legal Rationale Generation
Yue, L., Du, Y., & Gao, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0378-9_8
Extracting user profile via large language models and ontologies
Safari, P., & Shamsfard, M. (2026) · PLoS ONE · doi:10.1371/journal.pone.0329934
Arous, I., Chehbouni, K., Cheng, Z., & Dossou, B. F. P. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_85
Rote Memorization or Intelligence: An Assessment of Inferential Reasoning in Large Language Models
Mehmood, R., Rehman, E., & Habib, M. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01262-7 · Cited by 1
“This behavior aligns with prior findings that transformer-based models may rely on statistically reinforced heuristics rather than strict rule enforcement, even in domains governed by simple symbolic principles [ 4 , 5 ].”
idSCD: Identifying Training Datasets through Semantic Correlation DescriptorsPreprint
Gobeaja, A., Hodoroaga, I., Leordeanu, M., & Burceanu, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.30462
“…cues, such as backgrounds, textures, or secondary objects in vision tasks [Geirhos et al., 2020, Wang et al., 2021, Yang et al., 2022, Lin et al., 2024] and keywords in language tasks [Gururangan et al., 2018, McCoy et al., 2019, Niven and Kao, 2019, Du et al., 2023], instead of causal concepts.”
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response FrameworkPreprint
Huang, D., Chang, J., Shah, A., Mittal, P., & Sitawarin, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.16242
“…as a way to deliver poisoned samples into the final training set (2) shortcut learning : a well-known concept we use for the utility degradation attack (Geirhos et al., 2020; Du et al., 2023), (3) Omission Attack : a novel backdoor attack we introduce specifically for the safety degradation attack.”
Confusion-Aware In-Context-Learning for Vision-Language Models in Robotic Manipulation
He, Y., Kang, Z., Zhao, B., Wu, Z., Peng, J., & Wang, J. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582198
“For instance, a VLM might over-rely on superficial visual features or specific text patterns frequently associated with certain labels in the training data, leading to poor generalization in new scenarios [44].”
Rethinking the Rationalization: A Two-Step Decomposition for Graph Rationalization
Xia, R., & Yue, L. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582241
Early Stopping Without Validation Data in Weakly Supervised Learning
Yuan, S., Li, M., Feng, L., Han, B., Niu, G., Sugiyama, M., & Liu, T. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3712551
“This phenomenon connects to the broader understanding of overfitting where models may exploit non-robust “shortcuts” [110], [111] to minimize training loss [17].”
Zhu, D., Shen, P., & Shen, S. (2026) · Memetic Computing · doi:10.21203/rs.3.rs-10492304/v1
HEART-Synth: A Synthetic Resident-Patient Dialogue Dataset for Medical Error Disclosure Assessment
Ghimire, A., Amsaad, F., & Adkins, S. M. (2026) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon70028.2026.11674942
“This utility depends on auditing, because a generated dataset can accidentally reward a model for recognizing construction artifacts, fixed phrases, or scenario labels rather than the communication behavior being scored [6].”
Guan, X., Cao, J., Cao, B., Gao, Q., & Liu, B. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3724403
“Despite their remarkable capabilities, LLMs often suffer from “shortcut learning,” relying on spurious correlations or surface-level semantic priors rather than robust logical deduction [25], [26].”
Inoshita, K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7412289
Understanding or Imitation? Auditing Conceptual Understanding and Reasoning in Large Language Models
Hassan, M. S., Jan, E., Kaur, R., Yeh, E., Zaffar, M. F., & Gehani, A. (2026) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3820002.3828594
A pragmatic workflow for integrating large language models into toxicological risk assessment
Bassan, A., Amberg, A., Barreca, M. L., Bradley, P. M., Bringezu, F., Souza, J. D. P., Guicheney, C., Piparo, E. L., Kovarich, S., Massarsky, A., et al. (2026) · Frontiers in Toxicology · doi:10.3389/ftox.2026.1877128
“Notably, LLMs often rely on keywords and familiar phrasings rather than robust domain reasoning, a phenomenon known as shortcut learning (Geirhos et al., 2020; Du et al., 2024).”
Herramienta para la generación sintética de datos
Abdellaoui, A. (2026) · Deposito de Investigacion Universidad de Sevilla (University of Seville)
Wu, A., Shuang, K., Zhou, J., Liu, X., & Bao, R. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117149
CoBe: A Benchmark For Conversational Counterfactual Text EditingPreprint
Raghavan, A., Hwang, I., Havaldar, S., Lee, K.-Z., Maiti, A., Pan, Y., Wu, J., Li, M., & Bareinboim, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.13057
“…on associational signals rather than genuine causal structure, learning surface heuristics from their pretraining distribution (Joshi et al., 2024; Wu et al., 2024) and often learning shortcuts to predict “right but for the wrong reasons” (Du et al., 2023; Geirhos et al., 2020; McCoy et al., 2019).”
Detection and Exploration of References between Scholarly Texts and Literary Works
Arnold, F. (2026) · edoc Publication server (Humboldt University of Berlin) · doi:10.18452/39577
Gu, F., & Boyd-Graber, J. L. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1018041
Foundational challenges in assuring alignment and safety of large language models
Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 12
Comparison of LLM Prompting Strategies for Identifying Indonesian-Language Hoax Narratives
Ardiansyah, S. Z. (2026) · Journal Of World Science · doi:10.58344/jws.v5i9.1715
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI SafetyPreprint
Barez, F., Wu, T.-Y., Batra, H., Dominici, G., Draye, F., Elazar, Y., Fornasiere, D., García, A., Harrasse, A., Harris, R., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.08438 · Cited by 1
Advancing Language and Literacy Assessments through Technological Integrations
Hunte, M. R. (2024) · TSpace (University of Toronto)
Metadata
- Authors
- Du, M., He, F., Zou, N., Tao, D., & Hu, X.
- Affiliations
- New Jersey Institute of Technology (US)
- University of Edinburgh (GB)
- Texas A&M University (US)
- The University of Sydney (AU)
- Rice University (US)
- Venue
- Communications of the ACM
- Published
- 2023-12-21
- Volume
- 67
- Issue
- 1
- Pages
- 110-120
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W4390043316
- Cited by
- 94
- References
- 40
- Citations by year
- 2026: 22 · 2025: 34 · 2024: 22 · 2023: 16
- FWCI
- 8.86
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- shortcut learning, large language models, natural language understanding, robustness, model interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.