The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
HellaSwag: Can a Machine Really Finish Your Sentence?
Open accessAbstract
Recent work by Zellers et al. (2018) introduced a new task of commonsense natural language inference: given an event description such as "A woman sits at a piano," a machine must select the most likely followup: "She sets her fingers on the keys."With the introduction of BERT (Devlin et al., 2018), near human-level performance was reached.Does this mean that machines can perform human level commonsense inference?In this paper, we show that commonsense inference still proves di cult for even stateof-the-art models, by presenting HellaSwag, a new challenge dataset.Though its questions are trivial for humans (°95% accuracy), state-of-the-art models struggle ( †48%).We achieve this via Adversarial Filtering (AF), a data collection paradigm wherein a series of discriminators iteratively select an adversarial set of machine-generated wrong answers.AF proves to be surprisingly robust.The key insight is to scale up the length and complexity of the dataset examples towards a critical 'Goldilocks' zone wherein generated text is ridiculous to humans, yet often misclassified by state-of-the-art models.Our construction of HellaSwag, and its resulting di culty, sheds light on the inner workings of deep pretrained models.More broadly, it suggests a new path forward for NLP research, in which benchmarks co-evolve with the evolving state-of-the-art in an adversarial way, so as to present ever-harder challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is no 'ground-truth' implementation from the original benchmark authors, and many popular benchmarks (Paperno et al., 2016; Clark et al., 2018; Zellers et al., 2019) that have been 'retrofitted' for compatibility with autoregressive LMs.”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Zellers et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, benchmarks are often adversarially selected for tasks that current models struggle with compared to humans, 1 biasing the comparison to human performance.”
From Measuring AI Ability to Complete Long Software Tasks (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“During RLHF fine-tuning, we observe performance regressions compared to GPT-3 on certain public NLP datasets, notably SQuAD (Rajpurkar et al., 2018), DROP (Dua et al., 2019), HellaSwag (Zellers et al., 2019), and WMT 2015 French to English translation (Bojar et al., 2015).”
From Training language models to follow instructions with human feedback (2022) · cites this paper as Zellers et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Traditional language model benchmarks, such as HellaSwag (Zellers et al., 2019), WinoGrande (Sakaguchi et al., 2021), MMLU (Hendrycks et al., 2021), GSM-8K (Cobbe et al., 2021), and BIG-Bench (Srivastava et al., 2023), primarily assess LLMs on tasks like commonsense reasoning and standardized exams.”
From LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints (2024) · cites this paper as Zellers et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“for evaluating the following tasks that measure common sense reasoning: LAMBADA (OpenAI version in LM Evaluation Harness) (Paperno et al., 2016), HellaSwag (Zellers et al., 2019), PIQA (Bisk et al., 2020), ARC-challenge, ARC-easy (Clark et al., 2018), WinoGrande (Sakaguchi et al., 2021).”
From xLSTM: Extended Long Short-Term Memory (2024) · cites this paper as Zellers et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. Z. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1152 · Cited by 1,315
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Self-citationDense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,279
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
Research works printed but not linked: 2
- Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for e cient text classification. In Proceedings of the 15th Confer- ence of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 427-431.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), volume 1, pages 2227-2237.
Cited by
Sorted by citations
Items 801–900 of 908
Construction of a Large Language Model for Integrated Energy Systems
Chen, C., Lin, X., Zhou, Z., & Jin, Z. (2026) · Asia Energy and Electrical Engineering Symposium (AEEES) · doi:10.1109/aeees69423.2026.11556623
“Finally, to evaluate instruction-following and commonsense reasoning, we include Hellaswag [26].”
Sun, Y., Yao, W., Zhang, J., Song, W., Zhao, X., Hao, C., Chen, X. S., Zeng, S., Jia, S., Yang, Y., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.12.732007
Fu, W., Lu, Z., Qu, G., & Guo, X. (2026) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3787109.3815216
SLaB: Sparse-Lowrank-Binary Decomposition for Efficient Large Language Models
Li, Z., Ma, Y., & Kang, Y. (2026) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas66217.2026.11562018
“* factor refers to the quantization factor vector.”
Abidin, M. I. Z., Ghasemi, M. H., Razak, N. A., & Razak, A. A. (2026) · International Conference on Integrated Intelligence and Cognitive Engineering (ICIICE) · doi:10.1109/iciice69672.2026.11565422
Security-aware post-training quantization for Mixture-of-Experts large language models
Liu, B., Ge, S., Zhu, Z., Li, C., Yang, L., & Zhang, B. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114286
AceQuant: An Activation-Centric Framework for High-Fidelity Low-Bit Quantization of LLMs
Wei, M., Yan, Y., & Wang, D. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3706023
MGS: Markov Greedy Sums for Low-Power DNN Accumulation
Natesh, V., Kung, H. T., & Kong, D. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00054
“We synthesize designs using Cadence Genus [8], perform implementation using Cadence Innovus, run gate-level simulations using Synopsys VCS [40], and characterize power consumption based on transient behavior using Cadence Voltus.”
Matsuoka, D., Kawahara, S., Murakami, K., Matsumoto, R., Ito, R., Sugimoto, S., Sugiyama, D., Hara, M., Hayashida, M., Nguyen, K., et al. (2026) · Journal of Geophysical Research Machine Learning and Computation · doi:10.1029/2025jh001205
Three Birds, One Stone: Fast, Accurate-aware and Cost-Efficient Accelerator for Ternary LLM
정원석, Kang, J. K., Shin, S., Um, H., Lim, J., Park, Y., Koo, G., Park, S., & Suh, T. (2026) · International Conference on Supercomputing · doi:10.1145/3797905.3807877
“(Scores are averaged over HellaSwag [46], WinoGrande [36], WiC [34] and ANLI-R2 [30])”
Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference
Huang, Y., Di, S., & Li, G. (2026) · International Conference on Supercomputing · doi:10.1145/3797905.3800513 · Cited by 2
“Multi-bit cases for memory faults are further explored in Section 5.”
Sayah, B., Nehar, A., Cherroun, H., Bellaouar, S., & Alam, F. (2026) · BenchCouncil Transactions on Benchmarks Standards and Evaluations · doi:10.66834/byjwsd23
DynSpAttn: Efficient Attention via Dual-Side Dynamic Sparsity on Sparse Tensor Cores
Yu, X., Fan, R., Li, Z., Luo, W., Gong, G., & Chu, X. (2026) · International Conference on Supercomputing · doi:10.1145/3797905.3800548
D, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7069191
Lee, S.-C., Lee, T. H., & Chang, L.-H. (2025) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps68340.2025.11591138
“In addition to the fine-tuning datasets, we also select several non-target datasets including arc_challenge [23], arc_easy [23], hellaswag [24], openbookqa [25], piqa [26], and winogrande [27].”
DWI: Efficient On-Device LLM Decoding Through Dynamic Width Selection on Nested Models
Chen, W. P., Deng, Y., Wang, X., Fu, Z., Li, X. F., Cao, T., Liu, Y., & Ren, J. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3711495
Orlić, R., Šegota, S. B., Novak, A., & Lorencin, I. (2026) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro70003.2026.11591709
Can LLMs Understand Baseball Rules? A Benchmark for Long-Context Rule Reasoning
Matsuzaki, Y., Hirao, Y., Perusquía-Hernández, M., Uchiyama, H., & Kiyokawa, K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-20897-2_43
Ahmad, N., Nasir, A., Khan, H., Perveen, L., Hayat, M., & Sultan, N. (2026) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse68839.2026.11596924
“Recent studies have shown that even minor textual modi-fications can significantly reduce the performance of existing detection systems [35–37].”
TM-Bench: Benchmarking Large Language Models on Low-Resource Traditional Mongolian
Gao, Z., Bao, F., Bai, A., Hou, R., Ji, X., Wang, D., Ming, H., & Li, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808632
Accelerating Block Low-Rank Foundation Model Inference on Memory-Constrained GPUs
Abillama, P., Lee, C. S. G., Dong, J., Blaauw, D. T., Sylvester, D., & Kim, H.-S. (2026) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3806645.3807580
“Language models are evaluated with WikiText-103/2 perplexity and zero-shot classification accuracy on common sense reasoning benchmarks [4, 8, 9, 31, 39, 51].”
NOVA: Noise-to-variation guided pruning of large language models
Ding, A., Xu, P., & Jin, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116564
Hildebrand, D., Kiefer, B., & Zell, A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11614-6
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Wu, Y., Wang, X., Zhao, K., Wang, Z., Guo, Q., & Tang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3447-9_43
“We evaluate on language modeling (Wikitext103 test set) and six commonsense reasoning benchmarks: ARC-Challenge (AC), ARC-Easy (AE) [5], HellaSwag (HS) [30], PIQA (PI) [1], WinoGrande (WG) [19], and OpenBookQA (OB) [16].”
Zhu, H., Wang, F., Zhang, J., Cl, W., Zhang, H., Chen, K., Guo, W., & Zhu, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3441-7_38
Outlier-Aware and Task-Oriented Quantization-Based Low-Rank Adaptation of Large Language Models
Zhang, C., Zhou, G., & Fu, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3444-8_30
ALP: Adaptive Layerwise Pruning in Large Language Models
Mu, Y., Chen, Y., Cheng, B. H., Li, P. L., & Hu, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3435-6_14
Bsqat: block-wise shared quantization-aware training for large language models
Hou, S., Hou, L., Liu, T., Wang, L., & Yan, B. (2026) · Complex & Intelligent Systems · doi:10.1007/s40747-026-02394-1
A Survey of Commonsense Reasoning in LLMs
Teo, N., Huang, D., & Wang, Z. (2026) · ACM Computing Surveys · doi:10.1145/3832753
“Task-speciic models like T5 [165], BART [112], and UniiedQA [172] adopt a text-to-text framework and are ine-tuned on benchmark datasets such as SWAG, PIQA, and CommonsenseQA.”
InfluentialLAGR: Compensate Layer Pruning with Adaptive Gated Residuals for Efficient Compression
Mu, Y., Chen, Y., Cheng, B. H., Li, P., & Hu, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3520-9_23
Capability Degradation in Small Language Models Under Low-Signal Data Corruption
Qamar, R., Appasani, B., Bizon, N., Verma, S., & Jha, A. V. (2026) · International Conference on Electronics, Computers and Artificial Intelligence (ECAI) · doi:10.1109/ecai69016.2026.11613657
“MMLU, HellaSwag, and TruthfulQA motivate future evaluation of knowledge, commonsense, and truthfulness [13], [14], [15].”
From Video QoE to LLM QoE: Analogies, Gaps, and Challenges in Modeling Interactive QoE
Pérez, P., Orduna, M., & Koniuch, K. (2026) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex69967.2026.11618379
An Expert Routing Method Based on Positive Sample Semantic Distribution via BGMM-LSE
Ren, W., Zheng, T., Zhang, J., Chen, Y., Wei, E., Liu, C., Fan, B., Ji, Z., Lü, Y., He, J., et al. (2026) · Information · doi:10.3390/info17080736
Characterizing LLM performance via a Bayesian lens
Manousaridis, G., Samuelsson, J. G., Emir, B., Dasgupta, S., Jain, G., Madhavan, S., & Khan, S. (2026) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2026.1854495 · Cited by 1
“Future work can extend this research by applying the framework to more diverse benchmarks (e.g., HellaSwag, BoolQ) [54, 55], and perturbation types (e.g., adversarial, multilingual) [56, 57].”
iBRAHIMOĞLU, N., Demirezen, M. U., & Yıldız, F. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01490-x · Cited by 1
Fu, Y., Zhou, Z., Huang, C., Wang, J., Zeng, B., Dongke, H., Wang, X., Hong, W., & Fan, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770854.3783937
PP-LLMs: A progressive pruning approach with medium-granularity for large language models
Du, K., Wang, Y., Zhang, B., Wang, J., Zhang, F., Cao, L., & Guo, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133770
Xu, M., Xu, Y., & Mandic, D. P. (2026) · Machine Intelligence Research · doi:10.1007/s11633-026-1653-x
StableSkip: Stability-Guided Dynamic Token Skipping for Efficient Large Language Model Inference
Wu, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31673-8_17
UniCore: A Bit-Width Scalable GEMM Unit for Unified LLM Inference
Chen, Y., Zou, J., Chen, X., Xu, C., Guo, J., & Chen, X. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00176
“Tables I–II summarize WikiText-2 [29] perplexity and zero-shot accuracy [4], [8], [37], [47] for U NI C ORE and prior accelerators under several W/A/KV bit configurations on OPT-6.”
Triplet Decomposition and Extensions: A General Framework for Parameter-Efficient Fine-Tuning
Du, Z., Min, Y., Zhang, D., Lü, K., Li, J., & Shen, H. T. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3720458
“After fine-tuning, we evaluate the models on eight commonsense reasoning benchmarks, i.e., BoolQ [61], PIQA [62], SIQA [63], HellaSwag [64], WinoGrande [65], ARC-e/ ARC-c [66], OBQA [67].”
Kowalik, M., Poruba, M., Grzymkowski, Ł., & Stefański, T. P. (2026) · IEEE Access · doi:10.1109/access.2026.3720484
KV Cache Reuse for Elastic LLM Inference on Edge Devices
Wang, P., Zheng, Z., Huang, X., Wu, J., Wu, F., & Chen, G. (2026) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/2575-8411.2026.00035
“10 shows that the priority computation accounts for only about 4% of the total prefilling time, indicating that the additional overhead introduced by our method is negligible.”
Omni-LUT: Energy-Efficient LUT-Based Accelerator with Hardware-Aware KV Cache Quantization
Tsai, C.-H., Chou, K.-C., Wang, Y.-H., Wen, C.-D., & Yeh, T. T. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00037
Hong, S., Kong, H., Lee, J., Kim, S., & Yoo, H. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00079
“For commonsense reasoning, we report the average zero-shot accuracy across six multiple-choice benchmarks: PIQA [54], ARC-Easy, ARC-Challenge [55], BoolQ [56], HellaSwag [57], and Wino-Grande [58].”
Bridging Cognitive Architecture and Developmental Measurement for Artificial General Intelligence
Fu, Y., Wang, Y., Xie, H., Zhao, S., Liu, M., Lu, Y., He, X., Cheng, Z., Peng, Y., & Zhang, Z. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14080179
EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression
Xu, K., Wan, J., Hu, W., Han, P., & Wang, X. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817817
AzText: Curating Web-Scale Pretraining Data for a Low-Resource Language
Mahammadli, E., & Rustamov, S. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-31319-5_23
SimDiff: Depth pruning via similarity and difference
Chen, Y., Zhang, S., Meng, F., Cheng, B. H., Han, J., Tong, Q., & Liu, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133944 · Cited by 1
Lee, B.-S., & Gong, Y. (2026) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3816440.3818526
Fast LM Head via Coarse-Fine Decomposition and Tile-Wise Weight Clustering
Kim, J.-W., Yoon, S.-H., Lee, H., Lee, H.-B., Oh, S.-M., Kang, S.-Y., & Seo, Y.-H. (2026) · IEEE Transactions on Computers · doi:10.1109/tc.2026.3725429
“[ 34], GSM8K [35], IFEval [36], TruthQA [37], PIQA [38], WinoG.”
An Evaluation Framework for Post‐Trained Bookkeeping Language Models
Župan, M. (2026) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.70046
It Takes Two: Embracing Sparsity and Speculative Decoding for Efficient LLM Inference
Chu, H., Chen, C., Liu, W., Luan, J., Deng, J., Liu, L., Ma, H., & Zheng, X. (2026) · International Workshop on Quality of Service · doi:10.1109/iwqos70441.2026.11661086
Pick and Spin: Cold-Start-Aware Routing for Self-Hosted LLM Serving
Vangala, B. P., & Malik, T. (2026) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud72782.2026.00050
“We evaluate on 31,019 queries from eight standard benchmarks covering diverse task types: HumanEval [21] (164 queries), MBPP [22] (500), GSM8K [23] (1,319), MATH [24] (5,000), TruthfulQA [25] (790), MMLU-Pro [38] (12,032), ARC [26] (1,172), and HellaSwag [27] (10,042).”
Machine Common Sense: A Grand Challenge in Artificial Intelligence
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789272
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789276
Commonsense Knowledge versus Reasoning
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789275
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789277
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789274
(2026) · ACM eBooks · doi:10.1145/3789270.3789271
Multi-disciplinary Perspectives on Defining Common Sense
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789273
(2026) · ACM eBooks · doi:10.1145/3789270.3844419
Common Sense in Large Language Models
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789279
(2026) · ACM eBooks · doi:10.1145/3789270.3844418
Kejriwal, M. (2026) · ACM eBooks · doi:10.1145/3789270.3789278
Trainable Vector Quantization for Large Language Models
Prutianova, A., Bogdanova, A., Shutov, M., Poimanov, D. R., Gorkov, A., & Zharikov, I. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37667-1_15
“We also provide zero-shot accuracy on Wino-Grande [30], PiQA [3], HellaSwag [44], ARC-easy and ARC-challenge Question Answering (QA) benchmarks [6], using the Language Model Evaluation Harness library [12] for evaluation.”
The Effect of Quantization on Memorization in Large Language Models
Stahl, B., Laue, S., & Regneri, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37664-0_10
Costa, C. O., Bhaskar, B., Ewetz, H., Sleimi, A., Runonen, M., Oravecz, C., Hengchen, S., Ceresa, M., Brasoveanu, M. C., & Rausch, I. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.014
Developing GlassGPT: A Modern Day GPT-2 Alternative
Ravi, A. G., Devanand, A., Changethu, J. J., & Kavitha, K. V. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30438-4_3
Song, Q., & Tang, G. (2026) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3832810.3832814
Jiang, L., Sun, J., Lin, J., Li, H., Lu, M., Zhou, Y. C., & Sun, G. (2026) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3832810.3832877
MigMoE: Task-Aware Expert Migration for Faster and More Balanced Expert-Parallel MoE Inference
Han, X., Cai, Z., Jiang, Z., Zhang, R., & Ma, R. (2026) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3832810.3832811
“We use the same 12 public datasets as in Section 2, coveringfourtasktypes:(1) Math(GSM8K [10],HendrycksMATH[20], MathQA [1]); (2) QA/CR (ARC-C [9], BoolQ [8], HellaSwag [47]); (3) Code (HumanEval [6], MBPP [3], DS-1000 [32]); and (4) Sum-marization (BillSum [31], CNN/DailyMail [41], XSum [38]).”
Xia, J., Zhao, M., & Zhang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_18
Do Mixture-of-Experts Improve Energy Efficiency? An Early-Stage Evaluation
Gil, A., Bolón‐Canedo, V., Cancela, B., & Alonso‐Betanzos, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_23
Valisoa, M. V. A. R., & Matio, R. (2026) · IEEE Access · doi:10.1109/access.2026.3736829
Measuring Reproducibility in LLM Research: A Rubric and Cross-Topic Corpus Study
Chong, C. J., Hou, C., Liu, S., & Yao, Z. (2026) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3820002.3828584
“[23];FormalTrust[117];JailbreakBench[13];Many-shotJailbreaking[2];NotWhatYou’veSignedUpFor[30];SleeperAgents[43];Casper[18]; PACE[91] 14 1.07 1.64 1.36 2.64 1.86 1.69 Eval/Domain HellaSwag[120];Cobbeetal.”
InfluentialBit-sliced quantization for robust memristor-based compute-in-memory
Yu, Y., Yang, J., Ye, Z., Han, S., Wang, B., Wang, S., Wang, H., Qi, X., & Wang, Z. (2026) · npj Unconventional Computing · doi:10.1038/s44335-026-00099-9
“At 3-bit, BSQ consistently outperforms GPTQ on all reasoning tasks in both clean and noisy conditions; representative gains include PIQA (79.38% vs. 73.56%), HellaSwag (74.39% vs. 47.24%), and ARC-Challenge (46.76% vs. 37.80%), with clear margins retained under noise (for example, HellaSwag 55.36% vs. 35.28%).”
InfluentialTDG-LoRA: Token-Level Dynamic Gating for Mitigating Catastrophic Forgetting
Zhu, S., Huang, S., & Chen, X. (2026) · IEEE Access · doi:10.1109/access.2026.3737439
Knowledge transmission and fusion in large models: research advances and challenges
Fan, Y., Wang, Y., Yang, J., Wu, H., Zhang, Z., Zhu, P., Ding, W., & Hu, Q. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104819
Large Language Models Create Hallucinations in Response to Negated Text
Seo, J., Moon, H., & Lim, H. (2026) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3849694
“To track hallucination phenomena in LLMs caused by negated text, we utilize the HellaSwag [66] and HaluEval [35].”
InfluentialPark, S., Kim, C., Rho, S., Lee, M., & Chung, K. (2026) · IEEE Access · doi:10.1109/access.2026.3738900
Wei, M., Yan, Y., & Wang, D. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3739195
Arbiter: Unifying Executable Verification and Semantic Assessment
Lu, Y., Huang, Y., Ying, Y., Li, L., Yang, T., & Tie, Y. (2026) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr70517.2026.00067
“Traditional evaluation of large language models has relied mainly on benchmark-based paradigms, where models are assessed on static collections of predefined tasks with reference answers, such as HellaSwag, MMLU, GSM8K, and HumanEval [8]–[11].”
Chronologically consistent large language models
He, S., Lv, L., Manela, A., & Wu, J. (2026) · Journal of Financial Economics · doi:10.1016/j.jfineco.2026.104374 · Cited by 2
“While both models achieve chronological consistency, their underlying architectures make them suited to different types of tasks, with ChronoBERT excelling at classification and understanding benchmarks (GLUE) and ChronoGPT being better adapted to generative and completion tasks (HellaSwag).”
InfluentialScalable Entity-Centric Data Filtration for LLM Pretraining
Asatryan, A. A., Hovhannisyan, O., Hayrapetyan, A., Loukachevitch, N. V., Ordukhanyan, A., Baghdasaryan, L., Aleksanyan, H., Wenshuai, Y., Yujian, J., & Avetisyan, K. R. (2026) · IEEE Access · doi:10.1109/access.2026.3741148
Seeing Helps Reasoning in Language Models
Gan, Y., Zhao, K. I., Poggio, T., & Isola, P. (2026) · doi:10.1109/cvprf71909.2026.00680
“The Platonic Representation Hypothesis [18] observes a positive correlation between language-vision representation alignment in off-the-shelf models and language model performance on Hellaswag [62].”
PoM: A Linear-Time Replacement for Attention with the Polynomial Mixer
Picard, D., Dufour, N., Degeorge, L., Ghosh, A., Allegro, D., Ravaud, T., Perron, Y., Sautier, C., Baltaci, Z. S., Meng, F., et al. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00248
“We evaluate models using validation loss, accuracy on the fol-5 lowing benchmarks: ARC-E [12], HellaSwag [78], Wino-grande [61], and MMLU [32].”
Efficient zero-shot post-training quantization of large language models
Haq, S. U., El-Maleh, A. H., & Alsuwaiyan, A. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109750
Belkahla, M., & Hains, G. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28469-3_25
A Theoretical Analysis of Migration Strength Selection in SmoothQuant
Liu, Z., Sham, C., Li, J., Ma, S. L., & Fu, C. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728525
SWIS: An Efficient Structured Pruning Framework for LLMs with Spectral Geometric Alignment
Xu, L., & Li, P. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729617
“As summarized in Table I, our method surpasses SlimLLM, LLM-Pruner, and FLAP in average accuracy across all sparsity levels, under both pruning-only and fine-tuning recovery settings.”
ORCA: Outlier-aware Rotational Cluster Factorization for Efficient Low-Rank LLM Compression
Cho, H., Camacho, F., & Mukhopadhyay, S. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729006
SLoRA: Shared Low-Rank Adaptation for Parameter-Efficient Fine-Tuning of Large Language Models
Zhang, Q., Ji, F., & Ran, R. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729517
“(2) Natural Language Understanding (NLU): Training was conducted on an aggregated corpus comprising eight standard benchmarks: BoolQ [10], PIQA [11], SocialIQA [12], ARC-Challenge [13], ARC-Easy [13], OpenBookQA [14], Hel-laSwag [15], and Winogrande [16].”
DFQ: A Data-Free Quantization Algorithm for Large Language Models with Superior Scale and Zero Point
He, J., Ju, X., Yang, J., Wen, M., & Li, Z. (2026) · Learning and analytics in intelligent systems · doi:10.1007/978-3-032-26620-0_26
LC-DPO: Length-Controlled Direct Preference Optimization via Dynamic Token Selection
Dong, J., Zhang, B., & Zhang, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729428
“In addition, we test LC-DPO on six conditional generation benchmarks: GSM8K [27], IFEval [28], PiQA [29], MMLU [30], COQA [31], and HellaSwag [32], which evaluate abilities ranging from arithmetic and commonsense reasoning to factual accuracy.”
OBEQuant: Outlier-aware Batch-norm Equivalent Quantization for Large Language Models
Zhong, Y., Ji, B., Liu, X., Li, S., Ma, J., & Yu, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11730032
“These tasks cover multiple cognitive dimensions, including BoolQ [27], HellaSwag [28], LAMBADA [29], OpenBookQA [30], PIQA [31], SIQA [32], WinoGrande [33], ARC-Easy and ARC-Challenge [34], comprehensively reflecting the model’s ability to maintain its capabilities in real-world scenarios.”
CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge Servers
Zhang, C., Tao, Y., Zou, Y., Yang, Y., Zhang, R., Zhang, X., Dressler, F., Cheng, X., & Yu, D. (2026) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3795866.3844472
“For accuracy evaluation, we measure perplexity on WikiText-103 [24] and C4 [25] datasets, and report accuracy on downstream tasks including MMLU [15], HellaSwag [47], ARC-Challenge [5], and WinoGrande [27].”
UniNL: Unifying Fragmented Non-Linear Operators for Efficient Edge LLM Inference
Hu, Z. H., Fan, Z., Qin, S., Mu, Y., Li, W., & Ye, X. (2026) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3770743.3806382
Gu, F., & Boyd-Graber, J. L. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1018041
Metadata
- Authors
- Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 4791-4800
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2946609015
- Cited by
- 830
- References
- 20
- Citations by year
- 2026: 200 · 2025: 263 · 2024: 109 · 2023: 93 · 2022: 50 · 2021: 61 · 2020: 44 · 2019: 10
- FWCI
- 15.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation (1703166, 1256082, 1637479, 1524371)
- Nvidia
- Allen Institute for Artificial Intelligence
- Defense Advanced Research Projects Agency
- Intelligence Advanced Research Projects Activity
- Army Research Office (W911NF-15-1-, W911NF)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- commonsense reasoning, natural language inference, AI-generated text, BERT, benchmark design, pre-trained language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
