Measure and Improve Robustness in NLP Models: A Survey
Open accessAbstract
As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios.Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research.In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP.We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models' robustness.Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models.Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data distribution shifts are shown to lead to model failures.
“Such data distribution shifts are shown to lead to model failures.”
From A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures (2023) · cites this paper as [62]
Mitigation approaches include learning invariant representations and preventing models from relying on spurious patterns through methods such as augmentation, reweighting, ensembles, inductive priors, and causal interventions.
“Mitigation approaches involve learning invariant representations, and ensuring models do not rely on spurious patterns using techniques like data augmentation, reweighting, ensembling, inductiveprior design, and causal intervention [117].”
From Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) (2024) · cites this paper as [117]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776 · see citing works
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · see citing works
Theoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., & Jordan, M. I. (2019) · arXiv · doi:10.48550/arxiv.1901.08573 · Cited by 1,560 · see citing works
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410 · see citing works
Domain Generalization with Adversarial Feature Learning
Li, H., Pan, S. J., Wang, S., & Kot, A. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00566 · Cited by 1,251 · see citing works
Robustness May Be at Odds with AccuracyPreprint
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 1,240 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01501 · Cited by 1,090 · see citing works
Practical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
BLEURT: Learning Robust Metrics for Text Generation
Sellam, T., Das, D., & Parikh, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.704 · Cited by 1,005 · see citing works
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019) · arXiv · doi:10.48550/arxiv.1906.02530 · Cited by 989 · see citing works
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987 · see citing works
Adversarial Examples Are Not Bugs, They Are Features
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., & Madry, A. (2019) · arXiv · doi:10.48550/arxiv.1905.02175 · Cited by 971 · see citing works
Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation
Ghifary, M., Kleijn, W. B., Zhang, M., Balduzzi, D., & Li, W. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_36 · Cited by 948 · see citing works
HotFlip: White-Box Adversarial Examples for Text Classification
Ebrahimi, J., Rao, A., Lowd, D., & Dou, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2006 · Cited by 948 · see citing works
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944 · see citing works
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927 · see citing works
A Survey on Neural Network Interpretability
Zhang, Y., Tino, P., Leonardis, A., & Tang, K. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3100641 · Cited by 923 · see citing works
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907 · see citing works
Generating Natural Language Adversarial Examples
Alzantot, M., Sharma, Y., Elgohary, A., Ho, B.-J., Srivastava, M. B., & Chang, K.-W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1316 · Cited by 895 · see citing works
AugMix: A Simple Data Processing Method to Improve Robustness and UncertaintyPreprint
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02781 · Cited by 893 · see citing works
Beyond accuracy: behavioral testing of NLP models with CheckList
Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.442 · Cited by 890 · see citing works
Visual Domain Adaptation: A survey of recent advances
Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2347059 · Cited by 883 · see citing works
Unified Deep Supervised Domain Adaptation and Generalization
Motiian, S., Piccirilli, M., Adjeroh, D., & Doretto, G. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.609 · Cited by 835 · see citing works
A Survey of Data Augmentation Approaches for NLP
Feng, S. Y., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., & Hovy, E. H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.84 · Cited by 802 · see citing works
Domain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771 · see citing works
Robust Solutions of Optimization Problems Affected by Uncertain Probabilities
Ben‐Tal, A., den Hertog, D., Waegenaere, A. M. B. D., Melenberg, B., & Rennen, G. (2012) · Management Science · doi:10.1287/mnsc.1120.1641 · Cited by 753 · see citing works
Adversarial Training Methods for Semi-Supervised Text ClassificationPreprint
Miyato, T., Dai, A. M., & Goodfellow, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07725 · Cited by 713 · see citing works
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1009 · Cited by 712 · see citing works
Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
Gao, J., Lanchantin, J., Soffa, M. L., & Qi, Y. (2018) · 2018 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00016 · Cited by 693 · see citing works
Hypothesis Only Baselines in Natural Language Inference
Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., & Durme, B. V. (2018) · International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-2023 · Cited by 685 · see citing works
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · see citing works
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Wang, H., Wu, X., Huang, Z., & Xing, E. P. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00871 · Cited by 679 · see citing works
Self-citationGenerating Natural Language Adversarial Examples through Probability Weighted Word Saliency
Ren, S., Deng, Y., He, K., & Che, W. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1103 · Cited by 666 · see citing works
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656 · see citing works
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · see citing works
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks
Iyyer, M., Wieting, J., Gimpel, K., & Zettlemoyer, L. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1170 · Cited by 636 · see citing works
TextBugger: Generating Adversarial Text Against Real-world Applications
Li, J., Ji, S., Du, T., Li, B., & Wang, T. (2019) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2019.23138 · Cited by 634 · see citing works
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Li, L., Ma, R., Guo, Q., Xue, X., & Qiu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.500 · Cited by 623 · see citing works
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
Morris, J. X., Lifland, E., Yoo, J. Y., Grigsby, J., Jin, D., & Qi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2020.emnlp-demos.16 · Cited by 570 · see citing works
Synthetic and Natural Noise Both Break Neural Machine TranslationPreprint
Belinkov, Y., & Bisk, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02173 · Cited by 533 · see citing works
BAE: BERT-based Adversarial Examples for Text Classification
Garg, S., & Ramakrishnan, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.498 · Cited by 518 · see citing works
Generating Natural Adversarial Examples
Zhao, Z., Dua, D., & Singh, S. (2017) · arXiv · doi:10.48550/arxiv.1710.11342 · Cited by 508 · see citing works
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492 · see citing works
Beyond Sharing Weights for Deep Domain Adaptation
Rozantsev, A., Salzmann, M., & Fua, P. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2814042 · Cited by 466 · see citing works
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · see citing works
In Search of Lost Domain Generalization
Gulrajani, I., & Lopez-Paz, D. (2020) · arXiv · doi:10.48550/arxiv.2007.01434 · Cited by 440 · see citing works
Semantically Equivalent Adversarial Rules for Debugging NLP models
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1079 · Cited by 435 · see citing works
Stress Test Evaluation for Natural Language InferencePreprint
Naik, A., Ravichander, A., Sadeh, N., Rosé, C., & Neubig, G. (2018) · International Conference on Computational Linguistics · doi:10.48550/arxiv.1806.00692 · Cited by 431 · see citing works
Assessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410 · see citing works
Word-level Textual Adversarial Attacking as Combinatorial Optimization
Zang, Y., Qi, F., Yang, C., Liu, Z., Zhang, M., Liu, Q., & Sun, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.540 · Cited by 386 · see citing works
Intuition and Reasoning: A Dual-Process Perspective
Evans, J. S. B. T. (2010) · Psychological Inquiry · doi:10.1080/1047840x.2010.521057 · Cited by 385 · see citing works
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Clark, C., Yatskar, M., & Zettlemoyer, L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1418 · Cited by 381 · see citing works
Jiang, H., He, P., Chen, W., Liu, X., Gao, J., & Zhao, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.197 · Cited by 372 · see citing works
Chen, J., Yang, Z., & Yang, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.194 · Cited by 363 · see citing works
Self-citationWeight Poisoning Attacks on Pretrained Models
Kurita, K., Michel, P., & Neubig, G. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.249 · Cited by 354 · see citing works
Learning the Difference that Makes a Difference with Counterfactually-Augmented DataPreprint
Kaushik, D., Hovy, E. H., & Lipton, Z. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12434 · Cited by 346 · see citing works
Learning Robust Global Representations by Penalizing Local Predictive Power
Wang, H., Ge, S., Xing, E. P., & Lipton, Z. C. (2019) · arXiv · doi:10.48550/arxiv.1905.13549 · Cited by 337 · see citing works
Self-citationBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · see citing works
Neural Text Summarization: A Critical Evaluation
Kryscinski, W., Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1051 · Cited by 335 · see citing works
Pretrained Transformers Improve Out-of-Distribution Robustness
Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., & Song, D. X. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.244 · Cited by 312 · see citing works
Demographic dialectal variation in social media: a case study of African-American English
Blodgett, S. L., Green, L., & O'Connor, B. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1120 · Cited by 291 · cited 2× · see citing works
“els’ performance should not degrade due to the differences in grammar errors, dialects, speakers, languages (Craig and Washington, 2002; Blodgett et al., 2016; Demszky et al., 2021), or newly collected datasets for the same task but in different”
BackgroundCombating Adversarial Misspellings with Robust Word Recognition
Pruthi, D., Dhingra, B., & Lipton, Z. C. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1561 · Cited by 285 · see citing works
Geva, M., Goldberg, Y., & Berant, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1107 · Cited by 284 · see citing works
PAWS: Paraphrase Adversaries from Word Scrambling
Zhang, Y., Baldridge, J., & He, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1131 · Cited by 272 · see citing works
Evaluating Models’ Local Decision Boundaries via Contrast Sets
Gardner, M., Artzi, Y., Basmov, V., Berant, J., Bogin, B., Chen, S., Dasigi, P., Dua, D., Elazar, Y., Gottumukkala, A., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.117 · Cited by 265 · see citing works
Pathologies of Neural Models Make Interpretations Difficult
Feng, S., Wallace, E., Grissom, A., Iyyer, M., Rodríguez, P., & Boyd-Graber, J. L. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1407 · Cited by 260 · see citing works
Dynabench: Rethinking Benchmarking in NLP
Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.324 · Cited by 254 · see citing works
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A SurveyPreprint
Zhang, W. E., Sheng, Q. Z., Alhazmi, A., & Li, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06796 · Cited by 253 · see citing works
ToTTo: A Controlled Table-To-Text Generation Dataset
Parikh, A., Wang, X., Gehrmann, S., Faruqui, M., Dhingra, B., Yang, D., & Das, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.89 · Cited by 252 · see citing works
Self-citationPAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Yang, Y., Zhang, Y. E., Tar, C., & Baldridge, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1382 · Cited by 231 · see citing works
Robustness beyond shallowness: incremental deep parsing
Aı̈t-Mokhtar, S., Chanod, J.-P., & ROUX, C. (2002) · Natural Language Engineering · doi:10.1017/s1351324902002887 · Cited by 226 · see citing works
Certified Robustness to Adversarial Word Substitutions
Jia, R., Raghunathan, A., Göksel, K., & Liang, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1423 · Cited by 225 · see citing works
Select-additive learning: Improving generalization in multimodal sentiment analysis
Wang, H., Meghawat, A., Morency, L., & Xing, E. P. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019301 · Cited by 224 · see citing works
Self-citationContextualized Perturbation for Textual Adversarial Attack
Li, D., Zhang, Y., Peng, H., Chen, L., Brockett, C., Sun, M., & Dolan, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.400 · Cited by 213 · see citing works
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
Wu, T., Ribeiro, M. T., Heer, J., & Weld, D. S. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.523 · Cited by 210 · see citing works
Measuring Robustness to Natural Distribution Shifts in Image Classification
Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., & Schmidt, L. (2020) · arXiv · doi:10.48550/arxiv.2007.00644 · Cited by 206 · see citing works
Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach
Duchi, J. C., Glynn, P. W., & Namkoong, H. (2021) · Mathematics of Operations Research · doi:10.1287/moor.2020.1085 · Cited by 196 · see citing works
Distributionally Robust Neural Networks
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2020) · International Conference on Learning Representations · Cited by 164 · see citing works
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
He, H., Zha, S., & Wang, H. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6115 · Cited by 153 · see citing works
Self-citationIs BERT Really Robust? Natural Language Attack on Text Classification and EntailmentPreprint
Jin, D., Jin, Z., Zhou, J. T., & Peter Szolovits, B. (2019) · arXiv (Cornell University) · Cited by 152 · see citing works
Learning Robust Representations by Projecting Superficial Statistics OutPreprint
Wang, H., He, Z., Lipton, Z. C., & Xing, E. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06256 · Cited by 151 · see citing works
Self-citationHow Can We Accelerate Progress Towards Human-like Linguistic Generalization?
Linzen, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.465 · Cited by 148 · see citing works
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
Lewis, P., Stenetorp, P., & Riedel, S. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.86 · Cited by 145 · see citing works
Adversarial Training for Large Neural Language ModelsPreprint
Liu, X., Cheng, H., He, P., Chen, W., Wang, Y., Poon, H., & Gao, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08994 · Cited by 143 · see citing works
Just Train Twice: Improving Group Robustness without Training Group InformationPreprint
Liu, E. Z., Haghgoo, B., Chen, A., Raghunathan, A., Koh, P. W., Sagawa, S., Liang, P., & Finn, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09044 · Cited by 142 · see citing works
A Closer Look at Accuracy vs. RobustnessPreprint
Yang, Y.-Y., Rashtchian, C., Change Institutions to: University of Waterloo, Salakhutdinov, R., & Chaudhuri, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02460 · Cited by 138 · see citing works
Fairness without Demographics through Adversarially Reweighted LearningPreprint
Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., Wang, X., & Ed, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13114 · Cited by 135 · see citing works
Self-citationTrick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering
Wallace, E., Rodríguez, P., Feng, S., Yamada, I., & Boyd-Graber, J. L. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00279 · Cited by 130 · see citing works
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Tu, L., Lalwani, G., Gella, S., & He, H. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00335 · Cited by 125 · see citing works
Detecting Hallucinated Content in Conditional Neural Sequence Generation
Zhou, C., Neubig, G., Gu, J., Diab, M., Guzmán, F., Zettlemoyer, L., & Ghazvininejad, M. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.120 · Cited by 122 · see citing works
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension
Bartolo, M., Roberts, A., Welbl, J., Riedel, S., & Stenetorp, P. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00338 · Cited by 120 · see citing works
AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
Wallace, E., Tuyls, J., Wang, J., Subramanian, S., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations · doi:10.18653/v1/d19-3002 · Cited by 116 · see citing works
From Lifestyle Vlogs to Everyday Interactions
Fouhey, D. F., Kuo, W., Efros, A. A., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00524 · Cited by 115 · see citing works
Concealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · see citing works
Improving the Robustness of Question Answering Systems to Question Paraphrasing
Gan, W. C., & Ng, H. T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1610 · Cited by 112 · see citing works
Robustness Gym: Unifying the NLP Evaluation Landscape
Goel, K., Rajani, N. F., Vig, J., Taschdjian, Z., Bansal, M., & Ré, C. (2021) · doi:10.18653/v1/2021.naacl-demos.6 · Cited by 109 · see citing works
Robustness to Spurious Correlations via Human AnnotationsPreprint
Srivastava, M., Hashimoto, T. B., & Liang, P. (2020) · International Conference on Machine Learning · doi:10.48550/arxiv.2007.06661 · Cited by 105 · see citing works
The Risks of Invariant Risk MinimizationPreprint
Rosenfeld, E., Ravikumar, P., & Risteski, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05761 · Cited by 103 · see citing works
Beat the ai: Investigating adversarial human annotation for reading comprehension
Bartolo, M., Roberts, A., Welbl, J., Riedel, S., & Stenetorp, P. (2020) · UCL Discovery (University College London) · Cited by 102 · see citing works
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations
Tan, S., Joty, S., Kan, M., & Socher, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.263 · Cited by 101 · see citing works
Towards Debiasing NLU Models from Unknown Biases
Utama, P. A., Moosavi, N. S., & Gurevych, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.613 · Cited by 100 · see citing works
Social Bias in Elicited Natural Language Inferences
Rudinger, R., May, C., & Durme, B. V. (2017) · doi:10.18653/v1/w17-1609 · Cited by 98 · see citing works
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation
Karpukhin, V., Levy, O., Eisenstein, J., & Ghazvininejad, M. (2019) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/d19-5506 · Cited by 98 · see citing works
Reevaluating Adversarial Examples in Natural Language
Morris, J. X., Lifland, E., Lanchantin, J., Ji, Y., & Qi, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.341 · Cited by 96 · see citing works
Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance
Utama, P. A., Moosavi, N. S., & Gurevych, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.770 · Cited by 94 · see citing works
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets
Liu, N. F., Schwartz, R., & Smith, N. A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1225 · Cited by 88 · see citing works
(2019) · doi:10.18653/v1/d19-1432 · Cited by 88 · see citing works
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
Wang, B., Xu, C., Wang, S., Gan, Z., Cheng, Y., Gao, J., Awadallah, A. H., & Li, B. (2021) · arXiv · doi:10.48550/arxiv.2111.02840 · Cited by 88 · see citing works
Discrete Adversarial Attacks and Submodular Optimization with Applications to Text ClassificationPreprint
Lei, Q., Wu, L., Chen, P., Dimakis, A. G., Dhillon, I. S., & Witbrock, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00151 · Cited by 83 · see citing works
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Chen, S., Zhang, F., Sone, K., & Roth, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.475 · Cited by 83 · see citing works
Wang, Z., & Culotta, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17651 · Cited by 81 · see citing works
Oral Language Expectations for African American Preschoolers and Kindergartners
Craig, H. K., & Washington, J. A. (2002) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2002/007 · Cited by 80 · see citing works
InfoBERT: Improving Robustness of Language Models from An Information Theoretic PerspectivePreprint
Wang, B., Wang, S., Cheng, Y., Gan, Z., Jia, R., Li, B., & Liu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02329 · Cited by 78 · see citing works
The Effect of Natural Distribution Shift on Question Answering Models
Miller, J., Krauth, K., Recht, B., & Schmidt, L. (2020) · arXiv · doi:10.48550/arxiv.2004.14444 · Cited by 74 · see citing works
Adversarial Examples for Natural Language Classification Problems
Kuleshov, V., Thakoor, S., Lau, T., & Ermon, S. (2018) · Cited by 72 · see citing works
Exploring misclassifications of robust neural networks to enhance adversarial attacks
Schwinn, L., Raab, R., Nguyen, A., Zanca, D., & Eskofier, B. M. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04532-5 · Cited by 70 · see citing works
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation
Wang, T., Wang, X., Qin, Y., Packer, B., Li, K., Chen, J., Beutel, A., & Ed, H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.417 · Cited by 68 · see citing works
Self-citationIdentifying Spurious Correlations for Robust Text Classification
Wang, Z., & Culotta, A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.308 · Cited by 64 · see citing works
Adversarial Domain Adaptation for Machine Reading Comprehension
Wang, H., Gan, Z., Liu, X., Liu, J., Gao, J., & Wang, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1254 · Cited by 60 · see citing works
Self-citationNatural Language Adversarial Attacks and Defenses in Word LevelPreprint
Wang, X., Jin, H., & He, K. (2019) · arXiv (Cornell University) · Cited by 60 · see citing works
Self-citationMore Bang for Your Buck: Natural Perturbation for Robust Question Answering
Khashabi, D., Khot, T., & Sabharwal, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.12 · Cited by 60 · see citing works
Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints
Wang, Z., Wang, X., An, B., Yu, D., & Chen, C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.101 · Cited by 59 · see citing works
Self-citationImproving Robustness of Machine Translation with Synthetic Noise
Vaibhav, V., Singh, S., Stewart, C., & Neubig, G. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1190 · Cited by 57 · see citing works
Learning from Failure: Training Debiased Classifier from Biased ClassifierPreprint
Nam, J., Cha, H., Ahn, S. S., Lee, J., & Shin, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02561 · Cited by 57 · see citing works
Increasing Robustness to Spurious Correlations using Forgettable Examples
Yaghoobzadeh, Y., Mehri, S., Combes, R. T. D., Hazen, T. J., & Sordoni, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.291 · Cited by 53 · see citing works
Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress TestsPreprint
Veitch, V., D’Amour, A., Yadlowsky, S., & Eisenstein, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.00545 · Cited by 52 · see citing works
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models
Wang, T., Sridhar, R., Yang, D., & Wang, X. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.130 · Cited by 52 · see citing works
Self-citationUniversal Adversarial Attacks with Natural Triggers for Text Classification
Song, L., Yu, X., Peng, H.-T., & Narasimhan, K. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.291 · Cited by 49 · see citing works
Equalizing Gender Biases in Neural Machine Translation with Word Embeddings TechniquesPreprint
Font, J. E., & Costa‐jussà, M. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.03116 · Cited by 45 · see citing works
Tailor: Generating and Perturbing Text with Semantic Controls
Ross, A., Wu, T., Peng, H. M., Peters, M., & Gardner, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.228 · Cited by 41 · see citing works
Lin, B. Y., Gao, W., Yan, J., Moreno, R., & Ren, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.302 · Cited by 39 · see citing works
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
Tan, S., Joty, S., Varshney, L. R., & Kan, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.455 · Cited by 35 · see citing works
An Investigation of the (In)effectiveness of Counterfactually Augmented Data
Joshi, N., & He, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.256 · Cited by 34 · see citing works
Multi-granularity Textual Adversarial Attack with Behavior Cloning
Chen, Y., Su, J., & Wei, W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.371 · Cited by 33 · see citing works
End-to-End Bias Mitigation by Modelling Biases in Corpora
Mahabadi, R. K., Belinkov, Y., & Henderson, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.769 · Cited by 30 · see citing works
Huang, W., Liu, H., & Bowman, S. R. (2020) · doi:10.18653/v1/2020.insights-1.13 · Cited by 29 · see citing works
Learning to Recognize Dialect Features
Demszky, D., Sharma, D., Clark, J. H., Prabhakaran, V., & Eisenstein, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.184 · Cited by 29 · see citing works
Neural Machine Translation of Text from Non-Native Speakers
Anastasopoulos, A., Lui, A., Nguyen, T. Q., & Chiang, D. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1311 · Cited by 27 · see citing works
Earlier Isn’t Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization
Jung, T., Kang, D., Mentch, L., & Hovy, E. H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1327 · Cited by 27 · see citing works
Reliability Testing for Natural Language Processing Systems
Tan, S., Joty, S., Baxter, K., Taeihagh, A., Bennett, G. A., & Kan, M. (2021) · doi:10.18653/v1/2021.acl-long.321 · Cited by 27 · see citing works
Robustness to Modification with Shared Words in Paraphrase Identification
Shi, Z., & Huang, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.16 · Cited by 26 · see citing works
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
Tan, S., & Joty, S. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.282 · Cited by 23 · see citing works
SeqAttack: On Adversarial Attacks for Named Entity Recognition
Simoncini, W., & Spanakis, G. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.35 · Cited by 23 · see citing works
Launch and Iterate: Reducing Prediction Churn
Fard, M. M., Cormier, Q., Canini, K. R., & Gupta, M. R. (2016) · neural information processing systems · Cited by 22 · see citing works
Cross Lingual Transfer Learning for Zero-Resource Domain Adaptation
Abad, A., Bell, P., Carmantini, A., & Renals, S. J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054468 · Cited by 20 · see citing works
An Empirical Study of Invariant Risk MinimizationPreprint
Choe, Y. J., Ham, J., & Park, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05007 · Cited by 20 · see citing works
Unlearn Dataset Bias in Natural Language Inference by Fitting the ResidualPreprint
He, H., Zha, S., & Wang, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10763 · Cited by 19 · see citing works
Self-citationIdentifying and Mitigating Spurious Correlations for Improving Robustness in NLP ModelsPreprint
Wang, T., Sridhar, R., Yang, D., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07736 · Cited by 17 · see citing works
Self-citationDoes Data Augmentation Improve Generalization in NLP?Preprint
Jha, R., Lovering, C., & Pavlick, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.15012 · Cited by 12 · see citing works
Robust Semantic Parsing with Adversarial Learning for Domain Generalization
Marzinotto, G., Damnati, G., Béchet, F., & Favre, B. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-2021 · Cited by 12 · see citing works
Uncertainty Calibration for Ensemble-Based Debiasing MethodsPreprint
Xiong, R., Chen, Y., Pang, L., Cheng, X., & Lan, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04104 · Cited by 12 · see citing works
A robust parser based on syntactic information
Lee, K., Kweon, C.-J., Seo, J., & Kim, G. C. (1995) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/976973.977005 · Cited by 11 · see citing works
Domain Adversarial Fine-Tuning as an Effective Regularizer
Vernikos, G., Margatina, K., Chronopoulou, A., & Androutsopoulos, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.278 · Cited by 11 · see citing works
On Robustness of Neural Semantic Parsers
Huang, S., Li, Z., Qu, L., & Pan, L. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.292 · Cited by 11 · see citing works
Wang, H., Wu, Z., & Xing, E. P. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/442442 · Cited by 10 · see citing works
Self-citationCan We Improve Model Robustness through Secondary Attribute Counterfactuals?
Balashankar, A., Wang, X., Packer, B., Thain, N., Ed, H., & Beutel, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.386 · Cited by 10 · see citing works
Self-citationExploring Misclassifications of Robust Neural Networks to Enhance Adversarial AttacksPreprint
Schwinn, L., Raab, R., Nguyen, A., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10304 · Cited by 9 · see citing works
Word Shape Matters: Robust Machine Translation with Visual EmbeddingPreprint
Wang, H., Zhang, P., & Xing, E. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09997 · Cited by 7 · see citing works
Self-citationSemantics Altering Modifications for Evaluating Comprehension in Machine Reading
Schlegel, V., Nenadić, G., & Batista-Navarro, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17622 · Cited by 7 · see citing works
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned FeaturesPreprint
Wang, H., Huang, Z., Zhang, H., Lee, Y. J., & Xing, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03740 · Cited by 7 · see citing works
Self-citationIRM---when it works and when it doesn't: A test case of natural language inference
Dranker, Y., He, H., & Belinkov, Y. (2021) · neural information processing systems · Cited by 7 · see citing works
Unsupervised Domain Adaptation for Cross-lingual Text Labeling.
Zhang, D., Nallapati, R. M., Zhu, H., Feng, N., dos Santos, C. N., McKeown, K. R., & Xiang, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 16
- Faruk Ahmed, Yoshua Bengio, Harm van Seijen, and Aaron Courville. 2020. Systematic generalisation with group invariant predictions. In International Conference on Learning Representations.
- Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1613-1622, Lille, France.
- PMLR. N. Boucher, I. Shumailov, R. Anderson, and N. Pa- pernot. 2022. Bad characters: Imperceptible nlp attacks. In 2022 2022 IEEE Symposium on Secu- rity and Privacy (SP), pages 773-790, Los Alamitos, CA, USA. IEEE Computer Society. · DOI
- Ronan Le Bras, Swabha Swayamdipta, Chandra Bhaga- vatula, Rowan Zellers, Matthew Peters, Ashish Sab- harwal, and Yejin Choi. 2020. Adversarial filters of dataset biases. In Proceedings of the 37th Inter- national Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1078-1088. PMLR.
- Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. 2021a. Hiddencut: Simple data augmentation for natural language understanding with better gener- alizability. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Nat- ural Language Processing (Volume 1: Long Papers), pages 4380-4390.
- Christopher Clark, Mark Yatskar, and Luke Zettle- moyer. 2020. Learning to model and ignore dataset bias with mixed capacity ensembles. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 3031-3045.
- Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. 2021. Natural adver- sarial examples. In CVPR.
- Durk P Kingma, Tim Salimans, and Max Welling. 2015. Variational dropout and the local reparame- terization trick. In Advances in Neural Information Processing Systems, volume 28.
- Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. 2021. Just train twice: Im- proving group robustness without training group in- formation. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 6781-6792. PMLR.
- PMLR.
- Yonatan Oren, Shiori Sagawa, Tatsunori B Hashimoto, and Percy Liang. 2019. Distributionally robust lan- guage modeling. In EMNLP/IJCNLP (1).
- Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. 2020a. Distributionally robust neu- ral networks. In International Conference on Learn- ing Representations.
- Victor Veitch, Alexander D'Amour, Steve Yadlowsky, and Jacob Eisenstein. 2021. Counterfactual invari- ance to spurious correlations in text classification. In Advances in Neural Information Processing Sys- tems.
- Giorgos Vernikos, Katerina Margatina, Alexandra Chronopoulou, and Ion Androutsopoulos. 2020. Do- main adversarial fine-tuning as an effective regular- izer. CoRR, abs/2009.13366. · arXiv
- Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, and Kai-Wei Chang. 2021. On the transferability of adversarial attacks against neural text classifier. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 1612-1625, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
- Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Thomas Goldstein, and Jingjing Liu. 2020. Freelb: En- hanced adversarial training for language understand- ing. In ICLR.
Cited by
Sorted by citations
Javed, H., El–Sappagh, S., & Abuhmed, T. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11005-9 · Cited by 205 · see citing works
Safeguarding large language models: a survey
Dong, Y., Mu, R., Zhang, Y., Sun, S., Zhang, T., Wu, C., Jin, G., Qi, Y., Hu, J., Meng, J., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11389-2 · Cited by 161 · see citing works
State of What Art? A Call for Multi-Prompt LLM Evaluation
Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., & Stanovsky, G. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00681 · Cited by 113 · see citing works
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate
Wang, B., Yue, X., & Sun, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.795 · Cited by 71 · see citing works
Advances in Pruning and Quantization for Natural Language Processing
Bibi, U., Mazhar, M., Sabir, D., Butt, M. F. U., Hassan, A., Ghazanfar, M. A., Khan, A. A., & Abdul, W. (2024) · IEEE Access · doi:10.1109/access.2024.3465631 · Cited by 47 · see citing works
Shi, S., Hu, K., Xie, J., Guo, Y., & Wu, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103531 · Cited by 42 · see citing works
Unsupervised Concept Drift Detection From Deep Learning Representations in Real-Time
Greco, S., Vacchetti, B., Apiletti, D., & Cerquitelli, T. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3593123 · Cited by 49 · see citing works
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38 · see citing works
Romberg, J., & Escher, T. (2023) · Digital Government Research and Practice · doi:10.1145/3603254 · Cited by 43 · see citing works
Yang, L., Zhang, S., Qin, L., Li, Y., Wang, Y., Liu, H., Wang, J., Xie, X., & Zhang, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.806 · Cited by 43 · see citing works
A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures
Baughan, A., Wang, X., Liu, A., Mercurio, A., Chen, J., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581152 · Cited by 32 · see citing works
“NLP robustness can be defned as understanding how model performance changes when testing on a new dataset, which has a diferent distribution from the dataset the model is trained on [62].”
BackgroundSelf-citationKenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36 · see citing works
Can language representation models think in bets?
Tang, Z., & Kejriwal, M. (2023) · Royal Society Open Science · doi:10.1098/rsos.221585 · Cited by 21 · see citing works
Specification overfitting in artificial intelligence
Roth, B., de Araujo, P. H. L., Xia, Y., Kaltenbrunner, S., & Korab, C. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11040-6 · Cited by 21 · see citing works
Robust Recommender System: A Survey and Future Directions
Zhang, K., Cao, Q., Sun, F., Wu, Y., Tao, S., Shen, H., & Cheng, X. (2025) · ACM Computing Surveys · doi:10.1145/3757057 · Cited by 40 · see citing works
Evolutionary Multi-objective Optimization for Contextual Adversarial Example Generation
Zhou, S., Huang, M., Sun, Y., & Li, K. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660808 · Cited by 17 · see citing works
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs
Wu, T., Zhu, H., Albayrak, M., Axon, A., Bertsch, A., Deng, W., Ding, Z., Guo, B., Gururaja, S., Kuo, T.-S., et al. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706690 · Cited by 38 · see citing works
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation
Dixit, T., Paranjape, B., Hajishirzi, H., & Zettlemoyer, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.216 · Cited by 20 · see citing works
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future
Yang, L., Song, Y., Ren, X., Lyu, C., Wang, Y., Zhuo, J., Liu, L., Wang, J., Foster, J., & Zhang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.276 · Cited by 16 · see citing works
From text to multimodal: a survey of adversarial example generation in question answering systems
Yiğit, G., & Amasyalı, M. F. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02199-z · Cited by 11 · see citing works
RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations
Zhao, Y., Zhao, C., Nan, L., Qi, Z., Zhang, W., Tang, X., Mi, B., & Radev, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.334 · Cited by 13 · see citing works
Jiang, S., Wu, Z., Yang, H., Xiang, K., Ding, W., & Chen, Z. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120705 · Cited by 10 · see citing works
Towards Generalizable and Robust Text-to-SQL Parsing
Gao, C., Li, B., Zhang, W., Lam, W. P., Li, B., Huang, F., Si, L., & Li, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.155 · Cited by 9 · see citing works
Don’t Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text
Gupta, A., Blum, C., Choji, T., Fei, Y., Shah, S., Vempala, A., & Srikumar, V. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.781 · Cited by 10 · see citing works
Robustness of Demonstration-based Learning Under Limited Data Scenario
Zhang, H., Zhang, Y., Zhang, R., & Yang, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.116 · Cited by 10 · see citing works
Self-citationLEAP: Efficient and Automated Test Method for NLP Software
Xiao, M., Xiao, Y., Dong, H., Ji, S., & Zhang, P. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00052 · Cited by 8 · see citing works
Robustness of models addressing Information Disorder: A comprehensive review and benchmarking study
Fenza, G., Loia, V., Stanzione, C., & Gisi, M. D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127951 · Cited by 8 · see citing works
SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
Nagireddy, M., Chiazor, L., Singh, M., & Baldini, I. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30142 · Cited by 8 · see citing works
Laurer, M., van Atteveldt, W., Casas, A., & Welbers, K. (2024) · Communication Methods and Measures · doi:10.1080/19312458.2024.2378690 · Cited by 9 · see citing works
TextJuggler: Fooling text classification tasks by generating high-quality adversarial examples
Peng, H., Wang, Z., Wei, C., Zhao, D., Xu, G., Han, J., Guo, S., Zhong, M. J., & Ji, S. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112188 · Cited by 7 · see citing works
Efficient text-based evolution algorithm to hard-label adversarial attacks on text
Peng, H., Wang, Z., Zhao, D., Wu, Y., Han, J., Guo, S., Ji, S., & Zhong, M. J. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.03.017 · Cited by 7 · see citing works
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Hagström, L., Saynova, D., Norlund, T., Johansson, M., & Johansson, R. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.332 · Cited by 6 · see citing works
NLP verification: towards a general methodology for certifying robustness
Casadio, M., Dinkar, T., Komendantskaya, E., Arnaboldi, L., Daggitt, M. L., Isac, O., Katz, G., Rieser, V., & Lemon, O. (2025) · European Journal of Applied Mathematics · doi:10.1017/s0956792525000099 · Cited by 5 · see citing works
Unsupervised Dialogue Topic Segmentation in Hyperdimensional Space
Park, S., Seo, J., & Lee, J. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1859 · Cited by 6 · see citing works
ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verification
Casadio, M., Arnaboldi, L., Daggitt, M. L., Isac, O., Dinkar, T., Kienitz, D., Rieser, V., & Komendantskaya, E. (2023) · Kalpa publications in computing · doi:10.29007/7wxb · Cited by 5 · see citing works
Daily activity-travel pattern identification using natural language processing and semantic matching
Nayak, S., & Pandit, D. (2024) · Journal of Transport Geography · doi:10.1016/j.jtrangeo.2024.104057 · Cited by 5 · see citing works
Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering
Ho, X., Nguyen, A.-K. D., Sugawara, S., & Aizawa, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.87 · Cited by 5 · see citing works
Wang, X., Lu, Z., Wang, X., He, M., & Wang, X. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3400825 · Cited by 5 · see citing works
Self-citationTAPE: Assessing Few-shot Russian Language Understanding
Taktasheva, E., Shavrina, T., Fenogenova, A., Shevelev, D., Katricheva, N., Tikhonova, M., Akhmetgareeva, A., Zinkevich, O., Bashmakova, A., Iordanskaia, S., et al. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.183 · Cited by 10 · see citing works
Li, D., Hu, B., Chen, Q., & He, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.1 · Cited by 5 · see citing works
Evaluating the adversarial robustness of Arabic spam classifiers
Alajmi, A., Ahmad, I., & Mohammed, A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10778-y · Cited by 4 · see citing works
Robust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study
Ye, H., Ding, Y., Li, J., & Ng, H. T. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.460 · Cited by 4 · see citing works
RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation
Zhang, Y., Cui, L., Zhao, E., Bi, W., & Shi, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1043 · Cited by 4 · see citing works
Implications of Minimum Description Length for Adversarial Attack in Natural Language Processing
Tiwari, K., & Zhang, L. (2024) · Entropy · doi:10.3390/e26050354 · Cited by 4 · see citing works
Bug report priority prediction using social and technical features
Huang, Z., Shao, Z., Fan, G., Yu, H., Yang, K., & Zhou, Z. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2616 · Cited by 3 · see citing works
FATRER: Full-Attention Topic Regularizer for Accurate and Robust Conversational Emotion Recognition
Mao, Y., Lu, D., Zhang, Y., & Wang, X. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230453 · Cited by 3 · see citing works
Self-citationMao, T., Yoshie, O., Fu, J., & Mao, W.-X. (2023) · Neural computing & applications (Print) · doi:10.1007/s00521-023-09250-0 · Cited by 4 · see citing works
Zhang, H., Wang, J., Gao, H., Zhang, X., Wang, H., & Li, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128917 · Cited by 3 · see citing works
Self-citationCascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models
Malaviya, C., Bhatia, S., & Yatskar, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.438 · Cited by 3 · see citing works
Chen, N., Sun, Q., Wang, J., Gao, M. Y., Li, X., & Li, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.991 · Cited by 3 · see citing works
STTA: enhanced text classification via selective test-time augmentation
Xiong, H., Zhang, X., Yang, L., Xiang, Y., & Zhang, Y. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1757 · Cited by 2 · see citing works
Dingeto, H., & Kim, J. (2024) · Electronics · doi:10.3390/electronics13132534 · Cited by 2 · see citing works
Diacritical Manipulations as Adversarial Attacks in Arabic NLP Systems
Alshemali, B. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-025-10266-y · Cited by 2 · see citing works
Caselli, T., & Bos, J. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09643-6 · Cited by 2 · see citing works
The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models
Tran, S. Q., Nguyen-Thuan, P., Le, U., & Kretchmar, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.113 · Cited by 3 · see citing works
Beware of Model Collapse! Fast and Stable Test-time Adaptation for Robust Question Answering
Su, Y., Ji, Y., Li, J., Ye, H., & Zhang, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.803 · Cited by 2 · see citing works
Robust Self-Supervised Learning with Contrast Samples for Natural Language Understanding
Liu, J., Han, X., Deng, C., & Feng, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448238 · Cited by 3 · see citing works
Ansari, S. A., Wajid, M. A., Arif, M., & Wajid, M. S. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131450 · Cited by 2 · see citing works
Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answering
Ye, H., Xie, Q., & Ng, H. T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.537 · Cited by 4 · see citing works
When and Why Does Bias Mitigation Work?
Ravichander, A., Stacey, J., & Rei, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.619 · Cited by 1 · see citing works
Salido, E. S., Gonzalo, J. A., & Marco, G. (2026) · IEEE Access · doi:10.1109/access.2026.3651579 · Cited by 7 · see citing works
Gupta, A. P. D. K., Mittal, A., & Jain, R. (2025) · Intelligent Decision Technologies · doi:10.1177/18724981251384966 · Cited by 1 · see citing works
Future-Proofing Deepfake Detection by Integrating Audio, Video, and Text
Lisena, P., Lee, K. A., Wang, Y., Todisco, M., Galdi, C., Raphaël Troncy, A., Evans, N., Lin, W., Chau, L., & Mak, M. (2026) · ACM AI Letters · doi:10.1145/3797958 · Cited by 1 · see citing works
Causal Inference and Natural Language Processing
Chen, W., & Chu, Z. (2023) · Springer eBooks · doi:10.1007/978-3-031-35051-1_9 · Cited by 1 · see citing works
Effects of Human Adversarial and Affable Samples on BERT Generalizability
Elangovan, A., He, E., Li, Y., & Verspoor, K. M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.512 · Cited by 1 · see citing works
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Sugawara, S., & Tsugita, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.861 · Cited by 1 · see citing works
Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting
Feng, K., Tang, Z., Li, J., & Zhang, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_53 · Cited by 1 · see citing works
Enriching large language models with semantic lexicons and analogies
Zervakis, G. (2023) · theses.fr (ABES) · doi:10.70675/4f4cd0e5z5987z422az813bzc6e431d1a918 · Cited by 4 · see citing works
Chen, Y., Zhang, C., Luo, D., D'Haro, L. F., Tan, R. T., & Li, H. (2024) · arXiv · doi:10.48550/arxiv.2405.14646
Robust NLP for Finance (RobustFin)
Shah, S., Zhu, X., de Melo, G., Nourbakhsh, A., Liu, X., Ma, Z., Smiley, C., & Chen, Z. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599211
Making Pre-trained Language Models both Task-solvers and Self-calibrators
Chen, Y., Wang, X., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.624
Self-citationProbabilistic Robustness for Data Filtering
Yu, Y., Khan, A. R., Khadivi, S., & Xu, J. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.214
Robustness Tests for Automatic Machine Translation Metrics with Adversarial Attacks
Huang, Y., & Baldwin, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.340
Examining the Role of Natural Language Processing in Generating Topics from Web Content
Altarturi, H., Saadoon, M., & Anuar, N. B. (2023) · IEEE International Conference on Computing (ICOCO) · doi:10.1109/icoco59262.2023.10398059
A Simple and Effective Test-Time Augmentation Strategy
Zhang, X., & Xiang, Y. (2023) · International Conference on Artificial Intelligence and Intelligent Information Processing (AIIIP) · doi:10.1109/aiiip61647.2023.00040
Are Machine Reading Comprehension Systems Robust to Context Paraphrasing?
Wu, Y., Schlegel, V., & Batista-Navarro, R. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-short.21
Processamento de linguagem natural aplicado às anotações elaboradas com o DLNotes
da Silva, L. G. S., & Santos, E. R. (2024) · ABRALIN · doi:10.25189/rabralin.v23i2.2199
AI-Q: a Framework to Measure the Robustness of Text Classification Models
Mysliwiec, K., Chinea-Ríos, M., Borrego-Obrador, I., & Franco-Salvador, M. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.468
Adversarially Exploring Vulnerabilities in LLMs to Evaluate Social Biases
Ong, Y. J., Gala, J. P., An, S., Moore, R. J., & Jadav, D. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825408
Prompt-Based Out-of-Distribution Intent Detection
Chow, R., & Lam, A. Y. S. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3372440
Neural Quantum Embedding Enhanced Hybrid Quantum-Classical Sentiment Analysis ClassificationPreprint
Djemmal, I., Belhadef, H., & Mutawa, A. M. (2025) · Research Square · doi:10.21203/rs.3.rs-8197450/v1
Specifying Safety Requirements for Machine Learning Components in Autonomous Systems: A Survey
Hawkins, R., & Shahbeigi, S. (2025) · Safety-Critical Systems Symposium (SSS) · doi:10.65391/r3097
Zhang, Z., Fu, J., Ge, B., Li, S., Luo, Q., Gao, Q., Zha, H., & Lin, Y. (2026) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2026.3667731
Liu, Y., Liu, X., Huang, H., Yu, F., Xiong, T., Xia, X., You, B., & Wu, J. (2026) · Artificial Intelligence and Law · doi:10.1007/s10506-026-09510-z
Ming, S., Menke, J. D., Smalheiser, N. R., & Kilicoglu, H. (2026) · IEEE International Conference on Healthcare Informatics · doi:10.1109/ichi69079.2026.00024 · Cited by 1 · see citing works
Robustness of Prompting: Enhancing Robustness of Large Language Models Against Prompt Attacks
Mu, L., Chu, G., Li, N., Sang, L., & Zhang, Y. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3697684 · Cited by 1 · see citing works
Robustness Gap in NLP Models for Vulnerability Descriptions: Benchmarking and Data Augmentation
Majd, A., Yousefikia, M., Ghasemzadeh, S., Asari, A., Khoshnavataher, A., & Mirtaheri, S. L. (2025) · International Conference on Information and Knowledge Technology (IKT) · doi:10.1109/ikt69652.2025.11626735
Lu, J., Xu, W., Song, Y., Zhang, H., Zhang, C., Wu, K., Wei, X., & Wong, R. C.-W. (2026) · The VLDB Journal · doi:10.1007/s00778-026-00992-6
Natural Language Processing for Market Sentiment Analysis
Gupta, D. K., Padmavathy, C., Dholariya, S., Mandal, L., & Dutta, R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8167-1.ch010
Andreas, K. M., Clement, K., & Puspita, R. (2026) · International Conference on Smart Data, Intelligence, and Analytics (ICoSDIA) · doi:10.1109/icosdia70683.2026.11690457
Enhancing Large Language Model Accuracy with Retrieval-Augmented Generation: A Comparative Study
Bhattacharya, T., Mohammad, I. A., Das, A. K., Reddy, J. S., & Meka, H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28469-3_30
- Authors
- Wang, X., Wang, H., & Yang, D.
- Affiliations
- Google (United States) (US)
- Carnegie Mellon University (US)
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 4569-4586
- OpenAlex
- W4226040778
- Cited by
- 90
- References
- 162
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 22 · 2023: 40 · 2022: 6
- FWCI
- 11.06
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- robustness, robustness metrics, robustness analysis, adversarial robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
