Improving the Reliability of Deep Neural Networks in NLP: A Review
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Abstract Deep learning models have achieved great success in solving a variety of natural language processing (NLP) problems. An ever-growing body of research, however, illustrates the vulnerability of deep neural networks (DNNs) to adversarial examples — inputs modified by introducing small perturbations to deliberately fool a target model into outputting incorrect results. The vulnerability to adversarial examples has become one of the main hurdles precluding neural network deployment into safety-critical environments. This paper discusses the contemporary usage of adversarial examples to foil DNNs and presents a comprehensive review of their use to improve the robustness of DNNs in NLP applications. In this paper, we summarize recent approaches for generating adversarial texts and propose a taxonomy to categorize them. We further review various types of defensive strategies against adversarial examples, explore their main challenges, and highlight some future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“DNNs are also the most popular option for challenging applications in speech recognition [7], voice synthesis [8], language translation and natural language processing (NLP) [9].”
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800 · cited 1×
“A PRN is trained to recover adversarial samples in such a way that the targeted model’s output becomes the same as it would be for the clean versions of the same samples.”
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“• Question Answering Corpora: The Stanford Question Answering Dataset (SQuAD) [25] has been created using 150,000 human-written questions on Wikipedia passages along with their answers.”
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 6×
“• Textual Entailment Corpora: The Stanford Natural Language Inference (SNLI) [28], Multi-Genre Natural Language Inference (MultiNLI), 10 and SciTail [29] are three preferred corpora used in textual entailment.”
InfluentialUniversal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · cited 1×
“Their method first attacks the model with adversarial source inputs, then defends the model with adversarial target inputs, aiming to improve its robustness against the adversarial source inputs.”
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 1×
“…sentence; (b) Negation: Append the tautology “and false is not true”, which contains a negation word “not” to the end of every hypothesis sentence; (c) Length mismatch: Append the tautology “and true is true” five times to the end of every premise sentence; [77], BiLSTM [78], and CBOW [79].”
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410 · cited 13×
“Jia and Liang [13] Belinkov and Bisk [40] Heigold et al.”
InfluentialDelving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310
chrF: character n-gram F-score for automatic MT evaluation
Popović, M. (2015) · Proceedings of the Tenth Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3049 · Cited by 1,220 · cited 1×
“They found that chrF [133], which is based on character n-gram F-score, correlated most with human judgment, followed by METEOR [134] and BLEU.”
Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., & Tygar, J. D. (2011) · Proceedings of the 4th ACM workshop on Security and artificial intelligence · doi:10.1145/2046684.2046692 · Cited by 1,194 · cited 1×
“AML also involves the study of effective designs for machine learning techniques that can resist adversarial samples [14].”
Visual7W: Grounded Question Answering in Images
Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.540 · Cited by 794 · cited 1×
“They also tested the model of Zhu et al.’s [74] using their dataset.”
Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
Gao, J., Lanchantin, J., Soffa, M. L., & Qi, Y. (2018) · 2018 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00016 · Cited by 693
Generating Natural Adversarial Examples
Zhao, Z., Dua, D., & Singh, S. (2017) · arXiv · doi:10.48550/arxiv.1710.11342 · Cited by 508 · cited 1×
“From a practical point of view, carefully designed imperceptible textual perturbations that are syntactically and semantically correct are needed.”
SoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502
Multi-layer Representation Learning for Medical Concepts
Choi, E., Bahadori, M. T., Searles, E., Coffey, C., Thompson, M., Bost, J., Tejedor-Sojo, J., & Sun, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939823 · Cited by 469 · cited 1×
“In addition, deep learning has been employed to learn effective representations for patient records [4] in order to conduct predictive modeling [5] and support disease phenotyping [6].”
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
Glockner, M., Shwartz, V., & Goldberg, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2103 · Cited by 319 · cited 2×
“[49] designed testing sets that showed the deficiency of natural language inference (NLI) models.”
Did the Model Understand the Question?
Mudrakarta, P. K., Taly, A., Sundararajan, M., & Dhamdhere, K. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1176 · Cited by 166 · cited 1×
“Mudrakarta et al. [56] analyzed the sensitivity of a reading-comprehension deep learning model.”
Digital watermarking for deep neural networks
Nagai, Y., Uchida, Y., Sakazawa, S., & Satoh, S. (2018) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-018-0147-1 · Cited by 152 · cited 1×
“Hitaj et al. [37] proposed stealing DNNs even with embedded watermarking techniques [39].”
Cited by
Natural language processing: state of the art, current trends and challenges
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13428-4 · Cited by 1,823
“(2020) [63] for detailed literature survey and technological challenges relevant to management research and NLP, and a recent review work by Alshemali and Kalita (2020) [3], and references cited there in.”
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
Jelodar, H., Wang, Y., Orji, R., & Huang, S. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3001216 · Cited by 400
Chow, J. C. L., & Li, K. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/64406 · Cited by 75
Creating a large language model of a philosopher
Schwitzgebel, E., Schwitzgebel, D., & Strasser, A. (2024) · Mind & Language · doi:10.1111/mila.12466 · Cited by 73
Metadata
- Authors
- Alshemali, B., & Kalita, J.
- Venue
- Knowledge-Based Systems
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 191
- Pages
- 105210
- ISSN
- 0950-7051 (ISSN-L)
- OpenAlex
- W2989473642
- Cited by
- 188
- References
- 32
- FWCI
- 6.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.