Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
The DOI matches its Crossref record; 237 of the 237 references it cites were found in a registry or an index.
Abstract
Deep learning methods employ multiple processing layers to learn hierarchical representations of data, and have produced state-of-the-art results in many domains. Recently, a variety of model designs and methods have blossomed in the context of natural language processing (NLP). In this paper, we review significant deep learning related models and methods that have been employed for numerous NLP tasks and provide a walk-through of their evolution. We also summarize, compare and contrast the various models and put forward a detailed understanding of the past, present and future of deep learning in NLP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The most recently developed DL techniques have obtained good outstanding performance across a variety of applications, including audio and speech processing, visual data processing, natural language processing (NLP), among others [11–14].”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–237 of 237
Structured Training for Neural Network Transition-Based ParsingPreprint
Weiss, D., Alberti, C., Collins, M., & Petrov, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06158 · Cited by 75
Combating Human Trafficking with Multimodal Deep Models
Tong, E., Zadeh, A., Jones, C., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1142 · Cited by 63
Su, P.-H., Vandyke, D., Gašić, M., Kim, D., Mrkšić, N., Wen, T.-H., & Young, S. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-456 · Cited by 63
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J., Gao, J., & Dolan, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06714 · Cited by 62
Lucy, L., & Gauthier, J. (2017) · Proceedings of the First Workshop on Language Grounding for Robotics · doi:10.18653/v1/w17-2810 · Cited by 59
Lexicon Infused Phrase Embeddings for Named Entity Resolution
Passos, A., Kumar, V., & McCallum, A. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1609 · Cited by 59
Modeling Relational Information in Question-Answer Pairs with Convolutional Neural NetworksPreprint
Severyn, A., & Moschitti, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01178 · Cited by 58
Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
Dodge, J., Gane, A., Zhang, X., Bordes, A., Chopra, S., Miller, A., Szlam, A. D., & Weston, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06931 · Cited by 46
Fast and Accurate Sequence Labeling with Iterated Dilated Convolutions
Strubell, E., Verga, P., Belanger, D., & Mccallum, A. (2017) · arXiv.org · doi:10.48550/arxiv.1702.02098 · Cited by 42
Fast and accurate part-of-speech tagging
Giménez, J., & Màrquez, L. (2004) · Amsterdam studies in the theory and history of linguistic science. Series 4, Current issues in linguistic theory · doi:10.1075/cilt.260.17gim · Cited by 41
Stochastic Answer Networks for Machine Reading ComprehensionPreprint
Liu, X., Shen, Y., Duh, K., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03556 · Cited by 34
Recursive Neural Networks for Learning Logical Semantics
Bowman, S. R., Potts, C., & Manning, C. D. (2014) · arXiv.org · doi:10.48550/arxiv.1406.1827 · Cited by 33
Fast and Accurate Sequence Labeling with Iterated Dilated Convolutions.Preprint
Strubell, E., Verga, P. W., Belanger, D., & McCallum, A. (2017) · arXiv (Cornell University) · Cited by 30
Radical-Based Hierarchical Embeddings for Chinese Sentiment Analysis at Sentence Level.
Peng, H., Cambria, E., & Zou, X. (2017) · The Florida AI Research Society · Cited by 30
Self-citationContext-Dependent Translation Selection Using Convolutional Neural Network
Hu, B., Tu, Z., Lu, Z., Li, H., & Chen, Q. (2015) · doi:10.3115/v1/p15-2088 · Cited by 26
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Upadhyay, S., Chang, K.-W., Taddy, M. A., Kalai, A. T., & Zou, J. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2613 · Cited by 21
Efficient Likelihood Learning of a Generic CNN-CRF Model for Semantic Segmentation.Preprint
Kirillov, A., Schlesinger, D., Forkel, W., Zelenin, A., Zheng, S., Torr, P. H. S., & Rother, C. (2015) · arXiv (Cornell University) · Cited by 15
Lucy, L., & Gauthier, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.11168 · Cited by 14
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual ContextPreprint
Upadhyay, S., Chang, K.-W., Taddy, M. A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08160 · Cited by 11
Context-Dependent Translation Selection Using Convolutional Neural NetworkPreprint
Tu, Z., Hu, B., Lu, Z., & Li, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.02357 · Cited by 9
A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing
Zhou, H., Zhang, Y., Cheng, C., Huang, S., Dai, X., & Chen, J. (2017) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5259 · Cited by 8
Boosting Named Entity Recognition with Neural Character EmbeddingsPreprint
dos Santos, C. N., & Guimarães, V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05008 · Cited by 7
Joint Training of Generic CNN-CRF Models with Stochastic Optimization
Kirillov, A., Schlesinger, D., Zheng, S., Savchynskyy, B., Torr, P. H. S., & Rother, C. (2015) · arXiv · doi:10.48550/arxiv.1511.05067 · Cited by 6
(2015) · doi:10.4230/oasics.slate.2021.8 · Cited by 1
Weston, J., Chopra, S., & Bordes, A. (2014) · arXiv · doi:10.48550/arxiv.1410.3916
Generating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., et al. (2015) · arXiv · doi:10.48550/arxiv.1511.06349
Deep contextualized word representations
Peters, M. E., Neumann, M., Iyyer, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.05365
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · arXiv · doi:10.48550/arxiv.1509.00685
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · arXiv · doi:10.48550/arxiv.1503.00075
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I., & Pineau, J. (2015) · arXiv · doi:10.48550/arxiv.1506.08909
Globally Normalized Transition-Based Neural Networks
Andor, D., Alberti, C., Weiss, D., Severyn, A., Presta, A., Ganchev, K., Petrov, S., & Collins, M. (2016) · arXiv · doi:10.48550/arxiv.1603.06042
High-risk learning: acquiring new word vectors from tiny data
Herbelot, A., & Baroni, M. (2017) · arXiv · doi:10.48550/arxiv.1707.06556
Mimicking Word Embeddings using Subword RNNs
Pinter, Y., Guthrie, R., & Eisenstein, J. (2017) · arXiv · doi:10.48550/arxiv.1707.06961
INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis
Ruder, S., Ghaffari, P., & Breslin, J. G. (2016) · arXiv · doi:10.48550/arxiv.1609.02748
Aspect Level Sentiment Classification with Deep Memory Network
Tang, D., Qin, B., & Liu, T. (2016) · arXiv · doi:10.48550/arxiv.1605.08900
Lexicon Infused Phrase Embeddings for Named Entity Resolution
Passos, A., Kumar, V., & McCallum, A. (2014) · arXiv · doi:10.48550/arxiv.1404.5367
Named Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J. P. C., & Nichols, E. (2015) · arXiv · doi:10.48550/arxiv.1511.08308
Research works printed but not linked: 1
- zhang, 0, Generating text via adversarial training, Proc Neural Information Processing Systems Workshop Adversarial Training
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2018) · arXiv · doi:10.48550/arxiv.1809.02165 · Cited by 2,965
Machine learning and deep learning
Janiesch, C., Zschech, P., & Heinrich, K. (2021) · Electronic Markets · doi:10.1007/s12525-021-00475-2 · Cited by 2,959
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952
Metadata
- Authors
- Young, T., Hazarika, D., Poria, S., & Cambria, E.
- Affiliations
- Beijing Institute of Technology (CN)
- National University of Singapore (SG)
- Nanyang Technological University (SG)
- Venue
- IEEE Computational Intelligence Magazine
- Published
- 2018-08
- Volume
- 13
- Issue
- 3
- Pages
- 55-75
- ISSN
- 1556-603X (ISSN-L)
- 1556-6048
- OpenAlex
- W2884001105
- Cited by
- 2,921
- References
- 181
- Citations by year
- 2026: 132 · 2025: 264 · 2024: 350 · 2023: 425 · 2022: 527 · 2021: 484 · 2020: 452 · 2019: 260 · 2018: 27
- FWCI
- 169.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Computational Intelligence Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- deep learning, natural language processing, hierarchical representation, deep learning models, model comparison, NLP tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.