CoChat

Exploring the frontiers of deep learning and natural language processing: A comprehensive overview of key challenges and emerging trends

Open access
Khan, Wahab, Daud, Ali, Khan, Khairullah, Muhammad, Shakoor, Haq, Rafiul · 2023 · Natural Language Processing Journal
Cited by 217

The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.

Abstract

In the recent past, more than 5 years or so, DL especially the large language models (LLMs) has generated extensive studies out of a distinctly average downturn field of knowledge made up of a traditional society of researchers. As a result, DL (DL) is now so pervasive that its use is widespread across the body of research related to machine learning computing. The rapid emergence and apparent dominance of DL architectures over traditional machine learning techniques on a variety of tasks have been truly astonishing to witness. DL models outperformed in a variety of areas, including natural language processing (NLP), image analysis, language understanding, machine translation, computer vision, speech processing, audio recognition, style imitation, and computational biology. In this study, the aim is to explain the rudiments of DL, such as neural networks, convolutional neural networks, deep belief networks, and various variants of DL. The study will explore how these models have been applied to NLP and delve into the underlying mathematics behind them. Additionally, the study will investigate the latest advancements in DL and NLP, while acknowledging the key challenges and emerging trends in the field. Furthermore, it will discuss the core component of DL, namely embeddings, from a taxonomic perspective. Moreover, a literature review will be provided focusing on the application of DL models for six popular pattern recognition tasks: speech recognition, question answering, part of speech tagging, named entity recognition, text classification, and machine translation. Finally, the study will demystify state-of-the-art DL libraries/frameworks and available resources. The outcome and implication of this study reveal that LLMs face challenges in dealing with pragmatic aspects of language due to their reliance on statistical learning techniques and lack of genuine understanding of context, presupposition, implicature, and social norms. Furthermore, this study provides a comprehensive analysis of the current state-of-the-art advancements and highlights significant obstacles and emerging developments. The article has the potential to enhance readers’ understanding of the subject matter.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.nlp.2023.100026

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 217

Metadata

Authors
Khan, W., Daud, A., Khan, K., Muhammad, S., & Haq, R.
Venue
Natural Language Processing Journal
Publisher
Elsevier BV
Published
2023-09
Volume
4
Pages
100026
ISSN
2949-7191 (ISSN-L)
OpenAlex
W4385188297
Cited by
217
References
39
FWCI
19.06
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.