From Word Embeddings to Pre-Trained Language Models: A State-of-the-Art Walkthrough

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

Glove: Global Vectors for Word Representation

Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works

Distributed Representations of Words and Phrases and their Compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works

Distilling the Knowledge in a Neural Network

Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works

Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics

Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works

Indexing by latent semantic analysis

Deerwester, S. C., Dumais, S., Furnas, G. W., Landauer, T. K., & Harshman, R. A. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6 · Cited by 12,758 · see citing works

Deep Contextualized Word Representations

Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

Enriching Word Vectors with Subword Information

Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · see citing works

A Neural Probabilistic Language Model

Bengio, Y., Ducharme, R., Vincent, P., & Janvin, C. (2003) · Journal of machine learning research · doi:10.1162/153244303322533223 · Cited by 8,257 · see citing works

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works

Unsupervised Cross-lingual Representation Learning at Scale

Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637 · see citing works

A unified architecture for natural language processing: deep neural networks with multitask learning

Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · see citing works

Distributional Structure

Harris, Z. S. (1954) · <i>WORD</i> · doi:10.1080/00437956.1954.11659520 · Cited by 4,807 · see citing works

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · see citing works

Bag of Tricks for Efficient Text Classification

Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545 · see citing works

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works

Universal Language Model Fine-tuning for Text Classification

Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · see citing works

Energy and Policy Considerations for Deep Learning in NLP

Strubell, E., Ganesh, A., & McCallum, A. (2019) · arXiv · doi:10.48550/arxiv.1906.02243 · Cited by 3,735

Energy and Policy Considerations for Deep Learning in NLP

Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · see citing works

A Neural Probabilistic Language Model

Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451 · see citing works

Transformer-XL: Attentive language models beyond a fixed-length context

Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works

A neural probabilistic language model

BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,646 · see citing works

PaLM: Scaling Language Modeling with Pathways

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works

Longformer: The Long-Document Transformer

Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281 · see citing works

Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books

Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · see citing works

Model compression

Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · see citing works

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.10555 · Cited by 1,844 · see citing works

TinyBERT: Distilling BERT for natural language understanding

Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., & Liu, Q. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.372 · Cited by 1,786 · see citing works

Green AI

Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020) · Communications of the ACM · doi:10.1145/3381831 · Cited by 1,647 · see citing works

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.41 · Cited by 1,629 · see citing works

Cross-lingual Language Model Pretraining

Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works

Neural Word Embedding as Implicit Matrix Factorization

Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,581 · see citing works

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529 · see citing works

Reformer: The Efficient Transformer

Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · see citing works

GLaM: Efficient Scaling of Language Models with Mixture-of-ExpertsPreprint

Du, N., Huang, Y.-P., Dai, A. M., Tong, S., Lepikhin, D., Xu, Y.-Z., Krikun, M., Zhou, Y.-Q., Yu, A. W., Firat, O., et al. (2021) · International Conference on Machine Learning · doi:10.48550/arxiv.2112.06905 · Cited by 1,379 · see citing works

DeBERTa: Decoding-enhanced BERT with Disentangled AttentionPreprint

He, P., Liu, X., Gao, J., & Chen, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03654 · Cited by 1,224 · see citing works

Multi-Task Deep Neural Networks for Natural Language Understanding

Liu, X., He, P., Chen, W., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1441 · Cited by 1,216 · see citing works

PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

Zhang, J., Zhao, Y., Saleh, M., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1912.08777 · Cited by 1,206 · see citing works

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · UCL Discovery (University College London) · Cited by 1,187 · see citing works

Contextual String Embeddings for Sequence Labeling

Akbik, A., Blythe, D. A. J., & Vollgraf, R. (2018) · International Conference on Computational Linguistics · Cited by 1,084 · see citing works

Semi-supervised Sequence Learning

Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994 · see citing works

ERNIE: Enhanced Representation through Knowledge IntegrationPreprint

Sun, Y., Wang, S., Li, Y., Feng, S., Chen, X., Zhang, H., Tian, X., Zhu, D., Tian, H., & Wu, H. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09223 · Cited by 983 · see citing works

Training Compute-Optimal Large Language Models

Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929 · see citing works

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv · doi:10.48550/arxiv.1909.08053 · Cited by 893 · see citing works

Transformer-XL: Attentive Language Models Beyond a Fixed-Length ContextPreprint

Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.4230/lipics.ecrts.2023.7 · Cited by 860 · see citing works

mT5: A massively multilingual pre-trained text-to-text transformerPreprint

Xue, L., Constant, N., Roberts, A., Kale, M., Al‐Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11934 · Cited by 852 · see citing works

LaMDA: Language Models for Dialog Applications

Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · see citing works

Patient Knowledge Distillation for BERT Model Compression

Sun, S., Cheng, Y., Gan, Z., & Liu, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1441 · Cited by 758 · see citing works

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

Sun, Z., Yu, H., Song, X., Liu, R., Yang, Y., & Zhou, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.195 · Cited by 757 · see citing works

CTRL: A Conditional Transformer Language Model for Controllable Generation

Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works

MPNet: Masked and Permuted Pre-training for Language UnderstandingPreprint

Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09297 · Cited by 713 · see citing works

Using latent semantic analysis to improve access to textual information

Dumais, S., Furnas, G. W., Landauer, T. K., Deerwester, S. C., & Harshman, R. A. (1988) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/57167.57214 · Cited by 682 · see citing works

KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation

Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00360 · Cited by 676 · see citing works

Semi-supervised sequence tagging with bidirectional language models

Peters, M., Ammar, W., Bhagavatula, C., & Power, R. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1161 · Cited by 655 · see citing works

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works

TaPas: Weakly Supervised Table Parsing via Pre-training

Herzig, J., Nowak, P. K., Müller, T., Piccinno, F., & Eisenschlos, J. M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.398 · Cited by 589 · see citing works

REALM: Retrieval-Augmented Language Model Pre-Training

Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M.-W. (2020) · arXiv · doi:10.48550/arxiv.2002.08909 · Cited by 574 · see citing works

Learned in Translation: Contextualized Word Vectors

McCann, B., Bradbury, J., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1708.00107 · Cited by 572 · cited 2× · see citing works

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · see citing works

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., & Chen, Z. (2020) · arXiv · doi:10.48550/arxiv.2006.16668 · Cited by 412 · see citing works

GPT-NeoX-20B: An Open-Source Autoregressive Language Model

Black, S., Biderman, S., Hallahan, E., Anthony, Q. G., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.9 · Cited by 407 · see citing works

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.11990 · Cited by 286 · see citing works

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · see citing works

Multi-Task Deep Neural Networks for Natural Language UnderstandingPreprint

Liu, X., He, P., Chen, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11504 · Cited by 218 · see citing works

BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages

Heinzerling, B., & Strube, M. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3eqtu58tsqay · Cited by 196 · see citing works

A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data

Liu, D., Cheng, P., Dong, Z., He, X., Pan, W., & Ming, Z. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401083 · Cited by 195 · see citing works

Dataset distillation

Wang, T., Zhu, J.-Y., Torralba, A., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1811.10959 · Cited by 194 · see citing works

Compressing Large-Scale Transformer-Based Models: A Case Study on BERT

Ganesh, P., Chen, Y., Lou, X., Khan, M. A., Yang, Y., Sajjad, H., Nakov, P., Chen, D., & Winslett, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00413 · Cited by 190 · see citing works

Pathways: Asynchronous Distributed Dataflow for MLPreprint

Barham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R.-M., Roy, S., et al. (2022) · Conference on Machine Learning and Systems · doi:10.48550/arxiv.2203.12533 · Cited by 165 · see citing works

Few-shot Learning with Multilingual Generative Language Models

Lin, X. V., Mihaylov, T., Artetxe, M., Wang, T., Chen, S., Simig, D., Ott, M., Goyal, N., Bhosale, S., Du, J., et al. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.616 · Cited by 134 · see citing works

TinyBERT: Distilling BERT for Natural Language UnderstandingPreprint

Jiao, X., Yin, Y., Shang, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10351 · Cited by 133 · see citing works

BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 LanguagesPreprint

Heinzerling, B., & Strube, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02187 · Cited by 125 · see citing works

Small and Practical BERT Models for Sequence Labeling

Tsai, H., Riesa, J., Johnson, M., Arivazhagan, N., Li, X., & Archer, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1374 · Cited by 120 · see citing works

Pre-training via ParaphrasingPreprint

Lewis, M., Ghazvininejad, M., Ghosh, G., Aghajanyan, A., Wang, S. I., & Zettlemoyer, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15020 · Cited by 120 · see citing works

Compression of Deep Learning Models for Text: A Survey

Gupta, M., & Agrawal, P. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3487045 · Cited by 112 · see citing works

Data-Centric Green AI An Exploratory Empirical Study

Verdecchia, R., Cruz, L., Sallou, J., Lin, M., Wickenden, J., & Hotellier, E. (2022) · International Conference on ICT for Sustainability (ICT4S) · doi:10.1109/ict4s55073.2022.00015 · Cited by 94 · see citing works

A Deep Learning Model Based on BERT and Sentence Transformer for Semantic Keyphrase Extraction on Big Social Data

Devika, R., Subramaniyaswamy, V., Mahenthar, C. S. J., Varadarajan, V., & Kotecha, K. V. (2021) · IEEE Access · doi:10.1109/access.2021.3133651 · Cited by 79 · see citing works

Few-shot Learning with Multilingual Language ModelsPreprint

Lin, X. V., Mihaylov, T., Artetxe, M., Wang, T., Chen, S., Simig, D., Ott, M., Goyal, N., Bhosale, S., Du, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.10668 · Cited by 79 · see citing works

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA

Chi, Z., Huang, S., Dong, L., Ma, S., Zheng, B., Singhal, S., Bajaj, P., Song, X., Mao, X.-L., Huang, H., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.427 · Cited by 78 · see citing works

ZeroPrompt: Scaling Prompt-Based Pretraining to 1, 000 Tasks Improves Zero-Shot GeneralizationPreprint

Xu, H., Chen, Y., Du, Y., Shao, N., Wang, Y.-G., Li, H., & Yang, Z.-L. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2201.06910 · Cited by 77 · see citing works

Stance detection with BERT embeddings for credibility analysis of information on social media

Karande, H., Walambe, R. A., Benjamin, V., Kotecha, K. V., & Raghu, T. S. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.467 · Cited by 69 · see citing works

GPT-NeoX-20B: An Open-Source Autoregressive Language Model

Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.06745 · Cited by 67 · see citing works

Extreme Language Model Compression with Optimal Subwords and Shared Projections

Zhao, S., Gupta, R., Song, Y., & Zhou, D. (2019) · arXiv.org · doi:10.48550/arxiv.1909.11687 · Cited by 54

NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence PredictionPreprint

Sun, Y., Zheng, Y., Hao, C., & Qiu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03564 · Cited by 48 · see citing works

Task-oriented Word Embedding for Text Classification.

Liu, Q., Huang, H., Gao, Y., Wei, X., Tian, Y., & Liu, L. (2018) · International Conference on Computational Linguistics · Cited by 44 · see citing works

ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization

Xu, H., Chen, Y., Du, Y., Shao, N., Yanggang, W., Li, H., & Yang, Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.312 · Cited by 39 · see citing works

Experiments on the application of IOHMMs to model financial returns series

Bengio, Y., Lauzon, V., & Ducharme, R. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.896800 · Cited by 35 · see citing works

XLM-E: Cross-lingual Language Model Pre-training via ELECTRAPreprint

Chi, Z., Huang, S., Li, D., Ma, S., Zheng, B., Singhal, S., Bajaj, P., Song, X., Mao, X.-L., Huang, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.16138 · Cited by 24 · see citing works

Kronecker Decomposition for GPT Compression

Edalati, A., Tahaei, M. S., Rashid, A., Nia, V. P., Clark, J. J., & Rezagholizadeh, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.24 · Cited by 21 · see citing works

Small and Practical BERT Models for Sequence LabelingPreprint

Tsai, H., Riesa, J., Johnson, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00100 · Cited by 16 · see citing works

Kronecker Decomposition for GPT CompressionPreprint

Edalati, A., Tahaei, M. S., Rashid, A., Nia, V. P., Clark, J. J., & Rezagholizadeh, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08152 · Cited by 7 · see citing works

Multi-level Knowledge Distillation via Knowledge Alignment and CorrelationPreprint

Ding, F., Yang, Y., Hu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00573 · Cited by 1 · see citing works

Weight Squeezing: Reparameterization for Knowledge Transfer and Model CompressionPreprint

Chumachenko, A., Gavrilov, D., Balagansky, N., & Kalaidin, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06993 · Cited by 1 · see citing works

OPT: Open Pre-trained Transformer Language Models

Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805

Universal Language Model Fine-tuning for Text Classification

Howard, J., & Ruder, S. (2018) · arXiv · doi:10.48550/arxiv.1801.06146

Unsupervised Cross-lingual Representation Learning at Scale

Conneau, A., Khandelwal, K., Goyal, N., et al. (2019) · arXiv · doi:10.48550/arxiv.1911.02116

Patient Knowledge Distillation for BERT Model Compression

Sun, S., Cheng, Y., Gan, Z., & Liu, J. (2019) · arXiv · doi:10.48550/arxiv.1908.09355

TAPAS: Weakly Supervised Table Parsing via Pre-training

Herzig, J., Nowak, P. K., Müller, T., Piccinno, F., & Eisenschlos, J. M. (2020) · arXiv · doi:10.48550/arxiv.2004.02349

Data-Centric Green AI: An Exploratory Empirical Study

Verdecchia, R., Cruz, L., Sallou, J., Lin, M., Wickenden, J., & Hotellier, E. (2022) · arXiv · doi:10.48550/arxiv.2204.02766

Large language models (LLMs): survey, technical frameworks, and future challenges

Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395 · see citing works

A Comprehensive Study of ChatGPT: Advancements, Limitations, and Ethical Considerations in Natural Language Processing and Cybersecurity

Alawida, M., Mejri, S., Mehmood, A., Chikhaoui, B., & Abiodun, O. I. (2023) · Information · doi:10.3390/info14080462 · Cited by 211 · see citing works

Evaluation of the performance of GPT-3.5 and GPT-4 on the Polish Medical Final Examination

Rosoł, M., Gąsior, J. S., Łaba, J., Korzeniewski, K., & Młyńczak, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-46995-z · Cited by 156 · see citing works

Evaluating large language models for use in healthcare: A framework for translational value assessment

Reddy, S. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101304 · Cited by 99 · see citing works

Natural language processing in the era of large language models

Zubiaga, A. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1350306 · Cited by 81 · see citing works

Text Classification: How Machine Learning Is Revolutionizing Text Categorization

Allam, H., Makubvure, L., Gyamfi, B. K., Kwadwo, G., & Akinwolere, K. (2025) · Information · doi:10.3390/info16020130 · Cited by 62 · see citing works

A survey on sentiment analysis and its applications

Al-Qablan, T. A., Noor, M. H. M., Al‐Betar, M. A., & Khader, A. T. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08941-y · Cited by 69 · see citing works

Enhancing knowledge retrieval with in-context learning and semantic search through generative AI

Ghali, M.-K., Farrag, A., Won, D., & Jin, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113047 · Cited by 53 · see citing works

LLM4VV: Developing LLM-driven testsuite for compiler validation

Munley, C., Jarmusch, A., & Chandrasekaran, S. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.05.034 · Cited by 49 · see citing works

Using large language models to investigate cultural ecosystem services perceptions: A few-shot and prompt method

Luo, H., Zhang, Z., Zhu, Q., Ameur, N. E. H. B., Liu, X., Fan, D., & Cai, Y. (2025) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2025.105323 · Cited by 46 · see citing works

Using generative AI in human resource development: an applied research study

Ardichvili, A. A., Dirani, K. M., Jabarkhail, S., Mansour, W. E., & Aboulhosn, S. (2024) · Human Resource Development International · doi:10.1080/13678868.2024.2337964 · Cited by 32 · see citing works

Detecting Reddit Users with Depression Using a Hybrid Neural Network SBERT-CNN

Chen, Z., Yang, R., Fu, S., Zong, N., Liu, H., & Huang, M. S. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00035 · Cited by 38 · see citing works

Optimising the use of electronic medical records for large scale research in psychiatry

Newby, D., Taylor, N., Joyce, D. W., & Winchester, L. M. (2024) · Translational Psychiatry · doi:10.1038/s41398-024-02911-1 · Cited by 29 · see citing works

Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine Translation

Al-Khalifa, H. S., Al-Khalefah, K., & Haroon, H. (2024) · Human-Centric Intelligent Systems · doi:10.1007/s44230-024-00061-7 · Cited by 32 · see citing works

A systematic review of AI-driven business models for advancing Sustainable Development Goals

Khan, M. A., Rahman, A., Mahmud, F. U., Bishnu, K. K., Ahmed, M., Mridha, M. F., & Aung, Z. (2025) · Array · doi:10.1016/j.array.2025.100539 · Cited by 22 · see citing works

Unmasking large language models by means of OpenAI GPT-4 and Google AI: A deep instruction-based analysis

Zahid, I. A., Joudar, S. S., Albahri, A., Albahri, O., Alamoodi, A., Santamaría, J., & Alzubaidi, L. (2024) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2024.200431 · Cited by 25 · see citing works

Motivational Computing: Transformer-Based Automation of Implicit Motive Coding

Pang, J. S., Ring, H., Huang, K. Y., & Khoo, A. X. (2026) · Journal of Personality Assessment · doi:10.1080/00223891.2026.2630930 · Cited by 19 · see citing works

Distributional Semantics: Meaning Through Culture and Interaction

Contreras Kallens, P., & Christiansen, M. H. (2025) · Topics in Cognitive Science · doi:10.1111/tops.12771 · Cited by 17 · see citing works

Prompts and Large Language Models: A New Tool for Drafting, Reviewing and Interpreting Contracts?

Wang, B. T. (2024) · Law Technology and Humans · doi:10.5204/lthj.3483 · Cited by 14 · see citing works

Soft cosine and extended cosine adaptation for pre-trained language model semantic vector analysis

Ijebu, F. F., Liu, Y., Sun, C., & Usip, P. U. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112551 · Cited by 14 · see citing works

Creating a Parallel Corpus for the Kazakh Sign Language and Learning

Yerimbetova, A., Sakenov, B., Sambetbayeva, M., Daiyrbayeva, E., Berzhanova, U., & Othman, M. (2025) · Applied Sciences · doi:10.3390/app15052808 · Cited by 12 · see citing works

Bridging Traditional-Statistics and Machine-Learning Approaches in Psychology: Navigating Small Samples, Measurement Error, Nonindependent Observations, and Missing Data

Lavelle-Hill, R. E., Smith, G., & Murayama, K. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251345696 · Cited by 11 · see citing works

From lazy to rich to exclusive task representations in neural networks and neural codes

Farrell, M., Recanatesi, S., & Shea‐Brown, E. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102780 · Cited by 11 · see citing works

CybAttT: A Dataset of Cyberattack News Tweets for Enhanced Threat Intelligence

Lughbi, H., Mars, M., & Almotairi, K. H. (2024) · Data · doi:10.3390/data9030039 · Cited by 10 · see citing works

Framework for automation of short answer grading based on domain-specific pre-training

Bonthu, S., Sree, S. R., & Prasad, M. H. M. K. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109163 · Cited by 11 · see citing works

Analysis of Government Policy Sentiment Regarding Vacation during the COVID-19 Pandemic Using the Bidirectional Encoder Representation from Transformers (BERT)

Yulita, I. N., Wijaya, V., Rosadi, R., Sarathan, I., Djuyandi, Y., & Prabuwono, A. S. (2023) · Data · doi:10.3390/data8030046 · Cited by 9 · see citing works

Convolutional Neural Network and Language Model-Based Sequential CT Image Captioning for Intracerebral Hemorrhage

Kim, G.-Y., Oh, B.-D., Kim, C., & Kim, Y.-S. (2023) · Applied Sciences · doi:10.3390/app13179665 · Cited by 9 · see citing works

Unifying Sentence Transformer Embedding and Softmax Voting Ensemble for Accurate News Category Prediction

Khosa, S., Mehmood, A., & Rizwan, M. (2023) · Computers · doi:10.3390/computers12070137 · Cited by 8 · see citing works

Characterising research trends in bronchiectasis through AI-powered analytics

Narayana, J. K., Ling, Y. K. W., Aogáin, M. M., & Chotirmall, S. H. (2025) · European Respiratory Journal · doi:10.1183/13993003.00894-2025 · Cited by 8 · see citing works

La-LoRA: Parameter-efficient fine-tuning with layer-wise adaptive low-rank adaptation

Gu, J., Yuan, J., Cai, J., Zhou, X., & Fan, L. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108095 · Cited by 12 · see citing works

ChatGPT, extended: large language models and the extended mind

Smart, P., Clowes, R., & Clark, A. (2025) · Synthese · doi:10.1007/s11229-025-05046-y · Cited by 7 · see citing works

A Novel NLP-Driven Dashboard for Interactive CyberAttacks Tweet Classification and Visualization

Lughbi, H., Mars, M., & Almotairi, K. H. (2024) · Information · doi:10.3390/info15030137 · Cited by 9 · see citing works

Chinese Named Entity Recognition for Bridge Damage and Defects Based on Text Mining and Natural Language Pretraining Models

Liu, J., Li, W., Li, F., & Zhao, X. (2025) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-16019 · Cited by 6 · see citing works

Implementing T5 for Text Summarization: An Algorithmic Approach

Jyoti, D., Srivastava, J., & Mahato, D. P. (2025) · International Conference on Information Networking · doi:10.1109/icoin63865.2025.10992766 · Cited by 6 · see citing works

Enhancing Text Summarization with a T5 Model and Bayesian Optimization

Lubis, A. R., Safitri, H. R., Irvan, I., Lubis, M., Hamzah, M. L., Al-Khowarizmi, A.-K., & Nugroho, O. (2023) · Revue d intelligence artificielle · doi:10.18280/ria.370513 · Cited by 6 · see citing works

Sustainable Topic Modeling for Legal Moroccan Arabic Language: A Challenging Study on BERTopic Technique

Aouichaty, S., Maleh, Y., Mohtadi, M. T., Hajami, A., & Allali, H. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.05.069 · Cited by 5 · see citing works

Goal Model Extraction from User Stories Using Large Language Models

Siddeshwar, V., Alwidian, S., & Makrehchi, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-70245-7_19 · Cited by 5 · see citing works

TwIdw—A Novel Method for Feature Extraction from Unstructured Texts

Nagy, K. S., & Kapusta, J. (2023) · Applied Sciences · doi:10.3390/app13116438 · Cited by 5 · see citing works

Latent Representation Learning for Geospatial Entities

Lee, W. J., & Lauw, H. W. (2024) · ACM Transactions on Spatial Algorithms and Systems · doi:10.1145/3663474 · Cited by 5 · see citing works

Generative artificial intelligence in academic medical writing

Al-Imam, A., Al-Hadithi, N., Alissa, F., & Michalak, M. (2023) · Medical Journal of Babylon · doi:10.4103/mjbl.mjbl_1176_23 · Cited by 5 · see citing works

Unravelling Lexical and Narrative Patterns in the Hikayat Lonthoir: A Computational Linguistics Approach

Kersapati, M. I., Cacciafoco, F. P., Sihite, B., Wu, S., Widyaningrum, K. P., Atqa, M., & Toni, E. A. B. (2025) · Information · doi:10.3390/info16121069 · Cited by 4 · see citing works

A novel interpretable regularized cnn with a modified xlnet transformer for segmenting and classifying the ovarian cancer

Aelgani, V., & Vadlakonda, D. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18106-1 · Cited by 4 · see citing works

Deep learning-based automatic action extraction from structured chemical synthesis procedures

Vaškevičius, M., Kapočiūtė-Dzikienė, J., Vaškevičius, A., & Šlepikas, L. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1511 · Cited by 4 · see citing works

F-ALBERT: A Distilled Model from a Two-Time Distillation System for Reduced Computational Complexity in ALBERT Model

Kim, K.-H., & Jeong, C. (2023) · Applied Sciences · doi:10.3390/app13179530 · Cited by 4 · see citing works

Fully Attentional Network for Low-Resource Academic Machine Translation and Post Editing

İlhami, S. E. L., & Hanbay, D. (2022) · Applied Sciences · doi:10.3390/app122211456 · Cited by 7 · see citing works

Machine Learning Meets Psychological Data: Challenges and Future DirectionsPreprint

Lavelle-Hill, R. E., Smith, G., & Murayama, K. (2023) · doi:10.31219/osf.io/6xt82 · Cited by 7 · see citing works

Navigating the Frontier of Finance

Haidar, A., & Abbass, A. (2025) · Wiley eBooks · doi:10.1002/9781394271078.ch12 · Cited by 4 · see citing works

Semantic Similarity Search for Source Code Plagiarism Detection: An Exploratory Study

Ebrahim, F., & Joy, M. S. (2024) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3649217.3653622 · Cited by 6 · see citing works

Using Large Language Models for Advanced and Flexible Labelling of Protocol Deviations in Clinical Development

Zou, M., Popko, L., & Gaudio, M. (2025) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-025-00785-z · Cited by 3 · see citing works

Interpretable large language models for early prediction of antimicrobial multidrug resistance

Carmona-Martos, L., Martín-Palomeque, P., Escudero-Arnanz, Ó., & Soguero-Ruíz, C. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00401-2 · Cited by 3 · see citing works

Characterization of SLM Conversational Systems Models, Overview

Mir, Y. O. V., Pupo, I. P., Herrera, R. Y., Pérez, P. Y. P., Acuña, L. A., & Bello, R. (2025) · Studies in computational intelligence · doi:10.1007/978-3-031-83643-5_1 · Cited by 3 · see citing works

E3W—A Combined Model Based on GreedySoup Weighting Strategy for Chinese Agricultural News Classification

Xiao, Z., Yang, S., Duan, X., Tang, D., Guo, Y. Q., & Li, Z. (2022) · Applied Sciences · doi:10.3390/app122312059 · Cited by 3 · see citing works

Envisioning the interactive convergence of Generative AI and Facial Expression Recognition

Florance, G., Nirmala, M., Khan, J., & M, S. H. K. (2024) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct61223.2024.10543745 · Cited by 3 · see citing works

A Comprehensive Study of Bengali Embedding Models: Insights and Evaluations

Bulbul, A. I., Das, S., & Noori, S. A. H. (2024) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct64283.2024.10749935 · Cited by 3 · see citing works

Evaluating cultural knowledge processing in large language models: a cognitive benchmarking framework integrating retrieval-augmented generation

Lee, H.-S., Chang, C.-C., Chen, C.-Y., & Hsu, Y.-H. (2025) · The Electronic Library · doi:10.1108/el-04-2025-0136 · Cited by 2 · see citing works

Knowledge Graph-Based Credibility Evaluation Method for Electric Grid Large Language Model Knowledge Question-Answering

Li, W., Qi, X., Zhao, Q., Wang, C., Wu, Q., & Tang, X. (2023) · International Conference on Electronic Information Technology and Computer Engineering (EITCE) · doi:10.1145/3650400.3650526 · Cited by 2 · see citing works

Bias in candidate sourcing communication: Investigating stereotypical gender- and age-related frames in online job advertisements at the sectoral level

Noon, M. F. A., Kroon, A. C., van der Goot, M. J., Vliegenthart, R., & van Selm, M. (2024) · Public Relations Review · doi:10.1016/j.pubrev.2024.102456 · Cited by 2 · see citing works

Response Delay Reduction in Large Language Model-Based Dialogue Systems

Kamioka, H., Maeda, S., & Hashimoto, M. (2024) · Journal of Machine Intelligence and Data Science · doi:10.11159/jmids.2024.017 · Cited by 2 · see citing works

NVCiM-PT: An NVCiM-Assisted Prompt Tuning Framework for Edge LLMs

Qin, R., Ren, P., Yan, Z., Liu, L., Liu, D., Nassereldine, A., Xiong, J., Ni, K., Hu, X. S., & Shi, Y. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10993249 · Cited by 3 · see citing works

Neural Coreference Resolution for Turkish

Demir, Ş. (2023) · Journal of Intelligent Systems Theory and Applications · doi:10.38016/jista.1225097 · Cited by 2 · see citing works

Pathogen-driven Infectious Disease Recognition and Classification: An In-depth Analysis using Machine Learning and Deep Learning Methods

Bai, L. N., & Devi, M. (2024) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom61295.2024.10498961 · Cited by 1 · see citing works

Narrative Machines

Asrifan, A., Hasa, M. F., Syafryadin, S., & Pelu, H. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-6255-6.ch004 · Cited by 1 · see citing works

Towards Accurate SDG Research Categorization: A Hybrid Deep Learning Approach Using Scopus Metadata

Al-Bayati, J. S. H., Tawfeeq, F. N., & Al-Shammaa, M. (2025) · Journal of Advances in Information Technology · doi:10.12720/jait.16.11.1604-1623 · Cited by 1 · see citing works

Justifying the selection of a neural network linguistic classifier

Barkovska, O., Voropaieva, K., & Ruskikh, O. (2023) · Innovative technologies and scientific solutions for industries · doi:10.30837/itssi.2023.25.005 · Cited by 1 · see citing works

Felügyelt gépi tanulási módszerek alkalmazása a pszichológiai kutatásokban

Hajdú, N., Szászi, B., Aczél, B., & Nagy, T. (2024) · Magyar Pszichológiai Szemle · doi:10.1556/0016.2024.00052 · Cited by 1 · see citing works

Deep Learning Techniques for Machine Translation: A Survey

Hameed, D. A., & Al‐Khateeb, B. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.339 · Cited by 1 · see citing works

Optimizing Single DGX-A100 System: Overcoming GPU Limitations via Efficient Parallelism and Scheduling for Large Language Models

Kim, K.-H., & Jeong, C. (2023) · Applied Sciences · doi:10.3390/app13169306

Using supervised machine learning methods in psychological researchPreprint

Hajdú, N., Szászi, B., Aczél, B., & Nagy, T. (2023) · doi:10.31234/osf.io/tjkug

Concentration of Measure and Global Optimization of Bayesian Multilayer Perceptron. Part I

Temyanov, B. K., & Nigmatullin, R. R. (2024) · Lobachevskii Journal of Mathematics · doi:10.1134/s1995080224600651

АНАЛІЗ РИТОРИЧНИХ ПРИЙОМІВ У НАУКОВИХ СТАТТЯХ ЗА ДОПОМОГОЮ ІНСТРУМЕНТІВ ШТУЧНОГО ІНТЕЛЕКТУ

Шестакова, С. (2024) · Вісник науки та освіти · doi:10.52058/2786-6165-2024-11(29)-715-729

Deep Learning-based English-Arabic Machine Translation for Sulfur Manufacture Texts

Hameed, D. A., & Al‐Khateeb, B. (2024) · Mesopotamian Journal of Big Data · doi:10.58496/mjbd/2024/018

The Iterative Query Knowledge Graph Completion Strategy

Zhang, Y., Xiao, G., & Shen, Y. (2024) · International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI) · doi:10.1109/cei63587.2024.10871566

Bridging Traditional Statistics and Machine-Learning Approaches in Psychology: Navigating Small Samples, Measurement Error, Non-independent Observations and Missing DataPreprint

Lavelle-Hill, R. E., Smith, G., & Murayama, K. (2023) · doi:10.31219/osf.io/6xt82_v1

Integrity in the Age of AI: Detecting Synthetic Text Across Platforms and Purposes

Shah, R., & Mahato, M. (2025) · International Conference on Cognitive Computing in Engineering, Communications, Sciences and Biomedical Health Informatics (IC3ECSBHI) · doi:10.1109/ic3ecsbhi63591.2025.10990518

Dynamic Layer-Wise Strategy for Parameter-Efficient Fine-Tuning

Peng, G., Yang, Y., Zhang, D., Xie, X., Jiang, A., & Ding, F. (2025) · International Conference on Software Engineering and Information Management (ICSIM) · doi:10.1145/3725899.3725924 · Cited by 1 · see citing works

Cross language entity alignment based on knowledge augmentation of large language model

Zhang, F., Liu, S., Sun, J., Liu, L., Fang, Y., & Li, Z. (2025) · International Conference on Computer and Multimedia Technology (ICCMT) · doi:10.1145/3757749.3757763

Machines flunking an exam: Evaluating large language models on course-related open questionsPreprint

Huang, J., Wei, Y., Zhang, L., Lai, R., Lai, F., & Zheng, Y. (2025) · Research Square · doi:10.21203/rs.3.rs-6679662/v1

Automated Taxonomy Construction Using Large Language Models: A Comparative Study of Fine-Tuning and Prompt Engineering

Vũ, B. D., Naik, R. G., Nguyen, B. K., Mehraeen, S., & Hemmje, M. (2025) · Eng—Advances in Engineering · doi:10.3390/eng6110283

Intelligent model for target identification from Urdu threatening tweets in low-resource language using deep learning and fine-tuning of language models

Raza, G. M., Naseem, A., Iqbal, S., Masood, T., Kim, B.-S., & Nguyen, T. N. (2025) · Journal of Information and Telecommunication · doi:10.1080/24751839.2025.2576951

Unsupervised Learning of Urban Spatial Representation with Multimodal Features from POI Data

Ding, W., Shao, Y.-H., Li, W., Wang, Y., Feng, R., & Fan, R. (2025) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss55030.2025.11243769

The Impact of Word Embedding on The Film Recommendation System Using Hybrid Filtering

Arsytania, I. H., & Setiawan, E. B. (2025) · International Conference on Software Engineering and Information Technology (ICoSEIT) · doi:10.1109/icoseit67010.2025.11290773

Vocabulary Semantic Similarity Calculation in Natural Language Processing

Xiao, H., Zheng, K., & Li, X. (2025) · Journal of advances in engineering and technology. · doi:10.62177/jaet.v2i4.946

Automated Health Economic Model using Artificial Intelligence for Evaluating the Large Language Models

Tripathi, N., Abubakirova, N., Sayfullaevna, K. S., Sapaeva, R., Masharipova, N., & Chauhan, A. (2025) · International Conference on Advancement in Electronics & Communication Engineering (AECE) · doi:10.1109/aece67531.2025.11386604

Language without body, meaning without world: The uncanny in the language of artificial intelligence

Godoi, B. S., Campos, J. P., Raimundo, L., Mendonça, M., Silveira, G., Gregori, M., & Iannini, G. D. P. M. (2026) · International Forum of Psychoanalysis · doi:10.1080/0803706x.2026.2619432

Modélisation d’offres d’emploi et de curriculums vitæ en vue d’un processus de présélection automatisée de candidats

Espinal, A. (2023) · theses.fr (ABES) · doi:10.70675/846d9ae7zcadez4ad6z95b9zb86d2fd3945f

Designing metaverse interaction systems for the Turkish language enhanced by fine-tuning and retrieval-augmented generation (RAG)

Özkal, İ., & Başçi̇ftçi̇, F. (2026) · Scientific Reports · doi:10.1038/s41598-026-35392-x

The Impact of Pre-trained Language Models for Success Prediction in Digital Negotiations - A Comparative Analysis

Rukower, L., Kaya, M.-F., & Schoop, M. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-29551-4_2

Cultural and Linguistic Contextualization in Ugandan English: Training Transformer Models for Enhanced Local Relevance

David, B., Isingoma, B., Nakatumba-Nabende, J., & Jjingo, D. (2026) · Cureus Journal of Computer Science. · doi:10.7759/s44389-026-00167-y

ARTIFICIAL INTELLIGENCE–BASED ANALYSIS OF DISCOURSES ON SUSTAINABLE AGRICULTURE

Kayakuş, M., Kabaş, Ö., VLĂDUT, N.-V., & Kabaş, A. (2026) · INMATEH Agricultural Engineering · doi:10.35633/inmateh-79-56

Design of Automatic Short-Answer Scoring Prediction Model in Academic Courses Based on Statistical and Deep Learning Algorithms

Farooq, M. U., Ahmad, T., & Aslam, M. (2026) · Algorithms · doi:10.3390/a19090777

Hungary’s democratic moment – A critical theoretical modelling of deliberative transition

Sík, D. (2026) · Journal of Contemporary Central and Eastern Europe · doi:10.1080/25739638.2026.2742214

A Study of Pre-trained Language Models in Natural Language Processing

Duan, J., Zhao, H. J., Zhou, Q., Qiu, M., & Liu, M. (2020) · IEEE International Conference on Smart Cloud (SmartCloud) · doi:10.1109/smartcloud49737.2020.00030 · Cited by 31 · see citing works

A Survey of Pretrained Language Models

Sun, K., Luo, X., & Luo, M. Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10986-7_36 · Cited by 48 · see citing works

Comprehensive analysis of embeddings and pre-training in NLP

Tripathy, J. K., Sethuraman, S. C., Cruz, M. V., Namburu, A., Mangalraj, P., Nandha Kumar, R., Sudhakar Ilango, S., & Vijayakumar, V. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100433 · Cited by 41 · see citing works

A Survey of Pretrained Language Models for NLP TasksPreprint

Heleen, B., Abubakar, M., & Dodda, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5198602

Pre-Trained Language Models and Their Applications

Wang, H., Li, J., Wu, H. R., Hovy, E. H., & Sun, Y. (2022) · Engineering · doi:10.1016/j.eng.2022.04.024 · Cited by 404 · see citing works