The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Transformers: State-of-the-Art Natural Language Processing
Open accessAbstract
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, Alexander Rush. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2020.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Almost all CLaMs have been made publicly accessible via online model repositories like HuggingFace57.”
From The shaky foundations of large language models and foundation models for electronic health records (2023) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Employing BERT, GPT-2, RoBERTa, and DistilBERT, we achieved robust results; BERT for deep contextual insights, GPT-2 for coherent text generation, RoBERTa for performance with large data, and DistilBERT for fast, cost-effective efficiency [39].”
From Generative AI-Powered Educational Alignment: A Framework for Matching Syllabus Course Topics with Web Description (2024) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Open-source ecosystems such as Hugging Face and BLOOM reduce technical barriers but can reproduce extractive defaults without Indigenous authority over datasets, objectives, and deployment pathways (BigScience Workshop, 2022;Wolf et al., 2020).”
From Towards a decolonial AI sovereignty model for indigenous educational settings (2026) · cites this paper as Wolf et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Before the emergence of ChatGPT, BERT is a commonly used pre-trained machine learning model for NLP tasks (Wolf et al., 2020), and a ubiquitous baseline in text classification experiments of research in recent years (Rogers et al., 2021).”
From Towards reliable generative AI-driven scaffolding: Reducing hallucinations and enhancing quality in self-regulated learning support (2026) · cites this paper as Wolf et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“Tamarin is implemented in roughly 9K lines of Python on top of HuggingFace Transformers (Wolf et al., 2020); it uses no custom CUDA and does not modify model weights.”
From Tamarin (HSQ): Reversible Hierarchical KV-Cache Compression for LLM Inference (2026) · cites this paper as Wolf et al., 2020
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“3 speedup on GPT-2 (seq. length 1K)”
From FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) · cites this paper as [87]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Table 2 shows up to 3 endto-end speedup compared to Huggingface and 1.7 speedup compared to Megatron-LM.”
From FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) · cites this paper as [87]
Experimental — accuracy not yet measuredIn the citing paper
“While these early explorations are promising, they do not explore how much or what kind of interactivity is beneficial.”
From Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022) · cites this paper as [73]
In the citing paper
“along with high-quality open-source libraries [113] have lowered the barrier to entry and make it possible to build conversational models that exhibit high generalization and coherence [90].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [113]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“The Transformer (Vaswani et al., 2017) has rapidly become the dominant architecture for natural language processing, surpassing alternative neural models such as convolutional and recurrent neural networks in performance for tasks in both natural language understanding and natural language…”
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 4×
“, 2018) and the original source code for BERT (Devlin et al., 2018), both from Google Research.”
InfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 3×
“…text corpora, leading to major gains in accuracy on downstream tasks including text classification (Yang et al., 2019), language understanding (Liu et al., 2019b; Wang et al., 2018, 2019), machine translation (Lample and Conneau, 2019a), coreference resolution (Joshi et al., 2019),…”
InfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 1×
“Model pretraining (McCann et al., 2017; Howard and Ruder, 2018; Peters et al., 2018; Devlin et al., 2018) allows models to be trained on generic corpora and subsequently be easily adapted to specific tasks with strong performance.”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2×
“, 2019) T5 (Raffel et al., 2019) MarianMT (J.”
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252 · cited 1×
“Two core libraries are NLTK (Loper and Bird, 2002) and Stanford CoreNLP (Manning et al., 2014), which collect a variety of different approaches to NLP in a single package.”
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 3×
“The Transformer architecture is particularly conducive to pretraining on large text corpora, leading to major gains in accuracy on downstream tasks including text classification (Yang et al., 2019), language understanding (Liu et al.”
InfluentialDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 2×
“While core models like BERT and GPT-2 continue to be popular, other specialized models including DistilBERT (Sanh et al., 2019), which was developed for the library, are now widely downloaded by the community.”
Self-citation
Research works printed but not linked: 2
- Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. Flair: An easy-to-use framework for state-of-the-art nlp. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics (Demonstrations), pages 54- 59. aclweb.org.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.
Cited by
Sorted by citations
Items 501–600 of 1,115
BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese
Tran, N. L., Le, D., & Nguyen, D. Q. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10177 · Cited by 80
Bess, A., Berglind, F., Mukhopadhyay, S., Bryliński, M., Griggs, N. W., Cho, T., Galliano, C., & Wasan, K. M. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.10.022 · Cited by 64
Machine translation from signed to spoken languages: state of the art and challenges
Coster, M. D., Shterionov, D., Herreweghe, M. V., & Dambre, J. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-023-00992-1 · Cited by 82
A sequence-to-sequence approach for document-level relation extraction
Giorgi, J., Bader, G. D., & Wang, B. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.2 · Cited by 67
Self-Edit: Fault-Aware Code Editor for Code Generation
Zhang, K., Li, Z., Li, J., Li, G., & Jin, Z. G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.45 · Cited by 141
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03714 · Cited by 911
ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering
Chen, S., Liu, Q., Yu, Z., Lin, C.-Y., Lou, J., & Jiang, F. X. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.39 · Cited by 64
MOOCCubeX: A Large Knowledge-centered Repository for Adaptive Learning in MOOCs
Yu, J., Wang, Y., Zhong, Q., Luo, G., Mao, Y., Sun, K., Feng, W., Xu, W. X., Cao, S., Zeng, K., et al. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482010 · Cited by 71
ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension
Barba, E., Procopio, L., & Roberto Navigli, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.112 · Cited by 60
Guillarme, N. L., & Thuiller, W. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13778 · Cited by 60
Discop: Provably Secure Steganography in Practice Based on "Distribution Copies"
Ding, J., Chen, K., Wang, Y., Zhao, N., Zhang, W., & Yu, N. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179287 · Cited by 71
Tuckute, G., Feather, J., Boebinger, D., & McDermott, J. H. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002366 · Cited by 78
Zhao, J., Han, X., Wu, Y., Wang, Z., & Burke, A. F. (2024) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2024.11.011 · Cited by 66
Evaluating the representational power of pre-trained DNA language models for regulatory genomics
Tang, Z., Somia, N., Yu, Y., & Koo, P. K. (2025) · Genome biology · doi:10.1186/s13059-025-03674-8 · Cited by 71
Classification aware neural topic model for COVID-19 disinformation categorisation
Song, X., Petrak, J., Jiang, Y., Singh, I., Maynard, D., & Bontcheva, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247086 · Cited by 68
Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark
Niklaus, J., Chalkidis, I., & Stuermer, M. (2021) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2021.nllp-1.3 · Cited by 97
Effective Convolutional Attention Network for Multi-label Clinical Document Classification
Liu, Y., Cheng, H. N., Klopfer, R., Gormley, M. R., & Schaaf, T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.481 · Cited by 71
Improving Abstractive Summarization of Legal Rulings through Textual Entailment
Feijo, D. D. V., & Moreira, V. P. (2023) · Artificial Intelligence and Law · doi:10.1007/s10506-021-09305-4 · Cited by 59
Common vulnerability scoring system prediction based on open source intelligence information sources
Kühn, P., Relke, D. N., & Reuter, C. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103286 · Cited by 68
Automation of Language Sample Analysis
Liu, H., MacWhinney, B., Fromm, D., & Lanzi, A. (2023) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2023_jslhr-22-00642 · Cited by 59
Probing Pre-Trained Language Models for Cross-Cultural Differences in Values
Arora, A., Kaffee, L.-A., & Augenstein, I. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.12 · Cited by 142
CLUE: A Chinese Language Understanding Evaluation BenchmarkPreprint
Xu, L., Zhang, X., Li, L., Hu, H., Cao, C., Liu, W., Li, J., Li, Y., Sun, K., Xu, Y., et al. (2020) · International Conference on Computational Linguistics · doi:10.48550/arxiv.2004.05986 · Cited by 485
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.12833 · Cited by 59
Lightweight Adapter Tuning for Multilingual Speech Translation
Le, H., Pino, J., Wang, C., Gu, J., Schwab, D., & Besacier, L. (2021) · doi:10.18653/v1/2021.acl-short.103 · Cited by 63
Matsui, T., Suzuki, K., Ando, K., Kitai, Y., Haga, C., Masuhara, N., & Kawakubo, S. (2022) · Sustainability Science · doi:10.1007/s11625-022-01093-3 · Cited by 71
Context-Based Fake News Detection Model Relying on Deep Learning Models
Amer, E., Kwak, K. S., & El–Sappagh, S. (2022) · Electronics · doi:10.3390/electronics11081255 · Cited by 58
DarkBERT: A Language Model for the Dark Side of the Internet
Jin, Y., Jang, E., Cui, J., Chung, J.-W., Lee, Y., & Shin, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.415 · Cited by 65
Generative AI tools can enhance climate literacy but must be checked for biases and inaccuracies
Atkins, C., Girgente, G., Shirzaei, M., & Kim, J. (2024) · Communications Earth & Environment · doi:10.1038/s43247-024-01392-w · Cited by 62
Citation Intent Classification Using Word Embedding
Roman, M., Shahid, A., Khan, S., Koubâa, A., & Yu, L. (2021) · IEEE Access · doi:10.1109/access.2021.3050547 · Cited by 59
Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research
Gruetzemacher, R., & Paradice, D. (2022) · ACM Computing Surveys · doi:10.1145/3505245 · Cited by 68
FiNER: Financial Numeric Entity Recognition for XBRL Tagging
Loukas, L., Fergadiotis, M., Chalkidis, I., Spyropoulou, E., Malakasiotis, P., Androutsopoulos, I., & Παλιούρας, Γ. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.303 · Cited by 84
Automated Attack Synthesis by Extracting Finite State Machines from Protocol Specification Documents
Pacheco, M. L., von Hippel, M., Weintraub, B., Goldwasser, D., & Nita-Rotaru, C. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833673 · Cited by 64
Oliaee, A. H., Das, S., Liu, J., & Rahman, M. A. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100007 · Cited by 65
Koenig, N., Tonidandel, S., Thompson, I. B., Albritton, B. H., Koohifar, F., Yankov, G. P., Speer, A. B., Hardy, J. H., Gibson, C., Frost, C., et al. (2023) · Personnel Psychology · doi:10.1111/peps.12608 · Cited by 78
NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task
Abdul-Mageed, M., Zhang, C., Elmadany, A., Bouamor, H., & Habash, N. Y. (2022) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2022.wanlp-1.9 · Cited by 67
StructCoder: Structure-Aware Transformer for Code Generation
Tipirneni, S., Zhu, M., & Reddy, C. K. A. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3636430 · Cited by 90
Robust Visual Question Answering: Datasets, Methods, and Future Challenges
Ma, J., Wang, P., Kong, D., Wang, Z., Liu, J., Pei, H., & Zhao, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3366154 · Cited by 64
Assessing and understanding creativity in large language models
Zhao, Y., Zhang, R., Li, W., & Li, L. (2025) · Machine Intelligence Research · doi:10.1007/s11633-025-1546-4 · Cited by 57
Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems
Huang, Y., Kuo, H.-K. J., Thomas, S., Kons, Z., Audhkhasi, K., Kingsbury, B. E. D., Hoory, R., & Picheny, M. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053281 · Cited by 52
Differential Privacy for Text Analytics via Natural Text Sanitization
Yue, X., Du, M., Wang, T., Li, Y., Sun, H., & Chow, S. S. M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.337 · Cited by 96
Transformer Based Language Identification for Malayalam-English Code-Mixed Text
Thara, S., & Poornachandran, P. (2021) · IEEE Access · doi:10.1109/access.2021.3104106 · Cited by 59
Dict-BERT: Enhancing Language Model Pre-training with Dictionary
Yu, W., Zhu, C., Fang, Y., Yu, D., Wang, S., Xu, Y., Zeng, M., & Jiang, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.150 · Cited by 70
Transformer based named entity recognition for place name extraction from unstructured text
Berragan, C., Singleton, A. D., Calafiore, A., & Morley, J. G. (2022) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2022.2133125 · Cited by 68
Improving Log-Based Anomaly Detection by Pre-Training Hierarchical Transformers
Huang, S., Liu, Y., Fung, C., Wang, H., Yang, H., & Luan, Z. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3257518 · Cited by 61
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
DynaSent: A Dynamic Benchmark for Sentiment Analysis
Potts, C., Wu, Z., Geiger, A., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.186 · Cited by 56
Deidentification of free-text medical records using pre-trained bidirectional transformers
Johnson, A. E. W., Bulgarelli, L., & Pollard, T. J. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384455 · Cited by 55
Zero-shot Natural Language Video Localization
Nam, J., Ahn, D., Kang, D., Ha, S. J., & Choi, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00150 · Cited by 2
Do, S., Ollion, É., & Shen, R. (2022) · Sociological Methods & Research · doi:10.1177/00491241221134526 · Cited by 68
An Explainable Vision Transformer Model Based White Blood Cells Classification and Localization
KATAR, O., & Yıldırım, Ö. (2023) · Diagnostics · doi:10.3390/diagnostics13142459 · Cited by 55
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
Xiao, H., Zhu, X., Yu, L., Zhang, L., Song, Y.-Z., & Xiang, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00262 · Cited by 58
Deep-Learning for Change Detection Using Multi-Modal Fusion of Remote Sensing Images: A Review
Saidi, S., Idbraim, S., Karmoude, Y., Masse, A., & Arbelo, M. (2024) · Remote Sensing · doi:10.3390/rs16203852 · Cited by 62
Efficient Multimodal Fusion via Interactive Prompting
Li, Y., Quan, R., Zhu, L., & Yang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00256 · Cited by 60
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains
Ben-David, E., Oved, N., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00468 · Cited by 54
SentNoB: A Dataset for Analysing Sentiment on Noisy Bangla Texts
Islam, K. I., Kar, S., Islam, M. S., & Amin, M. R. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.278 · Cited by 61
Schedl, M., Brandl, S., Lesota, O., Parada‐Cabaleiro, E., Penz, D., & Rekabsaz, N. (2022) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505791 · Cited by 54
Mei, K., Fereidooni, S., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594109 · Cited by 56
Self-Attention and Transformers: Driving the Evolution of Large Language Models
Luo, Q., Zeng, W., Chen, M., Peng, G., Yuan, X., & Yin, Q. (2023) · IEEE International Conference on Electronic Information and Communication Technology (ICEICT) · doi:10.1109/iceict57916.2023.10245906 · Cited by 62
A systematic review of current AI techniques used in the context of the SDGs
Greif, L., Röckel, F., Kimmig, A., & Ovtcharova, J. (2025) · International Journal of Environmental Research · doi:10.1007/s41742-024-00668-5 · Cited by 55
NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
Lu, X., Welleck, S., West, P., Jiang, L., Kasai, J., Khashabi, D., Bras, R. J. L., Qin, L., Yu, Y., Zellers, R., et al. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.57 · Cited by 86
Recommendation Systems: An Insight Into Current Development and Future Research Challenges
Marcuzzo, M., Zangari, A., Albarelli, A., & Gasparetto, A. (2022) · IEEE Access · doi:10.1109/access.2022.3194536 · Cited by 63
Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely
Kauf, C., Ivanova, A. A., Rambelli, G., Chersoni, E., She, J. S., Chowdhury, Z., Fedorenko, E. G., & Lenci, A. (2023) · Cognitive Science · doi:10.1111/cogs.13386 · Cited by 70
Noisy-Labeled NER with Confidence Estimation
Liu, K., Fu, Y., Tan, C., Chen, M., Zhang, N., Huang, S., & Gao, S. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.269 · Cited by 57
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding EvaluationPreprint
Li, L., Lei, J., Gan, Z., Yu, L., Chen, Y.-C., Pillai, R., Cheng, Y., Zhou, L., Wang, X. E., Wang, W. Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04632 · Cited by 90
A dataset for medical instructional video classification and question answering
Gupta, D., Attal, K., & Demner‐Fushman, D. (2023) · Scientific Data · doi:10.1038/s41597-023-02036-y · Cited by 62
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 71
Pretraining Federated Text Models for Next Word Prediction
Stremmel, J., & Singh, A. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-73103-8_34 · Cited by 53
Model Generalization on COVID-19 Fake News Detection
Bang, Y., Ishii, E., Cahyawijaya, S., Ji, Z., & Fung, P. (2021) · Communications in Computer and Information Science · doi:10.1007/978-3-030-73696-5_13 · Cited by 52
Coreference-Aware Dialogue Summarization
Liu, Z., Shi, K., & Chen, N. F. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.53 · Cited by 57
Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features
Lee, B. W., Jang, Y. S., & Lee, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.834 · Cited by 66
SLUE: New Benchmark Tasks For Spoken Language Understanding Evaluation on Natural Speech
Shon, S., Pasad, A., Wu, F., Brusco, P., Artzi, Y., Livescu, K., & Han, K. J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746137 · Cited by 60
Macêdo, J. B., Moura, M. D. C., Aichele, D., & Lins, I. D. (2021) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2021.12.025 · Cited by 57
State Graph Reasoning for Multimodal Conversational Recommendation
Wu, Y., Liao, L., Zhang, G., Lei, W., Zhao, G., Qian, X., & Chua, T. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3155900 · Cited by 52
Liu, Y., Zhang, X. F., Wegsman, D., Beauchamp, N., & Wang, L. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.101 · Cited by 77
From ChatGPT-3 to GPT-4: A Significant Advancement in AI-Driven NLP Tools
Rahaman, M. S., Ahsan, M. M. T., Anjum, N., Terano, H. J. R., & Rahman, M. M. (2023) · Journal of Engineering and Emerging Technologies · doi:10.52631/jeet.v1i1.188 · Cited by 74
Petals: Collaborative Inference and Fine-tuning of Large Models
Borzunov, A., Baranchuk, D., Dettmers, T., Riabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., & Raffel, C. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2023.acl-demo.54 · Cited by 52
Sparsity in transformers: A systematic literature review
Farina, M., Ahmad, U., Taha, A. T. M., Younes, H., Mesbah, Y., Yu, X., & Witold Pedrycz, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127468 · Cited by 59
Beyond Detection: Leveraging Large Language Models for Cyber Attack Prediction in IoT Networks
Diaf, A., Korba, A. A., Karabadji, N. E. I., & Ghamri-Doudane, Y. M. (2024) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss-iot61029.2024.00026 · Cited by 61
Probing the 3D Awareness of Visual Foundation Models
Banani, M. E., Raj, A., Maninis, K.-K., Kar, A., Li, Y., Rubinstein, M., Sun, D., Guibas, L., Johnson, J., & Jampani, V. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02059 · Cited by 52
Benchmarking large language models for genomic knowledge with GeneTuring
Shang, X., Liao, X., Ji, Z., & Hou, W. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf492 · Cited by 53
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
Baek, J., Aji, A., & Saffari, A. (2023) · Proceedings of the First Workshop on Matching From Unstructured and Structured Data (MATCHING 2023) · doi:10.18653/v1/2023.matching-1.7 · Cited by 51
Detecting dementia from speech and transcripts using transformers
Ilias, L., Askounis, D., & Psarras, J. E. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101485 · Cited by 55
Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization
Laskar, M. T. R., Hoque, E., & Huang, J. X. (2022) · Computational Linguistics · doi:10.1162/coli_a_00434 · Cited by 58
Wulff, P., Mientus, L., Nowak, A., & Borowski, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00290-6 · Cited by 65
Domain Adaptation for Deep Entity Resolution
Tu, J., Fan, J., Tang, N., Wang, P., Chai, C., Li, G., Fan, R., & Du, X. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517870 · Cited by 56
QuALITY: Question Answering with Long Input Texts, Yes!
Pang, R. Y., Parrish, A., Joshi, N., Nangia, N., Phang, J., Chen, A., Padmakumar, V., Ma, J., Thompson, J., He, H., et al. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.391 · Cited by 51
W-Transformers: A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
Sasal, L., Chakraborty, T., & Hadid, A. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00111 · Cited by 59
A Survey on Data-Driven Runoff Forecasting Models Based on Neural Networks
Sheng, Z., Wen, S., Feng, Z.-K., Gong, J., Shi, K., Guo, Z., Yang, Y., & Huang, T. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3259434 · Cited by 53
Privacy Policies across the Ages: Content of Privacy Policies 1996–2021
Wagner, I. (2023) · ACM Transactions on Privacy and Security · doi:10.1145/3590152 · Cited by 51
Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing
Wang, Y., Sun, D., Chen, K., Lai, F., & Chowdhury, M. (2023) · European Conference on Computer Systems · doi:10.1145/3552326.3587451 · Cited by 59
FashionGPT: LLM instruction fine-tuning with multiple LoRA-adapter fusion
Gao, D., Ma, Y., Liu, S., Song, M., Jin, L., Jiang, W. R., Wang, X., Ning, W., Yu, S., Xuan, Q., et al. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112043 · Cited by 58
Next-Gen Medical Imaging: U-Net Evolution and the Rise of Transformers
Zhang, C. H., Deng, X., & Ling, S. H. (2024) · Sensors · doi:10.3390/s24144668 · Cited by 64
Meléndez, R., Ptaszynski, M. E., & Masui, F. (2024) · Electronics · doi:10.3390/electronics13244877 · Cited by 63
Wu, Q., Lin, Z., Karlsson, B. F., Huang, B., & Lou, J. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/543 · Cited by 55
DrugGPT: A GPT-based Strategy for Designing Potential Ligands Targeting Specific ProteinsPreprint
Li, Y., Gao, C., Song, X. G., Xiangyu, W., Xu, Y., & Han, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.06.29.543848 · Cited by 56
Contrastive Learning Models for Sentence Representations
Xu, L., Xie, H., Li, Z., Wang, F. L., Wang, W., & Li, Q. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3593590 · Cited by 50
Kim, H., Kim, B., & Kim, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.65 · Cited by 55
Challenges, Techniques, and Trends of Simple Knowledge Graph Question Answering: A Survey
Yani, M., & Krisnadhi, A. A. (2021) · Information · doi:10.3390/info12070271 · Cited by 57
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts
Baheti, A., Sap, M., Ritter, A., & Riedl, M. O. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.397 · Cited by 60
ESPnet-SLU: Advancing Spoken Language Understanding Through ESPnet
Arora, S., Dalmia, S., Denisov, P., Chang, X., Ueda, Y., Peng, Y., Zhang, Y., Kumar, S., Ganesan, K., Yan, B., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747674 · Cited by 51
Metadata
- Authors
- Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., & Rush, A.
- Affiliations
- Hugging Face (US)
- ORCID
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Published
- 2020
- Pages
- 38-45
- OpenAlex
- W2979826702
- Cited by
- 8,741
- References
- 39
- Citations by year
- 2026: 851 · 2025: 1481 · 2024: 1405 · 2023: 2347 · 2022: 1686 · 2021: 872 · 2020: 94
- FWCI
- 477.50
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- transformer models, natural language processing, Hugging Face
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
