Energy and Policy Considerations for Deep Learning in NLP
Open accessThe DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index.
Abstract
Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.These models have obtained notable gains in accuracy across many NLP tasks.However, these accuracy improvements depend on the availability of exceptionally large computational resources that necessitate similarly substantial energy consumption.As a result these models are costly to train and develop, both financially, due to the cost of hardware and electricity or cloud compute time, and environmentally, due to the carbon footprint required to fuel modern tensor processing hardware.In this paper we bring this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training a variety of recently successful neural network models for NLP.Based on these findings, we propose actionable recommendations to reduce costs and improve equity in NLP research and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To enable real-world deployment of multimodal models in a variety of settings [142], there is a need to build lightweight models with cheap training and inference.”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [142]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Significant critiques of LLMs have been raised, including concerns about bias and fairness, misinformation, opaqueness due to a lack of interpretability and explainability, and environmental impact (Bender et al., 2021; Gebru et al., 2021; Strubell et al., 2019).”
From Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments (2023) · cites this paper as Strubell et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The training and development of algorithms generate a substantial portion of greenhouse gas emissions.”
From Artificial Intelligence and Sustainable Decisions (2023) · cites this paper as [137]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“criticism around large models has been growing lately [9], especially ethical concerns regarding undesirable and often inscrutable societal biases percolating the models [9, 120], carbon footprint [9, 99], misuse and misinterpretation [9], privatization of AI research [99], and even research opportunity costs [49].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [99]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A 2019 study indicates training a single LLM may generate as much carbon dioxide as the lifetime emission of five automobiles (70).”
From Can Generative AI improve social science? (2024) · cites this paper as [70]
Experimental — accuracy not yet measured
27 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“…enabled impressive accuracy improvements across many fundamental NLP tasks ( Bahdanau et al., 2015; Luong et al., 2015; Dozat and Man-ning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the highest scores (Peters et al., 2018; Devlin et al., 2019; Radford et…”
BackgroundInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“The BERT model (Devlin et al., 2019) provides a Transformer-based architecture for building contextual representations similar to ELMo, but trained with a different language modeling objective.”
BackgroundMethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“The BERT model (Devlin et al., 2019) provides a Transformer-based architecture for building contextual representations similar to ELMo, but trained with a different language modeling objective.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“…techniques and hardware for training deep neural networks have recently enabled impressive accuracy improvements across many fundamental NLP tasks ( Bahdanau et al., 2015; Luong et al., 2015; Dozat and Man-ning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the…”
BackgroundDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“The ELMo model (Peters et al., 2018) is based on stacked LSTMs and provides rich word representations in context by pre-training on a large amount of data using a language modeling objective.”
BackgroundMethodEffective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 2×
“…for training deep neural networks have recently enabled impressive accuracy improvements across many fundamental NLP tasks (Bahdanau et al., 2015; Luong et al., 2015; Dozat and Manning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the highest scores (Peters…”
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 2×
“…the energy associated with model tuning is by providing easy-to-use APIs implementing more efficient alternatives to brute-force grid search for hyperparameter tuning, e.g. random or Bayesian hyperparameter search techniques (Bergstra et al., 2011; Bergstra and Bengio, 2012; Snoek et al., 2012).”
Method(2011) · Cited by 3,939
An Analysis of Deep Neural Network Models for Practical Applications
Canziani, A., Paszke, A., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1605.07678 · Cited by 969 · cited 1×
“nergy use required fortraining and inference in popular convolutional models for image classification in computer vision, including fine-grained analysis comparing different neural network layer types. Canziani et al. (2016) assess image classification model accuracy as a function of model size and gigaflops required duri…”
Method
Research works printed but not linked: 1
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In NAACL.
Cited by
Items 401–500 of 1,042
Arslan, Y., Allix, K., Veiber, L., Lothritz, C., Bissyandé, T. F., Klein, J., & Goujon, A. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3451375 · Cited by 98
“Indeed, training the BERT-based model has been evaluated in the literature to cost approximately $7000 for 4 days usage of 4 Clouds TPUs [40].”
MethodAI Art is Theft: Labour, Extraction, and Exploitation: Or, On the Dangers of Stochastic Pollocks
Goetze, T. S. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658898 · Cited by 91
“…point that the required computational power means that the creation and use of generative AI comes at considerable environmental cost [15, 28, 57, 84]. artists have had their work appropriated en masse by more powerful technologists for the creation of generative AI tools that pose an…”
Moral consideration of nonhumans in the ethics of artificial intelligence
Owe, A., & Baum, S. D. (2021) · AI and Ethics · doi:10.1007/s43681-021-00065-0 · Cited by 73
Clairvoyant prefetching for distributed machine learning I/O
Dryden, N., Boehringer, R., Ben‐Nun, T., & Hoefler, T. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3476181 · Cited by 94
“Given the high cost of training, it is critical that every aspect be as efficient as possible (OpenAI, 2018; Strubell et al., 2019).”
BackgroundAdapting natural language processing for technical text
Dima, A. A., Lukens, S., Hodkiewicz, M. R., Sexton, R., & Brundage, M. P. (2021) · Applied AI Letters · doi:10.1002/ail2.33 · Cited by 89
Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI
Yarally, T., Cruz, L., Feitosa, D., Sallou, J., & van Deursen, A. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00012 · Cited by 97
Photonic Neural Networks Based on Integrated Silicon Microresonators
Biasi, S., Donati, G. P., Lugnan, A., Mancinelli, M., Staffoli, E., & Pavesi, L. (2024) · Intelligent Computing · doi:10.34133/icomputing.0067 · Cited by 80
“Electronic ANNs face challenges such as long training times and high power consumption [10, 11, 12] as well as signal interference, difficulty in handling floating point operations, and low parallel computing efficiency [13, 14, 15].”
BackgroundAIGC-empowered smart manufacturing: Prospects and challenges
Leng, J., Zheng, K., Li, R. C., Chen, C., Wang, B., Liu, Q., Chen, X., & Shen, W. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103076 · Cited by 84
Automated essay scoring using efficient transformer-based language modelsPreprint
Ormerod, C. M., Malhotra, A., & Jafari, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.13136 · Cited by 89
“Lastly, we seek models that impose a smaller computational expense, which in turn has been linked by [ 37 ] to a much smaller carbon footprint.”
MethodNLP methods in host-based intrusion detection systems: A systematic review and future directions
Sworna, Z. T., Mousavi, Z., & Babar, M. A. (2023) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2023.103761 · Cited by 87
An Overview of Language Models: Recent Developments and Outlook
Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000010 · Cited by 86
“Some work [116, 119, 117, 133] calculated the computational complexity, approximate financial, and environmental costs of training PLMs.”
BackgroundJurowetzki, R., Hain, D. S., Mateos-Garcia, J. C., & Stathoulopoulos, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01648 · Cited by 88
“Training deep neural networks requires enormous amounts of data and computing power (Marcus, 2018; Russell, 2019), often exclusively available to large industry players and costly in terms of energy use and carbon emissions (Strubell et al., 2019).”
BackgroundLuccioni, A. S., Strubell, E., & Crawford, K. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732007 · Cited by 71
“On the one hand, it is champi-oned as a driver of efficiency that could potentially invent new solutions to the climate crisis or reinvent old ones [5, 21, 99]; on the other hand, it is criticized for its steeply increasing resource demands and carbon footprint [68, 120, 134].”
Self-citationEarth for AI: A Political Ecology of Data-Driven Climate Initiatives
Nost, E., & Colven, E. (2022) · Geoforum · doi:10.1016/j.geoforum.2022.01.016 · Cited by 71
Mao, R., Wen, B., Kazemi, A., Zhao, Y., Laguna, A. F., Lin, R., Wong, N., Niemier, M., Hu, X. S., Sheng, X., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33629-7 · Cited by 71
Cross-Domain Label-Adaptive Stance Detection
Hardalov, M., Arora, A., Nakov, P., & Augenstein, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.710 · Cited by 113
“requires a lot of computations and the use of GPUs/TPUs for training, which contributes to global warming (Strubell et al., 2019).”
BackgroundGupta, S., Zeballos, J. C., Castro, G. D. R., Tomičić, A., Morales, S. A., Mahfouz, M., Osemwegie, I., Sessi, V. P. C., Schmitz, M. A., Mahmoud, N., et al. (2023) · Sustainability · doi:10.3390/su15086844 · Cited by 82
Energy Efficiency of Training Neural Network Architectures: An Empirical Study
Xu, Y., Martínez‐Fernández, S., Martínez, M., & Franch, X. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.098 · Cited by 82
Artificial intelligence and management education: A conceptualization of human-machine interaction
Clegg, S. R., & Sarkar, S. (2024) · The International Journal of Management Education · doi:10.1016/j.ijme.2024.101007 · Cited by 89
Ren, Y., Zhang, H., Yu, F. R., Li, W., Zhao, P., & He, Y. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3522130 · Cited by 85
“However,deployingLLMsintheIIoTpresentssignificant challengesduetotheirhighdemandforcomputationalresources andenergy[13].”
Rakova, B., & Dobbe, R. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594014 · Cited by 39
“[116] estimate the environmental costs of Large Language Models (LLMs), while Lottick et al.”
BackgroundEnvironmental Burden of United States Data Centers in the Artificial Intelligence EraPreprint
Guidi, G., Dominici, F., Gilmour, J., Butler, K. A., Bell, E. F., Delaney, S., & Bargagli-Stoffi, F. J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.09786 · Cited by 95
“This has exacerbated the strain on electricity resources and triggered an unprecedented increase in energy demand and consumption ( 11, 12 ).”
Shopper intent prediction from clickstream e-commerce data with minimal browsing information
Requena, B., Cassani, G., Tagliabue, J., Greco, C., & Lacasa, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-73622-y · Cited by 86
Zhang, C., Xu, Y., Elahi, H., Zhang, D., Tan, Y., Chen, J., & Zhang, Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3171926 · Cited by 75
Generative AI in teacher education: Teacher educators’ perception and preparedness
Akanzire, B. N., Nyaaba, M., & Nabang, M. (2025) · Journal of Digital Educational Technology · doi:10.30935/jdet/15887 · Cited by 78
“It is critical to address these resource limits to make GenAI models more accessible and sustainable (Strubell et al., 2019).”
OLMo: Accelerating the Science of Language Models
Groeneveld, D., Beltagy, I., Walsh, E., Bhagia, A., Kinney, R., Tafjord, O., Jha, A., Ivison, H., Magnusson, I., Wang, Y., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.841 · Cited by 70
Self-citationRecent Trends in Deep Learning Based Open-Domain Textual Question Answering Systems
Huang, Z., Xu, S., Hu, M., Wang, X., Qiu, J., Fu, Y., Zhao, Y., Peng, Y., & Wang, C. (2020) · IEEE Access · doi:10.1109/access.2020.2988903 · Cited by 85
“With the improvement of performance along with the increase of model size, it is also a problem that running these models consumes a lot of energy [129].”
BackgroundMinixhofer, B., Paischer, F., & Rekabsaz, N. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.293 · Cited by 165
“Recent work has called attention to the costs associated with training increasingly large LMs, including environmental and financial cost (Strubell et al., 2019; Bender et al., 2021).”
BackgroundBolte, L., Vandemeulebroucke, T., & van Wynsberghe, A. (2022) · Sustainability · doi:10.3390/su14084472 · Cited by 75
“To be sure, there are a range of environmental consequences associated with the training and usage of AI/ML; carbon emissions and electricity consumption from running the algorithm [3,4], mining of precious minerals [5,6] for the development of different technologies making up the AI creating further environmental dam…”
BackgroundA Robustness-Assured White-Box Watermark in Neural Networks
Lv, P., Li, P., Zhang, S., Chen, K., Liang, R., Ma, H., Zhao, Y., & Li, Y. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3242737 · Cited by 92
“For instance, training BERT [51], a language understanding model, takes 3.3 days, i.e., 79.2 hours, on 4 DGX-2H servers with 64 Tesla V100 GPUs, and receives a bill of $3,751-$12,571 from the cloud computing platform [52].”
BackgroundInfluentialDaneshvar, N., Pandita, D., Erickson, S. M., Sulmasy, L. S., & DeCamp, M. W. (2024) · Annals of Internal Medicine · doi:10.7326/m24-0146 · Cited by 80
“Well before the widespread proliferation of generative AI, it was reported that training just 1 AI model for NLP can emit 284 019.13 kilograms of carbon dioxide equivalent — nearly 5 times the lifetime emissions of the average American car (84).”
InfluentialLarge Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 102
AI for Everyone? Critical Perspectives
(2021) · University of Westminster Press eBooks · doi:10.16997/book55 · Cited by 51
Realistic galaxy image simulation via score-based generative models
Smith, M. J., Geach, J. E., Jackson, R. A., Arora, N., Stone, C., & Courteau, S. (2022) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stac130 · Cited by 81
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
Ansell, A., Ponti, E. M., Pfeiffer, J., Ruder, S., Glavaš, G., Vulić, I., & Korhonen, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.410 · Cited by 125
“MAD-G aims to enable resource-efficient adaptation of MMTs to a wide range of previously unseen, radically resource-poor languages,6 and contribute in this manner to more sustainable (Strubell et al., 2019; Moosavi et al., 2020) and more inclusive”
BackgroundA survey on artificial intelligence for reducing the climate footprint in healthcare
Das, K. P., & Chandra, J. (2022) · Energy Nexus · doi:10.1016/j.nexus.2022.100167 · Cited by 88
Learning Markets: An AI Collaboration Framework Based on Blockchain and Smart Contracts
Ouyang, L., Yuan, Y., & Wang, F. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3032706 · Cited by 73
NAAS: Neural Accelerator Architecture Search
Lin, Y., Yang, M., & Han, S. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586250 · Cited by 89
Incremental Learning in Semantic Segmentation from Image Labels
Cermelli, F., Fontanel, D., Tavera, A., Ciccone, M., & Caputo, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00433 · Cited by 89
“However, this approach is impractical in case of frequent updates because training on the entire augmented dataset would take too long, increasing the energy consumption and carbon footprint of machine learning models [51, 58, 61].”
BackgroundZendrikov, D., Solinas, S. M. G., & Indiveri, G. (2023) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ace64c · Cited by 73
Strengths, Weaknesses, Opportunities, and Threats of Using ChatGPT in Scientific Research
Giray, L., Jacob, J., & Gumalin, D. L. (2024) · International Journal of Technology in Education · doi:10.46328/ijte.618 · Cited by 44
“Essentially, it is often unreliable when it comes to interpreting meanings and understanding queries (Strubell et al., 2019).”
BackgroundLarge language models in psychiatry: Opportunities and challenges
Volkmer, S., Meyer-Lindenberg, A., & Schwarz, E. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116026 · Cited by 59
Lauscher, A., Ravishankar, V., Vulić, I., & Glavas, G. (2020) · arXiv (Cornell University) · doi:10.17863/cam.62210 · Cited by 76
Wang, H., Li, Y., Hutch, M. R., Naidech, A. M., & Luo, Y. (2021) · Journal of Medical Internet Research · doi:10.2196/26302 · Cited by 72
Zicari, R. V., Brusseau, J., Blomberg, S. N. F., Christensen, H. C., Coffee, M. P., Ganapini, M. B., Gerke, S., Gilbert, T. K., Hickman, E., Hildt, E., et al. (2021) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2021.673104 · Cited by 79
“A requirement for AI systems that is becomingmore andmore salient is that their computing power should be also estimated (Strubell et al., 2019).”
BackgroundAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Yu, Z., Xiong, C., Shi, Y., & Liu, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.136 · Cited by 113
Don't be lazy: CompleteP enables compute-efficient deep transformers
Dey, N., Zhang, B., Noci, L., Li, M. B., Bordelon, B., Bergsma, S., Pehlevan, C., Hanin, B., & Hestness, J. (2025) · neural information processing systems · doi:10.52202/085713-4597 · Cited by 104
Röttger, P., & Pierrehumbert, J. B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.206 · Cited by 103
Commercial Use of Emotion Artificial Intelligence (AI): Implications for Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01330-7 · Cited by 74
ChatGPT in Hydrology and Earth Sciences: Opportunities, Prospects, and Concerns
Foroumandi, E., Moradkhani, H., Sánchez‐Vila, X., Singha, K., Castelletti, A. F., & Destouni, G. (2023) · Water Resources Research · doi:10.1029/2023wr036288 · Cited by 66
Managing Artificial Intelligence in Archeology. An overview
Gattiglia, G. (2024) · Journal of Cultural Heritage · doi:10.1016/j.culher.2024.11.020 · Cited by 72
AI in Global Health: The View from the Front Lines
Ismail, A., & Kumar, N. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445130 · Cited by 59
“…require less data ( e.g., by using transfer learning [68]), be directed about collecting data when required ( i.e. avoid collecting unnecessary data), compute less on the devices of the underserved, and build for lean computation and power consumption ( e.g., by es-chewing deep learning [18, 219]).”
BackgroundAutonomous Weapons Systems, Artificial Intelligence, and the Problem of Meaningful Human Control
Schwarz, E. (2021) · Philosophical Journal of Conflict and Violence · doi:10.22618/tp.pjcv.20215.1.139004 · Cited by 80
Onen, M., Émond, N., Li, J., Yildiz, B., & del Alamo, J. A. (2021) · Nano Letters · doi:10.1021/acs.nanolett.1c01614 · Cited by 68
Alimisis, V., Gennis, G., Touloupas, K., Dimas, C., Gourdouparis, M., & Sotiriadis, P. P. (2022) · Microelectronics Journal · doi:10.1016/j.mejo.2022.105510 · Cited by 66
Metz, L., Maheswaranathan, N., Freeman, C. D., Poole, B., & Sohl‐Dickstein, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11243 · Cited by 74
“Machine learning training and inference is a major energy consumer, and training models likely dominates most individual machine learning researcher’s carbon emissions [46].”
BackgroundLyu, W., Zhang, S., Chung, T., Sun, Y., & Zhang, Y. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100383 · Cited by 68
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models
McDonald, J., Li, B., Frey, N., Tiwari, D., Gadepally, V., & Samsi, S. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.151 · Cited by 62
“For this reason, important works such as Patterson et al. (2021) and Strubell et al. (2019) rely on estimates extrapolated from industry averages for some language models in some of their analysis of large neural net training, including total floating point operations per second (FLOPS) and hardware…”
Memorisation versus Generalisation in Pre-trained Language Models
Tänzer, M., Ruder, S., & Rei, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.521 · Cited by 98
“On the other hand, this method reduces the training time and therefore energy consumption (Strubell et al., 2019) to one third of the original method on CoNLL03 and we expect the reduction to be even greater for larger datasets.”
MethodCurrent and Near-Term AI as a Potential Existential Risk Factor
Bucknall, B. S., & Dori-Hacohen, S. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534146 · Cited by 83
“Furthermore, it has been reported that training a single natural language processing (NLP) AI model produces 300,000 kilograms of carbon dioxide emissions [80].”
BackgroundCLAD: A realistic Continual Learning benchmark for Autonomous Driving
Verwimp, E., Yang, K., Parisot, S., Hong, L., McDonagh, S., Pérez-Pellitero, E., Lange, M. D., & Tuytelaars, T. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.02.001 · Cited by 91
Machine translation and its evaluation: a study
Mondal, S. K., Zhang, H., Kabir, H. M. D., Ni, K., & Dai, H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10423-5 · Cited by 72
“Another ethical issue is the sustainable AI (by extension sustainable NMT) development and deployment, as it has an adverse impact on the environment ecosystem due to vast power consumption and carbon emission (Kenny et al. 2020; Moorkens 2022; Strubell et al. 2019; van Wynsberghe 2021).”
BackgroundData science opportunities of large language models for neuroscience and biomedicine
Bzdok, D., Thieme, A., Levkovskyy, O., Wren, P. B., Ray, T. S., & Reddy, S. (2024) · Neuron · doi:10.1016/j.neuron.2024.01.016 · Cited by 72
“71 For the goal of training LLMs from scratch, the extent of necessary richness in compute-storage resources probably sidelines the large majority of institutions in industry, academia, and government on the planet.”
Text Analysis for Psychology: Methods, Principles, and PracticesPreprint
Kennedy, B., Ashokkumar, A., Boyd, R. L., & Dehghani, M. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/h2b8t · Cited by 71
Niazi, S. K. (2025) · Pharmaceuticals · doi:10.3390/ph18091271 · Cited by 61
“Training state-of-the-art foundation models can consume substantial computational resources, with carbon footprints comparable to those of major industrial processes [121].”
Beyond ‘AI for Social Good’ (AI4SG): social transformations—not tech-fixes—for health equity
Holzmeyer, C. (2021) · Interdisciplinary Science Reviews · doi:10.1080/03080188.2020.1840221 · Cited by 51
Toward Sustainable Artificial Intelligence: An Overview of Environmental Protection Uses and Issues
Pachot, A., & Patissier, C. (2023) · Green and Low-Carbon Economy · doi:10.47852/bonviewglce3202608 · Cited by 66
“Researchers measured the CO2 emissions associated with the development of a natural language processing model, which generated the same amount of CO2 as 5 cars over their lifetime, and the equivalent of 315 round trips by plane between New York and San Francisco [1].”
BackgroundAdvancing bioinformatics with language models: components, applications, and perspectives
Liu, J., Yang, M., Yu, Y., Xu, H., Wang, T., Li, K., & Zhou, X. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag367 · Cited by 72
Radiology AI and sustainability paradox: environmental, economic, and social dimensions
Koçak, B., Ponsiglione, A., Romeo, V., Ugga, L., Huisman, M., & Cuocolo, R. (2025) · Insights into Imaging · doi:10.1186/s13244-025-01962-2 · Cited by 74
“When considering both training and inference, the energy usage of the top 20 AI systems results in around 103 million metric tons of CO 2 -eq emissions, which corresponds to the yearly carbon footprint of up to 22 million average people [21, 22].”
InfluentialPutting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction
Rocca, R., & Yarkoni, T. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211026864 · Cited by 63
“This problem is widely acknowledged in the literature, and it is often framed in terms of effect sizes—that is, that significance isn’t per se a guarantee of predictive validity; effect size also matters (Cumming, 2013; Kline, 2013; Nakagawa & Cuthill, 2007; Sullivan & Feinn, 2012).”
BackgroundTripp, C. E., USDOE Office of Critical Minerals and Energy Innovation (CMEI), Perr‐Sauer, J., Gafur, J., Nag, A., Purkayastha, A., Zisman, S., & Bensen, E. A. (2026) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/3015048 · Cited by 67
da Cunha, W. L. M., Canuto, S., Viegas, F. A. R., Salles, T., Gomes, C., Mangaravite, V., Resende, E., Rosa, T. C., Gonçalves, M. A., & da Rocha, L. C. D. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102263 · Cited by 74
Drummond, D., & Coulet, A. (2022) · Journal of Medical Internet Research · doi:10.2196/39698 · Cited by 64
“Finally, it was recently shown that training a single AI model could generate CO 2 emissions equivalent to those of a passenger making 300 flights between New York and San Francisco [68].”
Sengupta, U., & Sengupta, U. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.995603 · Cited by 64
“Third, an emerging area of concern is the increased energy usage by large ICT projects, where the consumption of energy by more power intensive processes contributes to global warming (Strubell et al., 2019).”
BackgroundReporting electricity consumption is essential for sustainable AI
Debus, C., Piraud, M., Streit, A., Theis, F. J., & Götz, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00750-1 · Cited by 67
Value-laden disciplinary shifts in machine learning
Dotan, R., & Milli, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3373157 · Cited by 37
Metrics reloaded: Recommendations for image analysis validation
Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.01653 · Cited by 59
“R.M.S. was an active member of the ObD and InS, the biomedical and the cross-topic expert groups, reviewed the manuscript, and participated in surveys and workshops.”
Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
Kim, G., & Cho, K. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.508 · Cited by 59
“This level of excessive computation has further raised the concern over energy consumption as well (Schwartz et al., 2019; Strubell et al., 2019; Cao et al., 2020).”
BackgroundEnabling Design Methodologies and Future Trends for Edge AI: Specialization and Codesign
Hao, C., Dotzel, J., Xiong, J., Benini, L., Zhang, Z., & Chen, D. (2021) · IEEE Design and Test · doi:10.1109/mdat.2021.3069952 · Cited by 70
Stafford, I. S., Gosink, M. M., Mossotto, E., Ennis, S., & Hauben, M. (2022) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izac115 · Cited by 71
Beware of sustainable AI! Uses and abuses of a worthy goal
Heilinger, J., Kempt, H., & Nagel, S. K. (2023) · AI and Ethics · doi:10.1007/s43681-023-00259-8 · Cited by 69
“First attempts have been undertaken to check whether these hopes are actually likely to come to fruition (see [38]), and awareness for the environmental and human costs of these systems is increasing, e.g., with regard to the energy needed for training neural networks [3, 42].”
Lim, W. M. (2023) · Journal of Consumer Behaviour · doi:10.1002/cb.2188 · Cited by 62
Recent Advances in Generative AI for Healthcare Applications
Shokrollahi, Y., Colmenarez, J., Liu, W., Yarmohammadtoosky, S., Nikahd, M. M., Dong, P., Li, X., & Gu, L. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01908-0 · Cited by 66
Sparsity in transformers: A systematic literature review
Farina, M., Ahmad, U., Taha, A. T. M., Younes, H., Mesbah, Y., Yu, X., & Pedrycz, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127468 · Cited by 72
Pruning Deep Neural Networks for Green Energy-Efficient Models: A Survey
Tmamna, J., Ayed, E. B., Fourati, R., Gogate, M., Arslan, T., Hussain, A., & Ayed, M. B. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10313-0 · Cited by 75
“According to [45], training GPT-3 emits almost 500M carbons, equivalent to nearly five cars over their lifetime.”
Seven HCI Grand Challenges Revisited: Five-Year Progress
Stephanidis, C., Salvendy, G., Antona, M., Duffy, V. G., Gao, Q., Karwowski, W., Konomi, S., Nah, F. F., Ntoa, S., Rau, P. P., et al. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2450411 · Cited by 69
“Compounding the issue, the energy consumption of AI experiences exponential growth with an increase in model parameters (Strubell et al., 2019).”
Cristofaro, M., & Giardino, P. L. (2025) · Journal of Management History · doi:10.1108/jmh-01-2025-0002 · Cited by 71
Integrating Machine Learning Techniques in Semantic Fake News Detection
Brașoveanu, A., & Andonie, R. (2020) · Neural Processing Letters · doi:10.1007/s11063-020-10365-x · Cited by 63
“However, considering the fact that a lot of these new language models are quite large, expensive to train and potentially damaging to the environment [57], we think this is a task that is worth pursuing.”
BackgroundHow Far Are We From AGI: Are LLMs All We Need?
Feng, T., Jin, C., Liu, J., Zhu, K., Tu, H., Cheng, Z., Lin, G., & You, J. (2024) · Transactions on machine learning research · doi:10.48550/arxiv.2405.10313 · Cited by 95
“• Practical Limitations of LLMs and Foundation models : Addressing external challenges like system constraints, computational costs [25], data acquisition barriers, and privacy concerns [29].”
Sustainability of AI: The Case of Provision of Information to Consumers
Kindylidi, I., & Cabral, T. S. (2021) · Sustainability · doi:10.3390/su132112064 · Cited by 75
“concluded that the training process of a single, deep learning, natural language processing model can lead to approximately 600,000 lb of carbon dioxide emissions [6], which roughly amounts to as much carbon dioxide emissions as the ones produced by five cars in their lifetime [7].”
BackgroundTowards improving e-commerce customer review analysis for sentiment detection
Singh, U., Saraswat, A., Azad, H. K., Abhishek, K., & Selvarajan, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-26432-3 · Cited by 65
Electrocardiogram (ECG)-Based User Authentication Using Deep Learning Algorithms
Agrawal, V., Fard, S. M. H., Elmiligi, H., & Gebali, F. (2023) · Diagnostics · doi:10.3390/diagnostics13030439 · Cited by 67
“CNN has been used for various cognitive activities, including computer vision [29], natural language [30], and others.”
MethodAmunts, K., Axer, M., Banerjee, S., Bitsch, L., Bjaalie, J. G., Brauner, P., Brovelli, A., Calarco, N., Carrère, M., Caspers, S., et al. (2024) · Imaging Neuroscience · doi:10.1162/imag_a_00137 · Cited by 82
Learn to Code Sustainably: An Empirical Study on Green Code Generation
Vartziotis, T., Dellatolas, I., Dasoulas, G., Schmidt, M., Schneider, F., Hoffmann, T., Kotsopoulos, S. D., & Keckeisen, M. (2024) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3643795.3648394 · Cited by 75
“Several studies have estimated the training cost of AI models, including natural language processing (NLP) models that are highly computationally expensive [26, 29, 31].”
BackgroundInfluentialEfficiency Is Not Enough: A Critical Perspective on Environmentally Sustainable AI
Wright, D., Igel, C., Samuel, G. N., & Selvan, R. (2025) · Communications of the ACM · doi:10.1145/3724500 · Cited by 72
BoMaNet: boolean masking of an entire neural network
Dubey, A., Cammarota, R., & Aysu, A. (2020) · doi:10.1145/3400302.3415649 · Cited by 77
“are costly to develop and hence become valuable IPs for the companies [65].”
BackgroundExploring the Accuracy – Energy Trade-off in Machine Learning
Brownlee, A. E. I., Adair, J., Haraldsson, S. O., & Jabbo, J. (2021) · IEEE/ACM International Workshop on Genetic Improvement (GI) · doi:10.1109/gi52543.2021.00011 · Cited by 80
“As datacentres are predicted to consume 8–21% of global electricity by 2025, it is estimated that training a large deep-learning model produces 284 000 kgs of the greenhouse gas carbon dioxide, equivalent to the lifetime emissions of five cars [1].”
BackgroundUMLAUT: Debugging Deep Learning Programs using Program Structure and Model Behavior
Schoop, E., Huang, F., & Hartmann, B. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445538 · Cited by 77
“DNN training time can take days or weeks, often even requiring online tuning in order to converge [69], lengthening the feedback loop.”
BackgroundMehlin, V., Schacht, S., & Lanquillon, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.01980 · Cited by 69
“encourage researchers to consider these two values when evaluating models ([3], [22], [101], [102]).”
BackgroundThe Ethics of Advanced AI Assistants
Gabriel, I., Manzini, A., Keeling, G., Hendricks, L. A., Rieser, V., Iqbal, H., Tomašev, N., Ktena, I., Kenton, Z., Rodriguez, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.16244 · Cited by 57
Metadata
- Authors
- Strubell, E., Ganesh, A., & McCallum, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 3645-3650
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2948223045
- Cited by
- 3,684
- References
- 15
- Citations by year
- 2026: 759 · 2025: 577 · 2024: 496 · 2023: 567 · 2022: 420 · 2021: 496 · 2020: 298 · 2019: 69
- FWCI
- 510.80
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1514053)
- Chan Zuckerberg Initiative
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Green IT and Sustainability
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- energy consumption, carbon footprint, large language models, environmental impact of AI, energy policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.