Energy and Policy Considerations for Deep Learning in NLP
Open accessThe DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index.
Abstract
Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.These models have obtained notable gains in accuracy across many NLP tasks.However, these accuracy improvements depend on the availability of exceptionally large computational resources that necessitate similarly substantial energy consumption.As a result these models are costly to train and develop, both financially, due to the cost of hardware and electricity or cloud compute time, and environmentally, due to the carbon footprint required to fuel modern tensor processing hardware.In this paper we bring this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training a variety of recently successful neural network models for NLP.Based on these findings, we propose actionable recommendations to reduce costs and improve equity in NLP research and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To enable real-world deployment of multimodal models in a variety of settings [142], there is a need to build lightweight models with cheap training and inference.”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [142]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Significant critiques of LLMs have been raised, including concerns about bias and fairness, misinformation, opaqueness due to a lack of interpretability and explainability, and environmental impact (Bender et al., 2021; Gebru et al., 2021; Strubell et al., 2019).”
From Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments (2023) · cites this paper as Strubell et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The training and development of algorithms generate a substantial portion of greenhouse gas emissions.”
From Artificial Intelligence and Sustainable Decisions (2023) · cites this paper as [137]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“criticism around large models has been growing lately [9], especially ethical concerns regarding undesirable and often inscrutable societal biases percolating the models [9, 120], carbon footprint [9, 99], misuse and misinterpretation [9], privatization of AI research [99], and even research opportunity costs [49].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [99]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A 2019 study indicates training a single LLM may generate as much carbon dioxide as the lifetime emission of five automobiles (70).”
From Can Generative AI improve social science? (2024) · cites this paper as [70]
Experimental — accuracy not yet measured
27 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“…enabled impressive accuracy improvements across many fundamental NLP tasks ( Bahdanau et al., 2015; Luong et al., 2015; Dozat and Man-ning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the highest scores (Peters et al., 2018; Devlin et al., 2019; Radford et…”
BackgroundInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“The BERT model (Devlin et al., 2019) provides a Transformer-based architecture for building contextual representations similar to ELMo, but trained with a different language modeling objective.”
BackgroundMethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“The BERT model (Devlin et al., 2019) provides a Transformer-based architecture for building contextual representations similar to ELMo, but trained with a different language modeling objective.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“…techniques and hardware for training deep neural networks have recently enabled impressive accuracy improvements across many fundamental NLP tasks ( Bahdanau et al., 2015; Luong et al., 2015; Dozat and Man-ning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the…”
BackgroundDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“The ELMo model (Peters et al., 2018) is based on stacked LSTMs and provides rich word representations in context by pre-training on a large amount of data using a language modeling objective.”
BackgroundMethodEffective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 2×
“…for training deep neural networks have recently enabled impressive accuracy improvements across many fundamental NLP tasks (Bahdanau et al., 2015; Luong et al., 2015; Dozat and Manning, 2017; Vaswani et al., 2017), with the most computationally-hungry models obtaining the highest scores (Peters…”
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 2×
“…the energy associated with model tuning is by providing easy-to-use APIs implementing more efficient alternatives to brute-force grid search for hyperparameter tuning, e.g. random or Bayesian hyperparameter search techniques (Bergstra et al., 2011; Bergstra and Bengio, 2012; Snoek et al., 2012).”
Method(2011) · Cited by 3,939
An Analysis of Deep Neural Network Models for Practical Applications
Canziani, A., Paszke, A., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1605.07678 · Cited by 969 · cited 1×
“nergy use required fortraining and inference in popular convolutional models for image classification in computer vision, including fine-grained analysis comparing different neural network layer types. Canziani et al. (2016) assess image classification model accuracy as a function of model size and gigaflops required duri…”
Method
Research works printed but not linked: 1
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In NAACL.
Cited by
Items 901–1,000 of 1,042
Nagendra, S., Kifer, D., Mirus, B. B., Pei, T., Lawson, K., manjunatha, S. B., Li, W., Nguyen, H., Qiu, T., Tran, S., et al. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3177025 · Cited by 34
NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge
Reddy, R. G., Chinthakindi, S. C., Wang, Z., Fung, Y. R., Conger, K., ELsayed, A., Palmer, M. S., Nakov, P., Hovy, E. H., Small, K., et al. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.403 · Cited by 51
“Environmental Impact We would also like to warn that the use of large-scale Transformers requires a lot of computations and the use of GPUs for training, which contributes to global warming (Strubell et al., 2019).”
BackgroundDas, D., Guha, S., Brubaker, J. R., & Semaan, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642669 · Cited by 50
“Moreover, by examining if those tools exhibit identity-based biases and how NLP tool biases are related to their developers’ demographic backgrounds, in the second section, we draw similarities in how sentiment analysis reanimates colonial hierarchies and underlines the politics of design.”
BackgroundMUMBO: MUlti-task Max-Value Bayesian Optimization
Moss, H. B., Leslie, D. S., & Rayson, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67664-3_27 · Cited by 43
RubberBand: cloud-based hyperparameter tuning
Misra, U., Liaw, R., Dunlap, L., Bhardwaj, R., Kandasamy, K., Gonzalez, J. E., Stoica, I. G., & Tumanov, A. (2021) · European Conference on Computer Systems · doi:10.1145/3447786.3456245 · Cited by 34
Energy Consumption of Deep Generative Audio ModelsPreprint
Douwes, C., Esling, P., & Briot, J.-P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02621 · Cited by 30
Parameter Prediction for Unseen Deep ArchitecturesPreprint
Knyazev, B. A., Drozdzal, M., Taylor, G. W., & Romero, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13100 · Cited by 120
“One of the negative impacts of training networks using iterative optimization methods is the environmental footprint [11, 12].”
BackgroundOn the foundations of Earth foundation models
Zhu, X. X., Xiong, Z., Wang, Y., Stewart, A. J., Heidler, K., Wang, Y., Yuan, Z., Dujardin, T., Xu, Q., & Shi, Y. (2026) · Communications Earth & Environment · doi:10.1038/s43247-025-03127-x · Cited by 26
“Earth AI 91 supports multiple tasks related to population and environmental dynamics, fostering task-agnostic Earth FMs. OceanGPT 92 , the fi rst LLM for ocean science, demonstrates expertise across various ocean applications.”
InfluentialAI and Aphasia in the Digital Age: A Critical Review
Privitera, A. J., Ng, S. H. S., Kong, A. P.-H., & Weekes, B. S. (2024) · Brain Sciences · doi:10.3390/brainsci14040383 · Cited by 26
“According to a study by Strubell et al. [103], training a single AI model can emit as much carbon dioxide as the lifetime emissions of five cars.”
ResultTransformer-based models for combating rumours on microblogging platforms: a review
Anggrainingsih, R., Hassan, G. M., & Datta, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10837-9 · Cited by 29
Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning
Zhang, X., Du, J., Li, Y., Xie, W., & Zhou, J. T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02477 · Cited by 61
“However, learning from large-scale datasets is both time-consuming and financially prohibitive [5, 24, 36, 37, 42, 46].”
BackgroundYu, M., Yang, H., Wei, X., Fu, S., Xu, J., & Zhang, J. (2026) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2026.3671111 · Cited by 33
“In 2020, Strubell et al. [11] highlighted in the field of natural language processing that training deep neural networks incurs substantial energy consumption and carbon emissions, spurring extensive research in green computing.”
Multiscale Modeling for Application-Oriented Optimization of Resistive Random-Access Memory
Torraca, P. L., Puglisi, F. M., Padovani, A., & Larcher, L. (2019) · Materials · doi:10.3390/ma12213461 · Cited by 29
The Role of Context in Detecting Previously Fact-Checked Claims
Shaar, S., Alam, F., Martino, G. D. S., & Nakov, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.122 · Cited by 43
On the Gap between Adoption and Understanding in NLP
Bianchi, F., & Hovy, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.340 · Cited by 35
“In addition to this problem, consider the environmental concerns generated by the training of large models (Strubell et al., 2019), and bigger does not necessarily equate better for reproducibility.”
BackgroundRedundancy and Analog Slicing for Precise In-Memory Machine Learning—Part I: Programming Techniques
Pedretti, G., Mannocci, P., Li, C., Sun, Z., Strachan, J. P., & Ielmini, D. (2021) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2021.3095433 · Cited by 28
How much pretraining data do language models need to learn syntax?
Pérez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.118 · Cited by 26
“As the resources used to train the MiniBERTas are not publicly available, we rely on the data provided in (Strubell et al., 2019)”
BackgroundMethodInfluentialAssisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document
Shaar, S., Georgiev, N., Alam, F., Martino, G. D. S., Mohamed, A., & Nakov, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.151 · Cited by 34
Managing Sustainability Tensions in Artificial Intelligence: Insights from Paradox Theory
Mill, E., Garn, W., & Ryman-Tubb, N. F. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534175 · Cited by 34
“The training of AI algorithms is highly energy-intensive with conse-quentially high carbon footprints [35, 41, 45].”
BackgroundDogra, V., Alharithi, F. S., Álvarez, R. M., Singh, A. D., & Qahtani, A. M. (2022) · Systems · doi:10.3390/systems10060233 · Cited by 28
“Writing rules for each label did not require a substantial amount of expert labour when compared to the efforts required to train the classifier on a training set that includes a collection of documents for each label with specified rules.”
BackgroundTowards Energy Consumption and Carbon Footprint Testing for AI-driven IoT Services
Trihinas, D., Thamsen, L., Beilharz, J., & Symeonides, M. (2022) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e55432.2022.00011 · Cited by 31
“To account for other components (i.e., cooling) in a (micro) datacenter setting, one can rely on the Power Usage Effectiveness (PUE) factor [15].”
MethodZesch, T., Horbach, A., & Zehner, F. (2023) · Educational Measurement Issues and Practice · doi:10.1111/emip.12544 · Cited by 29
Thinking Geographically about AI Sustainability
Shi, M., Currier, K., Liu, Z., Janowicz, K., Wiedemann, N., Verstegen, J. A., McKenzie, G., Graser, A., Zhu, R., & Mai, G. (2023) · AGILE GIScience Series · doi:10.5194/agile-giss-4-42-2023 · Cited by 39
“…settings that are energy efficient (Anthony et al., 2020; Patterson et al., 2021; Gupta et al., 2022; Henderson et al., 2020; La-coste et al., 2019; Strubell et al., 2019; Shaikh et al., 2021; Lannelongue et al., 2021); and selecting cloud providers that operate facilities with an optimal power…”
BackgroundTowards a Methodology and Framework for AI Sustainability Metrics
Eilam, T., Bello-Maldonado, P. D., Bhattacharjee, B., Costa, C., Lee, E. K., & Tantawi, A. N. (2023) · Workshop on Sustainable Computer Systems (HotCarbon) · doi:10.1145/3604930.3605715 · Cited by 32
“, training, (such as, [43] and [42]), or consider each phase in isolation (such as, [49]), we develop a metric that factors in the entire life cycle, including data preparation, training, and fine-tuning, to amortize the life cycle carbon cost of inference jobs.”
InfluentialB2T Connection: Serving Stability and Performance in Deep Transformers
Takase, S., Kiyono, S., Kobayashi, S., & Suzuki, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.192 · Cited by 42
“As discussed in Strubell et al. (2019) and Schwartz et al. (2019), such deep neural networks consume substantial amounts of energy.”
BackgroundACM TechBrief: Generative Artificial Intelligence
Leslie, D., & Rossi, F. (2023) · ACM eBooks · doi:10.1145/3626110 · Cited by 28
Reusing Deep Learning Models: Challenges and Directions in Software Engineering
Davis, J. C., Jajal, P., Jiang, W., Schorlemmer, T. R., Synovic, N. M., & Thiruvathukal, G. K. (2023) · IEEE John Vincent Atanasoff International Symposium on Modern Computing (JVA) · doi:10.1109/jva60410.2023.00015 · Cited by 38
“Second, this approach is highly energy-consumptive [138].”
EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
Wang, Y., Yue, Y., Lu, R., Han, Y., Song, S., & Huang, G. M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3401036 · Cited by 39
“Moreover, the considerable power consumption for training deep learning models will lead to significant carbon emissions [6], [7].”
BackgroundTowards Green AI: Current Status and Future Research
Clemm, C., Stobbe, L., Wimalawarne, K., & Druschke, J. (2024) · Electronics Goes Green · doi:10.23919/egg62010.2024.10631247 · Cited by 39
“Prominently, reference [24] analyzed the carbon impact of training their own state-of-the-art models, estimating that training one large transformer model takes 274,120 GPU hours, consuming 656,347 kWh energy in the process, resulting in 284 metric tons CO 2 e emissions, based on the average U.S.…”
BackgroundVoting with random classifiers (VORACE): theoretical and experimental analysis
Cornelio, C., Donini, M., Loreggia, A., Pini, M. S., & Rossi, F. (2021) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-021-09504-y · Cited by 30
“The importance of this property is corroborated by a recent line of work by [52] that suggests how industry and academia should focus their efforts on developing tools that reduce or avoid hyperparameters’ optimization, resulting in simpler methods that are also more sustainable in terms of energy and time consumption.”
BackgroundSparse and Continuous Attention Mechanisms
Martins, A. F. T., Farinhas, A., Treviso, M. V., Niculae, V., Aguiar, P. M. Q., & Figueiredo, M. A. T. (2020) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2006.07214 · Cited by 60
“While there is nothing specific about our research that poses environmental concerns or that promises to alleviate such concerns, our models share the same problematic property as other neural network models in terms of their energy consumption to train models and tune hyperparameters [47].”
BackgroundSingle Shot Structured Pruning Before TrainingPreprint
van Amersfoort, J., Alizadeh, M., Farquhar, S., Lane, N. D., & Gal, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00389 · Cited by 32
“However they do nothing to address the speed and computational cost of training the initial model, which can have a significant CO2 footprint [22].”
BackgroundInfluentialXtremeDistilTransformers: Task Transfer for Task-agnostic DistillationPreprint
Mukherjee, S., Awadallah, A. H., & Gao, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04563 · Cited by 31
“co/microsoft/xtremedistil-l12-h384-uncased The huge size poses significant challenges for downstream applications in terms of energy consumption and cost of inference (Strubell et al., 2019).”
BackgroundTowards Sustainable Artificial Intelligence: An Overview of Environmental Protection Uses and IssuesPreprint
Pachot, A., & Patissier, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.11738 · Cited by 32
“Researchers measured the CO2 emissions associated with the development of a natural language processing model, which generated the same amount of CO2 as 5 cars over their lifetime, and the equivalent of 315 round trips by plane between New York and San Francisco [1].”
BackgroundOn permutation-invariant neural networksPreprint
Kimura, M., Shimizu, R., Hirakawa, Y., Goto, R., & Saito, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.17410 · Cited by 35
“…some of these applications include image recognition (Sonka et al., 2014; Minaee et al., 2021; Guo et al., 2016), natural language processing (Strubell et al., 2019; Young et al., 2018; Otter et al., 2020; Deng & Liu, 2018), and recommendation systems (Portugal et al., 2018; Melville &…”
BackgroundGunser, V. E., Gottschling, S., Brucker, B., Richter, S., & Gerjets, P. (2021) · Communications in Computer and Information Science · doi:10.1007/978-3-030-78635-9_67 · Cited by 25
Does income inequality undermine the carbon abatement benefits of artificial intelligence?
Dong, Z., Zhang, L., Tan, C. P.-P., Luo, Q., & Zhang, L. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.143437 · Cited by 27
The Sunk Carbon Fallacy: Rethinking Carbon Footprint Metrics for Effective Carbon-Aware Scheduling
Bashir, N., Gohil, V., Subramanya, A. B., Shahrad, M., Irwin, D., Olivetti, E., & Delimitrou, C. (2024) · ACM Symposium on Cloud Computing · doi:10.1145/3698038.3698542 · Cited by 29
“This section provides an overview of efforts to improve computing’s sustainability, the metrics used in carbon-aware optimizations, and the research gaps in understanding sustainability metrics.”
Walsh, J., Cave, J. A. K., & Griffiths, F. E. (2024) · Journal of Medical Internet Research · doi:10.2196/54321 · Cited by 27
“Despite the development of ever-larger language models, such as GPT-3, which can be extremely resource heavy [36,37], there is an argument that to try to move nearer to the ultimate goal of natural language understanding, which is required to understand the complexity of patient experiences,…”
Augmented and Programmatically Optimized LLM Prompts Reduce Chemical Hallucinations
Reed, S. M. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c02322 · Cited by 30
Ortloff, A.-M., Martius, F., Meier, M., Sans-Raimbault, T., Geierhaas, L., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713671 · Cited by 30
“Increasingly large numbers of parameters also increase the amount of energy needed during training and development [11, 129], as well as increasing 𝐶𝑂 2 emissions [129] and water consumption, e.g. from cooling in data centers [158].”
Wu, W., & Zhang, Y. (2025) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2025.124211 · Cited by 29
Opportunities and Challenges of Foundation Models in Industrial Manufacturing
Ayyat, M., Osman, M., & Nadeem, T. (2025) · IEEE Access · doi:10.1109/access.2025.3589777 · Cited by 31
“The carbon footprint of training and deploying large-scale models is increasingly at odds with corporate sustainability targets [211].”
Usman, Y., Ihejirika, C. J., Offor, S. N., Akl, R., & Chataut, R. (2025) · IEEE Access · doi:10.1109/access.2025.3602451 · Cited by 28
“Training advanced models such as GPT-3 can emit approximately 626,000 pounds (284 metric tons) of carbon dioxide, comparable to the lifetime emissions of five average passenger vehicles [5].”
Transforming Speech-Language Pathology with AI: Opportunities, Challenges, and Ethical Guidelines
Georgiou, G. P. (2025) · Healthcare · doi:10.3390/healthcare13192460 · Cited by 36
“Training and serving large models can carry significant energy and carbon costs; rigorous studies show that model design, hardware, data-center efficiency, and grid mix can change the carbon footprint by orders of magnitude, and recommend reporting and reducing emissions [67,68].”
Artificial intelligence and robotics in predictive maintenance: a comprehensive review
Azeta, J., Omeche, T. T., Daniyan, I. A., Abiola, J. O., Daniyan, L., Phuluwa, H. S., & Muvunzi, R. (2026) · Frontiers in Mechanical Engineering · doi:10.3389/fmech.2025.1722114 · Cited by 35
“Publish energy and material accounting for both the PdM system (training and inference) and the avoided resource use (reduced downtime, extended asset life) using standard lifecycle or carbon-accounting methods (Strubell et al., 2019; Firdaus, 2023). iii.”
Reducing energy consumption of Neural Architecture Search: An inference latency prediction framework
Lu, L., & Lyu, B. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2021.102747 · Cited by 26
How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact
Jin, Z., Chauhan, G., Tse, B., Sachan, M., & Mihalcea, R. F. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.273 · Cited by 44
Quantifying relevance in learning and inference
Marsili, M., & Roudi, Y. (2022) · Physics Reports · doi:10.1016/j.physrep.2022.03.001 · Cited by 28
“in automatic translation) are limited to very specific tasks and require energy costs orders of magnitude greater than those employed by the human brain [16].”
BackgroundHampau, R. M., Kaptein, M. C., van Emden, R., Rost, T., & Malavolta, I. (2022) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3530019.3530025 · Cited by 36
“It is important to assess the potential implication of the fastpaced development of AI on our society, in a scenario where most interest in the subject of AI focuses towards improving the predictive accuracy of models [32].”
BackgroundFrom Demo to Design in Teaching Machine Learning
Bilstrup, K.-E. K., Kaspersen, M. H., Assent, I., Enni, S. A., & Petersen, M. G. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534634 · Cited by 30
Exploring Dimensionality Reduction Techniques in Multilingual Transformers
Huertas-García, Á., Martín, A. G., Huertas‐Tato, J., & Camacho, D. (2022) · Cognitive Computation · doi:10.1007/s12559-022-10066-8 · Cited by 33
The Role of AI in Driving the Sustainability of the Chemical Industry
Toniato, A., Schilter, O. T., & Laino, T. (2023) · CHIMIA International Journal for Chemistry · doi:10.2533/chimia.2023.144 · Cited by 28
Benchmarking Language Models for Code Syntax Understanding
Shen, D., Chen, X., Wang, C., Sen, K., & Song, D. X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.224 · Cited by 32
Anthropocentrism and Environmental Wellbeing in AI Ethics Standards: A Scoping Review and Discussion
Rigley, E., Chapman, A. P., Evers, C., & McNeill, W. (2023) · AI · doi:10.3390/ai4040043 · Cited by 32
“For example, the BERTlarge model was estimated to produce as much greenhouse gas emissions as a commercial flight between San Francisco and New York [93,94].”
MethodGenerative AI and Social Media May Exacerbate the Climate Crisis
van der Ven, H., Corry, D., Elnur, R. I., Provost, V. J., & Syukron, M. (2024) · Global Environmental Politics · doi:10.1162/glep_a_00747 · Cited by 33
“While some believe that the carbon cost of machine learning will eventually plateau and decline (Patterson et al. 2022), others believe that growing computational complexity will require increased energy consumption (Strubell et al. 2019).”
BackgroundLeuthe, D., Meyer-Hollatz, T., Plank, T., & Senkmüller, A. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10526-6 · Cited by 26
“Algorithmic and hardware advances have led to new generations of ML models with higher accuracy yet substantial energy consumption (Strubell et al., 2019).”
Deng, Z., Sun, R., Xue, M., Wen, S., Camtepe, S. A., Nepal, ., & Xiang, Y. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3532957 · Cited by 25
“It is estimated that training a single transformer model can generate CO2 emissions roughly equal to the total lifetime carbon footprint of 5 cars [72].”
Watt for What: Rethinking Deep Learning’s Energy-Performance Relationship
Gowda, S. N., Hao, X., Li, G., Gowda, S. N., Jin, X., & Sevilla-Lara, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92089-9_24 · Cited by 36
A NIME Of The Times: Developing an Outward-Looking Political Agenda For This Community
Morreale, F., Bin, S., McPherson, A., Stapleton, P., & Wanderley, M. M. (2020) · Research Portal (Queen's University Belfast) · doi:10.5281/zenodo.4813293 · Cited by 40
Baradaran, R., & Amirkhani, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.095 · Cited by 25
Neural Architecture Generator OptimizationPreprint
Ru, R., Esperança, P. M., & Carlucci, F. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01395 · Cited by 38
“This is detrimental both for the community, which has focused many works on marginally improving performance in a shallow pond, but also for the environment [61].”
BackgroundTowards Training Billion Parameter Graph Neural Networks for Atomic SimulationsPreprint
Sriram, A., Das, A., Wood, B. M., Goyal, S., & Zitnick, C. L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.09697 · Cited by 41
“The carbon emissions from training large deep learning models are non-trivial and the work we have presented here is no exception (Strubell et al., 2019; Schwartz et al., 2019).”
LLeMpower: Understanding Disparities in the Control and Access of Large Language ModelsPreprint
Sathish, V., Lin, H.-F. P. C.-C., Kamath, A. K., & Nyayachavadi, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.09356 · Cited by 28
“Other works [31, 32, 33] have looked at the energy costs of these models and the potential environmental impact.”
BackgroundHULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
Zhou, X., Chen, Z., Jin, X., & Wang, W. Y. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2021.eacl-demos.39 · Cited by 24
Saleh, M. E., Wazery, Y. M., & Ali, A. A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124153 · Cited by 31
Ferrati, F., Kim, P. H., & Muffatto, M. (2024) · Foundations and Trends® in Entrepreneurship · doi:10.1561/0300000121 · Cited by 31
Multimodal Artificial Intelligence in Medicine
Judge, C. S., Krewer, F., O'Donnell, M. J., Kiely, L., Sexton, D. J., Taylor, G. W., Skorburg, J. A., & Tripp, B. (2024) · Kidney360 · doi:10.34067/kid.0000000000000556 · Cited by 28
Inteligencia Artificial en la Educación Avances y Desafíos Multidisciplinarios.
Barahona-Martínez, G. E., Gallardo-Chiluisa, N. N., Caiza, Y. E. Q., Jiménez-Rivas, D. E., Caicedo-Basurto, R. L., Guanotuña-Yaulema, J. A., Flores-Cruz, P. L., & Hernández, D. M. G. (2024) · Editorial Grupo AEA eBooks · doi:10.55813/egaea.l.101 · Cited by 23
Language Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy
Alizadeh, N., Belchev, B., Saurabh, N., Kelbert, P., & Castor, F. (2025) · Empirical Software Engineering · doi:10.1109/msr66628.2025.00109 · Cited by 33
German End-to-end Speech Recognition based on DeepSpeech.
Agarwal, A., & Zesch, T. (2019) · KONVENS · Cited by 37
AI, Norms, Big Data, and the Law
Hydén, H. (2020) · Asian Journal of Law and Society · doi:10.1017/als.2020.36 · Cited by 30
“Hao (2019); Strubell, Ganesh, & McCallumb (2019).”
Energy-Harvesting Distributed Machine Learning
Güler, B., & Yener, A. (2021) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit45174.2021.9518045 · Cited by 26
Neural architecture search of echocardiography view classifiers
Azarmehr, N., Zolgharni, M., Ye, X., Howard, J. P., Lane, E. S., Labs, R. B., Shun-Shin, M. J., Cole, G. D., Bidaut, L. M., & Francis, D. P. (2021) · Journal of medical imaging · doi:10.1117/1.jmi.8.3.034002 · Cited by 28
“Such complexity results in using massive amounts of energy and requiring thousands of GPU hours or million-dollar cloud computing bills 22 to design neural network architectures.”
BackgroundRobust Detection of Malicious URLs with Self-Paced Wide & Deep Learning
Liang, Y., Wang, Q., Xiong, K., Zheng, X., Yu, Z., & Zeng, D. D. (2021) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3121388 · Cited by 43
ICT and Environmental Sustainability: A Comparative Study
Shaaban-Nejad, S., & Shirazi, F. (2022) · Sustainability · doi:10.3390/su14148651 · Cited by 30
Putra, R. V. W., Hanif, M. A., & Shafique, M. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.937782 · Cited by 29
“We consider efficiently performing FAT for minimizing training time, energy consumption, and carbon emission during the retraining process (Strubell et al., 2019, 2020), by conducting a small yet effective number of iterations for the retraining process, while avoiding accuracy collapse .”
BackgroundMethodInfluentialDevedžić, V. (2022) · Discover Artificial Intelligence · doi:10.1007/s44163-022-00038-0 · Cited by 30
“have calculated that the carbon emissions of training one large NLP model is roughly equivalent to the carbon emissions of five cars over their entire lifetime [70].”
BackgroundMolecular Property Prediction with Photonic Chip‐Based Machine Learning
Zhang, H., Lau, J. W. Z., Wan, L., Shi, L., Shi, Y., Cai, H. N., Luo, X., Lo, G., Lee, C., Kwek, L. C., et al. (2022) · Laser & Photonics Review · doi:10.1002/lpor.202200698 · Cited by 27
“Thus, training of ever more sophisticated machine learning models is expected to lead to the exponential growth of energy consumption [8], which comes with a huge environmental cost [9].”
BackgroundFraming the News: From Human Perception to Large Language Model Inferences
del Barrio, D. A., & Gática-Pérez, D. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592278 · Cited by 39
“In connection with this topic, Strubell et al.[55] argue that improvements in the accu-”
BackgroundMitigating carbon footprint for knowledge distillation based deep learning model compression
Rafat, K., Islam, S., Mahfug, A. A., Hossain, M. I., Rahman, F., Momen, S., Rahman, S., & Mohammed, N. (2023) · PLoS ONE · doi:10.1371/journal.pone.0285668 · Cited by 31
“[23] deduced that training large NLP frameworks trends towards a hefty carbon footprint that is detrimental to the environment.”
BackgroundMethodInfluentialJames, A. E., Novick, A., Rizzo, A. J., Parsons, R., Jang, K., Hattink, M. H. N., & Bergman, K. (2023) · Optica · doi:10.1364/optica.491756 · Cited by 24
From Chaos to Clarity: Claim Normalization to Empower Fact-Checking
Sundriyal, M., Chakraborty, T., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.439 · Cited by 41
Large language models and academic writing: Five tiers of engagement
Bekker, M. (2024) · South African Journal of Science · doi:10.17159/sajs.2024/17147 · Cited by 29
“Moreover, real and unconscionable human exploitation 11 and high environmental costs 12 , both present in current LLM models, cannot be discounted or wished away.”
BackgroundMaking AI Ethical by Design: The UNESCO Perspective
Ramos, G., Squicciarini, M., & Lamm, E. (2024) · Computer · doi:10.1109/mc.2023.3325949 · Cited by 27
Knowledge distillation on individual vertebrae segmentation exploiting 3D U-Net
Serrador, L., Villani, F. P., Moccia, S., & Santos, C. P. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102350 · Cited by 24
Balancing computational chemistry's potential with its environmental impact
Schilter, O. T., Schwaller, P., & Laino, T. (2024) · Green Chemistry · doi:10.1039/d4gc01745e · Cited by 27
Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruning
Gupta, M., Camci, E., Keneta, V. R., Vaidyanathan, A., Kanodia, R., James, A., Foo, C.-S., Wu, M., & Lin, J. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00144 · Cited by 31
“However, this poses a significant cost to the environment [4] and makes deployment on edge devices difficult [5].”
BackgroundSupporting Trustworthy AI Through Machine Unlearning
Hine, E., Novelli, C., Taddeo, M., & Floridi, L. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00500-5 · Cited by 40
“…transformer with 213 million parameters is estimated to emit over 600,000 pounds of CO 2 , or the lifetime emissions of about five cars (Hao, 2019; Strubell et al., 2019); GPT‑4, one of the largest transformer‑based large language models (LLMs), is reported to have over 1 trillion parameters…”
BackgroundTen Research Challenge Areas in Data Science
Wing, J. M. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.c6577b1f · Cited by 28
Alonso, P., Shridhar, K., Kleyko, D. V., Osipov, E., & Liwicki, M. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534359 · Cited by 24
“One of them [1] recommends a concerted effort by industry and academia to promote research of more computationally efficient algorithms.”
BackgroundMethodKantipudi, J., Dubey, S. R., & Chakraborty, S. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2020.3046167 · Cited by 30
“It has shown very promising performance in various domains, such as computer vision [3], [4], natural language processing [5], [6], health”
BackgroundNew universal sustainability metrics to assess edge intelligence
Lenherr, N., Pawlitzek, R., & Michel, B. (2021) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2021.100580 · Cited by 30
Beyond Fairness Metrics: Roadblocks and Challenges for Ethical AI in PracticePreprint
Chen, J., Storchan, V., & Kurshan, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06217 · Cited by 34
“As such, competing financial incentives, coupled with a general absence of pressures for accountable and responsible development practices ex ante [60, 70],”
BackgroundBridging the Gap: Integrating Ethics and Environmental Sustainability in AI Research and PracticePreprint
Luccioni, A. S., Pistilli, G., Sefala, R., & Moorosi, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.00797 · Cited by 29
“[192].”
Influentialİnanlar, B., & Altay, F. (2025) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2025.105491 · Cited by 25
General-Purpose Models for the Chemical Sciences: LLMs and Beyond
Alampara, N., Aneesh, A., Ríos-García, M., Mirza, A., Schilling-Wilhelmi, M., Aghajani, A. A., Sun, M., Prastalo, G., & Jablonka, K. M. (2026) · Chemical Reviews · doi:10.1021/acs.chemrev.5c00583 · Cited by 37
The future of human-artificial intelligence nexus and its environmental costs
Špelda, P., & Střítecký, V. (2020) · Futures · doi:10.1016/j.futures.2020.102531 · Cited by 29
Adhikari, A., Ram, A., Tang, R., Hamilton, W. L., & Lin, J. (2020) · Workshop on Representation Learning for NLP · doi:10.18653/v1/2020.repl4nlp-1.10 · Cited by 27
“However, these models consume immense computational resources (Strubell et al., 2019).”
BackgroundArtificial Intelligence Needs Environmental Ethics
Baum, S. D., & Owe, A. (2023) · Ethics, Policy & Environment · doi:10.1080/21550085.2022.2076538 · Cited by 22
Metadata
- Authors
- Strubell, E., Ganesh, A., & McCallum, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 3645-3650
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2948223045
- Cited by
- 3,684
- References
- 15
- Citations by year
- 2026: 759 · 2025: 577 · 2024: 496 · 2023: 567 · 2022: 420 · 2021: 496 · 2020: 298 · 2019: 69
- FWCI
- 510.80
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1514053)
- Chan Zuckerberg Initiative
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Green IT and Sustainability
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- energy consumption, carbon footprint, large language models, environmental impact of AI, energy policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.