CoChat
Sign Up
PreprintCited by 405Green IT and SustainabilityEnergy Efficiency in ComputingEthics and Social Impacts of AI

Sustainable AI: Environmental Implications, Challenges and Opportunities

Open access
Wu, Carole-Jean, Raghavendra, Ramya, Gupta, Udit, Acun, Bilge, Ardalani, Newsha, Maeng, Kiwan, Chang, Gloria, Behram, Fiona Aga +17 more·2021·arXiv
DOI 10.48550/arxiv.2111.00364arXiv 2111.00364
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the carbon footprint of AI computing by examining the model development cycle across industry-scale machine learning use cases and, at the same time, considering the life cycle of system hardware. Taking a step further, we capture the operational and manufacturing carbon footprint of AI computing and present an end-to-end analysis for what and how hardware-software design and at-scale optimization can help reduce the overall carbon footprint of AI. Based on the industry experience and lessons learned, we share the key challenges and chart out important development directions across the many dimensions of AI. We hope the key messages and insights presented in this paper can inspire the community to advance the field of AI in an environmentally-responsible manner.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2111.00364

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 100 · Cited by 405

Research works printed but not linked: 94

  1. [4][4] C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo, C. Ho, W. Hu, T. Lavril, A. Palizhati, M. Riviere, M. Shuaibi, A. Sriram, K. Tran, B. Wood, J. Yoon, D. Parikh, and Z. Ulissi, 'An introduction to electrocatalyst design using machine learning for renewable energy storage,' arXiv preprint arXiv:2010.09435 , 2020. - · arXiv
  2. [8][8] D. Rolnick, P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. S. Ross, N. Milojevic-Dupont, N. Jaques, A. WaldmanBrown, A. Luccioni, T. Maharaj, E. D. Sherwin, S. K. Mukkavilli, K. P. Kording, C. Gomes, A. Y. Ng, D. Hassabis, J. C. Platt, F. Creutzig, J. Chayes, and Y. Bengio, 'Tackling climate change with machine learning,' arXiv:1906.05433 , 2019. - · arXiv
  3. [9][9] R. Nishant, M. Kennedy, and J. Corbett, 'Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda,' International Journal of Information Management , vol. 53, 2020. -
  4. [11][11] D. Mudigere, Y. Hao, J. Huang, A. Tulloch, S. Sridharan, X. Liu, M. Ozdal, J. Nie, J. Park, L. Luo, J. A. Yang, L. Gao, D. Ivchenko, A. Basant, Y. Hu, J. Yang, E. K. Ardestani, X. Wang, R. Komuravelli, C. Chu, S. Yilmaz, H. Li, J. Qian, Z. Feng, Y. Ma, J. Yang, E. Wen, H. Li, L. Yang, C. Sun, W. Zhao, D. Melts, K. Dhulipala, K. R. Kishore, T. Graf, A. Eisenman, K. K. Matam, A. Gangidi, G. J. Chen, M. Krishnan, A. Nayak, K. Nair, B. Muthiah, M. khorashadi, P. Bhattacharya, P. Lapukhov, M. Naumov, L. Qiao, M. Smelyanskiy, B. Jia, and V. Rao, 'Software-hardware co-design for fast and scalable training of deep learning recommendation models,' arXiv preprint arXiv:2104.05158 , 2021. - · arXiv
  5. [12][12] D. Hernandez and T. B. Brown, 'Measuring the algorithmic efficiency of neural networks,' arXiv preprint arXiv:2005.04305 , 2020. - · arXiv
  6. [13][13] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. HerbertVoss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, 'Language models are few-shot learners,' arXiv preprint arXiv:2005.14165 , 2020. - · arXiv
  7. [15][15] X. Yi, Y.-F. Chen, S. Ramesh, V. Rajashekhar, L. Hong, N. Fiedel, N. Seshadri, L. Heldt, X. Wu, and E. H. Chi, 'Factorized deep retrieval and distributed tensorflow serving,' in Proceedings of Machine Learning and Systems , 2018. -
  8. [16][16] W. Zhao, D. Xie, R. Jia, Y. Qian, R. Ding, M. Sun, and P. Li, 'Distributed hierarchical gpu parameter server for massive scale deep learning ads systems,' arXiv preprint arXiv:2003.05622 , 2020. - · arXiv
  9. [17][17] M. Lui, Y. Yetim, O. Ozkan, Z. Zhao, S.-Y. Tsai, C.-J. Wu, and M. Hempstead, 'Understanding capacity-driven scale-out neural recommendation inference,' in Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software , 2021. -
  10. [18][18] S. Rajbhandari, O. Ruwase, J. Rasley, S. Smith, and Y. He, 'Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning,' arXiv preprint arXiv:2104.07857 , 2021. - · arXiv
  11. [19][19] U. Gupta, Y. Kim, S. Lee, J. Tse, H. S. Lee, G. Wei, D. Brooks, and C. Wu, 'Chasing carbon: The elusive environmental footprint of computing,' in Proceedings of the IEEE International Symposium on High-Performance Computer Architecture , 2021. -
  12. [20][20] N. Tomasev, J. Cornebise, F. Hutter, S. Mohamed, A. Picciariello, B. Connelly, D. Belgrave, D. Ezer, F. C. van der Haert, F. Mugisha, G. Abila, H. Arai, H. Almiraat, J. Proskurnia, K. Snyder, M. OtakeMatsuura, M. Othman, T. Glasmachers, W. D. Wever, Y. Teh, M. E. Khan, R. D. Winne, T. Schaul, and C. Clopath, 'Ai for social good: unlocking the opportunity for positive impact,' Nature Communications , vol. 11, 2020. -
  13. [21][21] D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean, 'Carbon emissions and large neural network training,' arXiv preprint arXiv:2104.10350 , 2021. - · arXiv
  14. [24][24] K. Hazelwood, S. Bird, D. Brooks, S. Chintala, U. Diril, D. Dzhulgakov, M. Fawzy, B. Jia, Y. Jia, A. Kalro, J. Law, K. Lee, J. Lu, P. Noordhuis, M. Smelyanskiy, L. Xiong, and X. Wang, 'Applied machine learning at facebook: A datacenter infrastructure perspective,' in Proceedings of the IEEE International Symposium on High Performance Computer Architecture , 2018. -
  15. [25][25] A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzm´ an, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov, 'Unsupervised cross-lingual representation learning at scale,' arXiv preprint arXiv:1911.02116 , 2020. - · arXiv
  16. [26][26] M. Naumov, D. Mudigere, H.-J. M. Shi, J. Huang, N. Sundaraman, J. Park, X. Wang, U. Gupta, C.-J. Wu, A. G. Azzolini, D. Dzhulgakov, A. Mallevich, I. Cherniavskii, Y. Lu, R. Krishnamoorthi, A. Yu, V. Kondratenko, S. Pereira, X. Chen, W. Chen, V. Rao, B. Jia, L. Xiong, and M. Smelyanskiy, 'Deep learning recommendation model for personalization and recommendation systems,' arXiv preprint arXiv:1906.00091 , 2019. - · arXiv
  17. [27][27] U. Gupta, C.-J. Wu, X. Wang, M. Naumov, B. Reagen, D. Brooks, B. Cottel, K. Hazelwood, M. Hempstead, B. Jia, H.-H. S. Lee, A. Malevich, D. Mudigere, M. Smelyanskiy, L. Xiong, and X. Zhang, 'The architectural implications of facebook's dnn-based personalized recommendation,' in Proceedings of the IEEE International Symposium on High Performance Computer Architecture , 2020. -
  18. [28][28] E. Strubell, A. Ganesh, and A. McCallum, 'Energy and policy considerations for deep learning in nlp,' arXiv preprint arXiv:1906.02243 , 2019. - · arXiv
  19. [29][29] W. Fedus, B. Zoph, and N. Shazeer, 'Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,' CoRR , vol. abs/2101.03961, 2021. - · arXiv
  20. [33][33] L. Ke, U. Gupta, B. Y. Cho, D. Brooks, V. Chandra, U. Diril, A. Firoozshahian, K. Hazelwood, B. Jia, H.-H. S. Lee, M. Li, B. Maher, D. Mudigere, M. Naumov, M. Schatz, M. Smelyanskiy, X. Wang, B. Reagen, C.-J. Wu, M. Hempstead, and X. Zhang, 'Recnmp: Accelerating personalized recommendation with near-memory processing,' in Proceedings of the ACM/IEEE Annual International Symposium on Computer Architecture , 2020. -
  21. [34][34] Z. Deng, J. Park, P. T. P. Tang, H. Liu, J. Yang, H. Yuen, J. Huang, D. Khudia, X. Wei, E. Wen, D. Choudhary, R. Krishnamoorthi, C.-J. Wu, S. Nadathur, C. Kim, M. Naumov, S. Naghshineh, and M. Smelyanskiy, 'Low-precision hardware architectures meet recommendation model inference at scale,' IEEE Micro , vol. 41, no. 5, pp. 93-100, 2021. -
  22. [35][35] L. Wesolowski, B. Acun, V. Andrei, A. Aziz, G. Dankel, C. Gregg, X. Meng, C. Meurillon, D. Sheahan, L. Tian, J. Yang, P. Yu, and K. Hazelwood, 'Datacenter-scale analysis and optimization of gpu machine learning workloads,' IEEE Micro , vol. 41, no. 5, 2021. -
  23. [36][36] A. Sriraman, A. Dhanotia, and T. F. Wenisch, 'Softsku: Optimizing server architectures for microservice diversity @scale,' in Proceedings of the 46th International Symposium on Computer Architecture , Association for Computing Machinery, 2019. -
  24. [37][37] A. Sriraman and A. Dhanotia, 'Accelerometer: Understanding acceleration opportunities for data center overheads at hyperscale,' in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems , 2020. -
  25. [38][38] C. Tang, K. Yu, K. Veeraraghavan, J. Kaldor, S. Michelson, T. Kooburat, A. Anbudurai, M. Clark, K. Gogia, L. Cheng, B. Christensen, A. Gartrell, M. Khutornenko, S. Kulkarni, M. Pawlowski, T. Pelkonen, A. Rodrigues, R. Tibrewal, V. Venkatesan, and P. Zhang, 'Twine: A unified cluster management system for shared infrastructure,' in Proceedings of the USENIX Symposium on Operating Systems Design and Implementation , 2020. -
  26. [42][42] N. Sachdeva, C.-J. Wu, and J. McAuley, 'Svp-cf: Selection via proxy for collaborative filtering data,' arXiv preprint arXiv:2107.04984 , 2021. - · arXiv
  27. [44][44] M. Zhao, N. Agarwal, A. Basant, B. Gedik, S. Pan, M. Ozdal, R. Komuravelli, J. Pan, T. Bao, H. Lu, S. Narayanan, J. Langman, K. Wilfong, H. Rastogi, C. Wu, C. Kozyrakis, and P. Pol, 'Understanding and co-designing the data ingestion pipeline for industry-scale recsys training,' CoRR , vol. abs/2108.09373, 2021. - · arXiv
  28. [45][45] R. Turner, D. Eriksson, M. McCourt, J. Kiili, E. Laaksonen, Z. Xu, and I. Guyon, 'Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020,' CoRR , vol. abs/2104.10201, 2021. - · arXiv
  29. [46][46] P. Ren, Y. Xiao, X. Chang, P.-y. Huang, Z. Li, X. Chen, and X. Wang, 'A comprehensive survey of neural architecture search: Challenges and solutions,' ACM Comput. Surv. , vol. 54, no. 4, 2021. -
  30. [47][47] Q. Song, D. Cheng, H. Zhou, J. Yang, Y. Tian, and X. Hu, 'Towards automated neural interaction discovery for click-through rate prediction,' Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2020. -
  31. [48][48] M. R. Joglekar, C. Li, M. Chen, T. Xu, X. Wang, J. K. Adams, P. Khaitan, J. Liu, and Q. V. Le, 'Neural input search for large scale recommendation models,' in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2020. -
  32. [50][50] D. Eriksson, P. I. Chuang, S. Daulton, P. Xia, A. Shrivastava, A. Babu, S. Zhao, A. Aly, G. Venkatesh, and M. Balandat, 'Latency-aware neural architecture search with multi-objective bayesian optimization,' CoRR , vol. abs/2106.11890, 2021. - · arXiv
  33. [51][51] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, 'Once-for-all: Train one network and specialize it for efficient deployment,' arXiv preprint arXiv:1908.09791 , 2020. - · arXiv
  34. [52][52] D. Stamoulis, R. Ding, D. Wang, D. Lymberopoulos, B. Priyantha, J. Liu, and D. Marculescu, 'Single-path nas: Designing hardware-efficient convnets in less than 4 hours,' arXiv preprint arXiv:1904.02877 , 2019. - · arXiv
  35. [53][53] W. Chen, X. Gong, and Z. Wang, 'Neural architecture search on imagenet in four gpu hours: A theoretically inspired perspective,' arXiv preprint arXiv:2102.11535 , 2021. - · arXiv
  36. [54][54] J. Mellor, J. Turner, A. Storkey, and E. J. Crowley, 'Neural architecture search without training,' arXiv preprint arXiv:2006.04647 , 2021. - · arXiv
  37. [55][55] B. Acun, M. Murphy, X. Wang, J. Nie, C. Wu, and K. Hazelwood, 'Understanding training efficiency of deep learning recommendation models at scale,' in Proceedings of the IEEE International Symposium on High-Performance Computer Architecture , 2021. -
  38. [56][56] C. Yin, B. Acun, X. Liu, and C.-J. Wu, 'TT-Rec: Tensor train compression for deep learning recommendation models,' in Proceedings of the Conference on Machine Learning and Systems , 2021. -
  39. [57][57] W.-C. Kang, D. Z. Cheng, T. Yao, X. Yi, T. Chen, L. Hong, and E. H. Chi, 'Learning to embed categorical features without embedding tables for recommendation,' arXiv preprint arXiv:2010.10784 , 2021. - · arXiv
  40. [58][58] A. S. Nemirovskij and D. B. Yudin, Problem complexity and method efficiency in optimization . Wiley-Interscience, 1983. -
  41. [59][59] D. Choi, C. J. Shallue, Z. Nado, J. Lee, C. J. Maddison, and G. E. Dahl, 'On empirical comparisons of optimizers for deep learning,' arXiv preprint arXiv:1910.05446 , 2019. - · arXiv
  42. [60][60] P. T. Sivaprasad, F. Mai, T. Vogels, M. Jaggi, and F. Fleuret, 'Optimizer benchmarking needs to account for hyperparameter tuning,' in Proceedings of the International Conference on Machine Learning , 2020. -
  43. [61][61] P. Goyal, P. Doll´ ar, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He, 'Accurate, large minibatch sgd: Training imagenet in 1 hour,' arXiv preprint arXiv:1706.02677 , 2017. - · arXiv
  44. [62][62] M. Ott, S. Edunov, D. Grangier, and M. Auli, 'Scaling neural machine translation,' arXiv preprint arXiv:1806.00187 , 2018. - · arXiv
  45. [63][63] D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, 'Qsgd: Communication-efficient sgd via gradient quantization and encoding,' in Proceedings of the Advances in Neural Information Processing Systems , vol. 30, 2017. -
  46. [64][64] T. Vogels, S. P. Karinireddy, and M. Jaggi, 'Powersgd: Practical lowrank gradient compression for distributed optimization,' in Proceedings of the Advances In Neural Information Processing Systems , vol. 32, 2019. -
  47. [65][65] Y. Huang, Y. Cheng, A. Bapna, O. Firat, D. Chen, M. Chen, H. Lee, J. Ngiam, Q. V. Le, Y. Wu, et al. , 'Gpipe: Efficient training of giant neural networks using pipeline parallelism,' in Proceedings of the Advances in neural information processing systems , vol. 32, 2019. -
  48. [66][66] S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, 'Zero: Memory optimizations toward training trillion parameter models,' in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , 2020. -
  49. [67][67] J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He, 'Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,' in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2020. -
  50. [68][68] N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers, R. Boyle, P.-l. Cantin, C. Chao, C. Clark, J. Coriell, M. Daley, M. Dau, J. Dean, B. Gelb, T. V. Ghaemmaghami, R. Gottipati, W. Gulland, R. Hagmann, C. R. Ho, D. Hogberg, J. Hu, R. Hundt, D. Hurt, J. Ibarz, A. Jaffey, A. Jaworski, A. Kaplan, H. Khaitan, D. Killebrew, A. Koch, N. Kumar, S. Lacy, J. Laudon, J. Law, D. Le, C. Leary, Z. Liu, K. Lucke, A. Lundin, G. MacKean, A. Maggiore, M. Mahony, K. Miller, R. Nagarajan, R. Narayanaswami, R. Ni, K. Nix, T. Norrie, M. Omernick, N. Penukonda, A. Phelps, J. Ross, M. Ross, A. Salek, E. Samadiani, C. Severn, G. Sizikov, M. Snelham, J. Souter, D. Steinberg, A. Swing, M. Tan, G. Thorson, B. Tian, H. Toma, E. Tuttle, V. Vasudevan, R. Walter, W. Wang, E. Wilcox, and D. H. Yoon, 'In-datacenter performance analysis of a tensor processing unit,' in Proceedings of the ACM/IEEE International Symposium on Computer Architecture , 2017. -
  51. [73][73] M. Gschwind, T. Kaldewey, and D. Tam, 'Optimizing the efficiency of deep learning through accelerator virtualization,' IBM Journal of Research and Development , vol. 61, no. 4-5, 2017. -
  52. [74][74] S. Ghodrati, B. H. Ahn, J. Kyung Kim, S. Kinzer, B. R. Yatham, N. Alla, H. Sharma, M. Alian, E. Ebrahimi, N. S. Kim, C. Young, and H. Esmaeilzadeh, 'Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks,' in Proceedings of the IEEE/ACM International Symposium on Microarchitecture , 2020. -
  53. [75][75] S.-C. Kao and T. Krishna, 'Domain-specific genetic algorithm for multitenant dnnaccelerator scheduling,' arXiv preprint arXiv:2104.13997 , 2021. - · arXiv
  54. [76][76] M. Jeon, S. Venkataraman, A. Phanishayee, u. Qian, W. Xiao, and F. Yang, 'Analysis of large-scale multi-tenant gpu clusters for dnn training workloads,' in Proceedings of the USENIX Annual Technical Conference , 2019. -
  55. [77][77] P. Yu and M. Chowdhury, 'Salus: Fine-grained gpu sharing primitives for deep learning applications,' arXiv preprint arXiv:1902.04610 , 2019. - · arXiv
  56. [78][78] R. Jain and J. Wullert, 'Challenges: Environmental design for pervasive computing systems,' in Proceedings of the International Conference on Mobile Computing and Networking , 2002. -
  57. [79][79] J. Chang, J. Meza, P. Ranganathan, C. Bash, and A. Shah, 'Green server design: Beyond operational energy to sustainability,' in Proceedings of the International Conference on Power Aware Computing and Systems , 2010. -
  58. [80][80] M. Garcia Bardon, P. Wuytens, L.-A. Ragnarsson, G. Mirabelli, D. Jang, G. Willems, A. Mallik, A. Spessot, J. Ryckaert, and B. Parvais, 'DTCO including sustainability: Power-performance-area-cost-environmental score (PPACE) analysis for logic technologies,' in Proceedings of the IEEE International Electron Devices Meeting , 2020. -
  59. [81][81] A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. P. Gopal, J. Gray, M. Haselman, S. Hauck, S. Heil, A. Hormati, J.-Y. Kim, S. Lanka, J. Larus, E. Peterson, S. Pope, A. Smith, J. Thong, P. Y. Xiao, and D. Burger, 'A reconfigurable fabric for accelerating large-scale datacenter services,' IEEE Micro , 2015. -
  60. [82][82] Y.-H. Chen, J. Emer, and V. Sze, 'Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks,' in Proceedings of the ACM/IEEE International Symposium on Computer Architecture , 2016. -
  61. [83][83] A. Radovanovic, R. Koningstein, I. Schneider, B. Chen, A. Duarte, B. Roy, D. Xiao, M. Haridasan, P. Hung, N. Care, et al. , 'Carbon-aware computing for datacenters,' arXiv preprint arXiv:2106.11750 , 2021. - · arXiv
  62. [84][84] H. Cai, C. Gan, L. Zhu, and S. Han, 'Tinytl: Reduce memory, not parameters for efficient on-device learning,' arXiv preprint arXiv:2007.11622 , 2020. - · arXiv
  63. [86][86] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Koneˇ cn` y, S. Mazzocchi, H. B. McMahan, et al. , 'Towards federated learning at scale: System design,' arXiv preprint arXiv:1902.01046 , 2019. - · arXiv
  64. [87][87] A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage, 'Federated learning for mobile keyboard prediction,' arXiv preprint arXiv:1811.03604 , 2018. - · arXiv
  65. [88][88] T. Yang, G. Andrew, H. Eichner, H. Sun, W. Li, N. Kong, D. Ramage, and F. Beaufays, 'Applied federated learning: Improving google keyboard query suggestions,' arXiv preprint arXiv:1812.02903 , 2018. - · arXiv
  66. [89][89] S. Ramaswamy, R. Mathews, K. Rao, and F. Beaufays, 'Federated learning for emoji prediction in a mobile keyboard,' arXiv preprint arXiv:1906.04329 , 2019. - · arXiv
  67. [90][90] D. Huba, J. Nguyen, K. Malik, R. Zhu, M. Rabbat, A. Yousefpour, C.-J. Wu, H. Zhan, P. Ustinov, H. Srinivas, K. Wang, A. Shoumikhin, J. Min, and M. Malek, 'Papaya: Practical, private, and scalable federated learning,' arXiv:2111.04877 , 2021. - · arXiv
  68. [91][91] K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu, M. Martin, T. Nagarajan, I. Radosavovic, S. K. Ramakrishnan, F. Ryan, J. Sharma, M. Wray, M. Xu, E. Z. Xu, C. Zhao, S. Bansal, D. Batra, V. Cartillier, S. Crane, T. Do, M. Doulaty, A. Erapalli, C. Feichtenhofer, A. Fragomeni, Q. Fu, C. Fuegen, A. Gebreselasie, C. Gonzalez, J. Hillis, X. Huang, Y. Huang, W. Jia, W. Khoo, J. Kolar, S. Kottur, A. Kumar, F. Landini, C. Li, Y. Li, Z. Li, K. Mangalam, R. Modhugu, J. Munro, T. Murrell, T. Nishiyasu, W. Price, P. R. Puentes, M. Ramazanova, L. Sari, K. Somasundaram, A. Southerland, Y. Sugano, R. Tao, M. Vo, Y. Wang, X. Wu, T. Yagi, Y. Zhu, P. Arbelaez, D. Crandall, D. Damen, G. M. Farinella, B. Ghanem, V. K. Ithapu, C. V. Jawahar, H. Joo, K. Kitani, H. Li, R. Newcombe, A. Oliva, H. S. Park, J. M. Rehg, Y. Sato, J. Shi, M. Z. Shou, A. Torralba, L. Torresani, M. Yan, and J. Malik, 'Ego4d: Around the world in 3,000 hours of egocentric video,' arXiv:2110.07058 , 2021. - · arXiv
  69. [92][92] Y. G. Kim and C.-J. Wu, 'Autofl: Enabling heterogeneity-aware energy efficient federated learning,' in Proceedings of the IEEE/ACM International Symposium on Microarchitecture , 2021. -
  70. [93][93] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, 'Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,' in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems , 2017. -
  71. [94][94] Y. G. Kim and C.-J. Wu, 'Autoscale: Energy efficiency optimization for stochastic edge inference using reinforcement learning,' in Proceedings of the IEEE/ACM International Symposium on Microarchitecture , 2020. -
  72. [95][95] T.-J. Yang, Y.-H. Chen, and V. Sze, 'Designing energy-efficient convolutional neural networks using energy-aware pruning,' arXiv:1611.05128 , 2017. - · arXiv
  73. [96][96] D. Stamoulis, T.-W. R. Chin, A. K. Prakash, H. Fang, S. Sajja, M. Bognar, and D. Marculescu, 'Designing adaptive neural networks for energy-constrained image classification,' in Proceedings of the International Conference on Computer-Aided Design , 2018. -
  74. [97][97] C. Gao, A. Gutierrez, M. Rajan, R. G. Dreslinski, T. Mudge, and C.J. Wu, 'A study of mobile device utilization,' in Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software , 2015. -
  75. [99][99] K. Lottick, S. Susai, S. A. Friedler, and J. P. Wilson, 'Energy usage reports: Environmental awareness as part of algorithmic accountability,' Workshop on Tackling Climate Change with Machine Learning at NeurIPS 2019 , 2019. -
  76. [101][101] D. Hernandez and T. B. Brown, 'Measuring the algorithmic efficiency of neural networks,' arXiv preprint arXiv:2005.04305 , 2020. - · arXiv
  77. [102][102] P. Mattson, V. J. Reddi, C. Cheng, C. Coleman, G. Diamos, D. Kanter, P. Micikevicius, D. Patterson, G. Schmuelling, H. Tang, G.-Y. Wei, and C.-J. Wu, 'Mlperf: An industry standard benchmark suite for machine learning performance,' IEEE Micro , vol. 40, no. 2, pp. 8-16, 2020. -
  78. [103][103] V. J. Reddi, C. Cheng, D. Kanter, P. Mattson, G. Schmuelling, and C.-J. Wu, 'The vision behind mlperf: Understanding ai inference performance,' IEEE Micro , vol. 41, no. 3, pp. 10-18, 2021. -
  79. [104][104] V. J. Reddi, D. Kanter, P. Mattson, J. Duke, T. Nguyen, R. Chukka, K. Shiring, K.-S. Tan, M. Charlebois, W. Chou, M. El-Khamy, J. Hong, M. Buch, C. Trinh, T. Atta-fosu, F. Cakir, M. Charkhabi, X. Chen, J. Chiang, D. Dexter, W. Heo, G. Schmuelling, M. Shabani, and D. Zika, 'Mlperf mobile inference benchmark,' arXiv:2012.02328 , 2021. - · arXiv
  80. [105][105] P. Mattson, C. Cheng, G. Diamos, C. Coleman, P. Micikevicius, D. Patterson, H. Tang, G.-Y. Wei, P. Bailis, V. Bittorf, D. Brooks, D. Chen, D. Dutta, U. Gupta, K. Hazelwood, A. Hock, X. Huang, D. Kang, D. Kanter, N. Kumar, J. Liao, D. Narayanan, T. Oguntebi, G. Pekhimenko, L. Pentecost, V. Janapa Reddi, T. Robie, T. St John, C.J. Wu, L. Xu, C. Young, and M. Zaharia, 'Mlperf training benchmark,' in Proceedings of Machine Learning and Systems , vol. 2, 2020. -
  81. [106][106] V. J. Reddi, C. Cheng, D. Kanter, P. Mattson, G. Schmuelling, C.-J. Wu, B. Anderson, M. Breughe, M. Charlebois, W. Chou, R. Chukka, C. Coleman, S. Davis, P. Deng, G. Diamos, J. Duke, D. Fick, J. S. Gardner, I. Hubara, S. Idgunji, T. B. Jablin, J. Jiao, T. S. John, P. Kanwar, D. Lee, J. Liao, A. Lokhmotov, F. Massa, P. Meng, P. Micikevicius, C. Osborne, G. Pekhimenko, A. T. R. Rajan, D. Sequeira, A. Sirasao, F. Sun, H. Tang, M. Thomson, F. Wei, E. Wu, L. Xu, K. Yamada, B. Yu, G. Yuan, A. Zhong, P. Zhang, and Y. Zhou, 'Mlperf inference benchmark,' in Proceedings of the ACM/IEEE Annual International Symposium on Computer Architecture , 2020. -
  82. [107][107] M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, 'Model cards for model reporting,' Proceedings of the Conference on Fairness, Accountability, and Transparency , 2019. -
  83. [108][108] C.-J. Wu, S. Manne, P. Ranganathan, S. Bird, and S. Greenstein, 'Socio-technological challenges and opportunities: Paths forward,' arXiv preprint arXiv:2108.06738 , 2021. - · arXiv
  84. [109][109] R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, 'Green ai,' arXiv preprint arXiv:1907.10597 , 2019. - · arXiv
  85. [110][110] K. Maeng, S. Bharuka, I. Gao, M. C. Jeffrey, V. Saraph, B.-Y. Su, C. Trippel, J. Yang, M. Rabbat, B. Lucia, and C.-J. Wu, 'Cpr: Understanding and improving failure tolerant training for deep learning recommendation with partial recovery,' in Proceedings of the Conference on Machine Learning and Systems , 2021. -
  86. [111][111] A. Eisenman, K. K. Matam, S. Ingram, D. Mudigere, R. Krishnamoorthi, K. Nair, M. Smelyanskiy, and M. Annavaram, 'Check-n-run: A checkpointing system for training deep learning recommendation models,' arXiv preprint arXiv:2010.08679 , 2021. - · arXiv
  87. [112][112] H. D. Dixit, S. Pendharkar, M. Beadon, C. Mason, T. Chakravarthy, B. Muthiah, and S. Sankar, 'Silent data corruptions at scale,' arXiv preprint arXiv:2102.11245 , 2021. - · arXiv
  88. [113][113] P. H. Hochschild, P. Turner, J. C. Mogul, R. Govindaraju, P. Ranganathan, D. E. Culler, and A. Vahdat, 'Cores that don't count,' in Proceedings of the Workshop on Hot Topics in Operating Systems , 2021. -
  89. [114][114] X. Qiu, T. Parcollet, J. Fernandez-Marques, P. P. B. de Gusmao, D. J. Beutel, T. Topal, A. Mathur, and N. D. Lane, 'A first look into the carbon footprint of federated learning,' arXiv preprint arXiv:2102.07627 , 2021. - · arXiv
  90. [115][115] H. Wang, B. Kim, J. Xie, and Z. Han, 'How is energy consumed in smartphone deep learning apps? executing locally vs. remotely,' in Proceedings of the IEEE Global Communications Conference , 2019. -
  91. [116][116] C.-J. Wu, D. Brooks, K. Chen, D. Chen, S. Choudhury, M. Dukhan, K. Hazelwood, E. Isaac, Y. Jia, B. Jia, T. Leyvand, H. Lu, Y. Lu, L. Qiao, B. Reagen, J. Spisak, F. Sun, A. Tulloch, P. Vajda, X. Wang, Y. Wang, B. Wasti, Y. Wu, R. Xian, S. Yoo, and P. Zhang, 'Machine learning at facebook: Understanding inference at the edge,' in Proceedings of the IEEE International Symposium on High Performance Computer Architecture , 2019. -
  92. [118][118] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, 'A simple framework for contrastive learning of visual representations,' in Proceedings of the International conference on machine learning , pp. 1597-1607, 2020. -
  93. [119][119] M. Assran, M. Caron, I. Misra, P. Bojanowski, A. Joulin, N. Ballas, and M. Rabbat, 'Semi-supervised learning of visual features by nonparametrically predicting view assignments with support samples,' arXiv preprint arXiv:2104.13963 , 2021. - · arXiv
  94. [120][120] L. M. Dery, P. Michel, A. Talwalkar, and G. Neubig, 'Should we be pre-training? an argument for end-task aware training as an alternative,' arXiv preprint arXiv:2109.07437 , 2021. Fig. 12. Model quality of recommendation use cases improves as we scale up the amount of data and/or the number of model parameters (e.g., embedding cardinality or dimension), leading to higher energy and carbon footprint. Maximizing model accuracy for the specific recommendation use case comes with significant energy cost - Roughly 4 × energy saving can be achieved with only 0.004 model quality degradation (green vs. yellow stars). <!-- image --> · arXiv

Authors
Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Behram, F. A., Huang, J., Bai, C., Gschwind, M., Gupta, A., Ott, M., Melnikov, A., Candido, S., Brooks, D., Chauhan, G., Lee, B., Lee, H.-H. S., Akyildiz, B., Balandat, M., Spisak, J., Jain, R., Rabbat, M., & Hazelwood, K.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2111.00364
arXiv
2111.00364
OpenAlex
W3208860256
Cited by
405
References
100
Citations by year
2026: 18 · 2025: 113 · 2024: 178 · 2023: 73 · 2022: 23
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by-nc-nd/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Green IT and Sustainability
  • Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
sustainable AI, environmental impact of machine learning, hardware-software co-design, energy-efficient algorithms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review