Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
Abstract
Machine learning sits at the core of many essential products and services at Facebook. This paper describes the hardware and software infrastructure that supports machine learning at global scale. Facebook's machine learning workloads are extremely diverse: services require many different types of models in practice. This diversity has implications at all layers in the system stack. In addition, a sizable fraction of all data stored at Facebook flows through machine learning pipelines, presenting significant challenges in delivering data to high-performance distributed training flows. Computational requirements are also intense, leveraging both GPU and CPU platforms for training and abundant CPU capacity for real-time inference. Addressing these and other emerging challenges continues to require diverse efforts that span machine learning algorithms, software, and hardware design.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The hardware constraints involved in massive training runs (including memory limitations and communication bandwidths) force users to segment massive models into groups of layers, which are then trained in parallel.
“The hardware constraints involved in massive training runs (including memory limitations and communication bandwidths) force users to segment massive models into groups of layers, which are then trained in parallel (Hazelwood et al., 2018; Huang et al., 2019; Athlur et al., 2021).”
From Compute Trends Across Three Eras of Machine Learning (2022) · cites this paper as Hazelwood et al., 2018
The model may be integrated directly into the application code, traditionally first having to be rewritten for production.
“The model may be integrated directly into the application code [19,29,45], traditionally first having to be rewritten for production [21].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [21]
Another reported method of integration is to serialize the model and load it at runtime, possibly with an ML framework optimized for production.
“Another reported method of integration is to serialize the model (e.g., into ONNXfoot_2 , HDF5, Joblib 4 ) and load it at runtime, possibly with an ML framework that is optimized for production [9,21,39,40].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [21]
[21] reported solving the framework tradeoff by decoupling the frameworks from the model by transferring it from research to production using an exchange format for neural networks.
“[21] reported solving this tradeoff by decoupling the frameworks from the model by transferring it from research to production using an exchange format for neural networks.”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [21]
Cites
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · see citing works
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · see citing works
Self-citationCourbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106 · see citing works
Practical Lessons from Predicting Clicks on Ads at Facebook
He, X., Pan, J. J., Jin, O., Xu, T., Liu, B., Xu, T., Shi, Y., Atallah, A., Herbrich, R., Bowers, S., et al. (2014) · doi:10.1145/2648584.2648589 · Cited by 892 · see citing works
BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1Preprint
Courbariaux, M., & Bengio, Y. (2016) · arXiv (Cornell University) · Cited by 484 · see citing works
Ternary neural networks for resource-efficient AI applications
Alemdar, H. O., Leroy, V., Prost-Boucle, A., & Pétrot, F. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966166 · Cited by 236 · see citing works
Deep Learning for Computer Architects
Reagen, B., Adolf, R., Whatmough, P. N., Wei, G.-Y., & Brooks, D. M. (2017) · Synthesis lectures on computer architecture · doi:10.1007/978-3-031-01756-8 · Cited by 26 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · see citing works
Compute Trends Across Three Eras of Machine Learning
Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891914 · Cited by 363 · see citing works
“The hardware constraints involved in massive training runs (including memory limitations and communication bandwidths) force users to segment massive models into groups of layers, which are then trained in parallel (Hazelwood et al., 2018; Huang et al., 2019; Athlur et al., 2021).”
BackgroundMeasuring the Carbon Intensity of AI in Cloud Instances
Dodge, J., Prewitt, T., Tachet des Combes, R., Odmark, E., Schwartz, R., Strubell, E., Luccioni, A. S., Smith, N. A., DeCario, N., & Buchanan, W. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533234 · Cited by 257 · see citing works
Lwakatare, L. E., Raj, A., Crnkovic, I., Bosch, J., & Olsson, H. H. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106368 · Cited by 241 · see citing works
Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-19034-7_14 · Cited by 214 · see citing works
Zuboff, S. (2022) · Organization Theory · doi:10.1177/26317877221129290 · Cited by 209 · see citing works
“In 2022, Facebook, already a leader in real-time data processing and machine learning (Chen et al., 2016; Hazelwood et al., 2018), completed work on what it describes as the world’s most powerful artificial intelligence supercomputer (Bhattacharyya, 2022).”
BackgroundA survey of techniques for optimizing deep learning on gpus
Mittal, S., & Vaishay, S. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101635 · Cited by 197 · see citing works
fpgaConvNet: Mapping regular and irregular convolutional neural networks on FPGAs
Venieris, S. I., & Bouganis, C.-S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2844093 · Cited by 161 · see citing works
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads
Kwon, H., Lai, L., Pellauer, M., Krishna, T., Chen, Y.-H., & Chandra, V. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00016 · Cited by 144 · see citing works
Sponge Examples: Energy-Latency Attacks on Neural Networks
Shumailov, I., Zhao, Y., Bates, D., Papernot, N., Mullins, R., & Anderson, R. (2021) · 2021 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp51992.2021.00024 · Cited by 125 · see citing works
ModelOps: Cloud-Based Lifecycle Management for Reliable and Trusted AI
Hummer, W., Muthusamy, V., Rausch, T., Dube, P., El Maghraoui, K., Murthi, A., & Oum, P. (2019) · 2019 IEEE International Conference on Cloud Engineering (IC2E) · doi:10.1109/ic2e.2019.00025 · Cited by 105 · see citing works
<i>BinFI</i>: an efficient fault injector for safety-critical machine learning systems
Chen, Z., Li, G., Pattabiraman, K., & DeBardeleben, N. (2019) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3295500.3356177 · Cited by 91 · see citing works
Lazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference
Choi, Y., Kim, Y., & Rhu, M. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00049 · Cited by 69 · see citing works
Parity models: erasure-coded resilience for prediction serving systems
Kosaian, J., Rashmi, K. V., & Venkataraman, S. (2019) · Proceedings of the 27th ACM Symposium on Operating Systems Principles · doi:10.1145/3341301.3359654 · Cited by 48 · see citing works
Distributed Machine Learning through Heterogeneous Edge Systems
Hu, H., Wang, D., & Wu, C. (2019) · arXiv · doi:10.48550/arxiv.1911.06949 · Cited by 47 · see citing works
Operationalizing machine learning models: a systematic literature review
Kolltveit, A. B., & Li, J. (2022) · Proceedings of the 1st Workshop on Software Engineering for Responsible AI · doi:10.1145/3526073.3527584 · Cited by 34 · see citing works
A Case for Managed and Model-less Inference Serving
Yadwadkar, N. J., Romero, F., Li, Q., & Kozyrakis, C. (2019) · Proceedings of the Workshop on Hot Topics in Operating Systems · doi:10.1145/3317550.3321443 · Cited by 31 · see citing works
Missing the Forest for the Trees: End-to-End AI Application Performance in Edge Data Centers
Richins, D., Doshi, D., Blackmore, M., Thulaseedharan Nair, A., Pathapati, N., Patel, A., Daguman, B., Dobrijalowski, D., Illikkal, R., Long, K., et al. (2020) · 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00049 · Cited by 23 · see citing works
Migrating a Recommendation System to Cloud Using ML Workflow
Chahal, D., Ojha, R., Choudhury, S. R., & Nambiar, M. (2020) · Companion of the ACM/SPEC International Conference on Performance Engineering · doi:10.1145/3375555.3384423 · Cited by 10 · see citing works
Mímir: Building and Deploying an ML Framework for Industrial IoT
Peticolas, D., Kirmayer, R., & Turaga, D. (2019) · 2019 International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2019.00065 · Cited by 2 · see citing works
- Authors
- Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., Kalro, A., Law, J., Lee, K., Lu, J., Noordhuis, P., Smelyanskiy, M., Xiong, L., & Wang, X.
- Affiliations
- Meta (United States) (US)
- ORCID
- Venue
- 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)
- Publisher
- IEEE
- Published
- 2018-02
- Pages
- 620-629
- ISSN
- 1530-0897 (ISSN-L)
- OpenAlex
- W2794670651
- Cited by
- 590
- References
- 7
- Citations by year
- 2026: 22 · 2025: 46 · 2024: 59 · 2023: 72 · 2022: 80 · 2021: 113 · 2020: 102 · 2019: 72 · 2018: 24
- FWCI
- 61.02
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Symposium on High-Performance Computer Architecture (HPCA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- machine learning infrastructure, distributed training, GPU computing, CPU-only inference, machine learning pipelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
