The DOI matches its DataCite record; 41 of the 42 references it cites were found in a registry or an index.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Open accessAbstract
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“There is a safe form of (1), which is immune to this problem:”
From Online normalizer calculation for softmax (2018) · cites this paper as [13]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Self-citationLearning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Self-citationSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Self-citationHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 3,622
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Self-citationIsard, M., Budiu, M., Yu, Y., Birrell, A. D., & Fetterly, D. (2007) · European Conference on Computer Systems · doi:10.1145/1272996.1273005 · Cited by 2,433
Self-citationDeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309
Self-citationcuDNN: Efficient Primitives for Deep LearningPreprint
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 2,058
Vinyals, O., Fortunato, M., & Jaitly, N. (2015) · neural information processing systems · Cited by 2,043
Self-citationA set of level 3 basic linear algebra subprograms
Dongarra, J. J., Croz, J. D., Hammarling, S. J., & Duff, I. (1990) · ACM Transactions on Mathematical Software · doi:10.1145/77626.79170 · Cited by 1,822
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468 · cited 1×
“In our distributed environment, these different tasks are containers in jobs managed by a cluster scheduling sys-tem [51].”
One weird trick for parallelizing convolutional neural networksPreprint
Krizhevsky, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.5997 · Cited by 1,372
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211
(2013) · doi:10.1145/2491956.2462176 · Cited by 885
Multiple Object Recognition with Visual Attention
Ba, J., Mnih, V., & Kavukcuoglu, K. (2014) · arXiv · doi:10.48550/arxiv.1412.7755 · Cited by 808
Murray, D. G., McSherry, F., Isaacs, R., Isard, M., Barham, P., & Abadi, M. (2013) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/2517349.2522738 · Cited by 655
Self-citationBuilding high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 646
Self-citationProject Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 614
Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013) · arXiv · doi:10.48550/arxiv.1312.6082 · Cited by 582
Self-citationMassively Multitask Networks for Drug DiscoveryPreprint
Ramsundar, B., Kearnes, S. M., Riley, P., Webster, D. R., Konerding, D. E., & Pande, V. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02072 · Cited by 567
On rectified linear units for speech processing
Zeiler, M., Ranzato, M., Monga, R., Mao, M. Z., Yang, K., Le, Q. V., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., et al. (2013) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2013.6638312 · Cited by 552
Self-citationVinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., & Hinton, G. (2014) · arXiv · doi:10.48550/arxiv.1412.7449 · Cited by 464
Self-citationTorch: a modular machine learning software library
Collobert, R., Bengio, S., & Mariéthoz, J. (2002) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 443
Executing a program on the MIT tagged-token dataflow architecture
Arvind, & Nikhil, R. S. (1990) · IEEE Transactions on Computers · doi:10.1109/12.48862 · Cited by 401
Massively Parallel Methods for Deep Reinforcement LearningPreprint
Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04296 · Cited by 390
Building high-level features using large scale unsupervised learningPreprint
Le, Q. V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., & Ng, A. Y. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.6209 · Cited by 341
Self-citationAn Introduction to Computational Networks and the Computational Network Toolkit
Yu, D., Eversole, A., Seltzer, M., Yao, K., Kuchaiev, O., Zhang, Y., Seide, F. T. B., Huang, Z., Guenter, B. K., Wang, H., et al. (2014) · Cited by 332
Self-citationChambers, C., Raniwala, A., Perry, F., Adams, S., Henry, R. R., Bradshaw, R., & Weizenbaum, N. (2010) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1806596.1806638 · Cited by 330
Multilingual acoustic models using distributed deep neural networks
Heigold, G., Vanhoucke, V., Senior, A., Nguyen, P., Ranzato, M., Devin, M., & Dean, J. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639348 · Cited by 277
Self-citationCIEL: a universal execution engine for distributed data-flow computing
Murray, D. G., Schwarzkopf, M., Smowton, C., Smith, S. M., Madhavapeddy, A., & Hand, S. (2011) · doi:10.5555/1972457.1972470 · Cited by 230
Self-citationRossbach, C. J., Yu, Y., Currey, J., Martin, J.-P., & Fetterly, D. (2013) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/2517349.2522715 · Cited by 142
Self-citationMove Evaluation in Go Using Deep Convolutional Neural NetworksPreprint
Maddison, C. J., Huang, A., Sutskever, I., & Silver, D. (2014) · International Conference on Learning Representations · doi:10.48550/arxiv.1412.6564 · Cited by 135
Self-citationGlobal code motion/global value numbering
Click, C. (1995) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/207110.207154 · Cited by 109
Pedestrian detection with a Large-Field-Of-View deep network
Angelova, A., Krizhevsky, A., & Vanhoucke, V. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139256 · Cited by 86
Self-citationFrame-by-frame language identification in short utterances using deep neural networks
Gónzalez-Domínguez, J., Moreno, I. L., Moreno, P. J., & González-Rodríguez, J. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.08.006 · Cited by 58
Research works printed but not linked: 6
- [7][7] James Bergstra, Olivier Breuleux, Fr´ ed´ eric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: A CPU and GPU math expression compiler. In Proceedings of the Python for scientific computing conference (SciPy) , volume 4, page 3. Austin, TX, 2010. UMontreal PDF. -
- [8][8] Craig Chambers, Ashish Raniwala, Frances Perry, Stephen Adams, Robert R Henry, Robert Bradshaw, and Nathan Weizenbaum. FlumeJava: easy, efficient data-parallel pipelines. In ACM Sigplan Notices , volume 45, pages 363-375. ACM, 2010. research.google.com/pubs/archive/35650.pdf. -
- [22][22] Sepp Hochreiter and J¨ urgen Schmidhuber. Long shortterm memory. Neural computation , 9(8):1735-1780, 1997. ftp.idsia.ch/pub/juergen/lstm.pdf. -
- [24][24] Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In ACM SIGOPS Operating Systems Review , volume 41, pages 59-72. ACM, 2007. www.michaelisard.com/pubs/eurosys07.pdf. -
- [36][36] Derek G Murray, Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Mart´ ın Abadi. Naiad: a timely dataflow system. In Proceedings of the TwentyFourth ACM Symposium on Operating Systems Principles , pages 439-455. ACM, 2013. Microsoft Research PDF. -
- [44][44] Christopher J Rossbach, Yuan Yu, Jon Currey, JeanPhilippe Martin, and Dennis Fetterly. Dandelion: a compiler and runtime for heterogeneous systems. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles , pages 49-68. ACM, 2013. research-srv.microsoft.com/pubs/201110/sosp13dandelion-final.pdf. -
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Self-citationChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Metadata
- Authors
- Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1603.04467
- OpenAlex
- W2271840356
- Cited by
- 9,408
- References
- 42
- Citations by year
- 2026: 70 · 2025: 343 · 2024: 764 · 2023: 868 · 2022: 847 · 2021: 1427 · 2020: 1317 · 2019: 1246 · 2018: 1462 · 2017: 920 · 2016: 140
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- TensorFlow, machine learning framework, heterogeneous distributed systems, GPU computing, deep neural network, large-scale machine learning, mobile devices, open source software, distributed training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
