TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Open accessAbstract
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The safe form of Softmax is immune to overflow/underflow due to exponent.
“There is a safe form of (1), which is immune to this problem:”
From Online normalizer calculation for softmax (2018) · cites this paper as [13]
Section 6 (p. 10) discusses numerical instability and checking for nonfinite values, but does not describe a safe form of softmax. A search for softmax, exponent overflow or underflow, and subtracting the maximum found no passage addressing the claimed protection. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Self-citationLearning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Self-citationSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Self-citationBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Self-citationHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Self-citationLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · see citing works
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 3,622 · see citing works
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,189 · see citing works
Self-citationIsard, M., Budiu, M., Yu, Y., Birrell, A. D., & Fetterly, D. (2007) · European Conference on Computer Systems · doi:10.1145/1272996.1273005 · Cited by 2,433 · see citing works
Self-citationDeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309 · see citing works
Self-citationVinyals, O., Fortunato, M., & Jaitly, N. (2015) · neural information processing systems · Cited by 2,024 · see citing works
Self-citationA set of level 3 basic linear algebra subprograms
Dongarra, J. J., Croz, J. D., Hammarling, S. J., & Duff, I. (1990) · ACM Transactions on Mathematical Software · doi:10.1145/77626.79170 · Cited by 1,822 · see citing works
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468 · cited 1× · see citing works
“In our distributed environment, these different tasks are containers in jobs managed by a cluster scheduling sys-tem [51].”
One weird trick for parallelizing convolutional neural networksPreprint
Krizhevsky, A. (2014) · arXiv.org · doi:10.48550/arxiv.1404.5997 · Cited by 1,435 · see citing works
cuDNN: Efficient Primitives for Deep LearningPreprint
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 1,394 · see citing works
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211 · see citing works
(2013) · doi:10.1145/2491956.2462176 · Cited by 885 · see citing works
Multiple Object Recognition with Visual Attention
Ba, J., Mnih, V., & Kavukcuoglu, K. (2014) · arXiv · doi:10.48550/arxiv.1412.7755 · Cited by 808 · see citing works
Murray, D. G., McSherry, F., Isaacs, R., Isard, M., Barham, P., & Abadi, M. (2013) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/2517349.2522738 · Cited by 655 · see citing works
Self-citationBuilding high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 633 · see citing works
Self-citationProject Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 614 · see citing works
Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013) · arXiv · doi:10.48550/arxiv.1312.6082 · Cited by 582 · see citing works
Self-citationOn rectified linear units for speech processing
Zeiler, M., Ranzato, M., Monga, R., Mao, M. Z., Yang, K., Le, Q. V., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., et al. (2013) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2013.6638312 · Cited by 552 · see citing works
Self-citationVinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., & Hinton, G. (2014) · arXiv · doi:10.48550/arxiv.1412.7449 · Cited by 464 · see citing works
Self-citationTorch: a modular machine learning software library
Collobert, R., Bengio, S., & Mariéthoz, J. (2002) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 443 · see citing works
Executing a program on the MIT tagged-token dataflow architecture
Arvind, & Nikhil, R. S. (1990) · IEEE Transactions on Computers · doi:10.1109/12.48862 · Cited by 401 · see citing works
Massively Parallel Methods for Deep Reinforcement LearningPreprint
Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04296 · Cited by 390 · see citing works
Massively Multitask Networks for Drug DiscoveryPreprint
Ramsundar, B., Kearnes, S. M., Riley, P., Webster, D. R., Konerding, D. E., & Pande, V. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02072 · Cited by 383 · see citing works
An Introduction to Computational Networks and the Computational Network Toolkit
Yu, D., Eversole, A., Seltzer, M., Yao, K., Kuchaiev, O., Zhang, Y., Seide, F. T. B., Huang, Z., Guenter, B. K., Wang, H., et al. (2014) · Cited by 334 · see citing works
Self-citationChambers, C., Raniwala, A., Perry, F., Adams, S., Henry, R. R., Bradshaw, R., & Weizenbaum, N. (2010) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1806596.1806638 · Cited by 330 · see citing works
Multilingual acoustic models using distributed deep neural networks
Heigold, G., Vanhoucke, V., Senior, A., Nguyen, P., Ranzato, M., Devin, M., & Dean, J. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639348 · Cited by 277 · see citing works
Self-citationBuilding high-level features using large scale unsupervised learningPreprint
Le, Q. V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., & Ng, A. Y. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.6209 · Cited by 236 · see citing works
Self-citationCIEL: a universal execution engine for distributed data-flow computing
Murray, D. G., Schwarzkopf, M., Smowton, C., Smith, S. M., Madhavapeddy, A., & Hand, S. (2011) · doi:10.5555/1972457.1972470 · Cited by 230 · see citing works
Self-citationRossbach, C. J., Yu, Y., Currey, J., Martin, J.-P., & Fetterly, D. (2013) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/2517349.2522715 · Cited by 142 · see citing works
Self-citationMove Evaluation in Go Using Deep Convolutional Neural NetworksPreprint
Maddison, C. J., Huang, A., Sutskever, I., & Silver, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6564 · Cited by 110 · see citing works
Self-citationGlobal code motion/global value numbering
Click, C. (1995) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/207110.207154 · Cited by 109 · see citing works
Pedestrian detection with a Large-Field-Of-View deep network
Angelova, A., Krizhevsky, A., & Vanhoucke, V. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139256 · Cited by 86 · see citing works
Self-citationFrame-by-frame language identification in short utterances using deep neural networks
Gónzalez-Domínguez, J., Moreno, I. L., Moreno, P. J., & González-Rodríguez, J. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.08.006 · Cited by 58 · see citing works
Flagged first, then by citations
1 of the 42 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- [7][7] James Bergstra, Olivier Breuleux, Fr´ ed´ eric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: A CPU and GPU math expression compiler. In Proceedings of the Python for scientific computing conference (SciPy) , volume 4, page 3. Austin, TX, 2010. UMontreal PDF. -
- [8][8] Craig Chambers, Ashish Raniwala, Frances Perry, Stephen Adams, Robert R Henry, Robert Bradshaw, and Nathan Weizenbaum. FlumeJava: easy, efficient data-parallel pipelines. In ACM Sigplan Notices , volume 45, pages 363-375. ACM, 2010. research.google.com/pubs/archive/35650.pdf. -
- [22][22] Sepp Hochreiter and J¨ urgen Schmidhuber. Long shortterm memory. Neural computation , 9(8):1735-1780, 1997. ftp.idsia.ch/pub/juergen/lstm.pdf. -
- [24][24] Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In ACM SIGOPS Operating Systems Review , volume 41, pages 59-72. ACM, 2007. www.michaelisard.com/pubs/eurosys07.pdf. -
- [36][36] Derek G Murray, Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Mart´ ın Abadi. Naiad: a timely dataflow system. In Proceedings of the TwentyFourth ACM Symposium on Operating Systems Principles , pages 439-455. ACM, 2013. Microsoft Research PDF. -
- [44][44] Christopher J Rossbach, Yuan Yu, Jon Currey, JeanPhilippe Martin, and Dennis Fetterly. Dandelion: a compiler and runtime for heterogeneous systems. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles , pages 49-68. ACM, 2013. research-srv.microsoft.com/pubs/201110/sosp13dandelion-final.pdf. -
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Self-citationChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Harris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844 · see citing works
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Google Earth Engine: Planetary-scale geospatial analysis for everyone
Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S., Thau, D., & Moore, R. (2017) · Remote Sensing of Environment · doi:10.1016/j.rse.2017.06.031 · Cited by 14,964 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · see citing works
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00482 · Cited by 7,781 · see citing works
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
Self-citationA guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312 · see citing works
Self-citationArtificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · see citing works
Self-citationOn hyperparameter optimization of machine learning algorithms: Theory and practice
Yang, L., & Shami, A. A. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.061 · Cited by 3,347 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
MoleculeNet: a benchmark for molecular machine learning
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., & Pande, V. (2018) · Chemical Science · doi:10.1039/c7sc02664a · Cited by 3,107 · see citing works
Deep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076 · see citing works
PointNet: Deep Learning on Point Sets for 3D Classification and SegmentationPreprint
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00593 · Cited by 2,929 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219890 · Cited by 3,733 · see citing works
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Miyato, T., Maeda, S., Koyama, M., & Ishii, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858821 · Cited by 2,860 · see citing works
Deep learning with long short-term memory networks for financial market predictions
Fischer, T. G., & Krauß, C. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.11.054 · Cited by 3,000 · see citing works
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic ForecastingPreprint
Li, Y., Yu, R., Shahabi, C., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01926 · Cited by 2,597 · see citing works
The Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 3,711 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310 · see citing works
Temporal Convolutional Networks for Action Segmentation and Detection
Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.113 · Cited by 2,238 · see citing works
Samaniego, E. P., Anitescu, C., Goswami, S., Nguyen‐Thanh, V. M., Guo, H., Hamdia, K. M., Zhuang, X., & Rabczuk, T. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2019.112790 · Cited by 2,139 · see citing works
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016) · arXiv · Cited by 2,067 · see citing works
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021 · see citing works
Frid-Adar, M., Diamant, I., Klang, E., Amitai, M. M., Goldberger, J., & Greenspan, H. K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.013 · Cited by 1,952 · see citing works
Automatic Differentiation in Machine Learning: a Survey
Baydin, A. G., Pearlmutter, B. A., Radul, A., & Siskind, J. M. (2015) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 2,055 · see citing works
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1606.05328 · Cited by 3,066 · see citing works
Self-citationTacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746 · see citing works
Self-citationDeepDTA: deep drug–target binding affinity prediction
Öztürk, H., Özgür, A., & Özkırımlı, E. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty593 · Cited by 1,728 · see citing works
Image reconstruction by domain-transform manifold learning
Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., & Rosen, M. S. (2018) · Nature · doi:10.1038/nature25988 · Cited by 1,694 · see citing works
Machine learning for medical imaging
Erickson, B. J., Korfiatis, P., Akkus, Z., & Kline, T. L. (2017) · RadioGraphics · doi:10.1148/rg.2017160130 · Cited by 1,672 · see citing works
A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., & Cottrell, G. W. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/366 · Cited by 1,648 · see citing works
Yang, Q., Yan, P., Zhang, Y., Yu, H., Shi, Y., Mou, X., Kalra, M. K., Zhang, Y., Sun, L., & Wang, G. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2827462 · Cited by 1,630 · see citing works
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603 · see citing works
Future Frame Prediction for Anomaly Detection - A New Baseline
Liu, W., Luo, W., Lian, D., & Gao, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00684 · Cited by 1,601 · see citing works
Improved protein structure prediction using predicted interresidue orientations
Yang, J., Anishchenko, I., Park, H., Peng, Z., Ovchinnikov, S., & Baker, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1914677117 · Cited by 1,421 · see citing works
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
Anomaly Detection with Robust Deep Autoencoders
Zhou, C., & Paffenroth, R. C. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098052 · Cited by 1,548 · see citing works
Statistical and machine learning forecasting methods: Concerns and ways forward
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2018) · PLOS ONE · doi:10.1371/journal.pone.0194889 · Cited by 1,528 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Deep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,513 · see citing works
Deep learning for computational biology
Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016) · Molecular Systems Biology · doi:10.15252/msb.20156651 · Cited by 1,473 · see citing works
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471 · see citing works
The iNaturalist Species Classification and Detection Dataset
Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., & Belongie, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00914 · Cited by 1,451 · see citing works
Deep learning for universal linear embeddings of nonlinear dynamics
Lusch, B., Kutz, J., & Brunton, S. (2017) · Nature Communications · doi:10.1038/s41467-018-07210-0 · Cited by 1,834 · see citing works
Pouyanfar, S., Sadiq, S. B., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M., Chen, S., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,435 · see citing works
DeepSleepNet: a model for automatic sleep stage scoring based on raw single-channel EEG
Supratak, A., Dong, H., Wu, C., & Guo, Y. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2017.2721116 · Cited by 1,376 · see citing works
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353 · see citing works
SEGAN: Speech Enhancement Generative Adversarial Network
Pascual, S., Bonafonte, A., & Serrà, J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1428 · Cited by 1,351 · see citing works
Large Scale Incremental Learning
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., & Fu, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00046 · Cited by 1,313 · see citing works
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Ed, H. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220007 · Cited by 1,329 · see citing works
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00212 · Cited by 1,280 · see citing works
Large-Scale Evolution of Image ClassifiersPreprint
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 1,372 · see citing works
Self-citationMahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. Y. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.058 · Cited by 1,299 · see citing works
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,273 · see citing works
AMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323 · see citing works
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N. P., Patterson, D. A., Agrawal, G., Bajwa, R. S., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080246 · Cited by 1,431 · see citing works
Self-citationMachine learning on big data: Opportunities and challenges
Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.026 · Cited by 1,235 · see citing works
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180220 · Cited by 1,400 · see citing works
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., & Erhan, D. (2016) · arXiv · doi:10.48550/arxiv.1608.06019 · Cited by 1,208 · see citing works
Deep learning: new computational modelling techniques for genomics
Eraslan, G., Avsec, Ž., Gagneur, J., & Theis, F. J. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0122-6 · Cited by 1,229 · see citing works
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · see citing works
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,257 · see citing works
Malware traffic classification using convolutional neural network for representation learning
Wang, W., Zhu, M., Zeng, X., Ye, X., & Sheng, Y. (2017) · 2017 International Conference on Information Networking (ICOIN) · doi:10.1109/icoin.2017.7899588 · Cited by 1,177 · see citing works
Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN
Shi, H., Xu, M., & Li, R. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2686012 · Cited by 1,106 · see citing works
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057 · see citing works
Deep learning classifiers for hyperspectral imaging: A review
Paoletti, M. E., Haut, J. M., Plaza, J., & Plaza, A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.09.006 · Cited by 1,094 · see citing works
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · see citing works
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works
PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges
Unke, O. T., & Meuwly, M. (2019) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.9b00181 · Cited by 1,071 · see citing works
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · see citing works
Deep learning for short-term traffic flow prediction
Polson, N. G., & Sokolov, V. (2017) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2017.02.024 · Cited by 1,018 · see citing works
Zhang, J., Gajjala, S., Agrawal, P., Tison, G. H., Hallock, L. A., Beussink-Nelson, L., Lassen, M. H., Fan, E., Aras, M. A., Jordan, C., et al. (2018) · Circulation · doi:10.1161/circulationaha.118.034338 · Cited by 1,006 · see citing works
“17 Image manipulation (such as linear interpolation for resizing) was performed using OpenCV 3.0 and scikit image.”
MethodInfluentialEnd-to-end encrypted traffic classification with one-dimensional convolution neural networks
Wang, W., Zhu, M., Wang, J., Zeng, X., & Yang, Z. (2017) · 2017 IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2017.8004872 · Cited by 991 · see citing works
Zhang, Y., Wang, H., Chen, W., Zeng, J., Zhang, L., Wang, H., & Weinan, E. (2020) · Computer Physics Communications · doi:10.1016/j.cpc.2020.107206 · Cited by 979 · see citing works
High-performance brain-to-text communication via handwriting
Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2021) · Nature · doi:10.1038/s41586-021-03506-2 · Cited by 969 · see citing works
“We used the “Adam” gradient descent method (Abadi et al., 2016; Kingma and Ba, 2017).”
MethodRaissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10561 · Cited by 956 · see citing works
Greenwald, N. F., Miller, G., Moen, E., Kong, A., Kagel, A., Dougherty, T., Fullaway, C. C., McIntosh, B. J., Leow, K. X., Schwartz, M. S., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01094-0 · Cited by 953 · see citing works
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 969 · see citing works
Deep Learning Models for Wireless Signal Classification With Distributed Low-Cost Spectrum Sensors
Rajendran, S., Meert, W., Giustiniano, D., Lenders, V., & Pollin, S. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2835460 · Cited by 950 · see citing works
Chen, F., & Jahanshahi, M. R. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2764844 · Cited by 933 · see citing works
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Convolutional neural networks in medical image understanding: a survey
Sarvamangala, D. R., & Kulkarni, R. V. (2021) · Evolutionary Intelligence · doi:10.1007/s12065-020-00540-3 · Cited by 929 · see citing works
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Sener, O., & Savarese, S. (2017) · arXiv · doi:10.48550/arxiv.1708.00489 · Cited by 922 · see citing works
Enhancing Underwater Imagery Using Generative Adversarial Networks
Fabbri, C., Islam, J., & Sattar, J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460552 · Cited by 908 · see citing works
- Authors
- Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1603.04467
- OpenAlex
- W2271840356
- Cited by
- 9,408
- References
- 42
- Citations by year
- 2026: 70 · 2025: 343 · 2024: 764 · 2023: 868 · 2022: 847 · 2021: 1427 · 2020: 1317 · 2019: 1246 · 2018: 1462 · 2017: 920 · 2016: 140 · 2015: 2 · 2014: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- TensorFlow, machine learning framework, heterogeneous distributed systems, GPU computing, deep neural network, large-scale machine learning, mobile devices, open source software, distributed training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review