Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1,620

The DOI matches its Crossref record; 204 of the 204 references it cites were found in a registry or an index.

A State-of-the-Art Survey on Deep Learning Theory and Architectures

Open access
Alom, Md Zahangir, Taha, Tarek M., Yakopcic, Chris, Westberg, Stefan, Sidike, Paheding, Nasrin, Mst Shamima, Hasan, Mahmudul, Van Essen, Brian C., Awwal, Abdul A. S., Asari, Vijayan K.·2019·Electronics
DOI 10.3390/electronics8030292
Open this paper in CoChat — free account

Abstract

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/electronics8030292

Cites

204 of 204 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 129

  1. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (arXiv, 2013). Playing atari with deep reinforcement learning, arXiv.
  2. Simonyan, K., and Zisserman, A. (arXiv, 2014). deep convolutional networks for large-scale image recognition, arXiv.
  3. Canziani, A., Paszke, A., and Culurciello, E. (arXiv, 2016). An analysis of deep neural network models for practical applications, arXiv.
  4. Zweig, G. (2012, January 25–30). Classification and recognition with direct segment models. Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan. · doi:10.1109/icassp.2012.6288835
  5. He, Y., and Fosler-Lussier, E. (2012, January 9–13). Efficient segmental conditional random fields for one-pass phone recognition. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA. · doi:10.21437/interspeech.2012-515
  6. Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2015). Attention-based models for speech recognition. Advances in Neural Information Processing Systems, MIT Press.
  7. Li, Y., Yu, R., Shahabi, C., and Liu, Y. (arXiv, 2017). Graph Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting, arXiv.
  8. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, The MIT Press.
  9. Kaiser, L., Gomez, A.N., Shazeer, N., Vaswani, A., Parmar, N., Jones, L., and Uszkoreit, J. (arXiv, 2017). One model to learn them all, arXiv.
  10. Singh, K., Gupta, G., Vig, L., Shroff, G., and Agarwal, P. (arXiv, 2017). Deep Convolutional Neural Networks for Pairwise Causality, arXiv.
  11. Kim, T., Cha, M., Kim, H., Lee, J.K., and Kim, J. (arXiv, 2017). Learning to discover cross-domain relations with generative adversarial networks, arXiv.
  12. Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., and Lee, H. (arXiv, 2016). Generative adversarial text to image synthesis, arXiv.
  13. Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., and Cai, J. (arXiv, 2015). Recent advances in convolutional neural networks, arXiv.
  14. Li, Y. (arXiv, 2017). Deep reinforcement learning: An overview, arXiv.
  15. Schuman, C.D., Potok, T.E., Patton, R.M., Birdwell, J.D., Dean, M.E., Rose, G.S., and Plank, J.S. (arXiv, 2017). A survey of neuromorphic computing and neural networks in hardware, arXiv.
  16. Yoshua, B., Lamblin, P., Popovici, D., and Larochelle, H. (2007). Greedy Layer-Wise Training of Deep Network. Advances in Neural Information Processing Systems 19 (NIPS 2006), MIT Press.
  17. Lin, M., Chen, Q., and Yan, S. (arXiv, 2013). Network in network, arXiv.
  18. Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (arXiv, 2014). Striving for simplicity: The all convolutional net, arXiv.
  19. Larsson, G., Maire, M., and Shakhnarovich, G. (arXiv, 2016). FractalNet: Ultra-Deep Neural Networks without Residuals, arXiv.
  20. Abdi, M., and Nahavandi, S. (arXiv, 2016). Multi-Residual Networks: Improving the Speed and Accuracy of Residual Networks, arXiv.
  21. Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv.
  22. Sabour, S., Frosst, N., and Hinton, G.E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems (NIPS), MIT Press.
  23. Bansal, A., Chen, X., Russell, B., Gupta, A., and Ramanan, D. (arXiv, 2017). Pixelnet: Representation of the pixels, by the pixels, and for the pixels, arXiv.
  24. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., and Tu, Z. (2015, January 9–12). Deeply-supervised nets. Proceedings of the Artificial Intelligence and Statistics, San Diego, CA, USA.
  25. Pezeshki, M., Fan, L., Brakel, P., Courville, A., and Bengio, Y. (2016, January 20–22). Deconstructing the ladder network architecture. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
  26. Ba, J., and Caruana, R. (2014). Do deep nets really need to be deep?. Advances in Neural Information Processing Systems, MIT Press. NIPS Proceedings.
  27. Urban, G., Geras, K.J., Kahou, S.E., Aslan, O., Wang, S., Caruana, R., Mohamed, A., Philipose, M., and Richardson, M. (arXiv, 2016). Do deep convolutional nets really need to be deep and convolutional?, arXiv.
  28. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (arXiv, 2014). Fitnets: Hints for thin deep nets, arXiv.
  29. Mishkin, D., and Matas, J. (arXiv, 2015). All you need is a good init, arXiv.
  30. Pandey, G., and Dukkipati, A. (arXiv, 2014). To go deep or wide in learning?, arXiv.
  31. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (arXiv, 2016). Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5 mb model size, arXiv.
  32. Han, S., Mao, H., and Dally, W.J. (arXiv, 2015). Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding, arXiv.
  33. Niepert, M., Ahmed, M., and Kutzkov, K. (arXiv, 2016). Learning Convolutional Neural Networks for Graphs, arXiv.
  34. Jia, X., Xu, X., Cai, B., and Guo, K. (2017). Single Image Super-Resolution Using Multi-Scale Convolutional Neural Network. Pacific Rim Conference on Multimedia, Springer.
  35. Ahn, B., and Cho, N.I. (arXiv, 2017). Block-Matching Convolutional Neural Network for Image Denoising, arXiv.
  36. Hammerla, N.Y., Halloran, S., and Ploetz, T. (arXiv, 2016). Deep, convolutional, and recurrent models for human activity recognition using wearables, arXiv.
  37. Rad, 2018, Deep learning for automatic stereotypical motor movement detection using wearable sensors in autism spectrum disorders, Signal Process., 144, 180, 10.1016/j.sigpro.2017.10.011 · DOI
  38. Alom, M.Z., Yakopcic, C., Taha, T.M., and Asari, V.K. (arXiv, 2018). Microscopic Nuclei Classification, Segmentation and Detection with improved Deep Convolutional Neural Network (DCNN) Approaches, arXiv.
  39. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (arXiv, 2014). Semantic image segmentation with deep convolutional nets and fully connected crfs, arXiv.
  40. Badrinarayanan, V., Kendall, A., and Cipolla, R. (arXiv, 2015). Segnet: A deep convolutional encoder-decoder architecture for image segmentation, arXiv.
  41. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., and Asari, V.K. (arXiv, 2018). Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation, arXiv. · DOI · Paper page
  42. Tran, P.V. (arXiv, 2016). A fully convolutional neural network for cardiac segmentation in short-axis MRI, arXiv.
  43. Alom, M.Z., Yakopcic, C., Taha, T.M., and Asari, V.K. (arXiv, 2018). Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural Network, arXiv. · DOI
  44. Orr, G., and Müller, K. (2012). Efficient BackProp. Neural Networks: Tricks of the Trade, Springer.
  45. Clevert, D.-A., Unterthiner, T., and Hochreiter, S. (arXiv, 2015). Fast and accurate deep network learning by exponential linear units (elus), arXiv.
  46. Xu, B., Wang, N., Chen, T., and Li, M. (arXiv, 2015). Empirical evaluation of rectified activations in convolutional network, arXiv.
  47. Graham, B. (arXiv, 2014). Fractional max-pooling, arXiv.
  48. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.R. (arXiv, 2012). Improving neural networks by preventing co-adaptation of feature detectors, arXiv.
  49. Wan, L., Zeiler, M., Zhang, S., le Cun, Y., and Fergus, R. (2013, January 16–21). Regularization of neural networks using dropconnect. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA.
  50. Bulò, S.R., Porzi, L., and Kontschieder, P. (2016, January 20–22). Dropout distillation. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
  51. Ruder, S. (arXiv, 2016). An overview of gradient descent optimization algorithms, arXiv.
  52. Koushik, J., and Hayashi, H. (arXiv, 2016). Improving stochastic gradient descent with feedback, arXiv.
  53. Hochreiter, S., Bengio, Y., Frasconi, P., and Schmidhuber, J. (2001). Gradient Flow in Recurrent Nets: The Difficulty of Learning Long-Term Dependencies, IEEE Press.
  54. Schmidhuber, J. (1993). Habilitation Thesis: Netzwerkarchitekturen, Zielfunktionen und Kettenregel (Network architectures, objective functions, and chain rule). [Ph.D. Thesis, Technische Universität München].
  55. Gers, 2002, Learning precise timing with LSTM recurrent networks, J. Mach. Learn. Res., 3, 115
  56. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (arXiv, 2014). Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv.
  57. Yao, K., Cohn, T., Vylomova, K., Duh, K., and Dyer, C. (arXiv, 2015). Depth-gated recurrent neural networks, arXiv.
  58. Koutnik, J., Greff, K., Gomez, F., and Schmidhuber, J. (arXiv, 2014). A clockwork rnn, arXiv.
  59. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (arXiv, 2013). Efficient estimation of word representations in vector space, arXiv.
  60. Goldberg, Y., and Levy, O. (arXiv, 2014). word2vec Explained: Deriving Mikolov et al.’s negative-sampling word-embedding method, arXiv.
  61. Oord, A.v.d., Kalchbrenner, N., and Kavukcuoglu, K. (arXiv, 2016). Pixel recurrent neural networks, arXiv.
  62. Xue, W., Nachum, I.B., Pandey, S., Warrington, J., Leung, S., and Li, S. (2017). Direct estimation of regional wall thicknesses via residual recurrent neural network. International Conference on Information Processing in Medical Imaging, Springer. · doi:10.1007/978-3-319-59050-9_40
  63. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems (NIPS), MIT Press.
  64. Lakhani, V.A., and Mahadev, R. (arXiv, 2016). Multi-Language Identification Using Convolutional Recurrent Neural Network, arXiv.
  65. Malhotra, P., Vishnu, T.V., Vig, L., Agarwal, P., and Shroff, G. (arXiv, 2017). TimeNet: Pre-trained deep recurrent neural network for time series classification, arXiv.
  66. Olabiyi, O., Martinson, E., Chintalapudi, V., and Guo, R. (arXiv, 2017). Driver Action Prediction Using Deep (Bidirectional) Recurrent Neural Network, arXiv.
  67. Ng, 2011, Sparse autoencoder, CS294A Lect. Notes, 72, 1
  68. Lu, J., Deshpande, A., and Forsyth, D. (arXiv, 2016). CDVAE: Co-embedding Deep Variational Auto Encoder for Conditional Variational Generation, arXiv.
  69. Ahmad, M., Protasov, S., and Khan, A.M. (arXiv, 2017). Hyperspectral Band Selection Using Unsupervised Non-Linear Deep Auto Encoder to Train External Classifiers, arXiv.
  70. Alom, M.Z., Sidike, P., Taha, T.M., and Asari, V.K. (arXiv, 2017). Handwritten bangla digit recognition using deep learning, arXiv.
  71. Albalooshi, 2018, Deep Belief Active Contours (DBAC) with Its Application to Oil Spill Segmentation from Remotely Sensed Aerial Imagery, Photogramm. Eng. Remote Sens., 84, 451, 10.14358/PERS.84.7.451 · doi:10.14358/pers.84.7.451
  72. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (arXiv, 2016). Improved techniques for training gans, arXiv.
  73. Vondrick, C., Pirsiavash, H., and Torralba, A. (2016). Generating videos with scene dynamics. Advances in Neural Information Processing Systems, MIT Press.
  74. Radford, A., Metz, L., and Chintala, S. (arXiv, 2015). Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv.
  75. Im, D.J., Kim, C.D., Jiang, H., and Memisevic, R. (arXiv, 2016). Generating images with recurrent adversarial net- works, arXiv.
  76. Liu, M.-Y., and Tuzel, O. (2016). Coupled generative adversarial networks. Advances in Neural Information Processing Systems, MIT Press.
  77. Donahue, J., Krähenbühl, P., and Darrell, T. (arXiv, 2016). Adversarial feature learning, arXiv.
  78. Berthelot, D., Schumm, T., and Metz, L. (arXiv, 2017). Began: Boundary equilibrium generative adversarial networks, arXiv.
  79. Martin, A., Chintala, S., and Bottou, L. (arXiv, 2017). Wasserstein gan, arXiv.
  80. Zhao, J., Mathieu, M., and LeCun, Y. (arXiv, 2016). Energy-based generative adversarial network, arXiv.
  81. Park, N., Anand, A., Moniz, J.R.A., Lee, K., Chakraborty, T., Choo, J., Park, H., and Kim, Y. (arXiv, 2017). MMGAN: Manifold Matching Generative Adversarial Network for Generating Images, arXiv. · DOI
  82. Eghbal-zadeh, H., and Widmer, G. (arXiv, 2017). Probabilistic Generative Adversarial Networks, arXiv.
  83. Fowkes, J., and Sutton, C. (2016). A Bayesian Network Model for Interesting Itemsets. Joint European Conference on Machine Learning and Knowledge Disco in Databases, Springer International Publishing. · doi:10.1007/978-3-319-46227-1_26
  84. Mescheder, L., Nowozin, S., and Geiger, A. (arXiv, 2017). Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks, arXiv.
  85. Nowozin, S., Cseke, B., and Tomioka, R. (2016). f-gan: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Processing Systems, MIT Press.
  86. (arXiv, 2017). 242. Hoang, Quan, Tu Dinh Nguyen, Trung Le, and Dinh Phung. Multi-Generator Gernerative Adversarial Nets, arXiv.
  87. Kansky, K., Silver, T., Mély, D.A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, S., and George, D. (arXiv, 2017). Schema networks: Zero-shot transfer with a generative causal model of intuitive physics, arXiv.
  88. Dash, A., Gamboa, J.C.B., Ahmed, S., Liwicki, M., and Afzal, M.Z. (arXiv, 2017). TAC-GAN-text conditioned auxiliary classifier generative adversarial network, arXiv.
  89. Zhang, H., and Dana, K. (arXiv, 2017). Multi-style Generative Network for Real-time Transfer, arXiv.
  90. Zhang, H., Sindagi, V., and Patel, V.M. (arXiv, 2017). Image De-raining Using a Conditional Generative Adversarial Network, arXiv.
  91. Mardani, M., Gong, E., Cheng, J.Y., Vasanawala, S., Zaharchuk, G., Alley, M., Thakur, N., Han, S., Dally, W., and Pauly, J.M. (arXiv, 2017). Deep generative adversarial networks for compressed sensing automates MRI, arXiv.
  92. Choi, E., Biswal, S., Malin, B., Duke, J., Stewart, W.F., and Sun, J. (arXiv, 2017). Generating Multilabel Discrete Electronic Health Records Using Generative Adversarial Networks, arXiv.
  93. Esteban, C., Hyland, S.L., and Rätsch, G. (arXiv, 2017). Real-valued (medical) time series generation with recurrent conditional gans, arXiv.
  94. Hayes, J., Melis, L., Danezis, G., and de Cristofaro, E. (arXiv, 2017). LOGAN: evaluating privacy leakage of generative models using generative adversarial networks, arXiv.
  95. Gordon, J., and Hernández-Lobato, J.M. (arXiv, 2017). Bayesian Semisupervised Learning with Deep Generative Models, arXiv.
  96. Abbasnejad, M.E., Shi, Q., Abbasnejad, I., van den Hengel, A., and Dick, A. (arXiv, 2017). Bayesian conditional generative adverserial networks, arXiv.
  97. Grnarova, P., Levy, K.Y., Lucchi, A., Hofmann, T., and Krause, A. (arXiv, 2017). An online learning approach to generative adversarial networks, arXiv.
  98. Li, Y., Swersky, K., and Zemel, R. (2015, January 6–11). Generative moment matching networks. Proceedings of the International Conference on Machine Learning, Lille, France.
  99. Saeedi, A., Hoffman, M.D., DiVerdi, S.J., Ghandeharioun, A., Johnson, M.J., and Adams, R.P. (arXiv, 2017). Multimodal prediction and personalization of photo edits with deep generative models, arXiv.
  100. Liu, M.-Y., Breuel, T., and Kautz, J. (2017). Unsupervised image-to-image translation networks. Advances in Neural Information Processing Systems, MIT Press. · DOI · Paper page
  101. Mehrotra, A., and Dukkipati, A. (arXiv, 2017). Generative Adversarial Residual Pairwise Networks for One Shot Learning, arXiv.
  102. Oord, A.v.d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (arXiv, 2016). Wavenet: A generative model for raw audio, arXiv.
  103. Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., and Feng, J. (2017). Dual path networks. Advances in Neural Information Processing Systems, MIT Press.
  104. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
  105. Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A.S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., and Schrittwieser, J. (arXiv, 2017). Starcraft ii: A new challenge for reinforcement learning, arXiv.
  106. Schulman, J., Levine, S., Abbeel, P., Jordan, M.I., and Moritz, P. (2015, January 6–11). Trust Region Policy Optimization. Proceedings of the 32nd International Conference on Machine Learning (ICML-15), Lille, France.
  107. Steckelmacher, D., Roijers, D.M., Harutyunyan, A., Vrancx, P., Plisnier, H., and Nowé, A. (2018, January 2–7). Reinforcement learning in POMDPs with memoryless options and option-observation initiation sets. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA. · DOI
  108. Hu, H., Zhang, X., Yan, X., Wang, L., and Xu, Y. (arXiv, 2017). Solving a new 3d bin packing problem with deep reinforcement learning method, arXiv.
  109. Denil, M., Agrawal, P., Kulkarni, T.D., Erez, T., Battaglia, P., and de Freitas, N. (arXiv, 2016). Learning to perform physics experiments via deep reinforcement learning, arXiv.
  110. Islam, R., Henderson, P., Gomrokchi, M., and Precup, D. (arXiv, 2017). Reproducibility of benchmarked deep reinforcement learning tasks for continuous control, arXiv.
  111. Li, K., and Burdick, J.W. (arXiv, 2017). Inverse Reinforcement Learning in Large State Spaces via Function Approximation, arXiv.
  112. Kendall, A., Gal, Y., and Cipolla, R. (arXiv, 2017). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics, arXiv.
  113. Gal, Y., and Ghahramani, Z. (arXiv, 2015). Bayesian convolutional neural networks with Bernoulli approximate variational inference, arXiv.
  114. Kumar, S., Laumann, F., Maurin, A.L., Olsen, M., and Bayesian, M.L. (arXiv, 2018). Convolutional Neural Networks with Variational Inference, arXiv.
  115. Hu, S.X., Champs-sur-Marne, F., Moreno, P.G., Lawrence, N., and Damianou, A. (2018, January 7). β-BNN: A Rate-Distortion Perspective on Bayesian Neural Networks. Proceedings of the Bayesian Deep Learning Workshop during the Thirty-Second Conference on Neural Information Processing Systems (NIPS 2018), Montréal, QC, Canada.
  116. Salvator, L., Han, J., Schroers, C., and Mandt, S. (2018, January 7). Video Compression through Deep Bayesian Learning Bayesian. Proceedings of the Deep Learning Workshop during the Thirty-Second Conference on Neural Information Processing Systems (NIPS 2018), Montréal, QC, Canada.
  117. Krishnan, R., Subedar, M., and Tickoo, O. (arXiv, 2018). BAR: Bayesian Activity Recognition using variational inference, arXiv.
  118. Chen, T., Goodfellow, I., and Shlens, J. (arXiv, 2015). Net2net: Accelerating learning via knowledge transfer, arXiv.
  119. Ganin, Y., and Lempitsky, V. (arXiv, 2014). Unsupervised domain adaptation by backpropagation, arXiv.
  120. Wenyuan, D., Yang, Q., Xue, G., and Yu, Y. (2007, January 20–24). Boosting for transfer learning. Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, USA.
  121. Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., and Macherey, K. (arXiv, 2016). Google’s neural machine translation system: Bridging the gap between human and machine translation, arXiv.
  122. Lin, Z., Courbariaux, M., Memisevic, R., and Bengio, Y. (arXiv, 2015). Neural networks with few multiplications, arXiv.
  123. Courbariaux, M., David, J.-E., and Bengio, Y. (arXiv, 2014). Training deep neural networks with low precision multiplications, arXiv.
  124. Hubara, I., Soudry, D., and El Yaniv, R. (arXiv, 2016). Binarized Neural Networks, arXiv.
  125. Kim, M., and Smaragdis, P. (arXiv, 2016). Bitwise neural networks, arXiv.
  126. Dettmers, T. (arXiv, 2015). 8-Bit Approximations for Parallelism in Deep Learning, arXiv.
  127. Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., and Zou, Y. (arXiv, 2016). Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients, arXiv.
  128. Novikov, A., Podoprikhin, D., Osokin, A., and Vetrov, D.P. (2005). Tensorizing neural networks. Advances in Neural Information Processing Systems, MIT Press.
  129. Zhu, C., Han, S., Mao, H., and Dally, W.J. (arXiv, 2016). Trained ternary quantization, arXiv.

Cited by

See all 1,620

Metadata

Authors
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K.
Venue
Electronics
Publisher
MDPI AG
Published
2019-03-05
Volume
8
Issue
3
Pages
292
ISSN
2079-9292 (ISSN-L)
DOI
10.3390/electronics8030292
OpenAlex
W2919358988
Cited by
1,620
References
204
FWCI
91.75
Type
journal article
Language
en
Open access
gold
Licence
creativecommons.org/licenses/by/4.0/ (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.