The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index.
The dropout learning algorithm
Open accessAbstract
Dropout is a recently introduced algorithm for training neural network by randomly dropping units during training to prevent their co-adaptation. A mathematical analysis of some of the static and dynamic properties of dropout is provided using Bernoulli gating variables, general enough to accommodate dropout on units or connections, and with variable rates. The framework allows a complete analysis of the ensemble averaging properties of dropout in linear networks, which is useful to understand the non-linear case. The ensemble averaging properties of dropout in non-linear logistic networks result from three fundamental equations: (1) the approximation of the expectations of logistic functions by normalized geometric means, for which bounds and estimates are derived; (2) the algebraic equality between normalized geometric means of logistic functions with the logistic of the means, which mathematically characterizes logistic functions; and (3) the linearity of the means with respect to sums, as well as products of independent variables. The results are also extended to other classes of transfer functions, including rectified linear functions. Approximation errors tend to cancel each other and do not accumulate. Dropout can also be connected to stochastic neurons and used to predict firing rates, and to backpropagation by viewing the backward propagation as ensemble averaging in a dropout linear network. Moreover, the convergence properties of dropout can be understood in terms of stochastic gradient descent. Finally, for the regularization properties of dropout, the expectation of the dropout gradient is the gradient of the corresponding approximation ensemble, regularized by an adaptive weight decay term with a propensity for self-consistent variance minimization and sparse representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As a result, it is strong in the mid-range of weights, plays a less significant role when weights are small and gradually weakens with saturation.”
From ModDrop: Adaptive Multi-Modal Gesture Recognition (2016) · cites this paper as [52]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It has been shown though [52], that in dropout-like [53] systems activation units of complete models produce a weighted normalized geometric mean of per-model outputs.”
From ModDrop: Adaptive Multi-Modal Gesture Recognition (2016) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“Finally, as has been shown by [52] for dropout, the multiplier proportional to the derivative of the sigmoid activation makes the regularization effect adaptive to the magnitude of the weights.”
From ModDrop: Adaptive Multi-Modal Gesture Recognition (2016) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,482
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Russell, E. L., Chiang, L. H., & Braatz, R. D. (2000) · Advances in industrial control · doi:10.1007/978-1-4471-0409-4_3 · Cited by 7,938
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Spencer, J. (1991) · Medical Entomology and Zoology · Cited by 4,310
Cox, D. R. (2008) · Springer eBooks · doi:10.1007/978-0-387-32833-1_5 · Cited by 2,687
Flagged first, then by citations
2 of the 50 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Mercer, 2001, Improved upper and lower bounds for the difference an-gn, J. Math., 31
Cited by
Sorted by citations
Items 301–340 of 340
Improving Predictive Uncertainty Estimation using Dropout -- Hamiltonian Monte CarloPreprint
Vergara, D., Hernández, S., Valdenegro-Toro, M., & Jorquera, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04756
Otimização de CNN aplicada ao Diagnóstico de Melanoma
Rocha, C. V. M., Maia, L. B., Bráz, G., de Almeida, J. D. S., & de Paiva, A. C. (2019) · doi:10.5753/sbcas.2019.6272
Regularizing Neural Networks via Stochastic Branch LayersPreprint
Park, W., Seo, P. H., Han, B., & Cho, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01467
Understanding Deep Learning Decisions: the Explanatory Vector Decomposition (EVD) method
van den Dool, W. V. S. O. (2020) · Utrecht University Repository (Utrecht University)
Spatial Attention as an Interface for Image Captioning ModelsPreprint
Sadler, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11701
Real-time reconstruction of intense, ultrafast laser pulses using deep learningPreprint
Stanfield, M., Ott, J., Gardner, C., Beier, N. F., Farinella, D. M., Mancuso, C. A., Baldi, P., & Dollar, F. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08372
Self-citationLearning to Attend with Neural Networks
Ba, J. L. (2020) · TSpace (University of Toronto)
Efficient Estimation of Influence of a Training Instance
Kobayashi, S., Yokoi, S., Suzuki, J., & Inui, K. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.573 · Cited by 3
Incorporating additional knowledge into image captioners
Yang, X. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/151726
Neural Networks and Optical Character Recognition
Nwaigwe, D. (2021) · UA Campus Repository (The University of Arizona)
Du, K.-L., & Swamy, M. N. S. (2019) · Springer eBooks · doi:10.1007/978-1-4471-7452-3_23
Jing, J., Wang, Z., & Zhang, H. (2023) · World Conference on Computing and Communication Technologies (WCCCT) · doi:10.1109/wccct56755.2023.10052295
Hansson, A., & Andersen, M. S. (2023) · Wiley eBooks · doi:10.1002/9781119809180.ref
Miranda, F. M., Fajardo, A. C., & Medina, R. P. (2023) · AIP conference proceedings · doi:10.1063/5.0162438
A Regularization Ensemble Based on Levenberg–Marquardt Algorithm for Robot Calibration
Luo, X., Li, Z., Jin, L., & Li, S. (2023) · SpringerBriefs in computer science · doi:10.1007/978-981-99-5766-8_5
Soleimani‐Babakamali, M. H., Askari, M., Heravi, M. A., Şişman, R., Attarchian, N., Askan, A., Soleimani, R., & Taciroğlu, E. (2023) · Research Square · doi:10.21203/rs.3.rs-3375920/v1
Integrated Classification and Regression for Quantitative Biomarker Analysis in Mixture Samples
Xu, Q., Wang, Q., & Wang, C. Q. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-7545-7_30
Mao, J., Hu, Y., & Wan, R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4820797
Mao, J., Chan, S., Gao, H., & Hu, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4894162
Intelligent Data Processing for Alzheimer's Disease Using Deep Learning
Garg, N., Chutani, G., Bohra, H., Chaudhary, S., & Sharma, P. (2024) · Recent Advances in Electrical & Electronic Engineering (Formerly Recent Patents on Electrical & Electronic Engineering) · doi:10.2174/0123520965327853240724113549
MTP-NT: A Mobile Traffic Predictor Enhanced by Neighboring and Transportation Data
de Araújo, P. L., Carneiro, M. G., Contreras, L. M., & Pasquini, R. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3488568
A 1-D CNN Algorithm for Low-Background β Detection With Time Projection Chamber
Huang, Z., Feng, C. Q., Cheng, Y., Yang, K., Tang, S., Zhang, Z., Zhang, R., & Liu, S. (2024) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2024.3519609
A theory of synaptic neural balance: From local to global order
Baldi, P., Alexos, A., Domingo, I., & Rahmansetayesh, A. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2025.104360
Self-citationOCTOPUS: Across-social network user identification via multi-category spatio-temporal trajectories
Qu, Y., Xing, L., Deng, K., Wu, H., Ling, Y., & Jia, D. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00129-9
Towards Precise Embodied Dialogue Localization via Causality Guided Diffusion
Wang, H., Wang, L., Zhou, S., Tian, J., Qin, Z., Wang, Y., Hua, G., & Tang, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01246 · Cited by 1
Reliability analysis of equipment in natural gas pipelines based on Bayesian neural networks
Hou, D., & Gao, Y. (2025) · International Journal of Dynamics and Control · doi:10.1007/s40435-025-01818-z
Lecturer Evaluation from the Perspective of Speech Emotion Recognition with Deep Learning
Rosita, Y. D., Salsabila, Z., & Pamungkas, A. R. P. (2025) · International Conference on Data Science and Its Applications (ICoDSA) · doi:10.1109/icodsa67155.2025.11157430
Research on Polarity-Linear Attention-Based CNN Models for Emotion Recognition
Cao, Z., Zhao, W. Q., & Qu, Z. (2025) · International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA) · doi:10.1109/caibda65784.2025.11182652
Deep learning-based investigation of chloroplast translation regulatory sequences
Abbasi-Vineh, M. A., Ingvarsson, P. K., & Farrokhi, N. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1698951
Marwala, T. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-36322-1.00016-x
Integrated numerical and machine learning framework for the vibro-acoustic analysis of car doors
Garofalo, E., Isacchi, G., Olivieri, M., Ebri, M., & Ripamonti, F. (2026) · Applied Acoustics · doi:10.1016/j.apacoust.2026.111311
Deep learning for human motion analysis
Neverova, N. (2016) · theses.fr (ABES) · doi:10.70675/a8cda552zdc96z48c1z979bzac140403e58f · Cited by 13
Li, Q., Hong, J., Zhang, C., Shangguan, W., Wei, Z., Li, L., Dong, W., Zhu, J., Chen, X., Yan, Y., et al. (2026) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2026.111161
Fourure, D. (2017) · theses.fr (ABES) · doi:10.70675/b54dd90bz6e92z43fez8df3z313ad61607ea
Input noise injection for supervised machine learning, with applications on genomic and image data
Khalfaoui, B. (2019) · theses.fr (ABES) · doi:10.70675/2026ab7fzf56cz4bd4z85daz4cbfb2fbc88a
A Causal Lens on Non-RGB Vision Sensor Understanding in Vision-Language Models
Yu, Y., & Ro, Y. M. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3675431
Methods for determining the Hubbard U parameter in DFT+U calculations for new energy materialPreprint
Jiang, Y., Dong, S., Yang, T., Sun, Y., Liu, S., Zhou, L., Wang, E., Wang, K., & Hou, X. (2026) · Research Square · doi:10.21203/rs.3.rs-9485062/v1
Marwan, M., Temghart, A. A., & Lazaar, M. (2026) · Array · doi:10.1016/j.array.2026.100948
Hou, Y., Wei, X., Hu, D., & Jiang, C. (2026) · Applied Thermal Engineering · doi:10.1016/j.applthermaleng.2026.133190
Metadata
- Authors
- Baldi, P., & Sadowski, P.
- Affiliations
- University of California, Irvine (US)
- Corresponding author
- Baldi, Pierre
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2014-05
- Volume
- 210
- Pages
- 78-122
- ISSN
- 0004-3702 (ISSN-L)
- PMC
- PMC3996711
- OpenAlex
- W2164370980
- Cited by
- 340
- References
- 50
- Citations by year
- 2026: 12 · 2025: 23 · 2024: 39 · 2023: 38 · 2022: 19 · 2021: 40 · 2020: 45 · 2019: 31 · 2018: 26 · 2017: 21 · 2016: 24 · 2015: 10 · 2014: 11
- FWCI
- 21.93
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc-nd)
- Funders
- NSF (IIS-0513376, NSF-IIS-1321053)
- NIH (LM010235)
- NIH NLM (T15 LM07443)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- dropout, neural network training, ensemble averaging, logistic function, rectified linear units, stochastic neurons, stochastic gradient descent, regularization, weight decay, sparse representation, variance minimization, co-adaptation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
