On Bias Plus Variance
The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
This article presents several additive corrections to the conventional quadratic loss bias-plus-variance formula. One of these corrections is appropriate when both the target is not fixed (as in Bayesian analysis) and training sets are averaged over (as in the conventional bias plus variance formula). Another additive correction casts conventional fixed-trainingset Bayesian analysis directly in terms of bias plus variance. Another correction is appropriate for measuring full generalization error over a test set rather than (as with conventional bias plus variance) error at a single point. Yet another correction can help explain the recent counterintuitive bias-variance decomposition of Friedman for zero-one loss. After presenting these corrections, this article discusses some other loss function-specific aspects of supervised learning. In particular, there is a discussion of the fact that if the loss function is a metric (e.g., zero-one loss), then there is bound on the change in generalization error accompanying changing the algorithm's guess from h 1 to h 2, a bound that depends only on h 1 and h 2 and not on the target. This article ends by presenting versions of the bias-plus-variance formula appropriate for logarithmic and quadratic scoring, and then all the additive corrections appropriate to those formulas. All the correction terms presented are a covariance, between the learning algorithm and the posterior distribution over targets. Accordingly, in the (very common) contexts in which those terms apply, there is not a “bias-variance trade-off” or a “bias-variance dilemma,” as one often hears. Rather there is a bias-variance-covariance trade-off.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 6×
“The bias-variance formula in Geman et al. (1992) is a special case of equation 3.1, where the learning algorithm always guesses the same h given the same training set d (something that is not the case for backpropagation with a random initial weight, for example).”
BackgroundMethodInfluential
Cited by
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237
Self-citationEnsemble deep learning: A review
Ganaie, M. A., Hu, M., Malik, A. K., Tanveer, M., & Suganthan, P. N. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105151 · Cited by 2,143
Neural networks for classification: a survey
Zhang, G. P. (2000) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.897072 · Cited by 1,667
Overfitting and neural networks: conjugate gradient and backpropagation
Lawrence, S., & Giles, C. L. (2000) · Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium · doi:10.1109/ijcnn.2000.857823 · Cited by 491
“M uch has been written about controlling the bias/variance tradeoff (which affects generalization) in neural network and other machine learning models [2, 10, 5, 8, 6, 11].”
BackgroundThe Supervised Learning No-Free-Lunch Theorems
Wolpert, D. H. (2002) · Soft Computing and Industry · doi:10.1007/978-1-4471-0123-9_3 · Cited by 468
Self-citation
Metadata
- Authors
- Wolpert, D. H.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 1997-08-01
- Volume
- 9
- Issue
- 6
- Pages
- 1211-1243
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2098363035
- Cited by
- 112
- References
- 1
- FWCI
- 4.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On Bias Plus Variance. https://app.cochat.ai/papers/10.1162/neco.1997.9.6.1211
- The paper
- Wolpert, D. H. (1997). On Bias Plus Variance. Neural Computation. https://doi.org/10.1162/neco.1997.9.6.1211
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.