The limits of distribution-free conditional predictive inference
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
We consider the problem of distribution-free predictive inference, with the goal of producing predictive coverage guarantees that hold conditionally rather than marginally. Existing methods such as conformal prediction offer marginal coverage guarantees, where predictive coverage holds on average over all possible test points, but this is not sufficient for many practical applications where we would like to know that our predictions are valid for a given individual, not merely on average over a population. On the other hand, exact conditional inference guarantees are known to be impossible without imposing assumptions on the underlying distribution. In this work, we aim to explore the space in between these two and examine what types of relaxations of the conditional coverage property would alleviate some of the practical concerns with marginal coverage guarantees while still being possible to achieve in a distribution-free setting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Algorithmic Learning in a Random World
(2005) · doi:10.1007/b106715 · Cited by 1,119
Distribution-Free Predictive Inference for Regression
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1307116 · Cited by 1,049 · cited 8×
“As a concrete example, we will construct a variant of the split conformal prediction method and will generalize [7]’s results on the efficiency of split conformal prediction to establish conditions under which the resulting prediction intervals are asymptotically efficient.”
BackgroundMethodInfluentialSelf-citationInductive Confidence Machines for Regression
Papadopoulos, H., Proedrou, K., Vovk, V., & Gammerman, A. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36755-1_29 · Cited by 346 · cited 1×
“The split conformal prediction algorithm, introduced in [10,14] (under the name ‘inductive conformal prediction’) and studied further by [9,13,7], is a well-known method that achieves distribution-free marginal coverage guarantees.”
MethodConditional validity of inductive conformal predictors
Vovk, V. (2013) · Machine Learning · doi:10.1007/s10994-013-5355-6 · Cited by 225 · cited 7×
“2 [Rephrased from [13,8]] Suppose that Ĉn satisfies (1 − α)-CC (1.”
BackgroundMethodInfluential
Cited by
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Hüllermeier, E., & Waegeman, W. (2021) · Machine Learning · doi:10.1007/s10994-021-05946-3 · Cited by 1,834
“Roughly speaking, in the case of marginal coverage, one averages over both XN+1 and YN+1 (i.e., the probability P refers to a joint measure on X × Y), while in the case of conditional coverage, XN+1 is fixed and the average in taken over YN+1 only (Barber et al., 2020).”
BackgroundConformal Prediction: A Gentle Introduction
Angelopoulos, A. N., & Bates, S. (2023) · Foundations and Trends in Machine Learning · doi:10.1561/2200000101 · Cited by 740
Second opinion needed: Communicating uncertainty in medical machine learning
Kompa, B., Snoek, J., & Beam, A. L. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00367-3 · Cited by 514
Conformal Inference of Counterfactuals and Individual Treatment Effects
Lei, L., & Candès, E. J. (2021) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12445 · Cited by 166
Self-citationDistribution-free, Risk-controlling Prediction Sets
Bates, S., Angelopoulos, A., Lei, L., Malik, J., & Jordan, M. (2021) · Journal of the ACM · doi:10.1145/3478535 · Cited by 144
“…prediction is a generic approach, and much recent work has focused on designing specific conformal procedures to have good performance according to metrics such as small set sizes [14], approximate coverage in all regions of feature space [15–21], and errors balanced across classes [14, 22–24].”
Background
Metadata
- Authors
- Foygel Barber, R., Candès, E. J., Ramdas, A., & Tibshirani, R. J.
- Venue
- Information and Inference: A Journal of the IMA
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-08-25
- Volume
- 10
- Issue
- 2
- Pages
- 455-482
- ISSN
- 2049-8764 (ISSN-L)
- 2049-8772
- OpenAlex
- W2922110817
- Cited by
- 147
- References
- 4
- FWCI
- 11.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The limits of distribution-free conditional predictive inference. https://app.cochat.ai/papers/10.1093/imaiai/iaaa017
- The paper
- Foygel Barber, R., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2021). The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA. https://doi.org/10.1093/imaiai/iaaa017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.