A tutorial on conformal prediction

“408–410 of his Statistical Learning Theory [27], support-vector machines grew out of the idea of separating two groups of examples with a hyperplane in a way that makes as few mistakes as possible—i.”

“1 Example: Classifying iris flowers In 1936 [11], R.”

“, zn−1) can be framed as an application of the widely accepted Neyman-Pearson theory for hypothesis testing and confidence intervals [17].”

“Readers familiar with Bayes nets [4] will recognize this diagram as an example; conditional on each variable, a joint probability distribution is given for its children (the variables to which arrows from it point), and given the variable, its descendants are independent of its ancestors.”

“This gives the following protocol: The Backward-Looking Betting Protocol Players: Joe, Bill KN := 1.”

“82; [21], pp.”

“Our concept of validity is consistent with a tradition that can be traced back to Jerzy Neyman’s introduction of confidence intervals for parameters in 1937 [19] and even to work by Laplace and others in the late 18th century.”

“On the other hand, as we have already said, it will make no difference if we replace the absolute difference d(z, z′) = |z − z′| with the squared difference d(z, z′) = (z − z′)2, thus squaring A.”

“Split conformal prediction [41] order statistic of { R i } and return interval [ˆ y − ˆ q, ˆ y + ˆ q ] .”

“They are ubiquitous in statistical learning theory, as seen through several well-established methods including conformal prediction [4], [12], [13], scenario optimization [14]–[16], and the holdout method [17], [18].”

“…process sequences through a fixed-depth network rather than step-by-step recurrence, offer a promising alternative for approximating iterative set-valued computations, while conformal prediction provides distribution-free finite-sample coverage guarantees for the resulting predictions [18], [19].”

“Conformal prediction provides finite-sample, distribution-free guarantees by calibrating nonconformity scores using held-out data [1,2,14].”

“We benchmark against the split Conformal Prediction (CP) (Shafer and Vovk, 2008) and Conformal Unconditional Quantile Regression (CUQR) (Alaa, Hussain and Sontag, 2023).”

“Conformal Prediction (CP) [1, 29, 42, 43, 45] offers an appealing solu-tion by providing distribution-free, per-sample uncertainty sets that remain valid regardless of model architecture or miscalibration.”

“Finally, CoNBONet incorporates split conformal prediction [53–55] to further calibrate the confidence intervals, enabling practitioners to quantify the uncertainty associated with the model’s predictions, an essential requirement in safety-critical reliability applications.”

“Note that the coverage bounds (15) can be obtained using distribution-free UQ procedures such as conformal prediction [28] or scenario-optimization-based calibration [26].”

“…x n +1 , the objective of the conformal prediction (CP) framework is to generate a statistically valid prediction set C ( x n +1 ) for the unknown response y n +1 , ensuring coverage guarantees[Vovk et al., 2005, Shafer and Vovk, 2008], i.e., where α ∈ [0 , 1] determines the level of coverage.”

“CP is a model-agnostic technique that provides prediction sets with a guaranteed error rate under minimal assumptions (Shafer and Vovk, 2008; Fontana et al., 2023).”

“To overcome this issue, some recent works [12], [32]– [34] adopt a post-processing approach based on conformal prediction [35] to construct circular uncertainty regions around predicted trajectory points [36], [37] and have shown promising results when the uncertainty regions are integrated into…”

“Conformal prediction methods that output calibrated prediction sets [55] provide an alternative paradigm entirely.”

“This can be done either by calculating a bound such as the prediction interval or distribution-free tolerance limit if the rate-quality function is estimated using a traditional regression model or using tools such as conformal prediction [17] if the regression model is Machine Learning (ML)-based.”

“…the calibration side, CP has emerged as one of the most influential approaches to distribution-free predictive inference because it offers distribution-free marginal coverage while assuming very little about the data-generating mechanism [Vovk et al., 2005, Shafer and Vovk, 2008, Lei et al., 2018].”

“Adaptive Conformal Inference Recent developments in adaptive conformal prediction approaches for sequential or streaming data [Shafer and Vovk, 2008, Tibshirani et al., 2019, Gibbs and Candès, 2021, Zaffran et al., 2022] share our goal of real-time inference, but for a different target.”

“Conformal inference (Vovk et al., 2005; Shafer & Vovk, 2008) allows us to construct prediction sets for black-box models with a finite-sample coverage guarantee.”

“3) Conformal prediction and reliability guarantees: Con-formal prediction is a distribution-free framework for constructing prediction sets with guaranteed coverage [21, 22].”

“However, unlike supervised learning, where confidence estimation techniques such as conformal prediction (Shafer & Vovk, 2007) or calibrated probabilities (Guo et al., 2017) are common, clustering methods typically do not offer reliable confidence scores for individual data point assignments.”

“Conformal prediction constructs set-valued predictors with finite-sample coverage guarantees under ex-changeability (Vovk et al., 2005; Shafer & Vovk, 2008; Angelopoulos & Bates, 2023).”

“…recent efforts have been directed towards developing distribution-free, data-driven approaches to uncertainty quantification by leveraging conformal prediction [17, 18], a principled framework for producing distribution-free prediction regions with marginal frequentist coverage guarantees [19, 20].”

“Conformal prediction [29, 24, 2] provides a general framework for attaching statistically valid uncertainty quantification to black-box predictive models.”

“This method was selected as it provides validated prediction intervals with marginal coverage guarantees, independent of strict distributional assumptions [26].”

“Conformal prediction (CP) provides a lightweight and distribution-free alternative: under ex-changeability, CP yields finite-sample validity guarantees without assumptions on the underlying model [5, 23].”

“Conformal prediction (CP) has emerged as a principled and widely adopted framework for this purpose, providing finite-sample, distribution-free prediction sets with formal coverage guarantees under the mild assumption of data exchangeability [1], [2].”

“For example, the conformal quantile regression (CQR) score (Romano et al., 2019) is based on fitted conditional quantiles and nearest-neighbor-based scores can be used for binary outcomes (Shafer and Vovk, 2008).”

“The prediction set was defined as follows: The conformal framework ensures that the prediction set contains the actual OSA severity class with probability at least 1 − α provided that the calibration and test data are exchangeable [22,24].”

“To ensure the reliability required for scientific applications, we integrate Split Conformal Prediction [29].”

“We develop a prediction set inspired by the conformal prediction framework (Shafer and Vovk 2008; Fontana et al. 2023), ensuring validity without any distributional assumptions.”

“We use conformal prediction (Shafer and Vovk 2008; Fontana et al. 2023) to obtain the prediction interval.”

“To further improve the reliability of medium-and low-confidence detection results, this paper introduces the Conformal Prediction (CP) method [10] to perform secondary verification on yellow-and red-grade detection targets, outputting prediction sets that contain the true category with statistical…”

“To address calibration issues, conformal prediction (CP) provides a lightweight and distribution-free framework for obtaining prediction intervals with finite-sample coverage guarantees under data exchangeability [22, 23].”

“Conformal Prediction [9] is a distribution-free method that provides prediction sets with guaranteed coverage probabilities.”

“Moreover, uncertainty sources such as: Conformal Prediction (CP, also known as Conformal Inference) is a well established method for obtaining some formal statistical guarantees regarding predictor output values [10–12].”

“A key advantage of CP is its model-agnostic nature: it can be applied on top of any prediction model, from simple linear regression models to complex machine learning algorithms, without requiring model-specific modifications (Shafer and Vovk 2008; Vovk, Gammerman, and Shafer 2022).”

“…provide numerical outputs including calibration error (Guo et al. 2017), entropy (Kendall and Gal 2017), ensemble variance (Lakshmi-narayanan, Pritzel, and Blundell 2017), out-of-distirbution detection scores (Hendrycks and Gimpel 2016), and conformal prediction set sizes (Shafer and Vovk 2008).”

“While conformal prediction offers strong coverage guarantees (Vovk, Nouretdinov, and Gammerman 2009; Shafer and Vovk 2008; Angelopoulos, Bates et al. 2023; Lei et al. 2018; Baheri and Shahbazi 2025), its assumptions conflict with the nested structure of healthcare data, limiting its effectiveness…”

“On a final note, we want to point out that copulas can be used in conformal prediction (Shafer and Vovk 2008), when more than one variable is to be predicted.”

“Vovk et al. [2005] introduced conformal prediction with exchangeability guarantees; Shafer and Vovk [2008] provide an accessible tutorial.”

“Conformal Prediction, originally formalized by Vovk & Shafer [1], provides distribution-free coverage guarantees.”

“…Lower Bound require knowledge of the underlying model to fit the experimental data to, which is not known 37 , and distribution-free uncertainty estimation methods such as Conformal Predictors require access to underlying ground truth examples 38,39 , which are also inherently unavailable.”

“Conformal inference has emerged as a powerful framework for constructing distribution-free prediction intervals with finite-sample guarantees [13, 14].”

“Conformal prediction (CP) is a model-agnostic method that provides distribution-free, probabilistic error bounds using calibration data [9].”

“Building on this, informed ML [17] approaches were applied by using simulation data to train a multi-objective regression model, with conformal prediction [18] providing confidence intervals for fiber deposition predictions based on process and material parameters.”

“Con-formal Prediction (CP), a distribution-free, finite-sample-valid uncertainty estimation framework, offers rigorous statistical guarantees for model predictions, outputting calibrated predic-2025 tion sets rather than single-point predictions [15].”

“To obtain results with statistical guarantees and alignment with the real world, we leverage conformal prediction calibration [37].”

“Full conformal prediction is the original variant and is the most data-efficient (Vovk et al., 2005; Shafer & Vovk, 2008; Caprio et al., 2025a).”

“Conformal prediction (CP) [11], [12] offers an attractive alternative by providing distribution-free prediction sets with finite-sample validity guarantees.”

“There are multiple methods for measuring model con fi dence, such as evidential deep learning (EDL) 25 , and conformal prediction 26 .”

“Various set-valued prediction methods based on the conformal prediction framework (Vovk et al. 2005, Shafer & Vovk 2008) have been developed to address these limitations.”

“Conformal Prediction (CP), in particular, serves as a powerful method to fill this gap as a non-parametric approach that constructs valid prediction intervals without assuming a particular form of the predictive distribution [20]–[22].”

“Conformal Prediction (CP), in particular, serves as a powerful method to fill this gap as a non-parametric approach that constructs valid prediction intervals without assuming a particular form of the predictive distribution [20]–[22].”

“In contrast, conformal prediction provides a distribution-free framework for uncertainty estimation with provable coverage guarantees [25, 27].”

“Tools such as quantile regression and conformal prediction generate prediction intervals without strong distributional assumptions (Shafer & Vovk, 2008).”

“Uncertainty quantification methods (e.g., conformal prediction [23], Bayesian model averaging [24]) also aim to characterize uncertainty, but focus either on per-sample prediction sets or on averaging across multiple models, whereas our problem concerns the true loss of a single, fixed model.”

“In practice, uncertainty can be estimated via Monte-Carlo dropout and complemented with split-conformal prediction to target empirical coverage, while interpretability can be provided by SHAP (global summaries) and LIME (local rationales) [17, 18].”

“Conformal prediction [1], [2] is an effective method for uncertainty quantification that has been widely used to predict a set of candidate labels for income data.”

“Both thresholds τ and β , which stipulate whether a time series is considered as stress-inducing or not, were set by default to the 80 th percentile of their respective values: errors measured in SMAPE for the first; and the average size of the prediction intervals, obtained using Conformal Prediction (Shafer and Vovk…”

“Reliability is assessed within the Conformal Prediction framework (Vovk et al. 2005; Shafer and Vovk 2008; Burnaev and Vovk 2014; Lei et al. 2018; Angelopoulos and Bates 2021; Tibshirani 2023), which delivers distribution-free prediction intervals with guaranteed coverage and supports the…”

“To enable uncertainty-aware predictions, we adopt the conformal prediction framework [10, 14], which provides a formal mechanism to quantify predictive uncertainty with statistical guarantees.”

“Conformal prediction (CP) delivers finite-sample coverage under exchangeability (Vovk et al., 2005; Shafer and Vovk, 2008).”

“Conformal prediction (Vovk et al., 2005; Shafer and Vovk, 2008; Barber et al., 2020) is a statistical framework that turns black-box model outputs into predictive intervals.”

“Conformal prediction methods construct prediction sets that include the true target with a predefined probability (a property referred to as validity), employing past observations to determine precise levels of confidence in new predictions (Shafer & Vovk, 2008).”

“Comprehensive overviews of this field are provided by Vovk et al. [2005], Angelopoulos and Bates [2021], Shafer and Vovk [2008], and Angelopoulos et al. [2024].”

“To address the cumulative drawbacks of aforementioned GP competitors, recent work has focused on calibration of surrogate outputs via conformal prediction [11].”

“Shafer and Vovk laid the theoretical foundation of CP in financial contexts, but practical implementations in real-time trading systems are still lacking [15].”

“Conformal methods provide distribution-free uncertainty quantification with finite-sample guarantees under exchangeability [3, 4, 16].”

“Future work could also incorporate conformal prediction to quantify the prediction uncertainty [41].”

“To meet these demands, uncertainty quantification methods are essential [43], while techniques such as Exceptional Model Mining (EMM) can offer interpretable insights [12, 17, 34].”

“…estimates via the Cramér-Rao Lower Bound require knowledge of the underlying model to fit the experimental data to, which is not known 36 , and distribution-free methods such as Conformal Predictors require access to underlying ground truth examples 37,38 , which are also inherently unavailable.”

“To address the prevalent issue of overconfidence and lack of reliable error control in modern machine learning models, Conformal Prediction [20, 21, 22, 23, 24] offers a rigorous, distribution-free solution.”

“Inspired by the notion of conformal prediction in machine learning [8, 85], we refer to this set as the conformal set for rank 𝑖 .”

“For example, at prediction indices 5, 6, and 9, the point forecast falls outside the CP interval, yet the interval itself successfully captures the true value.”

“These maps represent a valuable tool for detecting areas of limited model reliability in policy-driven territorial analysis, as highlighted by the results from the real-world case study presented in this article.”

“Conformal prediction provides a prediction region that can include multiple possible labels with some specified confidence level, ensuring overall statistical validity [13-16].”

“Conformal Prediction (CP) [25], [72], [73] is a distribution-free framework for uncertainty quantification.”

“To address this issue, methods estimating prediction reliability, such as conformal prediction [13, 29], are widely used.”

“Conformal prediction (Shafer and Vovk, 2008) is an increasingly popular analytic framework to quantify uncertainty in predictive modeling.”

“CP, combined with the transformer, is a statistically rigorous framework that constructs prediction intervals with guaranteed coverage probability 1 − α for unseen data, assuming only exchangeability of samples (Shafer & Vovk, 2008).”

“Conformal Prediction (CP) provides a framework for generating statistically valid prediction intervals for the outputs of any machine learning model, including regression models [32, 33].”

“Addressing the challenge of obtaining statistically valid uncertainty estimates, especially when using approximations for scalability or non-stationarity, can be achieved using conformal inference [38; 48].”

“To extend MSLR for predicting the upper bounds of resistance degradation with high reliability at a specified confidence level, we incorporate Adaptive Conformal Inference (ACI) [8], a recent Conformal Prediction (CP) approach [9].”

“Conformal prediction offers a simple yet effective post-hoc solution to generate prediction sets from any trained classifier using calibration data [35, 2, 7, 18, 23].”

“Two broadly used measures in the literature, validity and efficiency [30–32] were evaluated.”

“The versatile approach of CP evaluates a given prediction based on the output of a machine learning model [9].”