Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
It sometimes happens (for instance in case control studies) that a classifier is trained on a data set that does not reflect the true a priori probabilities of the target classes on real-world data. This may have a negative effect on the classification accuracy obtained on the real-world data set, especially when the classifier's decisions are based on the a posteriori probabilities of class membership. Indeed, in this case, the trained classifier provides estimates of the a posteriori probabilities that are not valid for this real-world data set (they rely on the a priori probabilities of the training set). Applying the classifier as is (without correcting its outputs with respect to these new conditions) on this new data set may thus be suboptimal. In this note, we present a simple iterative procedure for adjusting the outputs of the trained classifier with respect to these new a priori probabilities without having to refit the model, even when these probabilities are not known in advance. As a by-product, estimates of the new a priori probabilities are also obtained. This iterative algorithm is a straightforward instance of the expectation-maximization (EM) algorithm and is shown to maximize the likelihood of the new data. Thereafter, we discuss a statistical test that can be applied to decide if the a priori class probabilities have changed from the training set to the real-world data. The procedure is illustrated on different classification problems involving a multilayer neural network, and comparisons with a standard procedure for a priori probability estimation are provided. Our original method, based on the EM algorithm, is shown to be superior to the standard one for a priori probability estimation. Experimental results also indicate that the classifier with adjusted outputs always performs better than the original one in terms of classification accuracy, when the a priori probability conditions differ from the training set to the real-world data. The gain in classification accuracy can be significant.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, if wellcalibrated fraud probabilities are needed, then the posterior probabilities can be adjusted (Saerens et al., 2002).”
From Data engineering for fraud detection (2021) · cites this paper as Saerens et al., 2002
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Probability, Random Variables, and Stochastic Processes.
Abrahams, J., & Papoulis, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288754 · Cited by 16,226
UCI Repository of machine learning databases
Blake, C. L. (1998) · Medical Entomology and Zoology · Cited by 10,582
The EM Algorithm and Extensions
Kushary, D. (1998) · Technometrics · doi:10.1080/00401706.1998.10485534 · Cited by 5,417
An Introduction to the Theory of Statistics
(1911) · Nature · doi:10.1038/087345a0 · Cited by 4,394
Probability, Random Variables, and Stochastic Processes
Papoulis, A., & Hoffman, J. G. (1967) · Physics Today · doi:10.1063/1.3034123 · Cited by 4,107
Journal of the Royal Statistical Society (B)
Fildes, R. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90088-5 · Cited by 3,437
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736 · cited 1×
“For instance, Nigam et al. (2000) use the EM algorithm to learn classiers that take advantage of both labeled and unlabeled data.”
MethodDiscriminant Analysis and Statistical Pattern Recognition.
Hawkins, D. M., & McLachlan, G. J. (1993) · Journal of the American Statistical Association · doi:10.2307/2290355 · Cited by 2,633
Mixture models : inference and applications to clustering
McLachlan, G. J., & Basford, K. E. (1988) · M. Dekker eBooks · Cited by 2,094
Cited by
The importance of intrinsic disorder for protein phosphorylation
Iakoucheva, L. M. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh253 · Cited by 1,429
“Predictors trained using a specific class distribution are known not to be optimal when applied to unlabeled data with different class priors ( 37 ).”
A unifying view on dataset shift in classification
Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.06.019 · Cited by 1,029
Learning when training data are costly: the effect of class distribution on tree induction
Weiss, G. M., & Provost, F. (2003) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1199 · Cited by 921
Calibrating Probability with Undersampling for Unbalanced Classification
Pozzolo, A. D., Caelen, O., Johnson, R. A., & Bontempi, G. (2015) · 2015 IEEE Symposium Series on Computational Intelligence · doi:10.1109/ssci.2015.33 · Cited by 667
“[21] show that, given different priors between the training and testing sets, the posterior probability can be corrected with the following equation:”
BackgroundMethodResultInfluentialIdentification, analysis, and prediction of protein ubiquitination sites
Radivojac, P., Vacic, V., Haynes, C., Cocklin, R. R., Mohan, A., Heyen, J. W., Goebl, M. G., & Iakoucheva, L. M. (2010) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.22555 · Cited by 583
Morocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942390 · Cited by 418
“This adaptation may be done either by the end user [43], [44], or automatically [45], [46].”
BackgroundTowards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., & Russakovsky, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00894 · Cited by 274
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 271
“Other distribution shifts like prior shift [32] are further discussed in Sec.”
InfluentialWILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
“…e.g., learning to map data points from source to target domains (Hoffman et al., 2018), or estimating the test label distribution from unlabeled test data (Saerens et al., 2002; Zhang et al., 2013; Lipton et al., 2018; Azizzadenesheli et al., 2019; Alexandari et al., 2020; Garg et al., 2020).”
Detecting and Correcting for Label Shift with Black Box Predictors
Lipton, Z. C., Wang, Y.-X., & Smola, A. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03916 · Cited by 103
“Post-submission, we learned of antecedents for our estimator in epidemiology (Buck et al., 1966) and revisited by Forman (2008); Saerens et al. (2002).”
BackgroundFailing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
Rabanser, S., Günnemann, S., & Lipton, Z. C. (2018) · arXiv · doi:10.48550/arxiv.1810.11953 · Cited by 204
“Owing to the impossibility of correcting for shift absent assumptions [3], these papers often assume either covariate shift q(x, y) = q(x)p(y|x) [15, 45, 49] or label shift q(x, y) = q(y)p(x|y) [7, 29, 38, 48, 56].”
BackgroundCMU DeepLens: deep learning for automatic image-based galaxy–galaxy strong lens finding
Lanusse, F., Ma, Q., Li, N., Collett, T. E., Li, C.-L., Ravanbakhsh, S., Mandelbaum, R., & Póczos, B. (2018) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stx1665 · Cited by 191
Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation
Cawley, G. C., Talbot, N. L. C., & Girolami, M. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0031 · Cited by 189
Federated Visual Classification with Real-World Data Distribution
Hsu, T.-M. H., Qi, H., & Brown, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_5 · Cited by 178
Quantifying counts and costs via classification
Forman, G. (2008) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-008-0097-y · Cited by 173
“…the less common work that acknowledges that the training and test distributions may differ, the estimate of the class distribution is used only in order to improve the individual classifications, e.g. via ROC analysis (Provost and Fawcett 2001; Fawcett 2003; Mei and Zhai 2005; Saerens et al. 2002).”
MethodData engineering for fraud detection
Baesens, B., Höppner, S., & Verdonck, T. (2021) · Decision Support Systems · doi:10.1016/j.dss.2021.113492 · Cited by 170
“However, if wellcalibrated fraud probabilities are needed, then the posterior probabilities can be adjusted (Saerens et al., 2002).”
BackgroundCorporate default forecasting with machine learning
Moscatelli, M., Parlapiano, F., Narizzano, S., & Viggiano, G. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113567 · Cited by 165
When is Undersampling Effective in Unbalanced Classification Tasks?
Pozzolo, A. D., Caelen, O., & Bontempi, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23528-8_13 · Cited by 167
“The first side-effect is that, by removing majority class instances, we perturb the a priori probability of the training set and we induce a warping in the posterior distribution [18, 8].”
Class distribution estimation based on the Hellinger distance
González‐Castro, V., Aláiz-Rodríguez, R., & Alegre, E. (2012) · Information Sciences · doi:10.1016/j.ins.2012.05.028 · Cited by 167
Class-prior estimation for learning from positive and unlabeled data
du Plessis, M. C., Niu, G., & Sugiyama, M. (2016) · Machine Learning · doi:10.1007/s10994-016-5604-6 · Cited by 126
“If a mixture of class-wise input data densities, q ′ ( x ; θ ) = θp ( x y = 1) + (1 − θ ) p ( x y = − 1) , is fitted to the unlabeled input data density p ( x ), the true class prior π can be obtained (Saerens et al., 2002; du Plessis and Sugiyama, 2012), as illustrated in Figure 1(a).”
MethodInfluentialAutomatic identification of bird targets with radar via patterns produced by wing flapping
Zaugg, S. A., Saporta, G., van Loon, E. E., Schmaljohann, H., & Liechti, F. (2008) · Journal of The Royal Society Interface · doi:10.1098/rsif.2007.1349 · Cited by 119
A Review on Quantification Learning
González, P., Castaño, A. B., Chawla, N. V., & del Coz, J. J. (2017) · ACM Computing Surveys · doi:10.1145/3117807 · Cited by 167
Domain Adaptation with Active Learning for Word Sense Disambiguation
Chan, Y. S., & Ng, H. T. (2007) · Cited by 106
A Supervised Learning Approach to Monaural Segregation of Reverberant Speech
Jin, Z., & Wang, D. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2010633 · Cited by 102
Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
Wu, Y., Winston, E., Kaushik, D., & Lipton, Z. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.01689 · Cited by 70
“Some consider the reverse setting label shift (Saerens et al., 2002; Zhang et al., 2013; Lipton et al., 2018), where although the label distribution shifts ( p S ( y ) (cid:54) = p T ( y )), the class-conditional input distribution is invariant ( p S ( x | y ) = p T ( x | y )).”
BackgroundTheoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative LearningPreprint
Niu, G., du Plessis, M. C., Sakai, T., Ma, Y., & Sugiyama, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.03130 · Cited by 82
“In practice, it can be effectively estimated from P, N and U data [20, 21, 22] or only P and U data [23, 24].”
BackgroundDomain Adaptive Transfer Learning with Specialist Models
Ngiam, J., Peng, D., Vasudevan, V., Kornblith, S., Le, Q. V., & Pang, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07056 · Cited by 80
“related3 to prior probability shift [28, 31] also known as target shift [39].”
BackgroundMethodEstimating class priors in domain adaptation for word sense disambiguation
Chan, Y. S., & Ng, H. T. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220187 · Cited by 82
“To estimate the sense priors, or a priori probabilities of the different senses in a new dataset, we used a confusion matrix algorithm (Vucetic and Obradovic, 2001) and an EM based algorithm (Saerens et al., 2002) in (Chan and Ng, 2005b).”
BackgroundMethodSemi-supervised learning of class balance under class-prior change by distribution matching
du Plessis, M. C., & Sugiyama, M. (2013) · Neural Networks · doi:10.1016/j.neunet.2013.11.010 · Cited by 85
Semi-Supervised Classification Based on Classification from Positive and Unlabeled DataPreprint
Sakai, T., du Plessis, M. C., Niu, G., & Sugiyama, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06955 · Cited by 95
From classification to quantification in tweet sentiment analysis
Gao, W., & Sebastiani, F. (2016) · Social Network Analysis and Mining · doi:10.1007/s13278-016-0327-z · Cited by 75
Sugiyama, M., Kanamori, T., Suzuki, T., du Plessis, M. C., Liu, S., & Takeuchi, I. (2013) · Neural Computation · doi:10.1162/neco_a_00492 · Cited by 72
A Prototype-Oriented Framework for Unsupervised Domain AdaptationPreprint
Tanwisuth, K., Fan, X., Zheng, H., Zhang, S., Zhang, H., Chen, B., & Zhou, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12024 · Cited by 125
“Over the years, researchers have viewed the question through different lenses: applying kernel distribution embedding [56, 57, 21], using an EM update [43, 58], placing a meta-prior over the target proportions [59], and casting the problem under causal and anti-causal learning [21–25].”
BackgroundEvaluation measures for quantification: an axiomatic approach
Sebastiani, F. (2019) · Information Retrieval · doi:10.1007/s10791-019-09363-y · Cited by 36
Quantifying trends accurately despite classifier error and class imbalance
Forman, G. (2006) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1150402.1150423 · Cited by 64
Automatic plankton quantification using deep features
González, P., Castaño, A., Peacock, E. E., Díez, J., del Coz, J. J., & Sosik, H. M. (2019) · Journal of Plankton Research · doi:10.1093/plankt/fbz023 · Cited by 74
Word sense disambiguation with distribution estimation
Chan, Y. S., & Ng, H. T. (2005) · National University of Singapore · Cited by 63
Automatic Fungi Recognition: Deep Learning Meets Mycology
Picek, L., Šulc, M., Matas, J., Heilmann‐Clausen, J., Jeppesen, T. S., & Lind, E. (2022) · Sensors · doi:10.3390/s22020633 · Cited by 77
“While others [28,68,69] focus on estimating new unknown priors pe(k), we assume”
BackgroundQuantification-oriented learning based on reliable classifiers
Tolosa, J. B., Díez, J., & del Coz, J. J. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.07.032 · Cited by 72
“On the one hand, most published papers focus on adapting the deployed models to the new conditions [24, 25, 26, 27, 28].”
BackgroundPlant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings
Picek, L., Sulc, M., Patel, Y. J., & Matas, J. E. S. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.787527 · Cited by 51
“The experiments with state-of-theart methods for prior shift estimation (Saerens et al., 2002; Sipka et al., 2022), evaluated in Table 6, show that all three compared methods improve the classification accuracy in all cases.”
MethodKluger, D. M., Wang, S., & Lobell, D. B. (2021) · Remote Sensing of Environment · doi:10.1016/j.rse.2021.112488 · Cited by 65
“The Prior Shift Adjusted (PSA for short) classifier presented in Equation 4 of Saerens et al., 2002, is given by 𝑙𝑃𝑆𝐴(𝑥) = argmax 𝑘∈{1,2,…,𝐾} {?”
MethodInfluentialClassifier adaptation at prediction time
Royer, A., & Lampert, C. H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298746 · Cited by 52
“If π were known, we could immediately derive an optimal adaptation rule in closed form, see [23] for a detailed derivation.”
BackgroundMethodMaximum Mean Discrepancy for Class Ratio Estimation: Convergence Bounds and Kernel Selection
Iyer, A. S., Nath, S., & Sarawagi, S. (2014) · International Conference on Machine Learning · Cited by 85
Subadult sex estimation from diaphyseal dimensions
Stull, K. E., L’Abbé, E. N., & Ousley, S. D. (2017) · American Journal of Physical Anthropology · doi:10.1002/ajpa.23185 · Cited by 52
Word-class embeddings for multiclass text classification
Moreo, A., Esuli, A., & Sebastiani, F. (2021) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00735-3 · Cited by 68
Online Optimization Methods for the Quantification Problem
Kar, P., Li, S., Narasimhan, H., Chawla, S., & Sebastiani, F. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939832 · Cited by 51
“Saerens et al. [29], followed by other authors [1, 32, 34], apply quantification to customize a trained classifier to the class prevalence exhibited in the test set, with the goal of improving classification accuracy on unlabeled data exhibiting a class distribution different from that of the training…”
BackgroundProtein function in precision medicine: deep understanding with machine learning
Rost, B., Radivojac, P., & Bromberg, Y. (2016) · FEBS Letters · doi:10.1002/1873-3468.12307 · Cited by 51
“…t ho sc r from p 1 ( x ); Figure 1B) and negative (a sample from p 0 ( x ); Figure 1B) data are available and representative of class-conditional distributions, the presence of representative unlabeled data (a sample from p ( x ); Figure 1B) guarantees a correct estimate of class priors [40].”
BackgroundAssessing the Impact of Changing Environments on Classifier Performance
Aláiz-Rodríguez, R., & Japkowicz, N. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-68825-9_2 · Cited by 71
Recovering True Classifier Performance in Positive-Unlabeled Learning
Jain, S., White, M., & Radivojac, P. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10937 · Cited by 47
“…Mitchell 1998) or unbias the labeled data (Cortes et al. 2008); e.g., to estimate class proportions that are necessary to calibrate the model and accurately estimate precision when class balances (but not class-conditional distributions) in labeled data are not representative (Saerens et al. 2002).”
BackgroundKavzoğlu, T., & Reis, S. (2008) · GIScience & Remote Sensing · doi:10.2747/1548-1603.45.3.330 · Cited by 49
“It should be also mentioned that some researchers (e.g., Saerens et al., 2002; Brenning et al., 2006) suggest using randomly selected data sets in the training stage to be independent of a particular partitioning.”
BackgroundA Supervised Learning Approach to Monaural Segregation of Reverberant Speech
Jin, Z., & Wang, D. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367221 · Cited by 47
Quantification and semi-supervised classification methods for handling changes in class distribution
Xue, J. C., & Weiss, G. M. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557117 · Cited by 61
“The problem of improving classifier performance in response to unknown changes in class distribution has been studied previously [1, 9, 13] and the most relevant and successful approach thus far has involved expectation maximization (EM) [9, 13] and this approach has also been adapted to the related problem of classif…”
MethodApproaches for credit scorecard calibration: An empirical analysis
Bequé, A., Coussement, K., Gayler, R. W., & Lessmann, S. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.07.034 · Cited by 60
Engels, A., Reber, K. C., Lindlbauer, I., Rapp, K., Büchele, G., Klenk, J., Meid, A. D., Becker, C., & König, H. (2020) · PLoS ONE · doi:10.1371/journal.pone.0232969 · Cited by 45
“[40] may have been imperfectly suited for incorporating the bias due to random undersampling.”
BackgroundMethodMayer, A. T., Holman, D., Sood, A., Tandon, U., Bhate, S. S., Bodapati, S., Barlow, G. L., Chang, J. H., Black, S., Crenshaw, E. C., et al. (2023) · Science Advances · doi:10.1126/sciadv.add1166 · Cited by 44
Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers
Zietlow, D., Lohaus, M., Balakrishnan, G., Kleindeßner, M., Locatello, F., Schölkopf, B., & Russell, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01016 · Cited by 86
“Given this, it is unsurprising that these methods for enforcing fairness on a training set are not popular in computer vision, and instead research typically centers around heuristics, such as reweighting samples [51], training multiple classifiers on data subsets, and averaging them [70, 71], or data augmentation [50…”
MethodA Unified View of Label Shift EstimationPreprint
Garg, S., Wu, Y., Balakrishnan, S., & Lipton, Z. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07554 · Cited by 196
“Maximum Likelihood Label Shift (MLLS) [Saerens et al., 2002]: an Expectation Maximization (EM) algorithm that estimates pt(y) but assumes access to a classi er that outputs the true source distribution conditional probabilities ps(y|x).”
BackgroundA Recurrent Neural Network for Sentiment Quantification
Esuli, A., Moreo, A., & Sebastiani, F. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3269287 · Cited by 50
“Yet another approach to quantification is exemplified by the EMQ method of [12], which consists in generating a probabilistic classifier while exploiting the EM algorithm to iteratively shift the estimation of pC(D) from the one of the training set to one that maximizes the likelihood on the test data.”
MethodExploring Diverse Features for Sentiment Quantification Using Machine Learning Algorithms
Ayyub, K., Iqbal, S., Munir, E. U., Nisar, M. W., & Abbasi, M. (2020) · IEEE Access · doi:10.1109/access.2020.3011202 · Cited by 53
Belokurov, V. A., Evans, N. W., & Du, Y. L. (2004) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2004.07917.x · Cited by 43
“It has been shown (e.g., Saerens et al. 2002) that a simple iterative procedure can help to tackle the problem.”
BackgroundThe effect of small disjuncts and class distribution on decision tree learning
Weiss, G. M., & Hirsh, H. (2003) · Cited by 52
Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution MatchingPreprint
du Plessis, M. C., & Sugiyama, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4677 · Cited by 45
Unsupervised Domain Adaptation for Joint Segmentation and POS-Tagging
Liu, Y., & Zhang, Y. (2012) · Cited by 39
Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
Fabris, A., Esuli, A., Moreo, A., & Sebastiani, F. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14033 · Cited by 67
“end end /* Generate output */ for s ∈ S do p̂SLD U (s)← p̂ (t) U (s) end Pseudocode 2: The SLD algorithm (Saerens et al., 2002).”
BackgroundMethodInfluentialLatinne, P., Saerens, M., & Decaestecker, C. (2001) · Cited by 36
Self-citationPosterior Re-calibration for Imbalanced DatasetsPreprint
Tian, J., Liu, Y., Glaser, N., Hsu, Y.-C., & Kira, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11820 · Cited by 82
“1 is not new and has been studied before in [15].”
BackgroundPaillassa, M., Bertin, E., & Bouy, H. (2020) · Astronomy & Astrophysics · doi:10.1051/0004-6361/201936345 · Cited by 35
“If one knows what class proportions are expected in the observation data, output probabilities can be updated to better match these priors (e.g., Saerens et al. 2002; Bailer-Jones et al. 2008).”
BackgroundA self-adaptive particle-tracking method for minerals processing
Pereira, L. A., Frenzel, M., Khodadadzadeh, M., Tolosana‐Delgado, R., & Gutzmer, J. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.123711 · Cited by 36
Finding Quasars behind the Galactic Plane. I. Candidate Selections with Transfer Learning
Fu, Y., Wu, X.-B., Yang, Q., Brown, A. G. A., Feng, X., Ma, Q., & Li, S. (2021) · The Astrophysical Journal Supplement Series · doi:10.3847/1538-4365/abe85e · Cited by 43
“The prior probabilities of stars are much higher than those of quasars (and galaxies) in the Galactic plane, which means the marginal probability P(Y) differs from that at high Galactic latitude (i.e., class-balance change; Saerens et al. 2002; Du Plessis & Sugiyama 2014).”
BackgroundDrift mining in data: A framework for addressing drift in classification
Hofer, V., & Krempl, G. (2012) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2012.07.007 · Cited by 34
Nonparametric semi-supervised learning of class proportionsPreprint
Jain, S., White, M., Trosset, M. W., & Predrag Radivojac, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.01944 · Cited by 34
“These methods assume the existence of both positives and negatives in the labeled set and estimate class priors using various forms of iterative learning (Latinne et al., 2001; Vucetic and Obradovic, 2001; Saerens et al., 2002).”
MethodEnsuring Fairness under Prior Probability Shifts
Biswas, A., & Mukherjee, S. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462596 · Cited by 46
Class and subclass probability re-estimation to adapt a classifier in the presence of concept drift
Aláiz-Rodríguez, R., Guerrero-Curieses, A., & Cid‐Sueiro, J. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.03.019 · Cited by 39
Transfer estimation of evolving class priors in data stream classification
Zhang, Z., & Zhou, J. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.03.021 · Cited by 32
“It was shown that BEM algorithm is more effective than confusion matrix method [23,24].”
MethodAutoregressive Planet Search: Methodology
Caceres, G. A., Feigelson, E. D., Babu, G. J., Bahamonde, N., Christen, A., Bertin, K., Meza, C., & Curé, M. (2019) · The Astronomical Journal · doi:10.3847/1538-3881/ab26b8 · Cited by 39
“2015) and, more generally, to apply classification models to test sets that have different class distributions than the original training set (Elkan 2001; Saerens et al. 2002).”
BackgroundMethodZhang, K., & Zhuang, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_16 · Cited by 50
QuaPy: A Python-Based Framework for Quantification
Moreo, A., Esuli, A., & Sebastiani, F. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482015 · Cited by 39
“The SaerensLatinne-Decaestecker (SLD) algorithm [8, 31] (sometimes also called EMQ, for Expectation Maximization Quantifier) is a probabilistic quantifier-generatingmethod.”
BackgroundMethodTargeting spam control on middleboxes: Spam detection based on layer-3 e-mail content classification
Marsono, M. N., El-Kharashi, M. W., & Gebali, F. (2008) · Computer Networks · doi:10.1016/j.comnet.2008.11.012 · Cited by 29
Tweet sentiment quantification: An experimental re-evaluation
Moreo, A., & Sebastiani, F. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263449 · Cited by 40
“A further method that [GS2016] uses is the one proposed in [31] (which we here call SLD, from the names of its proposers, and which was called EMQ in [GS2016]), which consists of training a probabilistic classifier and then using the EM algorithm (i) to update (in an iterative, mutually recursive way) the posterior pr…”
MethodComputing single-particle flotation kinetics using automated mineralogy data and machine learning
Pereira, L. A., Frenzel, M., Hoang, D. H., Tolosana‐Delgado, R., Rudolph, M., & Gutzmer, J. (2021) · Minerals Engineering · doi:10.1016/j.mineng.2021.107054 · Cited by 28
Prieto, C., Kavetski, D., Vine, N. L., Álvarez, C., & Medina, R. (2021) · Water Resources Research · doi:10.1029/2020wr028338 · Cited by 30
“The initial selection of hypotheses can also bias test results, for example, if some model components/mechanisms are overrepresented in the ensemble under consideration (see, e.g., Elkan, 2001; Saerens et al., 2002).”
BackgroundMethodQuantification in-the-wild: data-sets and baselinesPreprint
Beijbom, O., Hoffman, J., Yao, E., Darrell, T., Ramirez, A. R., González‐Rivero, M., & Hoegh‐Guldberg, O. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.04811 · Cited by 27
“The EM-algorithm of [16] is also evaluated (Fig.”
BackgroundMethodInfluentialNon-stationary data sequence classification using online class priors estimation
Yang, C., & Zhou, J. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2008.01.025 · Cited by 28
“In a recent study, Forman [18] focused on estimating the class priors using an improved form of the commonly used confusion matrix inversion technique, which was proved to be less effective than the BEM algorithm [17].”
MethodInfluentialComputational Methods for Identification of Functional Residues in Protein Structures
Xin, F., & Radivojac, P. (2011) · Current Protein and Peptide Science · doi:10.2174/138920311796957685 · Cited by 29
“Methods for estimating class priors have been proposed [130-132] as well as the algorithms that partition inhomogeneous data sets and separately estimate class priors on each partition [133].”
BackgroundThe Open World of Micro-VideosPreprint
Nguyen, P. X., Rogez, G., Fowlkes, C. C., & Ramanan, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.09439 · Cited by 33
“For example, if we assume that the popularity of tags changes over time but their appearance models do not, one can model efficiently model temporal dynamics with a statistical prior shift [42].”
BackgroundFisher consistency for prior probability shiftPreprint
Tasche, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.05512 · Cited by 29
“…matrix approach’ by Saerens et al. (2001), but in use since long before (Gart and Buck, 1966), • the EM (expectation maximisation) algorithm by Saerens et al. (2001), described before as maximum likelihood approach by Peters and Coberly (1976), and • CDE-Iterate (CDE for class distribution…”
BackgroundMethodInfluentialThe art of clustering bandits.
Li, S. (2016) · IrInSubria (University of Insubria) · Cited by 28
Computationally Efficient Class-Prior Estimation under Class Balance Change Using Energy Distance
Kawakubo, H., du Plessis, M. C., & Sugiyama, M. (2015) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2015edp7212 · Cited by 37
“A seminal method [4] adopts the expectation-maximization (EM) algorithm [5] to estimate the class ratio.”
MethodKernel density estimation for multiclass quantification
Moreo, A., González, P., & del Coz, J. J. (2025) · Machine Learning · doi:10.1007/s10994-024-06726-5 · Cited by 46
“The complete tables that include these omitted methods can be consulted in Appendix A.”
Uncertainty-aware generative models for inferring document class prevalence
Keith, K. A., & O’Connor, B. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1487 · Cited by 31
“Tasche (2017) presents theoretical results for ACC and Saerens et al.’s EM method (what we call the LR-Implicit MLE), arguing they correctly predict θ ∗ under class prior shift; we confirm that those two methods are indeed better than many alternatives in our empirical evaluation.”
BackgroundMethodInfluentialEsuli, A., Molinari, A., & Sebastiani, F. (2020) · ACM Transactions on Information Systems · doi:10.1145/3433164 · Cited by 39
“SLD,proposed by Saerens et al. [34], is an instance of Expectation Maximization [8], a well-known iterative algorithm for finding maximum-likelihood estimates of parameters (in our case: the class prior probabilities) for models that depend on unobserved variables (in our case: the class labels).”
BackgroundMethodInfluentialAn Exploration of Bayesian Identification of Dominant Hydrological Mechanisms in Ungauged Catchments
Prieto, C., Vine, N. L., Kavetski, D., Fenicia, F., Scheidegger, A., & Vitolo, C. (2022) · Water Resources Research · doi:10.1029/2021wr030705 · Cited by 27
Learning under nonstationarity: covariate shift and class‐balance change
Sugiyama, M., Yamada, M., & du Plessis, M. C. (2013) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.1275 · Cited by 32
“EM: The class-prior estimation method based on the expectation-maximization algorithm [42].”
BackgroundMethodInfluentialAliferis, C., Statnikov, A., Tsamardinos, I., Schildcrout, J. S., Shepherd, B. E., & Harrell, F. E. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004922 · Cited by 24
“It is thus unrealistic to expect in general that a classifier that is trained using data from a distribution where events and nonevents are equally likely will perform well, without adjustments [37], in a different distribution where this ratio is heavily distorted.”
BackgroundCoussement, K., & Buckinx, W. (2011) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.05.027 · Cited by 24
“In detail, the calibrated posterior probabilities of response for the customers in the test set of fold k are obtained by weighting the non-calibrated posterior probabilities, Pkorg, by the ratio of the response incidence of REALkN, i.e. the new prior probability distribution, to the response incidence in the training…”
MethodInfluentialMinimax Regret Classifier for Imprecise Class Distributions
Aláiz-Rodríguez, R., Guerrero-Curieses, A., & Cid‐Sueiro, J. (2007) · Journal of Machine Learning Research · doi:10.5555/1248659.1248663 · Cited by 24
Gentile, F., Tortora, C., Covone, G., Koopmans, L. V. E., Spiniello, C., Fan, Z., Li, R., Liu, D., Napolitano, N. R., Vaccari, M., et al. (2021) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stab3386 · Cited by 35
“Both CNNs take as input 101×101 pixels2 stamps (equivalent to 20×20 arcsec2) and give as output a single p-value in the range [0, 1], related to the probability that the object in the image is a strong gravitational lens (Saerens et al. 2002).”
BackgroundAccurately Quantifying a Billion Instances per Second
Hassan, W., Maletzke, A. G., & Batista, G. (2020) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa49011.2020.00012 · Cited by 28
“Expectation-Maximization Quantifier (EMQ) [19] is an iterative approach to estimate the class prevalence in imbalanced class distributions based on the Expectation-Maximization algorithm.”
MethodQuantification under prior probability shift: the ratio estimator and its extensionsPreprint
Vaz, A. F., Izbicki, R., & Stern, R. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03929 · Cited by 22
“In order to deal with this problem, several improvements have been proposed under an assumption named prior shift (Saerens et al., 2002; Forman, 2008; Bella et al., 2010; Barranquero et al., 2015).”
BackgroundInfluentialSoumya, M. P., Sasikumar, K., Pandey, A., & Nampoothiri, K. M. (2019) · Bioresource Technology Reports · doi:10.1016/j.biteb.2019.02.012 · Cited by 21
Metadata
- Authors
- Saerens, M., Latinne, P., & Decaestecker, C.
- Affiliations
- Université Libre de Bruxelles (BE)
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-01-01
- Volume
- 14
- Issue
- 1
- Pages
- 21-41
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2141253686
- Cited by
- 361
- References
- 26
- Citations by year
- 2026: 57 · 2025: 28 · 2024: 26 · 2023: 24 · 2022: 23 · 2021: 32 · 2020: 27 · 2019: 19 · 2018: 15 · 2017: 9 · 2016: 17 · 2015: 11 · 2014: 9 · 2013: 4 · 2012: 6
- FWCI
- 4.14
- Type
- journal article
- Language
- en
- Open access
- green
- dipot.ulb.ac.be
- Licence
- other-oa
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- posterior probability, expectation-maximization algorithm, likelihood maximization, multilayer neural networks, case-control study, classification accuracy
- MeSH terms
- Classification, Probability, Probability Theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.