The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
AUC Maximization in the Era of Big Data and AI: A Survey
Open accessAbstract
Area under the ROC curve, a.k.a. AUC, is a measure of choice for assessing the performance of a classifier for imbalanced data. AUC maximization refers to a learning paradigm that learns a predictive model by directly maximizing its AUC score. It has been studied for more than two decades dating back to late 90s, and a huge amount of work has been devoted to AUC maximization since then. Recently, stochastic AUC maximization for big data and deep AUC maximization (DAM) for deep learning have received increasing attention and yielded dramatic impact for solving real-world problems. However, to the best our knowledge, there is no comprehensive survey of related works for AUC maximization. This article aims to address the gap by reviewing the literature in the past two decades. We not only give a holistic view of the literature but also present detailed explanations and comparisons of different papers from formulations to algorithms and theoretical guarantees. We also identify and discuss remaining and emerging issues for DAM and provide suggestions on topics for future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 3×
“According to the probabilistic interpretation of AUC in (2), a non-parametric estimator can be computed as follows that corresponds to the Mann-Whitney U-statistic [63]: A (non-normalized) non-parametric estimator of pAUC can be computed by [37]: pAUC ( 𝑓 , 𝛼, 𝛽 ; S) = 1 where 𝑞 𝛼 denotes the 𝛼 quantile of 𝑓 ( x − ) ,…”
BackgroundMethodInfluentialAdvances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
(2010) · Cited by 8,261
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
MoleculeNet: a benchmark for molecular machine learning
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., & Pande, V. (2018) · Chemical Science · doi:10.1039/c7sc02664a · Cited by 3,107
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
2011 International Conference on Computer Vision
(2011) · doi:10.1109/iccv17851.2011 · Cited by 2,860
Improved boosting algorithms using confidence-rated predictions
Schapire, R. E., & Singer, Y. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279960 · Cited by 2,547
Smooth minimization of non-smooth functions
Nesterov, Y. (2005) · Mathematical Programming · doi:10.1007/s10107-004-0552-5 · Cited by 2,455
Advances in Neural Information Processing Systems 32 (NeurIPS 2019)
Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché–Buc, F., Fox, E. B., & Garnett, R. (2020) · HAL · Cited by 2,450
Artificial intelligence and statistics
Yu, B., & Kumbier, K. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700813 · Cited by 2,329
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276
Robust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Johnson, R., & Zhang, T. (2013) · Cited by 1,889
Advances in Large-Margin Classifiers
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/1113.001.0001 · Cited by 1,869
Random sampling with a reservoir
Vitter, J. S. (1985) · ACM Transactions on Mathematical Software · doi:10.1145/3147.3165 · Cited by 1,758
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Miller, A. J., & Mihm, M. C. (2006) · New England Journal of Medicine · doi:10.1056/nejmra052166 · Cited by 1,339
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000018 · Cited by 1,246
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
Ad click prediction: a view from the trenches
McMahan, H. B., Holt, G., Sculley, D., Young, M., Ebner, D., Grady, J., Nie, L., Phillips, T., Davydov, E., Golovin, D., et al. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2488200 · Cited by 867
Current Status of the Digital Database for Screening Mammography
Heath, M., Bowyer, K., Kopans, D., Kegelmeyer, P., Moore, R., Chang, K., & Munishkumaran, S. (1998) · Computational Imaging and Vision · doi:10.1007/978-94-011-5318-8_75 · Cited by 787
A support vector method for multivariate performance measures
Joachims, T. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102399 · Cited by 786
Approximation Theory and Methods
Powell, M. J. D. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9781139171502 · Cited by 771
Analyzing a Portion of the ROC Curve
McClish, D. K. (1989) · Medical Decision Making · doi:10.1177/0272989x8900900307 · Cited by 707
This Looks Like That: Deep Learning for Interpretable Image Recognition
Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C. (2018) · arXiv · doi:10.48550/arxiv.1806.10574 · Cited by 702
Big Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · cited 1×
“Two issues are of foremost importance, namely fairness [7, 10, 20, 113] and interpretability [3, 24, 64].”
Background(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760200704 · Cited by 636
Conference on Neural Information Processing Systems
Liu, L., Zhou, T., Long, G., Jiang, J., & Zhang, C. (2019) · UTS ePRESS (University of Technology Sydney) · Cited by 593
Online Learning: A Modern Introduction Using Convex OptimizationPreprint
Orabona, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13213 · Cited by 587
Lectures on Stochastic Programming: Modeling and Theory, Second Edition
Shapiro, A., Dentcheva, D., & Ruszczyński, A. (2014) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611973433 · Cited by 575
Identifying suspicious URLs: an application of large-scale online learning
Ma, J., Saul, L. K., Savage, S., & Voelker, G. M. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553462 · Cited by 498
Rotation Equivariant CNNs for Digital Pathology
Veeling, B. S., Linmans, J., Winkens, J., Cohen, T., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00934-2_24 · Cited by 476
Open Graph Benchmark: Datasets for Machine Learning on Graphs
Hu, W., Fey, M., Žitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., & Leskovec, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00687 · Cited by 470
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
Partial AUC Estimation and Regression
Dodd, L. E., & Pepe, M. S. (2003) · Biometrics · doi:10.1111/1541-0420.00071 · Cited by 346
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Borkan, D., Dixon, L., Sorensen, J., Thain, N., & Vasserman, L. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317593 · Cited by 309
CheXclusion: Fairness gaps in deep chest X-ray classifiers
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M. (2020) · Biocomputing 2021 · doi:10.1142/9789811232701_0022 · Cited by 275 · cited 1×
“Although these issues have received tremendous attention in the literature for medical image classification [26, 145, 146], developing fair and interpretable DAMmethods remains to be explored.”
BackgroundContrastive Learning of Medical Visual Representations from Paired Images and Text
Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00747 · Cited by 265
Stochastic Model-Based Minimization of Weakly Convex Functions
Davis, D., & Drusvyatskiy, D. (2019) · SIAM Journal on Optimization · doi:10.1137/18m1178244 · Cited by 258
2013 IEEE Symposium on Security and Privacy
(2013) · doi:10.1109/sp16516.2013 · Cited by 218
Optimising area under the ROC curve using gradient descent
Herschtal, A., & Raskutti, B. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015366 · Cited by 190
Wang, M., Fang, E. X., & Liu, H. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1017-3 · Cited by 178
Implicit Regularization in Deep LearningPreprint
Neyshabur, B. (2017) · arXiv.org · doi:10.48550/arxiv.1709.01953 · Cited by 169
Interpretable Image Recognition with Hierarchical Prototypes
Hase, P., Chen, C., Li, O., & Rudin, C. D. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5265 · Cited by 143
Efficient AUC Optimization for Classification
Calders, T., & Jaroszewicz, S. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74976-9_8 · Cited by 142
The P-Norm Push: A Simple Convex Ranking Algorithm that Concentrates at the Top of the List
Rudin, C. D. (2009) · DSpace@MIT (Massachusetts Institute of Technology) · doi:10.5555/1577069.1755861 · Cited by 136
Advanced graph and sequence neural networks for molecular property prediction and drug discovery
Wang, Z., Liu, M., Luo, Y., Zhao, X., Xie, Y., Wang, L., Cai, L., Qi, Q., Yuan, Z., Yang, T., et al. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac112 · Cited by 135
Self-citationWang, S., Ma, J.-Z., & Xu, J. (2016) · Bioinform. · doi:10.1093/bioinformatics/btw446 · Cited by 124
AUC-oriented Graph Neural Network for Fraud Detection
Huang, M., Liu, Y., Ao, X., Li, K., Chi, J., Feng, J., Yang, H., & He, Q. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512178 · Cited by 121
Learning With Multiclass AUC: Theory and Algorithms
Yang, Z.-Y., Xu, Q.-Q., Bao, S.-L., Cao, X.-C., & Huang, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3101125 · Cited by 103
The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC MetricPreprint
Kallus, N., & Zhou, A. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1902.05826 · Cited by 95
A survey of Algorithms and Analysis for Adaptive Online Learning
McMahan, H. B. (2017) · Journal of Machine Learning Research · doi:10.5555/3122009.3176834 · Cited by 87
A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization
Ghadimi, S., Ruszczyński, A., & Wang, M. (2020) · SIAM Journal on Optimization · doi:10.1137/18m1230542 · Cited by 79
Orseau, L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24412-4 · Cited by 75
AUC-Maximizing Ensembles through Metalearning
LeDell, E., van der Laan, M. J., & Petersen, M. L. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0035 · Cited by 73
Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006
Wang, D., Irani, D., & Pu, C. (2012) · IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom) · doi:10.4108/icst.collaboratecom.2012.250689 · Cited by 66
Maximization of AUC and Buffered AUC in binary classification
Norton, M., & Uryasev, S. (2018) · Mathematical programming · doi:10.1007/s10107-018-1312-2 · Cited by 61
Alternating Proximal-Gradient Steps for (Stochastic) Nonconvex-Concave Minimax Problems
Boţ, R. I., & Böhm, A. (2023) · SIAM Journal on Optimization · doi:10.1137/21m1465470 · Cited by 58
Online Nonlinear AUC Maximization for Imbalanced Data Sets
Hu, J., Yang, H., Lyu, M. R.-T., King, I., & So, A. M. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2610465 · Cited by 56
Agarwal, S. (2011) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972818.72 · Cited by 54
Optimizing area under the ROC curve using semi-supervised learning
Wang, S., Li, D., Petrick, N., Sahiner, B., Linguraru, M. G., & Summers, R. M. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.07.025 · Cited by 50
Generalization Bounds for Uniformly Stable AlgorithmsPreprint
Feldman, V., & Vondrák, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.09859 · Cited by 48
Efficient Pedestrian Detection by Directly Optimizing the Partial Area under the ROC Curve
Paisitkriangkrai, S., Shen, C., & van den Hengel, A. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.135 · Cited by 43
An Adaptive Gradient Method for Online AUC Maximization
Ding, Y., Zhao, P., Hoi, S. C. H., & Ong, Y.-S. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9577 · Cited by 42
ProtoAttend: Attention-Based Prototypical LearningPreprint
Arık, S. Ö., & Pfister, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06292 · Cited by 42
Hinge Rank Loss and the Area Under the ROC Curve
Steck, H. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_33 · Cited by 39
Recommending Positive Links in Signed Social Networks by Optimizing a Generalized AUC
Song, D., & Meyer, D. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9167 · Cited by 38
An Introduction to Neural Architecture Search for Convolutional NetworksPreprint
Kyriakides, G., & Margaritis, K. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11074 · Cited by 38
Ren, K., Yang, H., Zhao, Y., Chen, W., Xue, M., Miao, H., Huang, S., & Liu, J. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2870666 · Cited by 36
Asymmetric support vector machines: low false-positive learning under the user tolerance
Wu, S.-H., Lin, K., Chen, C., & Chen⋆, M.-S. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401980 · Cited by 35
Support Vector Algorithms for Optimizing the Partial Area under the ROC Curve
Narasimhan, H., & Agarwal, S. (2017) · Neural Computation · doi:10.1162/neco_a_00972 · Cited by 35
Machine-learning selection of optical transients in the Subaru/Hyper Suprime-Cam survey
Morii, M., Ikeda, S., Tominaga, N., Tanaka, M., Morokuma, T., Ishiguro, K., Yamato, J., Ueda, N., Suzuki, N., Yasuda, N., et al. (2016) · Publications of the Astronomical Society of Japan · doi:10.1093/pasj/psw096 · Cited by 32
Optimizing area under the ROC curve via extreme learning machines
Yang, Z., Zhang, T., Lu, J., Zhang, D., & Kalui, D. M. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.05.013 · Cited by 32
Direct AUC optimization of regulatory motifs
Zhu, L., Zhang, H., & Huang, D.-S. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx255 · Cited by 32
Technical Challenges for Training Fair Neural NetworksPreprint
Cherepanova, V., Nanda, V., Goldblum, M., Dickerson, J. P., & Goldstein, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06764 · Cited by 25
Large-Scale Nonlinear AUC Maximization via Triply Stochastic Gradients
Dang, Z., Li, X., Gu, B., Deng, C. D., & Huang, H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3024987 · Cited by 24
Large Scale Kernel Methods for Online AUC Maximization
Ding, Y., Liu, C., Zhao, P., & Hoi, S. C. H. (2017) · Proceedings · doi:10.1109/icdm.2017.18 · Cited by 23
Partial AUC maximization in a linear combination of dichotomizers
Ricamato, M. T., & Tortorella, F. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2011.03.022 · Cited by 19
An Adaptive Moment estimation method for online AUC maximization
Liu, X., Pan, Z., Yang, H., Zhou, X., Bai, W., & Niu, X. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215426 · Cited by 15
A Survey on ROC-based Ordinal Regression
Waegeman, W., & Bernard De Baets, C. (2010) · Springer eBooks · doi:10.1007/978-3-642-14125-6_7 · Cited by 13
Shi, W., Gu, B., Li, X., & Huang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6029 · Cited by 12
AUC maximization linear classifier based on active learning and its application
Han, G., & Zhao, C. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.01.001 · Cited by 11
Evolving Neural Networks with Maximum AUC for Imbalanced Data Classification
Lu, X., Tang, K., & Yao, X. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13769-3_41 · Cited by 7
Semi-Supervised Learning for Maximizing the Partial AUC
Iwata, T., Fujino, A., & Ueda, N. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5846 · Cited by 7
Multivariate two-sample hypothesis testing through AUC maximization for biomedical applications
Bargiotas, I., Kalogeratos, A., Limnios, M., Vidal, P., Ricard, D., & Vayatis, N. (2020) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3411408.3411422 · Cited by 6
Approximate Heavily-Constrained Learning with Lagrange Multiplier Models
Narasimhan, H., Cotter, A., Zhou, Y., et al. (2020) · neural information processing systems · Cited by 5
An Adaptive Robust Online Method for AUC Maximization
Cheng, F., Zhang, X., Zhang, C., Qiu, J., & Zhang, L. (2018) · IEEE Access · doi:10.1109/access.2018.2869860 · Cited by 2
Cited by
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242
(2024) · doi:10.1002/advs.202406305 · Cited by 55
Squidiff: predicting cellular development and responses to perturbations using a diffusion model
He, S., Zhu, Y., Tavakol, D. N., Ye, H., Lao, Y.-H., Zhu, Z., Xu, C., Chauhan, S., Garty, G., Tomer, R., et al. (2026) · Nature Methods · doi:10.1038/s41592-025-02877-y · Cited by 38
Kasim, S., Amir Rudin, P. N. F., Malek, S., Ibrahim, N., Kiew, X. N., Nasir, N. M., Ibrahim, K. S., & Raja Shariff, R. E. (2025) · PLOS One · doi:10.1371/journal.pone.0323949 · Cited by 3
Explainable AI for employee turnover prediction: a SHAP-based intelligent analytics approach
Jia, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-60901-3
Metadata
- Authors
- Yang, T., & Ying, Y.
- Venue
- ACM Computing Surveys
- Published
- 2022-12-23
- Volume
- 55
- Issue
- 8
- Pages
- 1-37
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4226211505
- Cited by
- 323
- References
- 94
- FWCI
- 19.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.