The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Performance assessment of multiobjective optimizers: an analysis and review
Abstract
An important issue in multiobjective optimization is the quantitative comparison of the performance of different algorithms. In the case of multiobjective evolutionary algorithms, the outcome is usually an approximation of the Pareto-optimal set, which is denoted as an approximation set, and therefore the question arises of how to evaluate the quality of approximation sets. Most popular are methods that assign each approximation set a vector of real numbers that reflect different aspects of the quality. Sometimes, pairs of approximation sets are also considered. In this study, we provide a rigorous analysis of the limitations underlying this type of quality assessment. To this end, a mathematical framework is developed which allows one to classify and discuss existing techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Multi-Objective Optimization Using Evolutionary Algorithms
Deb, K., & Kalyanmoy, D. (2001) · Andalas University Repository (Andalas University) · Cited by 17,328
Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach
Zitzler, E., & Thiele, L. (1999) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.797969 · Cited by 8,798
Self-citationMuiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms
Srinivas, N., & Deb, K. (1994) · Evolutionary Computation · doi:10.1162/evco.1994.2.3.221 · Cited by 6,935
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,697
Self-citationA Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II
Deb, K., Agrawal, S., Pratap, A., & Meyarivan, T. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45356-3_83 · Cited by 5,289
Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy
Knowles, J. D., & Corne, D. W. (2000) · Evolutionary Computation · doi:10.1162/106365600568167 · Cited by 2,286
Multiobjective optimization using evolutionary algorithms — A comparative case study
Zitzler, E., & Thiele, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0056872 · Cited by 2,262
Self-citationEvolutionary algorithms for multiobjective optimization: methods and applications
Zitzler, E. (1999) · doi:10.3929/ethz-a-003856832 · Cited by 1,965
Self-citationCombining Convergence and Diversity in Evolutionary Multiobjective Optimization
Laumanns, M., Thiele, L., Deb, K., & Zitzler, E. (2002) · Evolutionary Computation · doi:10.1162/106365602760234108 · Cited by 1,487
Self-citation
Cited by
MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition
Zhang, Q., & Li, H. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2007.892759 · Cited by 10,069
Deb, K., & Jain, H. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2281535 · Cited by 7,133
Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II
Li, H., & Zhang, Q. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.925798 · Cited by 2,543
Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy
Knowles, J. D., & Corne, D. W. (2000) · Evolutionary Computation · doi:10.1162/106365600568167 · Cited by 2,286
Indicator-Based Selection in Multiobjective Search
Zitzler, E., & Künzli, S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30217-9_84 · Cited by 2,259
Self-citationMultiobjective evolutionary algorithms: A survey of the state of the art
Zhou, A., Qu, B.-Y., Li, H., Zhao, S.-Z., Suganthan, P. N., & Zhang, Q. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.03.001 · Cited by 2,255
A review of multiobjective test problems and a scalable test problem toolkit
Huband, S., Hingston, P., Barone, L., & While, L. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.861417 · Cited by 2,090
HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization
Bader, J., & Zitzler, E. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00009 · Cited by 2,081
Self-citationThe irace package: Iterated racing for automatic algorithm configuration
López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016) · Operations Research Perspectives · doi:10.1016/j.orp.2016.09.002 · Cited by 1,761
“6 ) and the (cid:6) - measure [95] , assign a numerical value to a set of nondominated solutions, thus allowing the application of standard automatic con-figuration methods [58,91] .”
BackgroundCombining Convergence and Diversity in Evolutionary Multiobjective Optimization
Laumanns, M., Thiele, L., Deb, K., & Zitzler, E. (2002) · Evolutionary Computation · doi:10.1162/106365602760234108 · Cited by 1,487
Self-citationEvolutionary multi-objective optimization: a historical view of the field
Coello Coello, C. A. (2006) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2006.1597059 · Cited by 1,446
Knowles, J. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.851274 · Cited by 1,254
Deb, K., & Deb, K. (2014) · Search Methodologies · doi:10.1007/978-1-4614-6940-7_15 · Cited by 1,248
The balance between proximity and diversity in multiobjective evolutionary algorithms
Bosman, P. A. N., & Thierens, D. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810761 · Cited by 1,154
A faster algorithm for calculating hypervolume
While, L., Hingston, P., Barone, L., & Huband, S. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.851275 · Cited by 1,100
Multi-objective Optimisation Using Evolutionary Algorithms: An Introduction
Deb, K. (2011) · Multi-objective Evolutionary Optimisation for Product Design and Manufacturing · doi:10.1007/978-0-85729-652-8_1 · Cited by 995
Evolutionary many-objective optimization: A short review
Ishibuchi, H., Tsukamoto, N., & Nojima, Y. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4631121 · Cited by 914
Many-Objective Evolutionary Algorithms: A Survey
Li, B., Li, J., Tang, K., & Yao, X. (2015) · ACM Computing Surveys · doi:10.1145/2792984 · Cited by 815
Covariance Matrix Adaptation for Multi-objective Optimization
Igel, C., Hansen, N., & Roth, S. (2007) · Evolutionary Computation · doi:10.1162/evco.2007.15.1.1 · Cited by 788
Improving PSO-Based Multi-objective Optimization Using Crowding, Mutation and ∈-Dominance
Sierra, M. R., & Coello Coello, C. A. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31880-4_35 · Cited by 779
A New Dominance Relation-Based Evolutionary Algorithm for Many-Objective Optimization
Yuan, Y., Xu, H., Wang, B., & Yao, X. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2420112 · Cited by 752
Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model
Zhang, Q., Liu, W., Tsang, E., & Virginas, B. (2010) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2009.2033671 · Cited by 746
A tutorial on multiobjective optimization: fundamentals and evolutionary methods
Emmerich, M. T. M., & Deutz, A. H. (2018) · Natural Computing · doi:10.1007/s11047-018-9685-y · Cited by 731
MOEA/D with Adaptive Weight Adjustment
Qi, Y., Ma, X., Liu, F., Jiao, L., Sun, J., & Wu, J. (2014) · Evolutionary Computation · doi:10.1162/evco_a_00109 · Cited by 690
Review: Multi-objective optimization methods and application in energy saving
Cui, Y., Geng, Z., Zhu, Q., & Han, Y. (2017) · Energy · doi:10.1016/j.energy.2017.02.174 · Cited by 685
Maier, H. R., Kapelan, Z., Kasprzyk, J., Kollat, J., Matott, L. S., Cunha, M. C., Dandy, G. C., Gibbs, M. S., Keedwell, E., Marchi, A., et al. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.09.013 · Cited by 657
Deb, K., Mohan, M., & Mishra, S. (2005) · Evolutionary Computation · doi:10.1162/106365605774666895 · Cited by 637
A Competitive-Cooperative Coevolutionary Paradigm for Dynamic Multiobjective Optimization
Goh, C.-K., & Tan, K. C. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.920671 · Cited by 592
Evolutionary multiobjective optimization in water resources: The past, present, and future
Reed, P. M., Hadka, D., Herman, J. D., Kasprzyk, J. R., & Kollat, J. B. (2013) · Advances in Water Resources · doi:10.1016/j.advwatres.2012.01.005 · Cited by 541
Ma, X., Liu, F., Qi, Y., Wang, X., Li, L., Jiao, L., Yin, M., & Gong, M. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2455812 · Cited by 523
A Tutorial on Evolutionary Multiobjective Optimization
Zitzler, E., Laumanns, M., & Bleuler, S. (2004) · Lecture Notes in Economics and Mathematical Systems · doi:10.1007/978-3-642-17144-4_1 · Cited by 515
Self-citationSchutze, O., Esquivel, X., Lara, A., & Coello, C. A. C. (2012) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2011.2161872 · Cited by 511
Multiobjective Immune Algorithm with Nondominated Neighbor-Based Selection
Gong, M., Jiao, L., Du, H., & Bo, L. (2008) · Evolutionary Computation · doi:10.1162/evco.2008.16.2.225 · Cited by 486
Generalizing Surrogate-Assisted Evolutionary Computation
Lim, D., Jin, Y., Ong, Y.-S., & Sendhoff, B. (2010) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2009.2027359 · Cited by 484
One decade of multi-objective calibration approaches in hydrological modelling: a review
Efstratiadis, A., & Koutsoyiannis, D. (2010) · Hydrological Sciences Journal · doi:10.1080/02626660903526292 · Cited by 479
Recent Advances in Bayesian Optimization
Wang, X., Jin, Y., Schmitt, S., & Olhofer, M. (2023) · ACM Computing Surveys · doi:10.1145/3582078 · Cited by 477
A Survey of Multi-Objective Sequential Decision-Making
Roijers, D. M., Vamplew, P., Whiteson, S., & Dazeley, R. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3987 · Cited by 461
The bi-objective Pollution-Routing Problem
Demir, E., Bektaş, T., & Laporte, G. (2014) · European Journal of Operational Research · doi:10.1016/j.ejor.2013.08.002 · Cited by 460
On the Evolutionary Optimization of Many Conflicting Objectives
Purshouse, R. C., & Fleming, P. J. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2007.910138 · Cited by 451
Performance metrics in multi-objective optimization
Riquelme, N., Von Lucken, C., & Baran, B. (2015) · 2015 Latin American Computing Conference (CLEI) · doi:10.1109/clei.2015.7360024 · Cited by 444
An Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems
Tian, Y., Zhang, X., Wang, C., & Jin, Y. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2918140 · Cited by 440
Constraint Handling in Multiobjective Evolutionary Optimization
Woldesenbet, Y. G., Yen, G. G., & Tessema, B. G. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.2009032 · Cited by 420
Li, K., Fialho, A., Kwong, S., & Zhang, Q. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2239648 · Cited by 418
Preference-Inspired Coevolutionary Algorithms for Many-Objective Optimization
Wang, R., Purshouse, R. C., & Fleming, P. J. (2013) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2012.2204264 · Cited by 406
Balancing Convergence and Diversity in Decomposition-Based Many-Objective Optimizers
Yuan, Y., Xu, H., Wang, B., Zhang, B., & Yao, X. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2443001 · Cited by 400
Currin, A., Swainston, N., Day, P. J., & Kell, D. B. (2015) · Chemical Society Reviews · doi:10.1039/c4cs00351a · Cited by 388
A Survey on the Hypervolume Indicator in Evolutionary Multiobjective Optimization
Shang, K., Ishibuchi, H., He, L., & Pang, L. M. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.3013290 · Cited by 380
Kou, G., Xu, Y., Peng, Y., Shen, F., Chen, Y., Chang, K., & Kou, S. (2021) · Decision Support Systems · doi:10.1016/j.dss.2020.113429 · Cited by 380
Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point
Auger, A., Bader, J., Brockhoff, D., & Zitzler, E. (2009) · Proceedings of the tenth ACM SIGEVO workshop on Foundations of genetic algorithms · doi:10.1145/1527125.1527138 · Cited by 375
Self-citationMany-Objective Optimization: An Engineering Design Perspective
Fleming, P. J., Purshouse, R. C., & Lygoe, R. J. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31880-4_2 · Cited by 372
Localized Weighted Sum Method for Many-Objective Optimization
Wang, R., Zhou, Z., Ishibuchi, H., Liao, T., & Zhang, T. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2611642 · Cited by 362
Multiobjective Reinforcement Learning: A Comprehensive Overview
Liu, C., Xu, X., & Hu, D. (2015) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2014.2358639 · Cited by 361
Stable Matching-Based Selection in Evolutionary Multiobjective Optimization
Li, K., Zhang, Q., Kwong, S., Li, M., & Wang, R. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2293776 · Cited by 357
Vladislavleva, E. J., Smits, G. F., & den Hertog, D. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.926486 · Cited by 353
Quality Evaluation of Solution Sets in Multiobjective Optimisation: A Survey
Li, M., & Yao, X. (2020) · ACM Computing Surveys · doi:10.1145/3300148 · Cited by 342
Kollat, J. B., & Reed, P. M. (2006) · Advances in Water Resources · doi:10.1016/j.advwatres.2005.07.010 · Cited by 328
An easy-to-use real-world multi-objective optimization problem suite
Tanabe, R., & Ishibuchi, H. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106078 · Cited by 324
Tian, Y., Lu, C., Zhang, X., Tan, K. C., & Jin, Y. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2979930 · Cited by 318
Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization
Jiang, S., Ong, Y.-S., Zhang, J., & Feng, L. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2307319 · Cited by 314
Evolutionary Dynamic Multiobjective Optimization Via Kalman Filter Prediction
Muruganantham, A., Tan, K. C., & Vadakkepat, P. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2490738 · Cited by 314
Quality Assessment of Pareto Set Approximations
Zitzler, E., Knowles, J., & Thiele, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88908-3_14 · Cited by 313
Self-citationDifficulty Adjustable and Scalable Constrained Multiobjective Test Problem Toolkit
Fan, Z., Li, W., Cai, X., Li, H., Wei, C., Zhang, Q., Deb, K., & Goodman, E. (2020) · Evolutionary Computation · doi:10.1162/evco_a_00259 · Cited by 308
Wang, Y.-N., Wu, L.-H., & Yuan, X.-F. (2010) · Soft Computing · doi:10.1007/s00500-008-0394-9 · Cited by 306
Chugh, T., Sindhya, K., Hakanen, J., & Miettinen, K. (2019) · Soft Computing · doi:10.1007/s00500-017-2965-0 · Cited by 305
The r-Dominance: A New Dominance Relation for Interactive Evolutionary Multicriteria Decision Making
Ben Said, L., Bechikh, S., & Ghedira, K. (2010) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2010.2041060 · Cited by 301
A survey on multi-objective evolutionary algorithms for many-objective problems
von Lücken, C., Barán, B., & Brizuela, C. (2014) · Computational Optimization and Applications · doi:10.1007/s10589-014-9644-1 · Cited by 297
Fazzolari, M., Alcala, R., Nojima, Y., Ishibuchi, H., & Herrera, F. (2013) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2012.2201338 · Cited by 296
Performance assessment of the metaheuristic optimization algorithms: an exhaustive review
Halim, A. H., Ismail, I., & Das, S. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09906-6 · Cited by 293
A New Decomposition-Based NSGA-II for Many-Objective Optimization
Elarbi, M., Bechikh, S., Gupta, A., Ben Said, L., & Ong, Y.-S. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2654301 · Cited by 290
Methods for evaluating and covering the design space during early design development
Gries, M. (2004) · Integration, the VLSI Journal · doi:10.1016/j.vlsi.2004.06.001 · Cited by 289
Decomposition-Based Algorithms Using Pareto Adaptive Scalarizing Methods
Wang, R., Zhang, Q., & Zhang, T. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2521175 · Cited by 283
A critical survey of performance indices for multi-objective optimisation
Okabe, T., Jin, Y., & Sendhoff, B. (2004) · The 2003 Congress on Evolutionary Computation, 2003. CEC '03. · doi:10.1109/cec.2003.1299759 · Cited by 278
Empirical evaluation methods for multiobjective reinforcement learning algorithms
Vamplew, P., Dazeley, R., Berry, A., Issabekov, R., & Dekker, E. (2011) · Machine Learning · doi:10.1007/s10994-010-5232-5 · Cited by 275
Techniques for highly multiobjective optimisation: some nondominated points are better than others
Corne, D. W., & Knowles, J. D. (2007) · Proceedings of the 9th annual conference on Genetic and evolutionary computation · doi:10.1145/1276958.1277115 · Cited by 273
“This is not ideal or authoritative (there is a growing literature on performance metrics for MOEAs, e.g. see [10, 26, 28]); however, early investigations showed that, in the case of the experiments done here, the cover metric provided sufficient distinctions in performance in this case.”
ResultCai, X., Geng, S., Wu, D., Cai, J., & Chen, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040019 · Cited by 273
Tsai, J.-T., Fang, J.-C., & Chou, J.-H. (2013) · Computers & Operations Research · doi:10.1016/j.cor.2013.06.012 · Cited by 272
A Many-Objective Evolutionary Algorithm Using A One-by-One Selection Strategy
Liu, Y., Gong, D., Sun, J., & Jin, Y. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2638902 · Cited by 271
Bradford, E., Schweidtmann, A. M., & Lapkin, A. (2018) · Journal of Global Optimization · doi:10.1007/s10898-018-0609-2 · Cited by 266
Solving Rotated Multi-objective Optimization Problems Using Differential Evolution
Iorio, A. W., & Li, X. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30549-1_74 · Cited by 263
Diversity Management in Evolutionary Many-Objective Optimization
Adra, S. F., & Fleming, P. J. (2011) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2010.2058117 · Cited by 262
Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., & Reed, P. M. (2016) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0000570 · Cited by 257
Dhiman, G., & Kumar, V. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.03.011 · Cited by 257
Adaptive Replacement Strategies for MOEA/D
Wang, Z., Zhang, Q., Zhou, A., Gong, M., & Jiao, L. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2403849 · Cited by 257
Couckuyt, I., Deschrijver, D., & Dhaene, T. (2014) · Journal of Global Optimization · doi:10.1007/s10898-013-0118-2 · Cited by 254
Multi-Objective Scheduling of a Single Mobile Robot Based on the Grey Wolf Optimization Algorithm
Petrović, M., Jokić, A., Miljković, Z., & Kulesza, Z. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109784 · Cited by 50
Preference-Based Evolutionary Many-Objective Optimization for Agile Satellite Mission Planning
Li, L., Chen, H., Li, J., Jing, N., & Emmerich, M. (2018) · IEEE Access · doi:10.1109/access.2018.2859028 · Cited by 46
Metadata
- Authors
- Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & da Fonseca, V. G.
- Venue
- IEEE Transactions on Evolutionary Computation
- Published
- 2003-04
- Volume
- 7
- Issue
- 2
- Pages
- 117-132
- ISSN
- 1089-778X (ISSN-L)
- OpenAlex
- W2098907614
- Cited by
- 4,330
- References
- 22
- FWCI
- 63.37
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.