The DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Agent-based model calibration using machine learning surrogates
Open accessAbstract
Taking agent-based models (ABM) closer to the data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs combining supervised machine-learning and intelligent sampling to build a surrogate meta-model. The proposed approach provides a fast and accurate approximation of model behaviour, dramatically reducing computation time. In that, our machine-learning surrogate facilitates large scale explorations of the parameter-space, while providing a powerful filter to gain insights into the complex functioning of agent-based models. The algorithm introduced in this paper merges model simulation and output analysis into a surrogate meta-model, which substantially ease ABM calibration. We successfully apply our approach to the Brock and Hommes (1998) asset pricing model and to the “Island” endogenous growth model (Fagiolo and Dosi, 2003). Performance is evaluated against a relatively large outof-sample set of parameter combinations, while employing different user-defined statistical tests for output analysis. The results demonstrate the capacity of machine learning surrogates to facilitate fast and precise exploration of agent-based models’ behaviour over their often rugged parameter spaces
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053
(2012) · Cited by 8,009
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
A Simple Model of Herd Behavior
Banerjee, A. V. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118364 · Cited by 6,556
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 6,385
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,506
On sequential Monte Carlo sampling methods for Bayesian filtering
Doucet, A., Godsill, S., & Andrieu, C. (2000) · Statistics and Computing · doi:10.1023/a:1008935410038 · Cited by 4,411
Sources, Procedures, and Microeconomic Effects of Innovation
Dosi, G. (2000) · Innovation, Organization and Economic Dynamics · doi:10.4337/9781782541851.00008 · Cited by 4,272
Understanding the Metropolis-Hastings Algorithm
Chib, S., & Greenberg, E. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476177 · Cited by 4,145
More Is Different: Broken symmetry and the nature of the hierarchical structure of science.
Anderson, P. W. (1972) · Science · doi:10.1126/science.177.4047.393 · Cited by 4,018
Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., & Tarantola, S. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2009.09.018 · Cited by 3,623
Proceedings of the 23rd international conference on Machine learning
Cohen, W. W., & Moore, A. (2006) · Cited by 2,647
Classification and regression trees
Loh, W. (2011) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.8 · Cited by 2,285
Machine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Individual-based Modeling and Ecology
Grimm, V., & Railsback, S. F. (2005) · Princeton University Press eBooks · doi:10.1515/9781400850624 · Cited by 1,940
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Rational Route to Randomness
Brock, W. A., & Hommes, C. H. (1997) · Econometrica · doi:10.2307/2171879 · Cited by 1,711
Equation of state calculations by fast computing machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · doi:10.2172/4390578 · Cited by 1,701
Heterogeneous beliefs and routes to chaos in a simple asset pricing model
Brock, W. A., & Hommes, C. H. (1998) · Journal of Economic Dynamics and Control · doi:10.1016/s0165-1889(98)00011-6 · Cited by 1,693
Seung, H. S., Opper, M., & Sompolinsky, H. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130417 · Cited by 1,651
Improving generalization with active learning
Cohn, D., Atlas, L., & Ladner, R. (1994) · Machine Learning · doi:10.1007/bf00993277 · Cited by 1,591
Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
Cireşan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_51 · Cited by 1,536 · cited 2×
“Taking into consideration the upper bound of 1 and various practical applications (e.g. Petrovic et al., 2011; Cire¸san et al., 2013), we consider the former result satisfying.”
Background(1990) · Choice Reviews Online · doi:10.5860/choice.28-0304 · Cited by 1,495
From Factors to Actors: Computational Sociology and Agent-Based Modeling
Macy, M. W., & Willer, R. (2002) · Annual Review of Sociology · doi:10.1146/annurev.soc.28.110601.141117 · Cited by 1,443
Understanding variable importances in forests of randomized trees
Louppe, G., Wehenkel, L. A., Sutera, A., & Geurts, P. (2013) · ORBi (University of Liège) · Cited by 1,364
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Empirical characterization of random forest variable importance measures
Archer, K. J., & Kimes, R. V. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.08.015 · Cited by 1,123
A rigorous framework for optimization of expensive functions by surrogates
Booker, A. J., Dennis, J. E., Frank, P. D., Serafini, D. B., Torczon, V., & Trosset, M. W. (1999) · Structural Optimization · doi:10.1007/bf01197708 · Cited by 960
Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles
Dosi, G., Fagiolo, G., & Roventini, A. (2010) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2010.06.018 · Cited by 730
Self-citationHandbook of Computational Economics, Volume 2: Agent-Based Computational Economics
Tesfatsion, L., & Judd, K. L. (2006) · Cited by 572
Proceedings of the fifth annual workshop on Computational learning theory
(1992) · doi:10.1145/130385 · Cited by 560
The Supervised Learning No-Free-Lunch Theorems
Wolpert, D. H. (2002) · Soft Computing and Industry · doi:10.1007/978-1-4471-0123-9_3 · Cited by 468
Behavioral heterogeneity in stock prices
Boswijk, H. P., Hommes, C., & Manzan, S. (2007) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2007.01.001 · Cited by 462
Explaining the distribution of firm growth rates
Bottazzi, G., & Secchi, A. (2006) · The RAND Journal of Economics · doi:10.1111/j.1756-2171.2006.tb00014.x · Cited by 421
Bayesian emulation of complex multi-output and dynamic computer models
Conti, S., & O’Hagan, A. (2010) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2009.08.006 · Cited by 417
Path dependence and the validation of agent‐based spatial models of land use
Brown, D. G., Page, S., Riolo, R., Zellner, M., & Rand, W. (2005) · International Journal of Geographical Information Science · doi:10.1080/13658810410001713399 · Cited by 388
Quasi-Random Sequences and Their Discrepancies
Morokoff, W. J., & Caflisch, R. E. (1994) · SIAM Journal on Scientific Computing · doi:10.1137/0915077 · Cited by 377
RT to Win! Predicting Message Propagation in Twitter
Petrovic, S., Osborne, M., & Lavrenko, V. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14149 · Cited by 367
Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model
Alfarano, S., Lux, T., & Wagner, F. (2005) · Computational Economics · doi:10.1007/s10614-005-6415-1 · Cited by 350
Income Distribution, Credit and Fiscal Policies in an Agent-Based Keynesian Model
Dosi, G., Fagiolo, G., Napoletano, M., & Roventini, A. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.1990209 · Cited by 338
Self-citationAgent based-stock flow consistent macroeconomics: Towards a benchmark model
Caiani, A., Godin, A., Caverzasi, E., Gallegati, M., Kinsella, S., & Stiglitz, J. E. (2016) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2016.06.001 · Cited by 336
The Complexities of Agent-Based Modeling Output Analysis
Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C. (2015) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2897 · Cited by 324
Income distribution, credit and fiscal policies in an agent-based Keynesian model
Dosi, G., Fagiolo, G., Napoletano, M., & Roventini, A. (2013) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2012.11.008 · Cited by 321
Self-citationThiele, J. C., Kurth, W., & Grimm, V. (2014) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2503 · Cited by 306
Validation of Agent-Based Models in Economics and Finance
Fagiolo, G., Guerini, M., Lamperti, F., Moneta, A., & Roventini, A. (2019) · Simulation Foundations, Methods and Applications · doi:10.1007/978-3-319-70766-2_31 · Cited by 300
Self-citationFiscal and monetary policies in complex evolving economies
Dosi, G., Fagiolo, G., Napoletano, M., Roventini, A., & Treibich, T. (2015) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2014.11.014 · Cited by 282
Self-citationBalcan, M.-F., Beygelzimer, A., & Langford, J. C. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143853 · Cited by 280
Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead
Fagiolo, G., & Roventini, A. (2017) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.3280 · Cited by 273
Self-citationWhich Sensitivity Analysis Method Should I Use for My Agent-Based Model?
ten Broeke, G., van Voorn, G., & Ligtenberg, A. (2016) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2857 · Cited by 249
BioWar: scalable agent-based model of bioattacks
Carley, K. M., Fridsma, D. B., Casman, E., Yahja, A., Altman, N., Chen, L.-C., Kaminsky, B., & Nave, D. (2006) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2005.851291 · Cited by 223
Are output growth‐rate distributions fat‐tailed? some evidence from OECD countries
Fagiolo, G., Napoletano, M., & Andrea Roventini, B. (2008) · Journal of Applied Econometrics · doi:10.1002/jae.1003 · Cited by 220
A global optimization heuristic for estimating agent based models
Gilli, M., & Winker, P. (2003) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(02)00214-1 · Cited by 218
Agent-based economic models and econometrics
Chen, S.-H., Chang, C.-L., & Du, Y.-R. (2012) · The Knowledge Engineering Review · doi:10.1017/s0269888912000136 · Cited by 218
Lamperti, F., Dosi, G., Napoletano, M., Roventini, A., & Sapio, A. (2018) · Ecological Economics · doi:10.1016/j.ecolecon.2018.03.023 · Cited by 206
Self-citationBayesian estimation of agent-based models
Grazzini, J., Richiardi, M. G., & Tsionas, E. G. (2017) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2017.01.014 · Cited by 202
Solution and Estimation Methods for DSGE Models
Fernández-Villaverde, J., Ramírez, J. F. R., & Schorfheide, F. (2016) · National Bureau of Economic Research · doi:10.3386/w21862 · Cited by 197
Alternative Approaches to the Empirical Validation of Agent-Based Models
Moss, S. (2007) · RePEc: Research Papers in Economics · Cited by 190
Kucherenko, S. S., Albrecht, D., & Saltelli, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02350 · Cited by 180
Computational and Mathematical Modeling in the Social Sciences
de Marchi, S. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511510588 · Cited by 175
Fiscal and Monetary Policies in Complex Evolving Economies
Dosi, G., Fagiolo, G., Napoletano, M., Roventini, A., & Treibich, T. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2400648 · Cited by 168
Self-citationTipping points in macroeconomic agent-based models
Gualdi, S., Tarzia, M., Zamponi, F., & Bouchaud, J.-P. (2015) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2014.08.003 · Cited by 165
Giovanni Dosi, A., Pereira, M. C., Andrea Roventini, B., & Virgillito, M. E. (2017) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2017.02.005 · Cited by 165
Boosting a weak learning algorithm by majority
Freund, Y. (1990) · Colt Proceedings 1990 · doi:10.1016/b978-1-55860-146-8.50019-9 · Cited by 163
Macroeconomic Policy in DSGE and Agent-Based Models
Fagiolo, G., & Andrea Roventini, B. (2012) · Revue de l'OFCE/La Revue de l'OFCE · doi:10.2139/ssrn.2011717 · Cited by 158
Estimation of ergodic agent-based models by simulated minimum distance
Grazzini, J., & Richiardi, M. G. (2014) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2014.10.006 · Cited by 146
Economic convergence: Policy implications from a heterogeneous agent model
Dawid, H., Harting, P., & Neugart, M. (2014) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2014.04.004 · Cited by 135
Fagiolo, G., & Giovanni Dosi, A. (2003) · Structural Change and Economic Dynamics · doi:10.1016/s0954-349x(03)00022-5 · Cited by 133
Boosting a weak learning algorithm by majority
Freund, Y. (1990) · Conference on Learning Theory · doi:10.5555/92571.92640 · Cited by 130
(2002) · Springer eBooks · doi:10.1007/978-1-4471-0123-9 · Cited by 126
Easy Hyperparameter Search Using OptunityPreprint
Claesen, M., Simm, J., Popovic, D. D., Moreau, Y., & Moor, B. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1114 · Cited by 113
Artificial Life Models in Software
Komosiński, M., & Adamatzky, A. (2009) · Springer eBooks · doi:10.1007/978-1-84882-285-6 · Cited by 107
Individual‐based Modeling and Ecology
Huse, G. (2008) · Fish and Fisheries · doi:10.1111/j.1467-2979.2008.00286.x · Cited by 106
Validating and Calibrating Agent-Based Models: A Case Study
Bianchi, C., Cirillo, P., Gallegati, M., & Vagliasindi, P. A. (2007) · Computational Economics · doi:10.1007/s10614-007-9097-z · Cited by 96
Efficient Sampling and Meta-Modeling for Computational Economic Models
Salle, I. L., & Yıldızoğlu, M. (2013) · Computational Economics · doi:10.1007/s10614-013-9406-7 · Cited by 94
Estimation of an adaptive stock market model with heterogeneous agents
Amilon, H. (2007) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2006.06.007 · Cited by 92
Rock around the Clock : An agent-based model of low- and high-frequency tradingPreprint
Leal, S. J., Napoletano, M., Andrea Roventini, B., & Fagiolo, G. (1991) · HAL · doi:10.1007/s00191-015-0418-4 · Cited by 88
Empirical Validation in Agent-based Models: Introduction to the Special Issue
Fagiolo, G., Birchenhall, C. R., & Windrum, P. (2007) · Computational Economics · doi:10.1007/s10614-007-9109-z · Cited by 75
The Impact of Agent-Based Models in the Social Sciences after 15 Years of Incursions
Squazzoni, F. (2010) · Institutional Research Information System (Università degli Studi di Brescia) · doi:10.1400/148280 · Cited by 72
Thoughts on Massively Scalable Gaussian ProcessesPreprint
Wilson, A. G., Dann, C., & Nickisch, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.01870 · Cited by 71
Reconstructing economics: Agent based models and complexity
Gallegati, M., & Kirman, A. P. (2012) · Complexity Economics · doi:10.7564/12-coec2 · Cited by 71
An information theoretic criterion for empirical validation of simulation models
Lamperti, F. (2017) · Econometrics and Statistics · doi:10.1016/j.ecosta.2017.01.006 · Cited by 69
Self-citationAnalysis of the Emergent Properties: Stationarity and Ergodicity
Grazzini, J. (2012) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.1929 · Cited by 62
Network Calibration and Metamodeling of a Financial Accelerator Agent Based Model
Bargigli, L., Riccetti, L., Russo, A., et al. (2016) · Journal of Economic Interaction and Coordination · doi:10.1007/s11403-018-0217-8 · Cited by 57
A Practical, Accurate, Information Criterion for Nth Order Markov Processes
Barde, S. (2016) · Computational Economics · doi:10.1007/s10614-016-9617-9 · Cited by 52
Direct comparison of agent-based models of herding in financial markets
Barde, S. (2016) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2016.10.005 · Cited by 51
On the robustness of the fat-tailed distribution of firm growth rates: a global sensitivity analysis
Dosi, G., Pereira, M. C., & Virgillito, M. E. (2017) · Journal of Economic Interaction and Coordination · doi:10.1007/s11403-017-0193-4 · Cited by 46
Agent-Based Macroeconomic Modeling and Policy Analysis: The Eurace@Unibi Model
Dawid, H., Gemkow, S., Harting, P., van der Hoog, S., & Neugart, M. (2014) · PUB – Publications at Bielefeld University (Bielefeld University) · doi:10.4119/unibi/2915476 · Cited by 42
Organizational performance in hierarchies and communities of practice
Dupouët, O., & Yıldızoğlu, M. (2006) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2004.07.011 · Cited by 29
Estimation of Financial Agent-Based Models with Simulated Maximum LikelihoodPreprint
Kukačka, J., & Baruník, J. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2783663 · Cited by 29
The Effects of Labour Market Reforms upon Unemployment and Income Inequalities: An Agent Based ModelPreprint
Giovanni Dosi, A., Pereira, M., Andrea Roventini, B., & Virgillito, M. E. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2804220 · Cited by 22
Lamperti, F., & Mattei, C. E. (2018) · Journal of Evolutionary Economics · doi:10.1007/s00191-018-0551-y · Cited by 18
Self-citationDeep Learning in Agent-Based Models: A ProspectusPreprint
van der Hoog, S. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2711216 · Cited by 18
Simulating Nursing Unit Performance With OrgAhead
Effken, J. A., Carley, K. M., Lee, J., Brewer, B. B., & Verran, J. A. (2012) · CIN Computers Informatics Nursing · doi:10.1097/nxn.0b013e318261f1bb · Cited by 15
Collet, T., & Pietquin, O. (2015) · Computational Intelligence and Neuroscience · doi:10.1155/2015/973696 · Cited by 7
Towards agent-based integrated assessment models: examples, challenges, and future developmentsPreprint
Lamperti, F., Mandel, A., Napoletano, M., Sapio, A., Andrea Roventini, B., Bálint, T., & Khorenzhenko, I. (2018) · RePEc: Research Papers in Economics · doi:10.1016/j.jebo.2016.12.017 · Cited by 1
Self-citation(1996)
Cited by
Three green financial policies to address climate risks
Lamperti, F., Bosetti, V., Roventini, A., Tavoni, M., & Treibich, T. (2021) · Journal of Financial Stability · doi:10.1016/j.jfs.2021.100875 · Cited by 301
Self-citationValidation of Agent-Based Models in Economics and Finance
Fagiolo, G., Guerini, M., Lamperti, F., Moneta, A., & Roventini, A. (2019) · Simulation Foundations, Methods and Applications · doi:10.1007/978-3-319-70766-2_31 · Cited by 300
Self-citation
Metadata
- Authors
- Lamperti, F., Roventini, A., & Sani, A.
- Venue
- Journal of Economic Dynamics and Control
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 90
- Pages
- 366-389
- ISSN
- 0165-1889 (ISSN-L)
- OpenAlex
- W2604765399
- Cited by
- 261
- References
- 107
- FWCI
- 18.59
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.