Some Comparisons Among Quadratic, Spherical, and Logarithmic Scoring Rules
Abstract
Strictly proper scoring rules continue to play an important role in probability assessment. Although many such rules have been developed, relatively little guidance exists as to which rule is the most appropriate. In this paper, we discuss two important properties of quadratic, spherical, and logarithmic scoring rules. From an ex post perspective, we compare their rank order properties and conclude that both quadratic and spherical scoring perform poorly in this regard, relative to logarithmic. Second, from an ex ante perspective, we demonstrate that in many situations, logarithmic scoring is the method least affected by a nonlinear utility function. These results suggest that logarithmic scoring is superior when rank order results are important and/or when the assessor has a nonlinear utility function. In addition to these results, and perhaps more important, we demonstrate that nonlinear utility induces relatively little deviation from the optimal assessment under an assumption of risk neutrality. These results provide both comfort and guidance to those who would like to use scoring rules as part of the assessment process.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Log score is a proper scoring rule that assigns greater values to models that assign high confidence to correct predictions and low confidence to incorrect predictions.
“Log score is the logarithmic scoring rule for evaluating prediction models, this is a proper scoring rule that assigns greater values to models that assign high confidence to correct predictions, and low confidence to incorrect predictions (Bickel, 2007).”
From Sysrev: A FAIR Platform for Data Curation and Systematic Evidence Review (2021) · cites this paper as Bickel, 2007
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139 · see citing works
Elicitation of Personal Probabilities and Expectations
Savage, L. J. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482346 · Cited by 1,326 · see citing works
Conservatism in a simple probability inference task.
Phillips, L. D., & Edwards, W. (1966) · Journal of Experimental Psychology · doi:10.1037/h0023653 · Cited by 878 · see citing works
Axiomatic Characterization of the Quadratic Scoring Rule
Selten, R. (1998) · Experimental Economics · doi:10.1023/a:1009957816843 · Cited by 338 · see citing works
Scoring Rules and the Evaluation of Probability Assessors
Winkler, R. L. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501037 · Cited by 271 · see citing works
Admissible Probability Measurement Procedures
Shuford, E. H., Albert, A., & Massengill, H. E. (1966) · Psychometrika · doi:10.1007/bf02289503 · Cited by 265 · see citing works
Scoring rules and the evaluation of probabilities
Winkler, R., Muñoz, J., Cervera, J. L., Bernardo, J., Blattenberger, G., Kadane, J., Lindley, D., Murphy, A. H., Oliver, R. M., & Insua, D. R. (1996) · Test · doi:10.1007/bf02562681 · Cited by 238 · see citing works
Scoring rules in probability assessment and evaluation
Murphy, A. H., & Winkler, R. L. (1970) · Acta Psychologica · doi:10.1016/0001-6918(70)90023-5 · Cited by 154 · see citing works
Turbulence and Wind Speed Spectra within a Japanese Larch Plantation
Allen, L. H. (1968) · Journal of applied meteorology · doi:10.1175/1520-0450(1968)007 · Cited by 149 · see citing works
Probabilistic Prediction: Some Experimental Results
Winkler, R. L. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482329 · Cited by 137 · see citing works
Climatic Change as a Mathematical Problem
Lorenz, E. N. (1970) · Journal of applied meteorology · doi:10.1175/1520-0450(1970)009 · Cited by 135 · see citing works
Assessment and evaluation of subjective probability distributions
von Holstein, S., & Carl-Axel, S. (1970) · Medical Entomology and Zoology · Cited by 111 · see citing works
Approximating Risk Aversion in Decision Analysis Applications
Kirkwood, C. W. (2004) · Decision Analysis · doi:10.1287/deca.1030.0007 · Cited by 67 · see citing works
Effective Scoring Rules for Probabilistic Forecasts
Friedman, D. (1983) · Management Science · doi:10.1287/mnsc.29.4.447 · Cited by 65 · see citing works
Probability Elicitation, Scoring Rules, and Competition Among Forecasters
Lichtendahl, K. C., & Winkler, R. (2007) · Management Sciences · doi:10.1287/mnsc.1070.0729 · Cited by 64 · see citing works
Approximating Risk Aversion in Decision Analysis Applications
Kirkwood, C. W. (2003) · Decision Analysis · doi:10.1287/deca.1.1.55.17845 · Cited by 59 · see citing works
Should Scoring Rules be “Effective”?
Nau, R. F. (1985) · Management Science · doi:10.1287/mnsc.31.5.527 · Cited by 42 · see citing works
Forecasters and Probability Forecasts: Some Current Problems
Murphy, A. H., & Winkler, R. L. (1971) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1971)052 · Cited by 40 · see citing works
Belief states and the uses of evidence
Roby, T. B. (1965) · Systems Research and Behavioral Science · doi:10.1002/bs.3830100304 · Cited by 24 · see citing works
Psychological effects of proper scoring rules
Jensen, F. A., & Peterson, C. R. (1973) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(73)90054-8 · Cited by 23 · see citing works
BELIEF STATES: A PRELIMINARY EMPIRICAL STUDY
Roby, T. B. (1964) · doi:10.21236/ad0600441 · Cited by 15 · see citing works
MEASUREMENT OF SUBJECTIVE PROBABILITY DISTRIBUTIONS. TECHN DOCUM REP ESD-TDR-63-407.
Toda, M. (1963) · PubMed · Cited by 3 · see citing works
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2 · cited 1×
“For example, the Brier score (Brier 1950), which is popular in meteorological contexts, is Bi r = 1−Qi r .”
Flagged first, then by citations
Cited by
Sorted by citations
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Taieb, S. B., Bergmeir, C., Bessa, R., Bijak, J., Boylan, J., et al. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 630 · see citing works
Identifying Expertise to Extract the Wisdom of Crowds
Budescu, D. V., & Chen, E. (2014) · Management Science · doi:10.1287/mnsc.2014.1909 · Cited by 365 · see citing works
Bayesian dark knowledgePreprint
Balan, A. K., Rathod, V., Murphy, K. P., & Welling, M. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1506.04416 · Cited by 276 · see citing works
Maros, M., Capper, D., Jones, D. T. W., Hovestadt, V., von Deimling, A., Pfister, S., Benner, A., Zucknick, M., & Sill, M. (2020) · Nature Protocols · doi:10.1038/s41596-019-0251-6 · Cited by 114 · see citing works
Choosing a Strictly Proper Scoring Rule
Merkle, E., & Steyvers, M. (2013) · Decision Analytics · doi:10.1287/deca.2013.0280 · Cited by 112 · see citing works
Sysrev: A FAIR Platform for Data Curation and Systematic Evidence Review
Bozada, T., Borden, J., Workman, J., Del Cid, M., Malinowski, J., & Luechtefeld, T. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.685298 · Cited by 118 · see citing works
Scoring Rules, Generalized Entropy, and Utility Maximization
Jose, V. R., Nau, R., & Winkler, R. (2008) · Operational Research · doi:10.1287/opre.1070.0498 · Cited by 104 · see citing works
Teixeira, P. L., Wei, W., Cronin, R. M., Mo, H., VanHouten, J. P., Carroll, R. J., LaRose, E. R., Bastarache, L. A., Rosenbloom, S. T., Edwards, T. L., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw071 · Cited by 91 · see citing works
A Kullback-Leibler View of Linear and Log-Linear Pools
Abbas, A. E. (2009) · Decision Analysis · doi:10.1287/deca.1080.0133 · Cited by 90 · see citing works
THE USE OF ANNUAL MILEAGE AS A RATING VARIABLE
Lemaire, J., Park, S. C., & Wang, K. C. (2016) · ASTIN Bulletin · doi:10.1017/asb.2015.25 · Cited by 74 · see citing works
The Fundamental Nature of the Log Loss Function
Vovk, V. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-23534-9_20 · Cited by 71 · see citing works
Probability Forecasts and Their Combination: A Research Perspective
Winkler, R. L., Grushka‐Cockayne, Y., Lichtendahl, K. C., & Jose, V. R. R. (2019) · Decision Analysis · doi:10.1287/deca.2019.0391 · Cited by 71 · see citing works
A comparison between ensemble and deterministic hydrological forecasts in an operational context
Boucher, M., Anctil, F., Perreault, L., & Tremblay, D. (2011) · Advances in geosciences · doi:10.5194/adgeo-29-85-2011 · Cited by 55 · see citing works
Staffel, J. (2019) · doi:10.1093/oso/9780198833710.001.0001 · Cited by 48 · see citing works
An Overview of Applications of Proper Scoring Rules
Carvalho, A. (2016) · Decision Analytics · doi:10.1287/deca.2016.0337 · Cited by 55 · see citing works
How Do Beliefs Simplify Reasoning?
Staffel, J. (2019) · doi:10.1093/oso/9780198833710.003.0009 · Cited by 45 · see citing works
Probabilistic estimation of link travel times in dynamic road networks
Asghari, M., Emrich, T., Demiryurek, U., & Shahabi, C. (2015) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/2820783.2820836 · Cited by 39 · see citing works
Tailored Scoring Rules for Probabilities
Johnstone, D. J., Jose, V. R. R., & Winkler, R. L. (2011) · Decision Analysis · doi:10.1287/deca.1110.0216 · Cited by 36 · see citing works
Managing Project Failure Risk Through Contingent Contracts in Procurement Auctions
Chen, J., Xu, L., & Whinston, A. B. (2009) · Decision Analysis · doi:10.1287/deca.1090.0155 · Cited by 33 · see citing works
Financial risk assessment in shipping: a holistic machine learning based methodology
Clintworth, M., Lyridis, D. V., & Boulougouris, E. K. (2021) · Maritime Economics & Logistics · doi:10.1057/s41278-020-00183-2 · Cited by 33 · see citing works
Luo, Z., Xu, J. H., Zhao, P., Yang, D., Xu, L., & Luo, J. L. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.09.026 · Cited by 32 · see citing works
Scoring Rules and Decision Analysis Education
Bickel, J. E. (2010) · Decision Analysis · doi:10.1287/deca.1100.0184 · Cited by 31 · see citing works
Guidelines for effective evaluation and comparison of wildland fire occurrence prediction models
Phelps, N., & Woolford, D. G. (2021) · International Journal of Wildland Fire · doi:10.1071/wf20134 · Cited by 31 · see citing works
Contrasting probabilistic scoring rules
Machete, R. L. (2013) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2013.05.012 · Cited by 25 · see citing works
Bregman Divergence Bounds and Universality Properties of the Logarithmic Loss
Painsky, A., & Wornell, G. W. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2958705 · Cited by 25 · see citing works
Behrangi, A., Nguyen, H., & Granger, S. L. (2015) · Journal of Applied Meteorology and Climatology · doi:10.1175/jamc-d-14-0162.1 · Cited by 24 · see citing works
Proper scoring rules with arbitrary value functions
Fang, F., Stinchcombe, M. B., & Whinston, A. B. (2010) · Journal of Mathematical Economics · doi:10.1016/j.jmateco.2010.09.002 · Cited by 22 · see citing works
Fallis, D., & Lewis, P. J. (2015) · Australasian Journal of Philosophy · doi:10.1080/00048402.2015.1123741 · Cited by 20 · see citing works
Wu, Y., McLeod, C., Blyth, C. C., Bowen, A. C., Martin, A. C., Nicholson, A. E., Mascaro, S., & Snelling, T. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101895 · Cited by 18 · see citing works
VIBE: A Design Space for VIsual Belief Elicitation in Data Journalism
Mahajan, S., Chen, M., Karduni, A., Kim, Y., & Wall, E. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14556 · Cited by 18 · see citing works
The construction of causal networks to estimate coral bleaching intensity
Krug, L. A., Gherardi, D. F. M., Stech, J., Leão, Z. M. A. N., de Kikuchi, R. K. P., Junior, E. R. H., & Suggett, D. J. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.01.003 · Cited by 18 · see citing works
An IRT forecasting model: linking proper scoring rules to item response theory
Bo, Y. E., Budescu, D. V., Lewis, C., Tetlock, P. E., & Mellers, B. A. (2017) · Judgment and Decision Making · doi:10.1017/s1930297500005647 · Cited by 17 · see citing works
Evaluating the discrimination ability of proper multi-variate scoring rules
Alexander, C., Coulon, M., Han, Y., & Meng, X. (2022) · Annals of Operations Research · doi:10.1007/s10479-022-04611-9 · Cited by 16 · see citing works
A score-driven model of short-term demand forecasting for retail distribution centers
Hoeltgebaum, H., Borenstein, D., Fernandes, C. A. C., & Veiga, Á. (2021) · Journal of Retailing · doi:10.1016/j.jretai.2021.05.003 · Cited by 17 · see citing works
On the assessment of reliability in probabilistic hydrometeorological event forecasting
DeChant, C. M., & Moradkhani, H. (2015) · Water Resources Research · doi:10.1002/2014wr016617 · Cited by 16 · see citing works
Lisboa, D. S., de Kikuchi, R. K. P., & Leão, Z. M. A. N. (2018) · Journal of Geophysical Research Oceans · doi:10.1002/2017jc012824 · Cited by 16 · see citing works
Verification in the presence of observation errors: Bayesian point of view
Duc, L., & Saito, K. (2018) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3275 · Cited by 15 · see citing works
Fiat, A., Karlin, A. R., Koutsoupias, E., & Vidali, A. (2013) · Conference on Innovations in Theoretical Computer Science · doi:10.1145/2422436.2422463 · Cited by 14 · see citing works
Detotto, C., Paolini, D., & Tena, J. D. D. (2017) · Journal of the Operational Research Society · doi:10.1057/s41274-017-0215-6 · Cited by 13 · see citing works
Afflerbach, P., van Dun, C., Gimpel, H., Parak, D., & Seyfried, J. (2020) · Business & Information Systems Engineering · doi:10.1007/s12599-020-00664-x · Cited by 13 · see citing works
Roser, A. V., Enterkine, J., Requena-Mullor, J., Glenn, N., Boehm, A., de Graaff, M., Clark, P., Pierson, F., & Caughlin, T. (2022) · Ecosphere · doi:10.1002/ecs2.4330 · Cited by 12 · see citing works
Fernández, A., García, S., Galar, M., Prati, R. C., Krawczyk, B., & Herrera, F. (2018) · Springer eBooks · doi:10.1007/978-3-319-98074-4_3 · Cited by 12 · see citing works
A hierarchical Bayesian approach to negative binomial regression
Fu, S. (2015) · Methods and Applications of Analysis · doi:10.4310/maa.2015.v22.n4.a4 · Cited by 11 · see citing works
Decreasing Marginal Value of Information Under Symmetric Loss
Bikhchandani, S., & Mamer, J. W. (2013) · Decision Analysis · doi:10.1287/deca.2013.0274 · Cited by 11 · see citing works
Hierarchical Bayesian LASSO for a negative binomial regression
Fu, S. (2015) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2015.1106541 · Cited by 10 · see citing works
Azizi, S., & Quinn, A. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2608662 · Cited by 10 · see citing works
Douven, I. (2018) · Synthese · doi:10.1007/s11229-018-1867-8 · Cited by 9 · see citing works
Performance-Weighted Aggregation: Ferreting Out Wisdom Within the Crowd
Collins, R. N., Mandel, D. R., & Budescu, D. V. (2023) · International series in management science/operations research/International series in operations research & management science · doi:10.1007/978-3-031-30085-1_7 · Cited by 16 · see citing works
LogitBoost autoregressive networks
Goessling, M. (2017) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2017.03.010 · Cited by 8 · see citing works
A Mechanism for Eliciting Second-Order Beliefs and the Inclination to Choose
Karni, E. (2018) · American Economic Journal Microeconomics · doi:10.1257/mic.20160345 · Cited by 8 · see citing works
On the importance of the probabilistic model in identifying the most decisive games in a tournament
Corona, F., Tena, J. D. D., & Wiper, M. P. (2017) · Journal of Quantitative Analysis in Sports · doi:10.1515/jqas-2016-0013 · Cited by 7 · see citing works
A Probability Scoring Rule for Simultaneous Events
Grant, A. R., Johnstone, D. J., & Kwon, O. K. (2019) · Decision Analysis · doi:10.1287/deca.2019.0393 · Cited by 7 · see citing works
Plastic Surgeon Expertise in Predicting Breast Reconstruction Outcomes for Patient Decision Analysis
Sun, C. S., Reece, G., Crosby, M., Fingeret, M., Skoracki, R., Villa, M., Hanasono, M., Baumann, D., Chang, D., Cantor, S., et al. (2013) · Plastic and Reconstructive Surgery, Global Open · doi:10.1097/gox.0000000000000010 · Cited by 8 · see citing works
Integrating individual and social learning: accuracy and evolutionary viability
Douven, I., & Schurz, G. (2022) · Computational and Mathematical Organization Theory · doi:10.1007/s10588-022-09372-1 · Cited by 6 · see citing works
Modeling Fixed Odds Betting For Future Event Prediction1
Chen, W., Li, X., & Zeng, D. D. (2017) · MIS Quarterly · doi:10.25300/misq/2017/41.2.14 · Cited by 6 · see citing works
Bayesian Measures of Confirmation from Scoring Rules
van Enk, S. J. (2014) · Philosophy of Science · doi:10.1086/674205 · Cited by 6 · see citing works
Probability Forecasts and Their Combination: A Research PerspectivePreprint
Winkler, R. L., Grushka‐Cockayne, Y., Lichtendahl, K. C., & Jose, V. R. R. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3258627 · Cited by 7 · see citing works
Winkler, R. L., & Jose, V. R. R. (2010) · Wiley Encyclopedia of Operations Research and Management Science · doi:10.1002/9780470400531.eorms0749 · Cited by 6 · see citing works
Chung, K.-M., Lui, E., & Pass, R. (2013) · Conference on Innovations in Theoretical Computer Science · doi:10.1145/2422436.2422443 · Cited by 5 · see citing works
Gender, Confidence, and the Mismeasure of Intelligence, Competitiveness, and Literacy
Harrison, G. W., Ross, D., & Swarthout, J. T. (2025) · Journal of Political Economy · doi:10.1086/738345 · Cited by 4 · see citing works
Selection versus averaging of logistic credit risk models
Hayden, E., Stomper, A., & Westerkamp, A. (2014) · The Journal of Risk · doi:10.21314/jor.2014.285 · Cited by 4 · see citing works
Measuring and adjusting for overconfidence
Schanbacher, P. (2013) · Decisions in Economics and Finance · doi:10.1007/s10203-013-0153-y
Inclusive Flavour Tagging Algorithm
Likhomanenko, T., Деркач, Д., & Rogozhnikov, A. (2016) · Journal of Physics Conference Series · doi:10.1088/1742-6596/762/1/012045 · Cited by 5 · see citing works
Keller, L. R., Abbas, A. E., Baucells, M., Bier, V. M., Budescu, D. V., Butler, J. C., Delquié, P., Merrick, J. R. W., Salo, A. A., & Wu, G. Y. (2010) · Decision Analysis · doi:10.1287/deca.1100.0195 · Cited by 4 · see citing works
Predicting Lockean from gradational accuracy
Douven, I., & Kriegeskorte, N. (2026) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2026.109636 · Cited by 3 · see citing works
Assessing the quality of bioforensic signatures
Sego, L. H., Holmes, A. E., Gosink, L., Webb‐Robertson, B., Kreuzer, H. W., Anderson, R. M., Brothers, A. J., Corley, C. D., & Tardiff, M. R. (2013) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2013.6578856 · Cited by 3 · see citing works
The Wisdom of Strategic Voting
Han, Q., Schoenebeck, G. R., Tao, B., & Xia, L. (2023) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3580507.3597681 · Cited by 3 · see citing works
Weighted Scoring Rules and Convex Risk Measures
Smith, Z. J., & Bickel, J. E. (2022) · Operations Research · doi:10.1287/opre.2021.2190 · Cited by 3 · see citing works
How to Conclude a Suspended Sports League?
Hassanzadeh, A., Hosseini, M., & Turner, J. G. (2024) · Manufacturing & Service Operations Management · doi:10.1287/msom.2022.0558 · Cited by 3 · see citing works
Proper scoring rules with general preferences: A dual characterization of optimal reports
Chambers, C. P., Healy, P. J., & Lambert, N. S. (2019) · Games and Economic Behavior · doi:10.1016/j.geb.2019.07.012 · Cited by 3 · see citing works
Eliciting high quality feedback from crowdsourced tree networks using continuous scoring rules
Ray, R., Vallam, R. D., & Narahari, Y. (2013) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2484920.2484967 · Cited by 3 · see citing works
Obtaining Accurate Probabilities Using Classifier Calibration.
Naeini, M. P. (2016) · ProQuest LLC eBooks · Cited by 3 · see citing works
Proper Scoring Rules and Risk AversionPreprint
Peysakhovich, A., & Plagborg‐Møller, M. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2019078 · Cited by 3 · see citing works
Keller, L. R., Abbas, A. E., Bickel, J. E., Bier, V. M., Budescu, D. V., Butler, J. C., Diecidue, E., Dillon, R. L., Hämäläinen, R. P., Lichtendahl, K. C., et al. (2012) · Decision Analysis · doi:10.1287/deca.1120.0255 · Cited by 3 · see citing works
Keller, L. R., Abbas, A. E., Bickel, J. E., Bier, V. M., Budescu, D. V., Butler, J. C., Delquié, P., Lichtendahl, K. C., Merrick, J. R. W., Salo, A. A., et al. (2011) · Decision Analysis · doi:10.1287/deca.1110.0222 · Cited by 3 · see citing works
Keller, L. R. (2009) · Decision Analysis · doi:10.1287/deca.1090.0137 · Cited by 3 · see citing works
Epistemic Consequentialism, Veritism, and Scoring Rules
Daoust, M., & Côté-Bouchard, C. (2021) · Erkenntnis · doi:10.1007/s10670-021-00426-5 · Cited by 2 · see citing works
Yeh, C.-K., Rice, G., & Dubin, J. A. (2021) · The American Statistician · doi:10.1080/00031305.2021.1967781 · Cited by 2 · see citing works
Assessing performance and tradeoffs of bioforensic signature systems
Watkins, D. M., Sego, L. H., Holmes, A. E., Webb‐Robertson, B., Whiten, A. M., Wunschen, D. S., Kreuzer, H. W., Corley, C. D., & Tardiff, M. F. (2013) · IEEE Conference on Technologies for Homeland Security (HST) · doi:10.1109/ths.2013.6699019 · Cited by 2 · see citing works
Is the log score in line with forecasters' preferences?
Schanbacher, P. (2013) · International Journal of Applied Decision Sciences · doi:10.1504/ijads.2013.056882 · Cited by 2 · see citing works
Testing the Conditions for Acquiring Intuitive Expertise in Judgment
Deloatch, R., Marmarchi, A., & Kirlik, A. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571064 · Cited by 3 · see citing works
Will Coral Reefs in the North Atlantic Ocean Bleach During the Next Season? A Probabilistic Answer
Lisboa, D. S., & de Kikuchi, R. K. P. (2020) · Geophysical Research Letters · doi:10.1029/2019gl086442 · Cited by 2 · see citing works
Securities Based Decision Markets
Wang, W., & Pfeiffer, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-94662-3_6 · Cited by 1 · see citing works
Verification of Medium- to Long-Range Hydrological Forecasts
Perreault, L., Gaudet, J. M., Delorme, L., & Chatelain, S. (2019) · Springer eBooks · doi:10.1007/978-3-642-39925-1_6 · Cited by 2 · see citing works
Sarin, R. K. (2013) · Decision Analysis · doi:10.1287/deca.2013.0284 · Cited by 2 · see citing works
Sarin, R. K., & Keller, L. R. (2013) · Decision Analysis · doi:10.1287/deca.1120.0262 · Cited by 2 · see citing works
Modelling Fixed Odds Betting for Future Event PredictionPreprint
Chen, W., Li, X., & Zeng, D. D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2552447 · Cited by 2 · see citing works
The Future Prospects of Energy Technologies: Insights from Expert ElicitationsPreprint
Verdolini, E., Anadón, L., Baker, E., Bosetti, V., & Reis, L. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2815367 · Cited by 5 · see citing works
Staffel, J. (2019) · doi:10.1093/oso/9780198833710.003.0003 · Cited by 2 · see citing works
Keller, L. R. (2009) · Decision Analysis · doi:10.1287/deca.1090.0156 · Cited by 2 · see citing works
Starns, J. J., Cohen, A. L., & Rotello, C. M. (2021) · doi:10.31234/osf.io/az9xf · Cited by 2 · see citing works
Keller, L. R., Baucells, M., Butler, J. C., Delquié, P., Merrick, J. R. W., Parnell, G. S., & Salo, A. A. (2009) · Decision Analysis · doi:10.1287/deca.1090.0165 · Cited by 2 · see citing works
Keller, L. R. (2008) · Decision Analysis · doi:10.1287/deca.1080.0122 · Cited by 2 · see citing works
Pretis, F. D., Glielmo, A., & Landes, J. (2025) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2025.109388 · Cited by 1 · see citing works
A note on proper scoring rules and risk aversion
Peysakhovich, A., & Plagborg‐Møller, M. (2012) · Economics Letters · doi:10.1016/j.econlet.2012.05.030 · Cited by 1 · see citing works
EPISTEMIC VALUE AND EPISTEMIC COMPROMISE: A REPLY TO MOSS
Konigsberg, A. (2013) · Episteme · doi:10.1017/epi.2013.4 · Cited by 1 · see citing works
Verification of Medium- to Long-Range Hydrological Forecasts
Perreault, L., Gaudet, J. M., Delorme, L., & Chatelain, S. (2016) · Springer eBooks · doi:10.1007/978-3-642-40457-3_6-1 · Cited by 1 · see citing works
Forecaster evaluation as model comparison.
Merkle, E. C., & Flores, R. D. (2024) · Decision · doi:10.1037/dec0000230 · Cited by 1 · see citing works
Incentive compatibility in prediction markets: Costly actions and external incentives
Chen, D., Dimitrov, S., & He, Q. (2018) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2018.07.005 · Cited by 1 · see citing works
A Score for Probability Forecasts of Binary Events Based on the User’s Cost–Loss Relations
Ivarsson, K.-I. (2019) · Weather and Forecasting · doi:10.1175/waf-d-19-0013.1 · Cited by 1 · see citing works
- Authors
- Bickel, J. E.
- Affiliations
- Texas A&M University (US)
- Corresponding author
- Bickel, J. Eric
- ORCID
- Bickel, J. Eric
- Venue
- Decision Analysis
- Published
- 2007-06
- Volume
- 4
- Issue
- 2
- Pages
- 49-65
- ISSN
- 1545-8490 (ISSN-L)
- 1545-8504
- OpenAlex
- W2161873643
- Cited by
- 142
- References
- 24
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 3 · 2023: 3 · 2022: 8 · 2021: 13 · 2020: 6 · 2019: 21 · 2018: 9 · 2017: 8 · 2016: 7 · 2015: 14 · 2014: 4 · 2013: 19 · 2012: 5
- FWCI
- 5.70
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Decision Analysis (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Keywords
- strictly proper scoring rules, logarithmic scoring rule, quadratic scoring rule, probability assessment, nonlinear utility functions, risk neutrality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
