An Experiment on Prediction Markets in Science
Open accessAbstract
Prediction markets are powerful forecasting tools. They have the potential to aggregate private information, to generate and disseminate a consensus among the market participants, and to provide incentives for information acquisition. These market functionalities can be very valuable for scientific research. Here, we report an experiment that examines the compatibility of prediction markets with the current practice of scientific publication. We investigated three settings. In the first setting, different pieces of information were disclosed to the public during the experiment. In the second setting, participants received private information. In the third setting, each piece of information was private at first, but was subsequently disclosed to the public. An automated, subsidizing market maker provided additional incentives for trading and mitigated liquidity problems. We find that the third setting combines the advantages of the first and second settings. Market performance was as good as in the setting with public information, and better than in the setting with private information. In contrast to the first setting, participants could benefit from information advantages. Thus the publication of information does not detract from the functionality of prediction markets. We conclude that for integrating prediction markets into the practice of scientific research it is of advantage to use subsidizing market makers, and to keep markets aligned with current publication practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prediction markets can aggregate private information on reproducibility and can generate and disseminate a consensus among market participants.
“Prediction markets can aggregate private information on reproducibility, and can generate and disseminate a consensus among market participants.”
From Evaluating replicability of laboratory experiments in economics (2016) · cites this paper as Almenberg et al. [58]
The Introduction says prediction markets can aggregate private information and communicate a consensus estimate on scientific questions to market participants. But the paper studies a stylized hypothesis-testing problem, not predictions about whether research findings will reproduce; the reproducibility-specific part of the claim is unsupported. (LLM, conf=0.88)
Experimental — accuracy not yet measuredAlmenberg et al. (58) conducted a lab-based test.
“Almenberg et al. ( 58 ) conducted a lab-based test.”
From Evaluating replicability of laboratory experiments in economics (2016) · cites this paper as Almenberg et al. [58]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193 · see citing works
EFFICIENT CAPITAL MARKETS: A REVIEW OF THEORY AND EMPIRICAL WORK*
Malkiel, B. G., & Fama, E. F. (1970) · The Journal of Finance · doi:10.1111/j.1540-6261.1970.tb00518.x · Cited by 15,733 · see citing works
Efficient capital markets: A review of theory and empirical work
Fama, E. F. (1970) · The Journal of Finance · doi:10.2307/2325486 · Cited by 13,635 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
Rational Expectations and the Theory of Price Movements
Muth, J. F. (1961) · Econometrica · doi:10.2307/1909635 · Cited by 5,678 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Gode, D. K., & Sunder, S. (1993) · Journal of Political Economy · doi:10.1086/261868 · Cited by 1,695 · see citing works
Bubbles, Crashes, and Endogenous Expectations in Experimental Spot Asset Markets
Smith, V. L. S., Suchanek, G. L., & Williams, A. W. (1988) · Econometrica · doi:10.2307/1911361 · Cited by 1,343 · see citing works
Bubbles, Crashes, and Endogenous Expectations in Experimental Spot Asset Markets
Suchanek, G. L., & Williams, A. W. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511528354.022 · Cited by 940 · see citing works
Wolfers, J., & Zitzewitz, E. (2004) · Journal of Economic Perspectives · doi:10.1257/0895330041371321 · Cited by 889 · see citing works
Allen, F., & Gale, D. M. (1992) · Review of Financial Studies · doi:10.1093/rfs/5.3.503 · Cited by 652 · see citing works
Rational Expectations and the Aggregation of Diverse Information in Laboratory Security Markets
Plott, C. R., & Sunder, S. (1988) · Econometrica · doi:10.2307/1911360 · Cited by 608 · see citing works
Plott, C. R., & Sunder, S. (1982) · Journal of Political Economy · doi:10.1086/261084 · Cited by 565 · see citing works
Combinatorial Information Market Design
Hanson, R. D. (2003) · Inf. Syst. Frontiers · doi:10.1023/a:1022058209073 · Cited by 554 · see citing works
(2008) · doi:10.1126/science.1157679 · Cited by 406 · see citing works
LOGARITHMIC MARKETS CORING RULES FOR MODULAR COMBINATORIAL INFORMATION AGGREGATION
Hanson, R. D. (2012) · The Journal of Prediction Markets · doi:10.5750/jpm.v1i1.417 · Cited by 381 · see citing works
Interpreting the predictions of prediction markets
Manski, C. F. (2006) · Economics Letters · doi:10.1016/j.econlet.2006.01.004 · Cited by 330 · see citing works
Do Biases in Probability Judgment Matter in Markets? Experimental Evidence
Camerer, C. F. (1987) · American Economic Review · Cited by 316 · see citing works
Plott, C. R., & Chen, K. (2002) · CaltechAUTHORS (California Institute of Technology) · doi:10.7907/n9y0a-a5y79 · Cited by 299 · see citing works
Asset Valuation in an Experimental Market
Forsythe, R. S., Palfrey, T. R., & Plott, C. R. (1982) · Econometrica · doi:10.2307/1912600 · Cited by 270 · see citing works
Information Aggregation in an Experimental Market
Forsythe, R. S., & Lundholm, R. J. (1990) · Econometrica · doi:10.2307/2938206 · Cited by 224 · see citing works
Interpreting Prediction Market Prices as ProbabilitiesPreprint
Wolfers, J. J., & Zitzewitz, E. W. (2006) · SSRN Electronic Journal · doi:10.2139/ssrn.898597 · Cited by 202 · see citing works
Expectations and bubbles in asset pricing experiments
Hommes, C., Joep Sonnemans, A., Jan Tuinstra, B., & van de Velden, H. (2007) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2007.06.006 · Cited by 196 · see citing works
Information aggregation and manipulation in an experimental market
Hanson, R., Oprea, R. D., & Porter, D. (2006) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2004.09.011 · Cited by 186 · see citing works
Information Mirages in Experimental Asset Markets
Camerer, C. F., & Weigelt, K. (1991) · The Journal of Business · doi:10.1086/296548 · Cited by 157 · see citing works
Interpreting Prediction Market Prices as Probabilities
Wolfers, J. J., & Zitzewitz, E. W. (2006) · National Bureau of Economic Research · doi:10.3386/w12200 · Cited by 149 · see citing works
Can Asset Markets Be Manipulated? A Field Experiment With Racetrack Betting
Camerer, C. F. (1998) · Journal of Political Economy · doi:10.1086/250018 · Cited by 141 · see citing works
Parimutuel betting markets as information aggregation devices: experimental results
Plott, C. R., Wit, J., & Yang, W. C. (2003) · Economic Theory · doi:10.1007/s00199-002-0306-7 · Cited by 113 · see citing works
The Effect of Sequential Information Arrival on Asset Prices: An Experimental Study
Copeland, T. E., & Friedman, D. (1987) · The Journal of Finance · doi:10.1111/j.1540-6261.1987.tb04585.x · Cited by 107 · see citing works
Could gambling save science? Encouraging an honest consensus
Hanson, R. (1995) · Social Epistemology · doi:10.1080/02691729508578768 · Cited by 82 · see citing works
Do Asset Market Prices Reflect Traders' Judgment Biases?
Ganguly, A. R., Kagel, J. H., & Moser, D. V. (2000) · Journal of Risk and Uncertainty · doi:10.1023/a:1007848013750 · Cited by 65 · see citing works
Promoting Intellectual Discovery: Patents Versus Markets
Meloso, D. C., Čopič, J., & Bossaerts, P. L. (2009) · Science · doi:10.1126/science.1158624 · Cited by 64 · see citing works
Professional Traders as Intuitive Bayesians
Anderson, M. J., & Sunder, S. (1995) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1995.1099 · Cited by 61 · see citing works
Bayesian equilibrium in double auctions populated by biased heuristic traders
Jamal, K., & Sunder, S. (1996) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(96)00892-x · Cited by 50 · see citing works
Bluffing and Strategic Reticence in Prediction Markets
Chen, Y., Reeves, D. M., Pennock, D. M., Hanson, R., Fortnow, L., & Gonen, R. (2007) · Springer eBooks · doi:10.1007/978-3-540-77105-0_10 · Cited by 40 · see citing works
Forecasting dynamics and convergence to market fundamentals
Peterson, S. P. (1993) · Journal of Economic Behavior & Organization · doi:10.1016/0167-2681(93)90002-7 · Cited by 30 · see citing works
Price manipulation in an experimental asset market
Veiga, H., & Vorsatz, M. (2008) · European Economic Review · doi:10.1016/j.euroecorev.2008.05.004 · Cited by 28 · see citing works
Microstructure Bluffing with Nested Information
Archishman Chakraborty, A., & Yilmaz, B. (2008) · American Economic Review · doi:10.1257/aer.98.2.280 · Cited by 27 · see citing works
Market behavior in the presence of costly, imperfect information: Experimental evidence
Ackert, L. F., Church, B. K., & Shehata, M. (1997) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(97)00021-8 · Cited by 23 · see citing works
Decision-Making in Research Tasks with Sequential Testing
Pfeiffer, T., Rand, D. G., & Dreber, A. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004607 · Cited by 22 · see citing works
Self-citationAxelrod, B. S., Kulick, B., Plott, C. R., & Roust, K. A. (2008) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2007.09.010 · Cited by 20 · see citing works
Prediction Markets for Promoting the Progress of Science and the Useful Arts
Bell, T. W. (2006) · Chapman University Digital Commons (Chapman University) · Cited by 13 · see citing works
Flagged first, then by citations
1 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [5]Arrow K Forsythe R Gorham R Hahn R Hanson RD 2008 The promise of prediction markets. Science 320 877 878 18487176
- [11]Hanson RD 1999 Prediction markets. IEEE Intelligent Systems 14 16 19
- [32]Gode DK Sunder S 1993 Allocative efficiency of markets with zero intelligence traders. Journal of Political Economy 101 119 137
Cited by
Sorted by citations
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314 · see citing works
Self-citationUsing prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331 · see citing works
Self-citationA Review on Blockchain Technology and Blockchain Projects Fostering Open Science
Leible, S., Steffen Schlager, C., Schubotz, M., & Gipp, B. (2019) · Frontiers in Blockchain · doi:10.3389/fbloc.2019.00016 · Cited by 176 · see citing works
Predicting replication outcomes in the Many Labs 2 study
Forsell, E., Viganola, D., Pfeiffer, T., Almenberg, J., Wilson, B., Chen, Y.-L., Nosek, B. A., Johannesson, M., & Dreber, A. (2018) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.10.009 · Cited by 97 · see citing works
Self-citationModelling the effects of subjective and objective decision making in scientific peer review
Park, I.-U., Peacey, M. W., & Munafò, M. R. (2014) · Nature · doi:10.1038/nature12786 · Cited by 103 · see citing works
Blockchain for Science and Knowledge Creation
Bartling, S. H. (2018) · Springer eBooks · doi:10.1007/978-3-662-57611-3_10 · Cited by 32 · see citing works
Can social microblogging be used to forecast intraday exchange rates?
Papaioannou, P., Russo, L., Papaioannou, G. P., & Siettos, C. I. (2013) · NETNOMICS Economic Research and Electronic Networking · doi:10.1007/s11066-013-9079-3 · Cited by 27 · see citing works
Prediction Markets – A literature review 2014
Horn, C. F., Ivens, B. S., Ohneberg, M., & Brem, A. (2014) · The Journal of Prediction Markets · doi:10.5750/jpm.v8i2.889 · Cited by 25 · see citing works
The power of prediction markets
Mann, A. (2016) · Nature · doi:10.1038/538308a · Cited by 22 · see citing works
Ethics and Statistics: The AAA Tranche of Subprime Science
Gelman, A., & Loken, E. (2014) · CHANCE · doi:10.1080/09332480.2014.890872 · Cited by 19 · see citing works
Incentive mechanism for the listing item task in crowdsourcing
Wang, S., & Dang, D. (2019) · Information Sciences · doi:10.1016/j.ins.2019.09.067 · Cited by 16 · see citing works
PREDICTION MARKETS - A LITERATURE REVIEW 2014 FOLLOWING TZIRALIS AND TATSIOPOULOS
Horn, C. F., Ohneberg, M., Ivens, B. S., Brem, A., & Clausen, M. H. (2014) · Cited by 15 · see citing works
Prediction markets and their potential role in biomedical research – A review
Pfeiffer, T., & Almenberg, J. (2010) · Biosystems · doi:10.1016/j.biosystems.2010.09.005 · Cited by 14 · see citing works
Self-citationViganola, D., Buckles, G., Chen, Y., Diego‐Rosell, P., Johannesson, M., Nosek, B. A., Pfeiffer, T., Siegel, A. C., & Dreber, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.181308 · Cited by 11 · see citing works
Self-citationPrediction Markets for Science: Is the Cure Worse than the Disease?
Thicke, M. (2017) · Social Epistemology · doi:10.1080/02691728.2017.1346720 · Cited by 9 · see citing works
Using prediction markets to forecast research evaluations
Munafo, M. R., Pfeiffer, T., Altmejd, A., Heikensten, E., Almenberg, J., Bird, A., Chen, Y., Wilson, B., Johannesson, M., & Dreber, A. (2015) · Royal Society Open Science · doi:10.1098/rsos.150287 · Cited by 9 · see citing works
Self-citationDesign and analysis of a synthetic prediction market using dynamic convex sets
Nakshatri, N. S., Menon, A. M., Giles, C. L., Rajtmajer, S. M., & Griffin, C. H. (2021) · Results in Control and Optimization · doi:10.1016/j.rico.2021.100052 · Cited by 5 · see citing works
Hevner, A. R., & Yassin, A. M. (2010) · Cited by 3 · see citing works
How Knowledge Stock Exchanges can increase student success in Massive Open Online Courses
Heusler, A., Molitor, D., & Spann, M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223064 · Cited by 2 · see citing works
Thompson, J., Dreber, A., Gaunt, T. R., Gordon, M., Holzmeister, F., Huber, J., Johannesson, M., Kirchler, M., Lyon, M., Penton‐Voak, I. S., et al. (2022) · doi:10.31222/osf.io/gsc8f · Cited by 1 · see citing works
Self-citationScience-Market Analogies: A Philosophical Examination
Thicke, M. (2016) · TSpace (University of Toronto)
Using Prediction Markets as a Tool for Classroom and Civic Engagement
McGee, Z. A., & Hall, P. D. (2024) · Journal of Political Science Education · doi:10.1080/15512169.2024.2385366
Clinical Trial Prediction Markets
Zaorsky, N. G. (2026) · Journal of Clinical Oncology · doi:10.1200/jco-26-01303
- Authors
- Almenberg, J., Kittlitz, K., & Pfeiffer, T.
- Affiliations
- Stockholm School of Economics (SE)
- Harvard University (US)
- Corresponding author
- Pfeiffer, Thomas
- ORCID
- Pfeiffer, Thomas
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-12-30
- Volume
- 4
- Issue
- 12
- Pages
- e8500
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2794371
- OpenAlex
- W2000129516
- Cited by
- 24
- References
- 44
- Citations by year
- 2026: 1 · 2024: 1 · 2022: 1 · 2021: 2 · 2019: 3 · 2018: 2 · 2017: 1 · 2016: 3 · 2015: 2 · 2014: 4 · 2013: 2
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Harvard Business School
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Sports Analytics and Performance
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- prediction markets, scientific prediction, scientific publications, information disclosure, private information, market liquidity, information aggregation, consensus formation
- MeSH terms
- Female, Forecasting, Humans, Male, Publishing, Science, Marketing, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
