A solution to the single-question crowd wisdom problem
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Once considered provocative, the notion that the wisdom of the crowd is superior to any individual has become itself a piece of crowd wisdom, leading to speculation that online voting may soon put credentialed experts out of business. Recent applications include political and economic forecasting, evaluating nuclear safety, public policy, the quality of chemical probes, and possible responses to a restless volcano. Algorithms for extracting wisdom from the crowd are typically based on a democratic voting procedure. They are simple to apply and preserve the independence of personal judgment. However, democratic methods have serious limitations. They are biased for shallow, lowest common denominator information, at the expense of novel or specialized knowledge that is not widely shared. Adjustments based on measuring confidence do not solve this problem reliably. Here we propose the following alternative to a democratic vote: select the answer that is more popular than people predict. We show that this principle yields the best answer under reasonable assumptions about voter behaviour, while the standard 'most popular' or 'most confident' principles fail under exactly those same assumptions. Like traditional voting, the principle accepts unique problems, such as panel decisions about scientific or artistic merit, and legal or historical disputes. The potential application domain is thus broader than that covered by machine learning and psychometric methods, which require data across multiple questions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The latest research indicates that exploiting the knowledge from the crowd will facilitate the task (Prelec et al., 2017).”
From Deep learning for sensor-based activity recognition: A survey (2019) · cites this paper as Prelec et al., 2017
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Self-citationModels of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352
Experts In Uncertainty: Opinion and Subjective Probability in Science
Cooke, R. M. (1991) · doi:10.1093/oso/9780195064650.001.0001 · Cited by 1,107
GALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017
Infotopia How Many Minds Produce Knowledge
Sunstein, C. R. (2006) · doi:10.1093/oso/9780195189285.001.0001 · Cited by 913
Use (and abuse) of expert elicitation in support of decision making for public policy
Morgan, M. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319946111 · Cited by 702
A Bayesian Truth Serum for Subjective Data
Prelec, D. (2004) · Science · doi:10.1126/science.1102081 · Cited by 530 · cited 1×
“To discourage such behaviour, one can impose truth-telling incentives with the Bayesian truth serum, which also elicits respondents’ vote predictions 24,25 .”
BackgroundSelf-citation(2008) · doi:10.1126/science.1157679 · Cited by 406
Cited by
Items 101–200 of 312
Liu, Y., Wang, J., & Chen, Y. (2022) · ACM Transactions on Economics and Computation · doi:10.1145/3565559 · Cited by 44
Hanson, R. (2016) · Oxford University Press eBooks · doi:10.1093/oso/9780198754626.003.0021 · Cited by 11
Empirical methodology for crowdsourcing ground truth
Dumitrache, A., Inel, O., Timmermans, B., Ortiz, C., Sips, R.-J., Aroyo, L. M., & Welty, C. (2020) · Semantic Web · doi:10.3233/sw-200415 · Cited by 26
Liu, H., Xin, X., Song, J., & Peng, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128792 · Cited by 10
A consensus building model in group decision making with non-reciprocal fuzzy preference relations
Liu, F., Liu, T., & Chen, Y. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00675-z · Cited by 10
On an effective and efficient method for exploiting the wisdom of the inner crowd
Fujisaki, I., Yang, K., & Ueda, K. (2023) · Scientific Reports · doi:10.1038/s41598-023-30599-8 · Cited by 9
Improving the Wisdom of Crowds with Analysis of Variance of Predictions of Related Outcomes
Satopää, V. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.03.011 · Cited by 12
Almaatouq, A., Rahimian, M. A., Burton, J. W., & Alhajri, A. (2022) · Scientific Reports · doi:10.1038/s41598-022-20551-7 · Cited by 19
A Bayesian Hierarchical Model of Crowd Wisdom Based on Predicting Opinions of Others
McCoy, J. P., & Prelec, D. (2023) · Management Science · doi:10.1287/mnsc.2023.4955 · Cited by 17
Funk, P., Davis, A. L., Vaishnav, P., Dewitt, B., & Fuchs, E. R. H. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.119984 · Cited by 12
Testing the Ability of the Surprisingly Popular Method to Predict NFL GamesPreprint
Lee, M. D., Danileiko, I., & Vi, J. (2018) · doi:10.31234/osf.io/4a5bw · Cited by 9
Improving Judgments of Existential Risk: Better Forecasts, Questions, Explanations, PoliciesPreprint
Karger, E., Atanasov, P. D., & Tetlock, P. E. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4001628 · Cited by 10
Diversity of inference strategies can enhance the ‘wisdom-of-crowds’ effect
Fujisaki, I., Honda, H., & Ueda, K. (2018) · Humanities and Social Sciences Communications · doi:10.1057/s41599-018-0161-1 · Cited by 12
Design recommendations for augmenting creative tasks with computational priming
Oppenlaender, J., & Hosio, S. J. (2019) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3365610.3365621 · Cited by 14
Adaryukov, J. A., Grunevski, S., Reed, D. D., & Pleskac, T. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269625 · Cited by 7
How to detect high-performing individuals and groups: decision similarity predicts accuracyPreprint
Kurvers, R. H. J. M., Herzog, S. M., Hertwig, R., Krause, J., Moussaïd, M., Argenziano, G., Zalaudek, I., Carney, P., & Wolf, M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ryvnw · Cited by 13
Richardson, E., & Keil, F. C. (2022) · Scientific Reports · doi:10.1038/s41598-021-04680-z · Cited by 8
Deep Learning for Sensor-Based Activity Recognition: Recent Trends
Ahad, M. A. R., Antar, A. D., & Ahmed, M. (2020) · Intelligent systems reference library · doi:10.1007/978-3-030-51379-5_9 · Cited by 7
Ronzio, L., Campagner, A., Cabitza, F., & Gensini, G. F. (2021) · Journal of Intelligence · doi:10.3390/jintelligence9020017 · Cited by 7
Yasmin, R., Hassan, M. M., Grassel, J. T., Bhogaraju, H., Escobedo, A. R., & Fuentes, O. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.848056 · Cited by 11
Chang, J.-P., Zheng, H.-L., Mardani, A., Witold Pedrycz, C., & Chen, Z. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2024.3351703 · Cited by 7
“Deliberated Intuition” in Stock Price Forecasting
Endreß, T. (2018) · Economics & Sociology · doi:10.14254/2071-789x.2018/11-3/1 · Cited by 7
Behavioral pitfalls of product proliferation in supply chains: An experimental study
Hyndman, K., & Menezes, M. B. C. (2021) · Decision Sciences · doi:10.1111/deci.12542 · Cited by 7
Schoenegger, P. (2021) · Review of Philosophy and Psychology · doi:10.1007/s13164-021-00571-4 · Cited by 9
Zhang, H., He, A., Sun, J., & Chen, Y. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06345-0 · Cited by 6
Semisupervised Clustering by Queries and Locally Encodable Source Coding
Mazumdar, A., & Pal, S. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3037533 · Cited by 6
Accurate wisdom of the crowd from unsupervised dimension reduction
Wang, L., & Michoel, T. (2019) · Royal Society Open Science · doi:10.1098/rsos.181806 · Cited by 6
Aggregating diverse evaluations in group decision making: An approach based on wisdom of crowds
Wang, H., Yang, G., & Xu, Z.-S. (2023) · Information Sciences · doi:10.1016/j.ins.2023.120005 · Cited by 6
Radas, S., & Prelec, D. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225432 · Cited by 7
Towards Stable Principles of Collective Intelligence under an Environment-Dependent FrameworkPreprint
Almaatouq, A. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ahr7y · Cited by 6
Collective wisdom in polarized groups
Bak-Coleman, J. B., Tokita, C. K., Morris, D. H., Rubenstein, D. I., & Couzin, I. D. (2022) · Collective Intelligence · doi:10.1177/26339137221104788 · Cited by 5
Predicting the future with humans and AI
Mellers, B. A., Lu, L., & McCoy, J. P. (2022) · Consumer Psychology Review · doi:10.1002/arcp.1089 · Cited by 7
Robust recalibration of aggregate probability forecasts using meta-beliefs
Peker, C., & Wilkening, T. S. (2024) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2024.09.005 · Cited by 6
Truth-Tracking via Approval Voting: Size Matters
Allouche, T., Lang, J., & Yger, F. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20403 · Cited by 5
Fujisaki, I., Honda, H., & Ueda, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-16584-7 · Cited by 6
Dominantly Truthful Peer Prediction Mechanisms with a Finite Number of Tasks
Kong, Y. (2024) · Journal of the ACM · doi:10.1145/3638239 · Cited by 16
Face matching as a majority: Getting the best from a crowd
Kramer, R. S. S., & Javorková, N. (2024) · Perception · doi:10.1177/03010066241303705 · Cited by 4
Endreß, T., & Gear, T. E. (2018) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2018.514 · Cited by 4
Yasmin, R., Grassel, J. T., Hassan, M. M., Fuentes, O., & Escobedo, A. R. (2021) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v9i1.18949 · Cited by 5
Zhang, Q., & Zhen, T. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-06938-z · Cited by 4
Powell, B., Satopää, V., MacKay, N. J., & Tetlock, P. E. (2024) · Decision · doi:10.1037/dec0000191 · Cited by 4
Testing the Social Function of Metacognition for Common‐Pool Resource Use
Fischer, H., Wijermans, N., & Schlüter, M. (2023) · Cognitive Science · doi:10.1111/cogs.13212 · Cited by 5
Ratner, N., Kagan, E., Kumar, P., & Ben‐Gal, I. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110551 · Cited by 4
Identifying Wisdom (of the Crowd): A Regression Approach
Libgober, J. (2024) · Journal of Political Economy Microeconomics · doi:10.1086/733781 · Cited by 6
Performance as a Constraint: An Improved Wisdom of Crowds Using Performance Regularization
Li, J., Kawase, Y., Baba, Y., & Kashima, H. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/213 · Cited by 4
Incentive-Compatible Surveys via Posterior Probabilities
Cvitanić, J., Prelec, D., Radas, S., & Šikić, H. (2020) · Theory of Probability and Its Applications · doi:10.1137/s0040585x97t989957 · Cited by 5
When is the Crowd Wise?Preprint
Arieli, I., Babichenko, Y., & Smorodinsky, R. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3083608 · Cited by 4
Expert Selection Within a Single Judgment ProblemPreprint
Palley, A. B., & Satopää, V. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3504286 · Cited by 4
Comparing Human and Machine Deepfake Detection with Affective and Holistic Processing.Preprint
Groh, M., Epstein, Z., Firestone, C., & Picard, R. W. (2021) · arXiv (Cornell University) · Cited by 5
Collective Intelligence as Infrastructure for Reducing Broad Global Catastrophic RisksPreprint
Yang, V. C., & Sandberg, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4101966 · Cited by 9
Three and a half stages of crowd wisdom
Douven, I., Kriegeskorte, N., & Stinson, P. (2026) · Collective Intelligence · doi:10.1177/26339137261435121 · Cited by 3
Game of Duels: Information-Theoretic Axiomatization of Scoring Rules
Cvitanić, J., Prelec, D., Radas, S., & Šikić, H. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2867469 · Cited by 4
Wisdom of Crowds in Quantum Machine Learning
Krisnanda, T., Dini, K., Xu, H., Verstraelen, W., & Liew, T. C. H. (2023) · Physical Review Applied · doi:10.1103/physrevapplied.19.034010 · Cited by 4
The epistemology of democracy: The epistemic virtues of democracy
Samaržija, S. P. (2020) · Filozofija i drustvo · doi:10.2298/fid2001056p · Cited by 3
Wisdom of crowds: much ado about nothing
Reia, S. M., & Fontanari, J. F. (2021) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/abfa1f · Cited by 4
Garnier, S., & Moussaïd, M. (2022) · Collective Intelligence · doi:10.1177/26339137221133400 · Cited by 10
A normative model for Bayesian combination of subjective probability estimates
Trick, S., Rothkopf, C. A., & Jäkel, F. (2023) · Judgment and Decision Making · doi:10.1017/jdm.2023.39 · Cited by 5
Measuring probabilistic coherence to identify superior forecasters
Ho, E., Budescu, D. V., & Himmelstein, M. (2024) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2024.02.005 · Cited by 12
The Social Psychology of the Wisdom of Crowds (with a New Section on Recent Advances)
Larrick, R. P., Mannes, A. E., & Soll, J. B. (2024) · International series in management science/operations research/International series in operations research & management science · doi:10.1007/978-3-031-44424-1_7 · Cited by 4
Karger, E., Rosenberg, J., Jacobs, Z., Hickman, M. G., & Tetlock, P. E. (2025) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2024.11.008 · Cited by 12
Discovering User Bias in Ordinal Voting Systems
Lees, A., & Welty, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3317080 · Cited by 3
Modeling Biological Systems Using Crowdsourcing
Wąsik, S. (2018) · Foundations of Computing and Decision Sciences · doi:10.1515/fcds-2018-0012 · Cited by 3
Read-Agree-Predict: A Crowdsourced Approach to Discovering Relevant Primary Sources for Historians
Wang, N.-C., Hicks, D. L., Quigley, P., & Luther, K. (2019) · Human Computation · doi:10.15346/hc.v6i1.102 · Cited by 3
Terrón, A. M., Barroso, C. S., & Ariza, J. M. R. (2020) · IJERI International Journal of Educational Research and Innovation · doi:10.46661/ijeri.4903 · Cited by 4
Campagner, A., Ciucci, D., & Cabitza, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57524-3_5 · Cited by 4
Convex Combinations in Optimal Judgment Aggregation
Jaspersen, J. G. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.09.050 · Cited by 3
Eliciting and Aggregating Forecasts When Information is SharedPreprint
Palley, A. B., & Soll, J. B. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2636376 · Cited by 4
The Wisdom of the Crowd and Higher-Order BeliefsPreprint
Chen, Y.-C., Mueller-Frank, M., & Pai, M. M. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2102.02666 · Cited by 13
Identify Experts through Revealed Confidence: Application to Wisdom of CrowdsPreprint
Zhang, Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3739192 · Cited by 3
On the Quantification of Crowd Wisdom, Optimal Crowd Size, and the Mean-Median DilemmaPreprint
Lorenz, J. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6ydg4 · Cited by 3
Li, R., Wang, L., Sang, H., Du, Y., & Yao, L. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3645403 · Cited by 2
Optimal linear aggregation of correlated expert judgments
Peker, C. (2026) · Economics Letters · doi:10.1016/j.econlet.2026.112855 · Cited by 3
Lenience breeds strictness: The generosity-erosion effect in hiring decisions
Vives, M., Navia, T. F., Teixido-Figueras, J. J., & Serra‐Burriel, M. (2021) · Science Advances · doi:10.1126/sciadv.abe2045 · Cited by 4
Predicted preference conjoint analysis
Radas, S., & Prelec, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0256010 · Cited by 4
Frustratingly Easy Truth Discovery
Meir, R., Amir, O., Ben-Porat, O., Ben-Shabat, T., Cohensius, G., & Xia, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25750 · Cited by 7
Improving geopolitical forecasts with 100 brains and one computer
Shinitzky, H., Shemesh, Y., Leiser, D., & Gilead, M. (2023) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2023.08.004 · Cited by 5
Using Cognitive Models to Improve the Wisdom of the Crowd
Lee, M. D. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241264292 · Cited by 2
A New Ensemble Strategy Based on Surprisingly Popular Algorithm and Classifier Prediction Confidence
Shi, H., Yuan, Z., Zhang, Y., Zhang, H., & Wang, X. (2025) · Applied Sciences · doi:10.3390/app15063003 · Cited by 3
Mechanisms for Belief Elicitation Without Ground Truth
Lehmann, N. V. (2025) · Journal of Economic Surveys · doi:10.1111/joes.70000 · Cited by 2
Billfeld, N., & Kim, M. (2017) · IEEE Access · doi:10.1109/access.2018.2888575 · Cited by 2
Sun, Q., Wu, Q., ZHANG, J., JIANG, C., Zhao, L., & Hu, F. (2019) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2019.00905 · Cited by 6
Mixed aggregation functions for outliers detection
Zhang, H., Chen, C., Chen, T., Wang, Z., & Chen, Y. (2021) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-200278 · Cited by 2
Zhu, Q., Li, Y., & Zhang, Z. (2023) · Mathematics · doi:10.3390/math11122630 · Cited by 2
What inclines us towards collective decisions
Zein, M. E., Rouault, M., Wurzer, D., Haggard, P., & Deroy, O. D. (2023) · Open Research Europe · doi:10.12688/openreseurope.16160.1 · Cited by 2
Exploiting Meta-cognitive Features for a Machine-Learning-Based One-Shot Group-Decision Aggregation
Shinitzky, H., Avraham, D., Vaisman, Y., Tsizer, Y., Leedon, Y., & D, Y. S. M. (2023) · Group Decision and Negotiation · doi:10.1007/s10726-023-09855-9 · Cited by 2
Hope, T., & Shahaf, D. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159670 · Cited by 2
When social influence promotes the wisdom of crowdsPreprint
Almaatouq, A., Rahimian, M. A., Burton, J. W., & Alhajri, A. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/f5ky3 · Cited by 3
Adaptive Life-Long Learning for an Inclusive Knowledge Economy
Arnold, A., Lindsey, A., McCoy, A. P., Khademian, A. M., Lockee, B. B., Adams, C., Amelink, C. T., Blankenship, C., Glover, C., Harris, C., et al. (2020) · doi:10.21061/adaptive-learning-report · Cited by 2
Hanson, R. (2016) · Oxford University Press eBooks · doi:10.1093/oso/9780198754626.003.0015 · Cited by 2
Bilateral Bayesian truth serum: The n x m signals casePreprint
Prelec, D. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3908446 · Cited by 3
The Wisdom of Strategically Diverse CrowdsPreprint
Jia, Y., Keppo, J., & Satopää, V. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4855714 · Cited by 3
Measuring Misinformation in Financial MarketsPreprint
Fan, J., Liu, Q., Song, Y., & Wang, Z. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4922648 · Cited by 11
Shirasuna, M., & Honda, H. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280984 · Cited by 1
Analysis of Surprisingly Popular Voting for Opinion Aggregation on Social Networks
Yamashita, Y., Sakurai, Y., Oyama, S., Onishi, M., & Morishima, A. (2025) · IEEE Access · doi:10.1109/access.2025.3532754 · Cited by 1
The meta-aggregation approaches: Correctly choosing how to choose correctly in groups
Shinitzky, H., Parpara, O., Ezrets, M., Klein, R., & D, Y. S. M. (2025) · Collective Intelligence · doi:10.1177/26339137251315410 · Cited by 3
Predictive Modeling and Voter Behavior in Electoral Studies: A Systematic Bibliometric Analysis
Κουτσουπιάς, Ν., & Nosios, M. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-1037-4.ch005 · Cited by 1
Deroy, O., & Bahrami, B. (2026) · La Presse Médicale · doi:10.1016/j.lpm.2026.104362 · Cited by 2
An Argument for Post-Hoc Collective Intelligence
Jones, D. J., & Mansingh, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91800-6_16 · Cited by 1
Experimenting With Online Governance
Tchernichovski, O., Frey, S., Jacoby, N., & Conley, D. (2021) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2021.629285 · Cited by 2
Wang, J., Chen, Y., & Hu, C. (2023) · Springer eBooks · doi:10.1007/978-3-031-24628-9_29 · Cited by 2
Metadata
- Authors
- Prelec, D., Seung, H. S., & McCoy, J.
- Affiliations
- Massachusetts Institute of Technology (US)
- Princeton University (US)
- Corresponding author
- Prelec, Dražen
- Venue
- Nature
- Published
- 2017-01
- Volume
- 541
- Issue
- 7638
- Pages
- 532-535
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2580299352
- Cited by
- 314
- References
- 26
- Citations by year
- 2026: 13 · 2025: 28 · 2024: 23 · 2023: 33 · 2022: 29 · 2021: 40 · 2020: 31 · 2019: 28 · 2018: 35 · 2017: 19
- FWCI
- 37.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- U.S. Department of the Interior
- Interior Business Center
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- wisdom of crowds, online voting, democratic voting, expert opinion, prediction markets, confidence calibration, prediction accuracy, group decision making
- MeSH terms
- Algorithms, Art, Bayes Theorem, Choice Behavior, Commerce, Democracy, Expert Testimony, Geography, Humans, Judgment, Public Opinion, Logistic Models, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.