A solution to the single-question crowd wisdom problem
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Once considered provocative, the notion that the wisdom of the crowd is superior to any individual has become itself a piece of crowd wisdom, leading to speculation that online voting may soon put credentialed experts out of business. Recent applications include political and economic forecasting, evaluating nuclear safety, public policy, the quality of chemical probes, and possible responses to a restless volcano. Algorithms for extracting wisdom from the crowd are typically based on a democratic voting procedure. They are simple to apply and preserve the independence of personal judgment. However, democratic methods have serious limitations. They are biased for shallow, lowest common denominator information, at the expense of novel or specialized knowledge that is not widely shared. Adjustments based on measuring confidence do not solve this problem reliably. Here we propose the following alternative to a democratic vote: select the answer that is more popular than people predict. We show that this principle yields the best answer under reasonable assumptions about voter behaviour, while the standard 'most popular' or 'most confident' principles fail under exactly those same assumptions. Like traditional voting, the principle accepts unique problems, such as panel decisions about scientific or artistic merit, and legal or historical disputes. The potential application domain is thus broader than that covered by machine learning and psychometric methods, which require data across multiple questions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The latest research indicates that exploiting the knowledge from the crowd will facilitate the task (Prelec et al., 2017).”
From Deep learning for sensor-based activity recognition: A survey (2019) · cites this paper as Prelec et al., 2017
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Self-citationModels of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352
Experts In Uncertainty: Opinion and Subjective Probability in Science
Cooke, R. M. (1991) · doi:10.1093/oso/9780195064650.001.0001 · Cited by 1,107
GALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017
Infotopia How Many Minds Produce Knowledge
Sunstein, C. R. (2006) · doi:10.1093/oso/9780195189285.001.0001 · Cited by 913
Use (and abuse) of expert elicitation in support of decision making for public policy
Morgan, M. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319946111 · Cited by 702
A Bayesian Truth Serum for Subjective Data
Prelec, D. (2004) · Science · doi:10.1126/science.1102081 · Cited by 530 · cited 1×
“To discourage such behaviour, one can impose truth-telling incentives with the Bayesian truth serum, which also elicits respondents’ vote predictions 24,25 .”
BackgroundSelf-citation(2008) · doi:10.1126/science.1157679 · Cited by 406
Cited by
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
Key issues in rigorous accuracy assessment of land cover products
Stehman, S. V., & Foody, G. M. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.05.018 · Cited by 651
Stratified Transfer Learning for Cross-domain Activity Recognition
Wang, J., Chen, Y., Hu, L., Peng, X., & Yu, P. S. (2018) · 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2018.8444572 · Cited by 253
Deepfake detection by human crowds, machines, and machine-informed crowds
Groh, M., Epstein, Z., Firestone, C., & Picard, R. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2110013119 · Cited by 248
Making better decisions in groups
Bang, D., & Frith, C. D. (2017) · Royal Society Open Science · doi:10.1098/rsos.170193 · Cited by 185
Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds
Navajas, J., Niella, T., Garbulsky, G., Bahrami, B., & Sigman, M. (2018) · Nature Human Behaviour · doi:10.1038/s41562-017-0273-4 · Cited by 183
Information aggregation and collective intelligence beyond the wisdom of crowds
Kameda, T., Toyokawa, W., & Tindale, R. S. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00054-y · Cited by 193
Li, R., Gong, W., Wang, L., Lu, C., & Zhuang, X. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3280175 · Cited by 138
Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research
Vaughan, J. W. (2018) · Journal of Machine Learning Research · Cited by 228
Shared responsibility in collective decisions
Zein, M. E., Bahrami, B., & Hertwig, R. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0596-4 · Cited by 161
Human social sensing is an untapped resource for computational social science
Galešić, M., de Bruin, W. B., Dalege, J., Feld, S. L., Kreuter, F., Olsson, H., Prelec, D., Stein, D. L., & van der Does, T. (2021) · Nature · doi:10.1038/s41586-021-03649-2 · Cited by 120
Modeling the competitive diffusions of rumor and knowledge and the impacts on epidemic spreading
Huang, H. Q., Chen, Y., & Ma, Y. (2020) · Applied Mathematics and Computation · doi:10.1016/j.amc.2020.125536 · Cited by 91
Guilbeault, D., & Centola, D. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227813 · Cited by 86
Cross-position activity recognition with stratified transfer learning
Chen, Y., Wang, J., Huang, M., & Yu, H. (2019) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2019.04.004 · Cited by 79
Aggregation models in ensemble learning: A large-scale comparison
Campagner, A., Ciucci, D., & Cabitza, F. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.09.015 · Cited by 80
Extracting the Wisdom of Crowds When Information Is Shared
Palley, A. B., & Soll, J. B. (2019) · Management Science · doi:10.1287/mnsc.2018.3047 · Cited by 117
Asking about social circles improves election predictions
Galešić, M., de Bruin, W. B., Dumas, M., Kapteyn, A., Darling, J. E., & Meijer, E. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0302-y · Cited by 82
Counteracting estimation bias and social influence to improve the wisdom of crowds
Kao, A. B., Berdahl, A. M., Hartnett, A. T., Lutz, M. J., Bak-Coleman, J. B., Ioannou, C. C., Giam, X., & Couzin, I. D. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2018.0130 · Cited by 74
The Age of Em: Work, Love, and Life when Robots Rule the Earth
Hanson, R. (2016) · doi:10.1093/oso/9780198754626.001.0001 · Cited by 218
From CNNs to Transformers in Multimodal Human Action Recognition: A Survey
Shaikh, M. B., Chai, D., Islam, S. M. S., & Akhtar, N. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3664815 · Cited by 88
Incorporating Surprisingly Popular Algorithm and Euclidean distance-based adaptive topology into PSO
Wu, X., Han, J., Wang, D., Gao, P., Cui, Q., Chen, L., Liang, Y., Huang, H. P., Lee, H. P., Miao, C., et al. (2022) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2022.101222 · Cited by 67
Mitigating the Attraction Effect with Visualizations
Dimara, E., Bailly, G., Bezerianos, A., & Franconeri, S. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865233 · Cited by 61
A Modified Consensus Model in Group Decision Making With an Allocation of Information Granularity
Liu, F., Wu, Y., & Pedrycz, W. (2018) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2018.2793885 · Cited by 60
Content Analysis by the Crowd: Assessing the Usability of Crowdsourcing for Coding Latent Constructs
Lind, F., Gruber, M., & Boomgaarden, H. G. (2017) · Communication Methods and Measures · doi:10.1080/19312458.2017.1317338 · Cited by 68
Bias, Information, Noise: The BIN Model of Forecasting
Satopää, V., Salikhov, M., Tetlock, P. E., & Mellers, B. A. (2020) · Management Science · doi:10.1287/mnsc.2020.3882 · Cited by 65
Danchin, A., Ouzounis, C., Tokuyasu, T., & Zucker, J. (2018) · Microbial Biotechnology · doi:10.1111/1751-7915.13284 · Cited by 56
Toward a Design Space for Mitigating Cognitive Bias in Vis
Wall, E., Stasko, J., & Endert, A. (2019) · Visualization · doi:10.1109/visual.2019.8933611 · Cited by 65
Ground truthing from multi-rater labeling with three-way decision and possibility theory
Campagner, A., Ciucci, D., Svensson, C., Figge, M. T., & Cabitza, F. (2020) · Information Sciences · doi:10.1016/j.ins.2020.09.049 · Cited by 62
Mai, E. S., & Liao, Y. (2021) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-05-2021-0269 · Cited by 65
Wise teamwork: Collective confidence calibration predicts the effectiveness of group discussion
Silver, I., Mellers, B. A., & Tetlock, P. E. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104157 · Cited by 60
Efficient Elicitation Approaches to Estimate Collective Crowd Answers
Chung, J. J. Y., Song, J. Y., Kutty, S., Hong, S. (., Kim, J., & Lasecki, W. S. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359164 · Cited by 46
How to detect high-performing individuals and groups: Decision similarity predicts accuracy
Kurvers, R. H. J. M., Herzog, S. M., Hertwig, R., Krause, J., Moussaïd, M., Argenziano, G., Zalaudek, I., Carney, P. A., & Wolf, M. (2019) · Science Advances · doi:10.1126/sciadv.aaw9011 · Cited by 54
Cabitza, F., Campagner, A., & Sconfienza, L. M. (2021) · Health Information Science and Systems · doi:10.1007/s13755-021-00138-8 · Cited by 39
Rescuing Collective Wisdom when the Average Group Opinion Is Wrong
Laan, A., Madirolas, G., & de Polavieja, G. G. (2017) · Frontiers in Robotics and AI · doi:10.3389/frobt.2017.00056 · Cited by 47
Mathematically aggregating experts’ predictions of possible futures
Hanea, A. M., Wilkinson, D. P., McBride, M. F., Lyon, A., van Ravenzwaaij, D., Thorn, F. S., Gray, C. T., Mandel, D. R., Willcox, A., Gould, E., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0256919 · Cited by 42
Group efficiency and individual fairness tradeoff in making wise decisions
Tang, M., & Liao, H. (2023) · Omega · doi:10.1016/j.omega.2023.103015 · Cited by 39
Game Theory for Data Science: Eliciting Truthful Information
Faltings, B. V., & Radanović, G. (2017) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00788ed1v01y201707aim035 · Cited by 44
Chatzidiakou, L., Archer, R. J., Beale, V., Bland, S., Carter, H., Castro-Faccetti, C., Edwards, H., Finneran, J., Hama, S. M. L., Jones, R. L., et al. (2023) · Developments in the Built Environment · doi:10.1016/j.dibe.2023.100266 · Cited by 48
The impact of social media signals on supplier selection: insights from two experiments
Banerjee, A., Ries, J. M., & Wiertz, C. (2020) · International Journal of Operations & Production Management · doi:10.1108/ijopm-05-2019-0413 · Cited by 40
Conflation of expert and crowd reference data to validate global binary thematic maps
Waldner, F., Schucknecht, A., Lesiv, M., Gallego, J., See, L., Pérez-Hoyos, A., d’Andrimont, R., Maet, T. D., Bayas, J. C. L., Fritz, S., et al. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.10.039 · Cited by 45
Knowledge fusion patterns: A survey
Smirnov, A., & Levashova, T. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.11.007 · Cited by 33
Literature review on collective intelligence: a crowd science perspective
Yu, C., Chai, Y., & Liu, Y. (2018) · International Journal of Crowd Science · doi:10.1108/ijcs-08-2017-0013 · Cited by 37
Ideology and Composition Among an Online Crowd: Evidence from Wikipedians
Greenstein, S. M., Gu, G., & Zhu, F. (2020) · Management Science · doi:10.1287/mnsc.2020.3661 · Cited by 39
Palley, A. B., & Satopää, V. (2019) · Management Science · doi:10.1287/mnsc.2022.4648 · Cited by 43
Modelling Cognitive Bias in Crowdsourcing Systems
Saab, F., Elhajj, I. H., Kayssi, A., & Chehab, A. (2019) · Cognitive Systems Research · doi:10.1016/j.cogsys.2019.04.004 · Cited by 32
Koriat, A. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000397 · Cited by 37
Foody, G. M., See, L., Fritz, S., Moorthy, I., Perger, C., Schill, C., & Boyd, D. S. (2018) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi7030080 · Cited by 28
Wang, W., & Zhao, J. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.03.013 · Cited by 27
A PSO-algorithm-based consensus model with the application to large-scale group decision-making
Liu, F., Zhang, J., & Liu, T. (2020) · Complex & Intelligent Systems · doi:10.1007/s40747-020-00144-5 · Cited by 26
Wilkening, T. S., Martinie, M. F. R. C., & Howe, P. D. L. (2021) · Management Science · doi:10.1287/mnsc.2020.3919 · Cited by 35
Xie, H., Li, Y., Pu, Y., Zhang, C., & Huang, J. (2024) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2024.102655 · Cited by 32
Arieli, I., Babichenko, Y., & Smorodinsky, R. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1813934115 · Cited by 45
Chen, F., Luo, C., Gong, W., & Lu, C. (2024) · Complex System Modeling and Simulation · doi:10.23919/csms.2024.0003 · Cited by 23
Studying the “Wisdom of Crowds” at Scale
Simoiu, C., Sumanth, C., Mysore, A., & Goel, S. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5271 · Cited by 25
The wisdom of crowds for visual search
Juni, M. Z., & Eckstein, M. P. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1610732114 · Cited by 24
Metaheuristics should be tested on large benchmark set with various numbers of function evaluations
Piotrowski, A., Napiorkowski, J. J., & Piotrowska, A. E. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101807 · Cited by 31
Crowdsourcing the assessment of wine quality: Vivino ratings, professional critics, and the weather
Kopsacheilis, O., Analytis, P. P., Kaushik, K., Herzog, S. M., Bahrami, B., & Deroy, O. D. (2024) · Journal of Wine Economics · doi:10.1017/jwe.2024.20 · Cited by 24
Long-term forecasts for energy commodities price: What the experts think
Zhou, F., Page, L., Perrons, R. K., Zheng, Z., & Washington, S. P. (2019) · Energy Economics · doi:10.1016/j.eneco.2019.104484 · Cited by 22
Cvitanić, J., Prelec, D., Riley, B., & Tereick, B. (2019) · American Economic Review Insights · doi:10.1257/aeri.20180227 · Cited by 32
Determinants of idea sharing in crowdsourcing: evidence from the automotive industry
Schäper, T., Foege, J. N., Nüesch, S., & Schäfer, S. (2020) · R and D Management · doi:10.1111/radm.12429 · Cited by 24
Hyper Questions: Unsupervised Targeting of a Few Experts in Crowdsourcing
Li, J., Baba, Y., & Kashima, H. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3132971 · Cited by 24
Untapped potential of collective intelligence in conservation and environmental decision making
Vercammen, A., & Burgman, M. A. (2019) · Conservation Biology · doi:10.1111/cobi.13335 · Cited by 24
Using meta-predictions to identify experts in the crowd when past performance is unknown
Martinie, M. F. R. C., Wilkening, T. S., & Howe, P. D. L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0232058 · Cited by 25
On the round number bias and wisdom of crowds in different response formats for numerical estimation
Honda, H., Kagawa, R., & Shirasuna, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-11900-7 · Cited by 30
The wisdom of many in few: Finding individuals who are as wise as the crowd.
Himmelstein, M., Budescu, D. V., & Ho, E. (2023) · Journal of Experimental Psychology General · doi:10.1037/xge0001340 · Cited by 28
Answer validation for generic crowdsourcing tasks with minimal efforts
Nguyen, Q. V. H., Thang, D. C., Nguyên, T. T., Weidlich, M., Aberer, K., Yin, H., & Zhou, X. (2017) · The VLDB Journal · doi:10.1007/s00778-017-0484-3 · Cited by 33
A consensus model for group decision making under additive reciprocal matrices with flexibility
Liu, F., Zou, S.-C., & Wu, Y. (2019) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2019.11.006 · Cited by 18
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.903 · Cited by 65
A crowdsourcing-based topic model for service matchmaking in Internet of Things
Liu, Y., Du, F., Sun, J., Jiang, Y., Jian-min, H., Zhu, T., & Sun, C. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.05.005 · Cited by 18
Beating the average forecast: Regularization based on forecaster attributes
Merkle, E. C., Saw, G., & Davis‐Stober, C. P. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102419 · Cited by 20
Forecast Aggregation via Peer Prediction
Wang, J., Liu, Y., & Chen, Y. (2021) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v9i1.18946 · Cited by 25
The wisdom of the crowd and prediction markets
Dai, M., Jia, Y., & Kou, S. (2020) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.07.016 · Cited by 20
Collective Intelligence: from the Enlightenment to the Crowd Science
Yu, C., Chai, Y., & Liu, Y. (2017) · International Conference on Crowd Science and Engineering (ICCSE) · doi:10.1145/3126973.3126993 · Cited by 16
Deutschmann, E. (2025) · Acta Sociologica · doi:10.1177/00016993241311523 · Cited by 15
Eliciting Expertise without Verification
Kong, Y., & Schoenebeck, G. R. (2018) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3219166.3219172 · Cited by 31
Danchin, A. (2024) · Microbial Biotechnology · doi:10.1111/1751-7915.70014 · Cited by 18
Liu, F., Chen, Y.-R., & Zhou, D.-H. (2023) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2022.12.030 · Cited by 14
Himmelstein, M., Budescu, D. V., & Han, Y. (2023) · International series in management science/operations research/International series in operations research & management science · doi:10.1007/978-3-031-30085-1_8 · Cited by 19
Certainty Is Primarily Determined by Past Performance During Concept Learning
Martí, L., Mollica, F., Piantadosi, S. T., & Kidd, C. (2018) · Open Mind · doi:10.1162/opmi_a_00017 · Cited by 24
Mellers, B. A., McCoy, J. P., Lu, L., & Tetlock, P. E. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231185339 · Cited by 16
Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding
Mazumdar, A., & Pal, S. (2017) · neural information processing systems · Cited by 13
A Novel Large Group Decision-Making Method via Normalized Alternative Prediction Selection
Zhang, H., Zhou, Y., Zheng, Q., Wang, Z., Chen, Y., Chen, C., & Liu, T. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2020.2991308 · Cited by 13
Model‐Based Wisdom of the Crowd for Sequential Decision‐Making Tasks
Thomas, B., Coon, J., Westfall, H. A., & Lee, M. D. (2021) · Cognitive Science · doi:10.1111/cogs.13011 · Cited by 19
Surprisingly Popular Voting Recovers Rankings, Surprisingly!
Hosseini, H., Mandal, D., Shah, N., & Shi, K. Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/35 · Cited by 20
Wang, W., Guo, L., Wu, Y. J., Goh, M., & Wang, S. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102832 · Cited by 12
The relationship between crowd majority and accuracy for binary decisions
Lee, M. D., & Lee, M. N. (2017) · Judgment and Decision Making · doi:10.1017/s1930297500006227 · Cited by 15
Surprisingly Popular Algorithm-Based Comprehensive Adaptive Topology Learning PSO
Cui, Q., Tang, C., Xu, G., Wu, C., Shi, X., Liang, Y., Chen, L., Lee, H. P., & Huang, H. P. (2019) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2019.8790002 · Cited by 13
Testing the ability of the surprisingly popular method to predict NFL games
Lee, M. D., Danileiko, I., & Vi, J. (2018) · Judgment and Decision Making · doi:10.1017/s1930297500009207 · Cited by 23
Truth in Disagreement: Crowdsourcing Labeled Data for Natural Language Processing
Dumitrache, A. (2019) · Data Archiving and Networked Services (DANS) · Cited by 48
Ami, D., Aprahamian, F., Chanel, O., & Luchini, S. (2018) · Journal of Choice Modelling · doi:10.1016/j.jocm.2018.09.001 · Cited by 15
Identifiable information structures
Arieli, I., Babichenko, Y., & Rann Smorodinsky, B. (2019) · Games and Economic Behavior · doi:10.1016/j.geb.2019.12.006 · Cited by 12
Does the “surprisingly popular” method yield accurate crowdsourced predictions?
Rutchick, A. M., Ross, B. J., Calvillo, D. P., & Mesick, C. C. (2020) · Cognitive Research Principles and Implications · doi:10.1186/s41235-020-00256-z · Cited by 12
Information markets and nonmarkets
Bergemann, D., & Ottaviani, M. (2021) · Handbook of Industrial Organization · doi:10.1016/bs.hesind.2021.11.008 · Cited by 24
Find truth in the hands of the few: acquiring specific knowledge with crowdsourcing
Han, T., Sun, H., Song, Y., Fang, Y., & Liu, X. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-020-9364-x · Cited by 13
Citizen science decisions: A Bayesian approach optimises effort
Mugford, J., Moltchanova, E. V., Plank, M. J., Sullivan, J. J., Byrom, A. E., & James, A. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101313 · Cited by 14
Crowds Improve Human Detection of AI ‐Synthesised Faces
Kramer, R. S. S., & Cartledge, C. (2024) · Applied Cognitive Psychology · doi:10.1002/acp.4245 · Cited by 11
Machine truth serum: a surprisingly popular approach to improving ensemble methods
Luo, T., & Liu, Y. (2019) · Machine-mediated learning · doi:10.1007/s10994-022-06183-y · Cited by 9
Extracting the collective wisdom in probabilistic judgments
Peker, C. (2022) · Theory and Decision · doi:10.1007/s11238-022-09899-4 · Cited by 14
Identification of influencers through the wisdom of crowds
Tanase, R., Tessone, C. J., & Algesheimer, R. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200109 · Cited by 22
Humans best judge how much to cooperate when facing hard problems in large groups
Guazzini, A., Stefanelli, F., Imbimbo, E., Vilone, D., Bagnoli, F., & Levnajić, Z. (2019) · Scientific Reports · doi:10.1038/s41598-019-41773-2 · Cited by 13
Metadata
- Authors
- Prelec, D., Seung, H. S., & McCoy, J.
- Affiliations
- Massachusetts Institute of Technology (US)
- Princeton University (US)
- Corresponding author
- Prelec, Dražen
- Venue
- Nature
- Published
- 2017-01
- Volume
- 541
- Issue
- 7638
- Pages
- 532-535
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2580299352
- Cited by
- 314
- References
- 26
- Citations by year
- 2026: 13 · 2025: 28 · 2024: 23 · 2023: 33 · 2022: 29 · 2021: 40 · 2020: 31 · 2019: 28 · 2018: 35 · 2017: 19
- FWCI
- 37.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- U.S. Department of the Interior
- Interior Business Center
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- wisdom of crowds, online voting, democratic voting, expert opinion, prediction markets, confidence calibration, prediction accuracy, group decision making
- MeSH terms
- Algorithms, Art, Bayes Theorem, Choice Behavior, Commerce, Democracy, Expert Testimony, Geography, Humans, Judgment, Public Opinion, Logistic Models, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.