A solution to the single-question crowd wisdom problem
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Once considered provocative, the notion that the wisdom of the crowd is superior to any individual has become itself a piece of crowd wisdom, leading to speculation that online voting may soon put credentialed experts out of business. Recent applications include political and economic forecasting, evaluating nuclear safety, public policy, the quality of chemical probes, and possible responses to a restless volcano. Algorithms for extracting wisdom from the crowd are typically based on a democratic voting procedure. They are simple to apply and preserve the independence of personal judgment. However, democratic methods have serious limitations. They are biased for shallow, lowest common denominator information, at the expense of novel or specialized knowledge that is not widely shared. Adjustments based on measuring confidence do not solve this problem reliably. Here we propose the following alternative to a democratic vote: select the answer that is more popular than people predict. We show that this principle yields the best answer under reasonable assumptions about voter behaviour, while the standard 'most popular' or 'most confident' principles fail under exactly those same assumptions. Like traditional voting, the principle accepts unique problems, such as panel decisions about scientific or artistic merit, and legal or historical disputes. The potential application domain is thus broader than that covered by machine learning and psychometric methods, which require data across multiple questions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The latest research indicates that exploiting the knowledge from the crowd will facilitate the task (Prelec et al., 2017).”
From Deep learning for sensor-based activity recognition: A survey (2019) · cites this paper as Prelec et al., 2017
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Self-citationModels of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352
Experts In Uncertainty: Opinion and Subjective Probability in Science
Cooke, R. M. (1991) · doi:10.1093/oso/9780195064650.001.0001 · Cited by 1,107
GALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017
Infotopia How Many Minds Produce Knowledge
Sunstein, C. R. (2006) · doi:10.1093/oso/9780195189285.001.0001 · Cited by 913
Use (and abuse) of expert elicitation in support of decision making for public policy
Morgan, M. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319946111 · Cited by 702
A Bayesian Truth Serum for Subjective Data
Prelec, D. (2004) · Science · doi:10.1126/science.1102081 · Cited by 530 · cited 1×
“To discourage such behaviour, one can impose truth-telling incentives with the Bayesian truth serum, which also elicits respondents’ vote predictions 24,25 .”
BackgroundSelf-citation(2008) · doi:10.1126/science.1157679 · Cited by 406
Cited by
Items 301–312 of 312
Wall, E. (2026) · Synthesis lectures on visualization · doi:10.1007/978-3-032-09307-3_7
Majority views and confidence information promote informed decisions
He, Y., Lien, J. W., & Zheng, J. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-06668-4 · Cited by 1
Aggregating Judgmental Demand Forecasts in Environments with Structural Breaks
Seifert, M., Lee, Y. S., & Shijith Kumar, P. M. (2026) · Decision Analysis · doi:10.1287/deca.2025.0417
Dimara, E. (2017) · theses.fr (ABES) · doi:10.70675/fa0c0a43z19acz4a60za48ez3ad8abc79047 · Cited by 2
Co-opétition et performance créative dans le crowdsourcing d'idées
Elmoukhliss, M.-A. (2018) · theses.fr (ABES) · doi:10.70675/59a81cf1z908dz4f94z81ffz23105ad0051a · Cited by 6
Epistemic Approval Voting : Applications to Crowdsourcing Data Labeling
Allouche, T. (2022) · theses.fr (ABES) · doi:10.70675/2f253320zfaf9z4dcdz82bezd5b7a0a4a725
A collective-intelligence-driven parametric model for measuring and improving group performance
Liu, F., Zhou, D.-H., & Wei, X. (2026) · Information Sciences · doi:10.1016/j.ins.2026.123485
Bayesian combination of correlated subjective probability estimates
Trick, S., Jäkel, F., & Rothkopf, C. A. (2026) · Judgment and Decision Making · doi:10.1017/jdm.2026.10035
Piotrowski, A., & Piotrowska, A. E. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132947
Wisdom of LLM Crowds: Aggregation and Contamination in Language Model Ensembles
Douven, I. (2026) · ACM Transactions on Social Computing · doi:10.1145/3839337 · Cited by 3
Leveraging Collective Advice-Taking Behavior to Infer Accuracy and Improve the Wisdom of Crowds
Zhang, Y. (2026) · Management Science · doi:10.1287/mnsc.2024.04499
Liang, F., Wang, X., Fan, J., He, S., & Chen, W. (2026) · Axioms · doi:10.3390/axioms15090639
Metadata
- Authors
- Prelec, D., Seung, H. S., & McCoy, J.
- Affiliations
- Massachusetts Institute of Technology (US)
- Princeton University (US)
- Corresponding author
- Prelec, Dražen
- Venue
- Nature
- Published
- 2017-01
- Volume
- 541
- Issue
- 7638
- Pages
- 532-535
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2580299352
- Cited by
- 314
- References
- 26
- Citations by year
- 2026: 13 · 2025: 28 · 2024: 23 · 2023: 33 · 2022: 29 · 2021: 40 · 2020: 31 · 2019: 28 · 2018: 35 · 2017: 19
- FWCI
- 37.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- U.S. Department of the Interior
- Interior Business Center
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- wisdom of crowds, online voting, democratic voting, expert opinion, prediction markets, confidence calibration, prediction accuracy, group decision making
- MeSH terms
- Algorithms, Art, Bayes Theorem, Choice Behavior, Commerce, Democracy, Expert Testimony, Geography, Humans, Judgment, Public Opinion, Logistic Models, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.