CoChat

A solution to the single-question crowd wisdom problem

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Abstract

Once considered provocative, the notion that the wisdom of the crowd is superior to any individual has become itself a piece of crowd wisdom, leading to speculation that online voting may soon put credentialed experts out of business. Recent applications include political and economic forecasting, evaluating nuclear safety, public policy, the quality of chemical probes, and possible responses to a restless volcano. Algorithms for extracting wisdom from the crowd are typically based on a democratic voting procedure. They are simple to apply and preserve the independence of personal judgment. However, democratic methods have serious limitations. They are biased for shallow, lowest common denominator information, at the expense of novel or specialized knowledge that is not widely shared. Adjustments based on measuring confidence do not solve this problem reliably. Here we propose the following alternative to a democratic vote: select the answer that is more popular than people predict. We show that this principle yields the best answer under reasonable assumptions about voter behaviour, while the standard 'most popular' or 'most confident' principles fail under exactly those same assumptions. Like traditional voting, the principle accepts unique problems, such as panel decisions about scientific or artistic merit, and legal or historical disputes. The potential application domain is thus broader than that covered by machine learning and psychometric methods, which require data across multiple questions.

Overview

Integrity signals

Registry

Crossref · 10.1038/nature21054

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 26 references

Cited by

Items 201–300 of 312

Metadata

Authors
Prelec, D., Seung, H. S., & McCoy, J.
Affiliations
  • Massachusetts Institute of Technology (US)
  • Princeton University (US)
Corresponding author
Prelec, Dražen
Venue
Nature
Published
2017-01
Volume
541
Issue
7638
Pages
532-535
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W2580299352
Cited by
314
References
26
Citations by year
2026: 13 · 2025: 28 · 2024: 23 · 2023: 33 · 2022: 29 · 2021: 40 · 2020: 31 · 2019: 28 · 2018: 35 · 2017: 19
FWCI
37.04
Type
journal article
Language
en
Open access
closed
Licence
www.springer.com/tdm (tdm)
Locations
  • Nature (publishedVersion)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • U.S. Department of the Interior
  • Interior Business Center
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
  • Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
Keywords
wisdom of crowds, online voting, democratic voting, expert opinion, prediction markets, confidence calibration, prediction accuracy, group decision making
MeSH terms
Algorithms, Art, Bayes Theorem, Choice Behavior, Commerce, Democracy, Expert Testimony, Geography, Humans, Judgment, Public Opinion, Logistic Models, Knowledge

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.