The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Identifying Expertise to Extract the Wisdom of Crowds
Abstract
Statistical aggregation is often used to combine multiple opinions within a group. Such aggregates outperform individuals, including experts, in various prediction and estimation tasks. This result is attributed to the “wisdom of crowds.” We seek to improve the quality of such aggregates by eliminating poorly performing individuals from the crowd. We propose a new measure of contribution to assess the judges' performance relative to the group and use positive contributors to build a weighting model for aggregating forecasts. In Study 1, we analyze 1,233 judges forecasting almost 200 current events to illustrate the superiority of our model over unweighted models and models weighted by measures of absolute performance. In Study 2, we replicate our findings by using economic forecasts from the European Central Bank and show how the method can be used to identify smaller crowds of the top positive contributors. We show that the model derives its power from identifying experts who consistently outperform the crowd. Data, as supplemental material, are available at http://dx.doi.org/10.1287/mnsc.2014.1909 . This paper was accepted by James Smith, decision analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Applied Linear Statistical Models
Ziegel, E. R., Neter, J., Wasserman, W., & Kutner, M. (1992) · Technometrics · doi:10.2307/1269588 · Cited by 9,907
Applied Linear Statistical Models
(1997) · Technometrics · doi:10.1080/00401706.1997.10485141 · Cited by 7,185
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T., Griffin, D., & Kahneman, D. (2004) · The Academy of Management Review · doi:10.2307/20159081 · Cited by 3,536
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1645 · Cited by 2,925
Constructive Consumer Choice Processes
Bettman, J. R., Luce, M. F., & Payne, J. W. (1998) · Journal of Consumer Research · doi:10.1086/209535 · Cited by 2,831
Heuristics and Biases: The Psychology of Intuitive Judgment
(2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511808098 · Cited by 2,192
Combining forecasts: A review and annotated bibliography
Clemen, R. T. (1989) · International Journal of Forecasting · doi:10.1016/0169-2070(89)90012-5 · Cited by 2,164
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,966
Combining forecasts: A review and annotated bibliography
(1990) · Long Range Planning · doi:10.1016/0024-6301(90)90316-v · Cited by 1,465
Applied Linear Statistical Models
John, R. C. S. (1986) · Journal of Quality Technology · doi:10.1080/00224065.1986.11978989 · Cited by 1,307
Cited by
A solution to the single-question crowd wisdom problem
Prelec, D., Seung, H. S., & McCoy, J. (2017) · Nature · doi:10.1038/nature21054 · Cited by 314
Will Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16553 · Cited by 303
Forecast combinations: An over 50-year review
Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2022.11.005 · Cited by 280
The gateway belief model: a large-scale replication
van der Linden, S., Leiserowitz, A., & Maibach, E. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.01.009 · Cited by 262
Schaerer, M., du Plessis, C., Nguyen, M. H. B., van Aert, R. C. M., Tiokhin, L., Lakens, D., Giulia Clemente, E., Pfeiffer, T., Dreber, A., Johannesson, M., et al. (2023) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2023.104280 · Cited by 87
Metadata
- Authors
- Budescu, D. V., & Chen, E.
- Venue
- Management Science
- Published
- 2015-02
- Volume
- 61
- Issue
- 2
- Pages
- 267-280
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W2007423920
- Cited by
- 360
- References
- 49
- FWCI
- 20.13
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying Expertise to Extract the Wisdom of Crowds. https://app.cochat.ai/papers/10.1287/mnsc.2014.1909
- The paper
- Budescu, D. V., & Chen, E. (2015). Identifying Expertise to Extract the Wisdom of Crowds. Management Science. https://doi.org/10.1287/mnsc.2014.1909
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
