Intuitions about combining opinions: misappreciation of the averaging principle
Abstract
Averaging estimates is an effective way to improve accuracy when combining expert judgments, integrating group members’ judgments, or using advice to modify personal judgments. If the estimates of two judges ever fall on different sides of the truth, which we term bracketing, averaging must outperform the average judge for convex loss functions, such as mean absolute deviation (MAD). We hypothesized that people often hold incorrect beliefs about averaging, falsely concluding that the average of two judges’ estimates would be no more accurate than the average judge. The experiments confirmed that this misconception was common across a range of tasks that involved reasoning from summary data (Experiment 1), from specific instances (Experiment 2), and conceptually (Experiment 3). However, this misconception decreased as observed or assumed bracketing rate increased (all three studies) and when bracketing was made more transparent (Experiment 2). Experiment 4 showed that flawed inferential rules and poor extensional reasoning abilities contributed to the misconception. We conclude by describing how people may face few opportunities to learn the benefits of averaging and how misappreciating averaging contributes to poor intuitive strategies for combining estimates.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If both opinions appear equally reliable, a rational decision would integrate them with equal weight, based on statistical principle that aggregating imperfect judgments reduces error.
“If both opinions appear equally reliable, a rational decision would be to integrate them with equal weight, based on the statistical principle that aggregating imperfect judgments reduces error [49] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [49]
Cites
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1× · see citing works
“We measured extensional reasoning ability by testing performance on two reasoning problems that are formally unrelated to combining opinions: The Will Rogers effect (Messick & Asuncion, 1993) and Simpson's paradox (Simpson, 1951).”
The Impact of Empirical Accuracy Studies on Time Series Analysis and Forecasting
Fildes, R., & Makridakis, S. (1995) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403481 · Cited by 172 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017 · see citing works
Making sense of recommendations
Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2118 · Cited by 459 · see citing works
Big Data and supply chain management: A review and bibliometric analysis
Mishra, D., Gunasekaran, A., Papadopoulos, T., & Childe, S. J. (2018) · Annals of Operations Research · doi:10.1007/s10479-016-2236-y · Cited by 382 · see citing works
Power, competitiveness, and advice taking: Why the powerful don’t listen
Tost, L. P., Gino, F., & Larrick, R. P. (2012) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2011.10.001 · Cited by 294 · see citing works
Self-citationCombining forecasts: An application to elections
Graefe, A., Armstrong, J. S., Jones, R. J., & Cuzán, A. G. (2014) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2013.02.005 · Cited by 160 · see citing works
Effective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85 · see citing works
Confidence Calibration in a Multiyear Geopolitical Forecasting Competition
Moore, D. A., Swift, S. A., Minster, A., Mellers, B., Ungar, L., Tetlock, P., Yang, H. H. J., & Tenney, E. R. (2017) · Management Science · doi:10.1287/mnsc.2016.2525 · Cited by 74 · see citing works
Do fit opinions matter? The impact of fit context on online product returns
Wang, Y., Ramachandran, V., & Liu Sheng, O. R. (2021) · Information Systems Research · doi:10.1287/isre.2020.0965 · Cited by 72 · see citing works
Enhancing human-AI collaboration: The case of colonoscopy
Introzzi, L., Zonca, J., Cabitza, F., Cherubini, P., & Reverberi, C. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2023.10.018 · Cited by 28 · see citing works
Surrogate Decision Making: Do We Have to Trade Off Accuracy and Procedural Satisfaction?
Frey, R., Hertwig, R., & Herzog, S. M. (2014) · Medical Decision Making · doi:10.1177/0272989x12471729 · Cited by 24 · see citing works
Quantifying machine influence over human forecasters
Abeliuk, A., Benjamin, D. M., Morstatter, F., & Galstyan, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72690-4 · Cited by 18 · see citing works
- Authors
- Larrick, R. P., & Soll, J. B.
- Affiliations
- Duke University (US)
- Venue
- Management Science
- Published
- 2006-01
- Volume
- 52
- Issue
- 1
- Pages
- 111-127
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W2145534813
- Cited by
- 410
- References
- 2
- Citations by year
- 2026: 15 · 2025: 21 · 2024: 24 · 2023: 20 · 2022: 23 · 2021: 28 · 2020: 27 · 2019: 29 · 2018: 27 · 2017: 27 · 2016: 24 · 2015: 27 · 2014: 15 · 2013: 25 · 2012: 12
- FWCI
- 5.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Management Science (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- averaging principle, bracketing, intuition, judgment accuracy, mean absolute deviation, convex loss functions, inferential reasoning, group judgment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
