How social influence can undermine the wisdom of crowd effect
Open accessAbstract
Social groups can be remarkably smart and knowledgeable when their averaged judgements are compared with the judgements of individuals. Already Galton [Galton F (1907) Nature 75:7] found evidence that the median estimate of a group can be more accurate than estimates of experts. This wisdom of crowd effect was recently supported by examples from stock markets, political elections, and quiz shows [Surowiecki J (2004) The Wisdom of Crowds ]. In contrast, we demonstrate by experimental evidence ( N = 144) that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks. In the experiment, subjects could reconsider their response to factual questions after having received average or full information of the responses of other subjects. We compare subjects’ convergence of estimates and improvements in accuracy over five consecutive estimation periods with a control condition, in which no information about others’ responses was provided. Although groups are initially “wise,” knowledge about estimates of others narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd effect in three different ways. The “social influence effect” diminishes the diversity of the crowd without improvements of its collective error. The “range reduction effect” moves the position of the truth to peripheral regions of the range of estimates so that the crowd becomes less reliable in providing expertise for external observers. The “confidence effect” boosts individuals’ confidence after convergence of their estimates despite lack of improved accuracy. Examples of the revealed mechanism range from misled elites to the recent global financial crisis.
AI snapshot
Written from the paper’s full text.
TL;DR
In a laboratory experiment with N = 144 participants, even mild social influence undermined the wisdom-of-crowd effect in factual estimation tasks. Social influence reduced diversity without improving collective accuracy, moved the truth toward the periphery of the estimate range, and increased individuals’ confidence despite no accuracy gain.
Key findings
- Social influence effect: information about others’ estimates caused convergence and significantly reduced group diversity without significantly reducing collective error; in only 21.3% of cases was the arithmetic mean closer to truth than individual first estimates, whereas after log transformation the arithmetic mean was closer in 77.1% of cases, supporting the geometric mean as the aggregation measure.
- Range reduction effect: the wisdom-of-crowd indicator was about one unit lower under information exchange than in the control condition, meaning the truth became less central and the group less reliable; reduction was stronger under aggregated information than full information.
- Confidence effect: individuals’ confidence was substantially and significantly boosted in both aggregated and full information conditions relative to the no-information control, despite lack of improved accuracy.
- Table 1 example: for population density of Switzerland (true value 184), arithmetic mean was 2,644 (+1,337.2%), geometric mean 132 (−28.1%), median 130 (−29.3%); for murders in Switzerland 2006 (true 198), arithmetic mean 838 (+323.2%), geometric mean 174 (−11.9%), median 170 (−14.1%).
Methods
- 144 participants recruited from more than 8,000 ETH Zurich students; 12 sessions of 12 subjects; six factual questions on geography and crime statistics; five consecutive estimates per question; confidence measured after first and final estimate on a six-point Likert scale.
- Three conditions: no information control, aggregated information (arithmetic mean of all 12 previous estimates), and full information (trajectories and numerical values of all subjects’ previous estimates); two questions per treatment per session, with randomized order.
- Monetary incentives: 1.40 CHF for 0–10% deviation, 0.70 CHF for 11–20%, 0.35 CHF for 21–40%, no reward beyond 40%; analysis used log-transformed estimates normalized by true values, geometric mean, collective error, group diversity, and wisdom-of-crowd indicator; 10 values were removed (outliers, fun moves, one zero for log analysis).
Limitations
- Stated: the experiment abstracted from real-life social influence effects such as competition, group pressure, and authority effects; negative effects were less severe for easier questions and for more confident individuals, which the authors say should be explored in follow-up studies.
- Stated: initial diversity appeared higher in the no-information condition, possibly because subjects anticipated unease about distant published estimates; this discrepancy was mainly caused by crime-statistics questions. Collective error slightly declined under social influence due to right-skewed distributions and arithmetic-mean feedback causing upward drift; authors say this deserves future studies.
- Evident from text: findings come from a controlled lab setting with ETH Zurich students, six questions, no communication, and mild information feedback, so generalization to other populations, questions, and stronger social influence is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 9 cited sources checked · 0 supported · 0 flagged
- Findingown data§ Acknowledgments
In only 1.5% of all cases, subjects responded at all five times in the most inner payment range of one particular question.
“in only 1.5% of all cases, subjects responded at all five times in the most inner payment range of one particular question.”
Quote in the paperSubstantial claim - Findingnone given§ Discussion
The convergence of estimates significantly boosts individuals' confidence.
“Psychologically, however, the convergence of estimates significantly boosts individuals’ confidence.”
Quote in the paperSubstantial claim - Findingown data§ Abstract
Social influence moves the position of the truth to peripheral regions of the range of estimates, making the crowd less reliable in providing expertise for external observers.
“The “range reduction effect” moves the position of the truth to peripheral regions of the range of estimates so that the crowd becomes less reliable in providing expertise for external observers.”
- Also stated in: § Abstract · plain
“The “range reduction effect” moves the position of the truth to peripheral regions. This corrupts the wisdom of the crowd from an observer's perspective in the sense that the group becomes less reliable in guiding decision-makers.”
Quote in the paperSubstantial claim- Rests on an assumption: Social influence in laboratory estimation tasks with students generalizes to real-world crowds and societies.
- Also stated in: § Abstract · plain
- Findingown data§ Results
When using logarithms of estimates, the arithmetic mean is closer to the logarithm of the truth than individuals' estimates in 77.1% of cases.
“When using logarithms of estimates, the arithmetic mean is closer to the logarithm of the truth than the individuals’ estimates in 77.1% of the cases.”
Quote in the paperSubstantial claim - Findingown data§ Results
The distribution of estimates changes significantly if social influence is allowed for.
“A Kolmogorov–Smirnov test confirms that the distribution of estimates changes significantly if social influence is allowed for.”
Quote in the paperSubstantial claim
- Findingown data§ Results
The wisdom-of-crowd indicator is about one unit lower in conditions with information exchange compared with the control condition.
“It is revealed that the wisdom of crowd indicator is about one unit lower in conditions with information exchange compared with the control condition.”
Quote in the paperSubstantial claim - Findingreasoning§ Abstract
Social influence boosts individuals' confidence after convergence of their estimates despite lack of improved accuracy.
“The “confidence effect” boosts individuals’ confidence after convergence of their estimates despite lack of improved accuracy.”
- Also stated in: § Abstract · plain
“The “confidence effect” boosts individuals’ confidence.”
Quote in the paperSubstantial claim - Also stated in: § Abstract · plain
- Findingown data§ Results
Social influence promotes a convergence of estimates.
“It is evident that social influence promotes a convergence of estimates.”
Quote in the paperSubstantial claim - Findingnone given§ Acknowledgments
The distributions of estimates are right-skewed.
“First, the distributions of estimates are right-skewed.”
Quote in the paperSubstantial claim - Findingown data§ Abstract
Even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks (N = 144).
“In contrast, we demonstrate by experimental evidence (N = 144) that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks.”
Quote in the paperSubstantial claim - Findingown data§ Results
The variance of the distribution changes significantly under social influence.
“This applies particularly to the variance of the distribution, as an F test shows.”
Quote in the paperSubstantial claim - Findingreasoning§ Results
The arithmetic mean is closer to the truth than individual first estimates in only 21.3% of cases.
“In only 21.3% of the cases is the arithmetic mean closer to the truth than the individual first estimates.”
Quote in the paperSubstantial claim - Findingown data§ Results
Individuals' confidence is substantially and significantly boosted in the aggregated and full information conditions compared with the control condition without social influence.
“This regression analysis demonstrates that individuals’ confidence is substantially and significantly boosted in the aggregated and full information conditions in comparison with the control condition without social influence.”
Quote in the paperSubstantial claim - Findingown data§ Acknowledgments
For our choice of questions, the geometric mean is always slightly lower than the true value.
“Second, it is an empirical fact for our choice of questions that the geometric mean (which is our aggregation measure to compute the collective error) is always slightly lower than the true value (Table 1).”
Quote in the paperSubstantial claim - Findingown data§ Results
The reduction in the wisdom-of-crowd indicator is stronger under the aggregated information condition compared with the full information condition.
“Note that the reduction is stronger under the aggregated information condition compared with the “full information condition.”
Quote in the paperNot a substantial claim - Findingreasoning§ Discussion
The remaining diversity is often so small that the correct value shifts from the center to outer regions of the range of estimates.
“The remaining diversity is often so small that the correct value shifts from the center to outer regions of the range of estimates.”
Quote in the paperSubstantial claim - Findingown data§ Results
The wisdom-of-crowd indicator tends to decline over time under conditions of social influence.
“The figure demonstrates that the wisdom-of-crowd indicator tends to decline over time under conditions of social influence.”
Quote in the paperSubstantial claim - Interpretationreasoning§ Results
The group becomes less reliable in estimating the truth if it has been exposed to social influence.
“Another interpretation of this effect is that the group becomes less reliable in estimating the truth if it has been exposed to social influence.”
Quote in the paperSubstantial claim - Interpretationreasoning§ Discussion
When taking committee decisions or following advice of an expert group exposed to social influence, their opinions may result in a set of predictions that does not even enclose the correct value anymore.
“Thus, when taking committee decisions or following the advise of an expert group that was exposed to social influence, their opinions may result in a set of predictions that does not even enclose the correct value anymore.”
Quote in the paperSubstantial claim- Rests on an assumption: It is hardly feasible to receive independent opinions in society because people are embedded in social networks and typically influence each other to a certain extent.
- Interpretationreasoning§ Results
If all predictions are narrowly distributed around a wrong value, a decision-maker would gain confidence in advice that is actually misleading.
“If all predictions are narrowly distributed around a wrong value, a decision-maker would gain confidence in advice that is actually misleading.”
Quote in the paperSubstantial claim- Rests on an assumption: Social influence in laboratory estimation tasks with students generalizes to real-world crowds and societies.
- Interpretationown data§ Discussion
The results underpin the value of collecting individuals' estimates in the absence of social influence.
“Our results underpin the value of collecting individuals’ estimates in the absence of social influence.”
Quote in the paperSubstantial claim- Rests on an assumption: Groups can be remarkably accurate in estimating vaguely known facts based on the wisdom-of-crowd effect.
- Interpretationreasoning§ Results
The close clustering around a wrong value makes the group less wise in the sense that the group delivers a wrong hint regarding the location of the truth.
“In fact, the close clustering around a wrong value makes the group less “wise” in the sense that the group delivers a wrong hint regarding the location of the truth.”
Quote in the paperSubstantial claim- Rests on an assumption: The six factual questions and the student sample are representative of estimation tasks and crowds in general.
- Interpretationown data§ Discussion
Little social influence is required to produce herding behavior and negative side effects for the mechanism underlying the wisdom of crowds.
“It is remarkable how little social influence is required to produce herding behavior and negative side effects for the mechanism underlying the wisdom of crowds.”
Quote in the paperSubstantial claim - Interpretationreasoning§ Discussion
Such advice may be thoroughly misleading because closely related, seemingly independent advice may pretend certainty despite substantial deviations from the correct solution.
“From the perspective of decision-makers, such advice may be thoroughly misleading, because closely related, seemingly independent advice may pretend certainty despite substantial deviations from the correct solution.”
Quote in the paperSubstantial claim- Rests on an assumption: It is hardly feasible to receive independent opinions in society because people are embedded in social networks and typically influence each other to a certain extent.
- Interpretationreasoning§ Results
The decline in the wisdom-of-crowd indicator demonstrates that the truth becomes less central if social influence is allowed for.
“This effect demonstrates that the truth becomes less central if social influence is allowed for.”
Quote in the paperSubstantial claim - Interpretationcitation§ Abstract
Social influence can have a substantial impact on the statistical wisdom of crowd effect in societies.
“As social influence among human group members may trigger individuals to revise their estimates (10), it can have a substantial impact on the statistical wisdom of crowd effect in societies.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10] Mason WA, Conrey FR, Smith ER. Situating social influence processes: dynamic, multidirectional flows of influence within social networks. Pers Soc Psychol Rev. 2007;11:279–300. doi: 10.1177/1088868307301032. [DOI] [PubMed] [Google Scholar] source
Not checked in the free run.
- Rests on an assumption: Social influence in laboratory estimation tasks with students generalizes to real-world crowds and societies.
- Interpretationown data§ Results
The geometric mean is an accurate measure of the wisdom of crowds for our data.
“This confirms that the geometric mean (i.e., exponential of the mean of the logarithmized data) is an accurate measure of the wisdom of crowds for our data (Table 1).”
Quote in the paperSubstantial claim - Interpretationreasoning§ Acknowledgments
The mechanism of presenting the arithmetic mean in the aggregated condition triggers an upward drift toward the true value.
“The mechanism of presenting the arithmetic mean in the aggregated condition thus triggers an upward drift toward the true value.”
Quote in the paperSubstantial claim - Premisecitation§ Abstract
The wisdom of crowds effect works if estimation errors of individuals are large but unbiased such that they cancel each other out.
“The wisdom of crowds effect works if estimation errors of individuals are large but unbiased such that they cancel each other out.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] Page SE. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton, NJ: Princeton Univ Press; 2007. [Google Scholar]
Not checked in the free run.
- Premisecitation§ Abstract
The collective error is equal to the average individual error minus the group's diversity.
“This can be quantified by the “diversity prediction theorem” (9), which states that the collective error is equal to the average individual error minus the group's diversity (Materials and Methods).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] Page SE. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton, NJ: Princeton Univ Press; 2007. [Google Scholar]
Not checked in the free run.
- Classified as probably not a substantial claim. 18
- May combine two claims. 17
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Materials and MethodsIn our empirical case, the choice of the geometric mean is not in conflict with the median as appropriate measure, as can be seen by the similarity of the geometric mean and the median in Table 1.
“In our empirical case this is not in conflict with the choice of the geometric mean as can be seen by the similarity of the geometric mean and the median in Table 1.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingreasoning§ AcknowledgmentsThe collective error slightly declines under social influence, especially in the aggregated information condition.
“Note that the collective error slightly declines under social influence, especially in the aggregated information condition, which is partially supported by the significance tests (SI Appendix).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ AbstractSocial influence diminishes the diversity of the crowd without improving its collective error.
“The “social influence effect” diminishes the diversity of the crowd without improvements of its collective error.”
- Also stated in: § Abstract · plain
“The “social influence effect” describes the fact that social influence diminishes the diversity of the group without improvements in accuracy.”
Quote in the paperSubstantial claim- Also stated in: § Abstract · plain
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ AcknowledgmentsThe initial diversity seems higher in the no information condition.
“It deserves to be mentioned that the initial diversity seems to be higher in the no information condition.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ DiscussionSocial influence triggers the convergence of individual estimates and substantially reduces the diversity of the group without improving its accuracy.
“Our experimental results show that social influence triggers the convergence of individual estimates and substantially reduces the diversity of the group without improving its accuracy.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ ResultsThe wisdom of crowd effect exists with respect to the geometric mean but not with respect to the arithmetic mean.
“The wisdom of crowd effect exists with respect to the geometric mean but not with respect to the arithmetic mean”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ DiscussionThe statistical effects of undermining are less severe for easier questions and if individuals are more confident in their answers.
“Nevertheless, the statistical effects of undermining are less severe for easier questions and if individuals are more confident in their answers (SI Appendix).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ ResultsThe decline in the wisdom-of-crowd indicator under social influence is substantial and statistically significant for all questions.
“This effect is substantial and statistically significant for all questions, which is confirmed by the regression model in Fig. 2B.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ AcknowledgmentsThe discrepancy in initial variance is mainly caused by the questions about crime statistics and not about geographical facts.
“Interestingly, this discrepancy in initial variance is mainly caused by the questions about crime statistics and not about geographical facts.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ ResultsThe estimates of our type of questions are not normally distributed but right-skewed, with a majority of estimates low and a minority scattered in a fat right tail.
“This is because the estimates of our type of questions are not normally distributed but right-skewed. In other words, the majority of estimates are low and a minority of estimates are scattered in a fat right tail, as it is the case for log-normal distributions.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ ResultsIn the exemplary sessions, individuals become more confident in the full information condition and less confident in the control condition.
“It can be seen that individuals in these exemplary sessions become more confident in the full information condition and less confident in the control condition.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ ResultsIn the control condition without social influence, the effects of social influence on diversity and collective error are almost null.
“In the control condition without social influence, these effects are almost null.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoning§ AbstractThe reward structure introduced incentives only to find the truth and avoided incentives to conform with others.
“This reward structure introduced incentives only to find the truth and avoided incentives to conform with others.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoning§ AbstractThe experimental design put subjects into a situation in which they would try to get as close to the truth as possible by using their own knowledge and the estimates of others.
“Therefore, our experimental design put subjects into a situation in which they would try to get as close to the truth as possible by using their own knowledge and the estimates of others.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoning§ AbstractThe allocation of subjects to a random sequence of questions and information treatments ruled out order effects with regard to questions and treatments.
“The allocation of subjects to a random sequence of questions and information treatments ruled out order effects with regard to questions and treatments.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ ResultsSocial influence undermines the wisdom of crowds by boosting the subjective and decreasing the objective reliability of the crowd.
“The comparison of both illustrates that social influence undermines the wisdom of crowds by boosting the subjective and decreasing the objective reliability of the crowd.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisecitation§ Materials and MethodsThe diversity prediction theorem states that collective error plus group diversity equals the average individual error, which is the average of the squared deviations from the truth.
“The diversity prediction theorem (see ref. 9, p. 211, and ref. 35) states that collective error plus group diversity equals the average individual error, which is the average of the squared deviations from the truth”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [9] Page SE. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton, NJ: Princeton Univ Press; 2007. [Google Scholar]
Not checked in the free run.
- cites [35] Krogh A, Vedelsby J. Neural network ensembles, cross validation, and active learning. Adv Neural Inform Process Syst. 1995;7:231–238. [Google Scholar]
Not checked in the free run.
May combine two claims.
Interpretationreasoning§ DiscussionThe confidence gain happens despite a lack of improvements, giving evidence for a psychological trap whereby individuals are led into the false belief of collective accuracy as a result of their convergence.
“This confidence gain happens despite a lack of improvements, giving evidence for a psychological trap whereby individuals are led into the false belief of collective accuracy as a result of their convergence.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ AbstractThe wisdom of crowd effect was recently supported by examples from stock markets, political elections, and quiz shows.
“This wisdom of crowd effect was recently supported by examples from stock markets, political elections, and quiz shows [Surowiecki J (2004) The Wisdom of Crowds].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Surowiecki J (2004) The Wisdom of Crowds
Not checked in the free run.
May combine two claims.
Interpretationown data§ AbstractKnowledge about estimates of others narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd effect in three different ways.
“Although groups are initially “wise,” knowledge about estimates of others narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd effect in three different ways.”
Quote in the paperSubstantial claim- Rests on an assumption: The six factual questions and the student sample are representative of estimation tasks and crowds in general.
Classified as probably not a substantial claim.
Premisecitation§ AbstractSocial groups can be remarkably smart and knowledgeable when their averaged judgements are compared with the judgements of individuals.
“Social groups can be remarkably smart and knowledgeable when their averaged judgements are compared with the judgements of individuals.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Galton F (1907) Nature 75:7
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductreasoning§ AbstractUsing two different operationalizations of social influence demonstrates the robustness of the effects with regard to the specific kind of social influence.
“The reason to use two different kinds of social influence was to demonstrate the robustness of our effects with regard to the specific kind of social influence.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ Materials and MethodsThe geometric mean and the median coincide for a log-normal distribution.
“A theoretical reason is that the geometric mean and the median coincide for a log-normal distribution.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ AcknowledgmentsAs a consequence of right-skewed distributions, the arithmetic mean is usually much larger than most estimates and also much larger than the true value.
“As a consequence, the arithmetic mean is usually much larger than most estimates and also much larger than the true value.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Study design and conductreasoning§ AcknowledgmentsOur comparably mild and parsimonious kind of information feedback enables a particularly controlled experimental setting in which there is little ambiguity about which kind of information feedback and social influence played a role.
“In contrast to such possibilities, our comparably mild and parsimonious kind of information feedback has the advantage that it enables a particularly controlled experimental setting in which there is little ambiguity about which kind of information feedback and social influence played a role.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ Materials and MethodsA low collective error combined with a high group diversity implies that the wisdom-of-crowd effect works well, because asking many instead of one drastically improves accuracy.
“A low collective error combined with a high group diversity implies that the wisdom of crowd effect works well, because asking many instead of one drastically improves accuracy.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisereasoning§ AbstractThe wisdom of crowd effect is a statistical phenomenon and not a social psychological effect, because it is based on a mathematical aggregation of individual estimates.
“The wisdom of crowd effect is a statistical phenomenon and not a social psychological effect, because it is based on a mathematical aggregation of individual estimates.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ ResultsPrevious findings on effects of social influence in terms of gossip on behavior indicate the effect is stronger if gossip comes from fewer sources.
“This matches previous findings on the effects of social influence in terms of gossip on the behavior, which seems to be stronger if gossip comes from fewer sources (34).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [34] Sommerfeld RD, Krambeck HJ, Milinski M. Multiple gossip statements and their effect on reputation and trustworthiness. Proc Biol Sci. 2008;275:2529–2536. doi: 10.1098/rspb.2008.0762. [DOI] [PMC free article] [PubMed] [Google Scholar] source
Not checked in the free run.
May combine two claims.
Interpretationown data§ ResultsOpinion convergence boosts individuals' confidence in their estimates despite a lack of collective improvements in accuracy.
“The confidence effect reflects that opinion convergence boosts individuals’ confidence in their estimates despite a lack of collective improvements in accuracy.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisecitation§ AbstractGalton found evidence that the median estimate of a group can be more accurate than estimates of experts.
“Already Galton [Galton F (1907) Nature 75:7] found evidence that the median estimate of a group can be more accurate than estimates of experts.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Galton F (1907) Nature 75:7
Not checked in the free run.
May combine two claims.
Interpretationreasoning§ DiscussionThe wisdom-of-crowd effect is valuable for society, but using it multiple times creates collective overconfidence in possibly false beliefs.
“The wisdom of crowd effect is valuable for society, but using it multiple times creates collective overconfidence in possibly false beliefs.”
Quote in the paperSubstantial claim- Rests on an assumption: Groups can be remarkably accurate in estimating vaguely known facts based on the wisdom-of-crowd effect.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ Materials and MethodsThe order of the questions and treatments was randomized among the experimental sessions.
“The order of the questions and treatments was randomized among the experimental sessions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown data§ DiscussionThe negative effects of social influence occur especially in a certain range of question difficulty and individuals' confidence.
“This gives weight to the conclusion that the negative effects of social influence occur especially in a certain range of question difficulty and individuals’ confidence, a conjecture that should be explored in follow-up studies.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Social influence in laboratory estimation tasks with students generalizes to real-world crowds and societies.
“Examples of the revealed mechanism range from misled elites to the recent global financial crisis.”
“it can have a substantial impact on the statistical wisdom of crowd effect in societies.”
The paper relies on this without support.
The six factual questions and the student sample are representative of estimation tasks and crowds in general.
“The participants had to solve six different estimation tasks testing their real-world knowledge regarding geographical facts and crime statistics.”
“A total of 144 participants were recruited from a sample of more than 8000 students at Eidgenössische Technische Hochschule Zürich in Zurich, Switzerland.”
The paper relies on this without support.
It is hardly feasible to receive independent opinions in society because people are embedded in social networks and typically influence each other to a certain extent.
“However, it is hardly feasible to receive independent opinions in society, because people are embedded in social networks and typically influence each other to a certain extent.”
The paper relies on this without support.
In democratic societies, it is difficult to accomplish a collection of independent estimates because the loss of diversity in estimates appears to be a necessary byproduct of transparent decision-making processes.
“However, in democratic societies, it is difficult to accomplish such a collection of independent estimates, because the loss of diversity in estimates appears to be a necessary byproduct of transparent decision-making processes.”
The paper relies on this without support.
Opinion polls and mass media largely promote information feedback and therefore trigger convergence of how we judge the facts.
“For example, opinion polls and the mass media largely promote information feedback and therefore trigger convergence of how we judge the facts.”
The paper relies on this without support.
Groups can be remarkably accurate in estimating vaguely known facts based on the wisdom-of-crowd effect.
“Based on the wisdom of crowd effect, groups can be remarkably accurate in estimating vaguely known facts.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
As members see one another's views, the majority exerts pressure to conform, and the group converges on a shared answer without becoming more accurate.
“As members see one another's views, the majority exerts pressure to conform (Asch, 1956), and the group converges on a shared answer without becoming more accurate (Lorenz et al., 2011).”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Lorenz et al., 2011
After social influence, error distributions tend to become widespread, covering a greater proportion of high error values, regardless of the aggregating method.
“We find that the error distributions tend to become widespread, now covering a greater proportion of also high error values after social influence, regardless of the aggregating method.”
From Social Influence and the Collective Dynamics of Opinion Formation (2013) · cites this paper as [29]
The reliability of high confidence judgments is undermined by social influence.
“Taking the revised estimates of Experiment 2 into account, we observe that the reliability of high confidence judgments is undermined by social influence [29].”
From Social Influence and the Collective Dynamics of Opinion Formation (2013) · cites this paper as [29]
The presence of these elements confirms that the experimental design captured the fundamental mechanisms of social influence, and factual questions can be used to some extent to study fundamental features of opinion dynamics.
“The presence of these elements confirms that our experimental design has indeed captured the fundamental mechanisms of social influence, and that factual questions can be used, to some extent, to study the fundamental features of opinion dynamics [29].”
From Social Influence and the Collective Dynamics of Opinion Formation (2013) · cites this paper as [29]
The increase of confidence supports the emergence of basins of attraction during collective opinion dynamics by boosting the attractive power of individuals sharing a similar opinion.
“Second, the increase of confidence supports the emergence of basins of attraction during collective opinion dynamics by boosting the attractive power of individuals sharing a similar opinion [29].”
From Social Influence and the Collective Dynamics of Opinion Formation (2013) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Collective error is conflated with the log geometric mean
MinorThe stated definition makes collective error a squared deviation from the truth. The logarithm of the geometric mean is the average of the logarithms, not that squared deviation.
“For the case of logarithmically transformed data, the collective error of the logarithms is the logarithm of the geometric mean and one SD is the logarithm of the geometric SD.”
“The collective error is the squared deviation of the average from the truth”
Indicator change has the wrong direction
MinorThe paper says needing four rather than two central estimates to enclose the truth gives a lower wisdom-of-crowd indicator. Needing one additional estimate on each side therefore corresponds to a decrease, not an increase, of one unit.
“An increase of one unit means that one has to consider one additional person in the upper range and one additional person in the lower range of sorted estimates so that the truth is included in the selected range.”
“If the four most central values are needed to enclose the true value, the level of wisdom is considered to be lower”
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · see citing works
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650 · see citing works
z-Tree: Zurich toolbox for ready-made economic experiments
Fischbacher, U. (2007) · Experimental Economics · doi:10.1007/s10683-006-9159-4 · Cited by 9,524 · see citing works
A Simple Model of Herd Behavior
Banerjee, A. V. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118364 · Cited by 6,556 · see citing works
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1645 · Cited by 2,928 · see citing works
Log-normal distributions across the sciences: keys and clues
LIMPERT, E., STAHEL, W. A., & ABBT, M. (2001) · BioScience · doi:10.1641/0006-3568(2001)051[0341:lndats]2.0.co;2 · Cited by 2,621 · see citing works
Asch, S. E. (1955) · Scientific American · doi:10.1038/scientificamerican1155-31 · Cited by 2,364 · see citing works
The difference: how the power of diversity creates better groups, firms, schools, and societies
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1534 · Cited by 2,280 · see citing works
Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006) · Science · doi:10.1126/science.1121066 · Cited by 2,219 · see citing works
A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades
Bikhchandani, S., Hirshleifer, D. A., & Welch, I. (1992) · Journal of Political Economy · doi:10.1086/261849 · Cited by 1,980 · see citing works
Neural Network Ensembles, Cross Validation, and Active Learning
Krogh, A., & Vedelsby, J. (1994) · Cited by 1,838 · see citing works
Social comparison : contemporary theory and research
Suls, J. M., & Wills, T. A. (1991) · Cited by 1,381 · see citing works
The psychology of judgment and decision making
(1994) · Choice Reviews Online · doi:10.5860/choice.31-2965 · Cited by 1,256 · see citing works
GALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080 · see citing works
Naïve Learning in Social Networks and the Wisdom of Crowds
Golub, B., & Jackson, M. O. (2010) · American Economic Journal Microeconomics · doi:10.1257/mic.2.1.112 · Cited by 1,036 · see citing works
Combining Probability Distributions: A Critique and an Annotated Bibliography
Genest, C., & Zidek, J. V. (1986) · Statistical Science · doi:10.1214/ss/1177013825 · Cited by 1,000 · see citing works
The Network Structure of Exploration and Exploitation
Lazer, D., & Friedman, A. (2007) · Administrative Science Quarterly · doi:10.2189/asqu.52.4.667 · Cited by 696 · see citing works
Log or Linear? Distinct Intuitions of the Number Scale in Western and Amazonian Indigene Cultures
Dehaene, S., Véronique Izard, A., Spelke, E. S., & Pica, P. (2008) · Science · doi:10.1126/science.1156540 · Cited by 634 · see citing works
Intuitions about combining opinions: misappreciation of the averaging principle
Larrick, R. P., & Soll, J. B. (2006) · Management Science · doi:10.1287/mnsc.1050.0459 · Cited by 410 · see citing works
Coordination of Expectations in Asset Pricing Experiments
Hommes, C., Joep Sonnemans, A., Jan Tuinstra, B., & van de Velden, H. (2004) · Review of Financial Studies · doi:10.1093/rfs/hhi003 · Cited by 404 · see citing works
Vul, E., & Pashler, H. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02136.x · Cited by 340 · see citing works
Mason, W. A., Conrey, F. R., & Smith, E. R. (2007) · Personality and Social Psychology Review · doi:10.1177/1088868307301032 · Cited by 335 · see citing works
The Wisdom of Many in One Mind
Herzog, S. M., & Hertwig, R. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02271.x · Cited by 304 · see citing works
Lorge, I. D., Fox, D. J., Davitz, J. R., & Brenner, M. (1958) · Psychological Bulletin · doi:10.1037/h0042344 · Cited by 302 · see citing works
The False Consensus Effect and Overconfidence: Flaws in Judgment or Flaws in How We Study Judgment?
Dawes, R. M., & Mulford, M. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0020 · Cited by 246 · see citing works
Using advice from multiple sources to revise and improve judgments
Yaniv, I., & Milyavsky, M. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.05.006 · Cited by 244 · see citing works
When Social Comparison Goes Awry: The Case of Pluralistic Ignorance
Miller, D. T., & McFarland, C. (2024) · Routledge eBooks · doi:10.4324/9781003469490-14 · Cited by 237 · see citing works
Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision
Mannes, A. E. (2009) · Management Science · doi:10.1287/mnsc.1090.1031 · Cited by 166 · see citing works
Multiple gossip statements and their effect on reputation and trustworthiness
Sommerfeld, R. D., Krambeck, H.-J., & Milinski, M. (2008) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2008.0762 · Cited by 158 · see citing works
The discovery of pluralistic ignorance: An ironic lesson
O'Gorman, H. J. (1986) · Journal of the History of the Behavioral Sciences · doi:10.1002/1520-6696(198610)22:4 · Cited by 144 · see citing works
Goldstone, R. L., & Gureckis, T. M. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2009.01038.x · Cited by 126 · see citing works
Rauhut, H., & Lorenz, J. (2010) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2010.10.002 · Cited by 89 · see citing works
Self-citationCoherent combination of experts' opinions
Dawid, A. P., DeGroot, M. H., Mortera, J., Cooke, R. M., French, S., Genest, C., Schervish, M. J., Lindley, D. V., McConway, K., & Winkler, R. L. (1995) · Test · doi:10.1007/bf02562628 · Cited by 87 · see citing works
Eliminating Public Knowledge Biases in Information-Aggregation Mechanisms
Chen, K., Fine, L. R., & Huberman, B. A. (2004) · Management Science · doi:10.1287/mnsc.1040.0247 · Cited by 63 · see citing works
The study of the undivided personality.
Āllport, G. W. (1924) · The Journal of Abnormal Psychology and Social Psychology · doi:10.1037/h0064744 · Cited by 62 · see citing works
Galton, F. (1907) · Nature · doi:10.1038/075509f0 · Cited by 43 · see citing works
Hooker, R. H. (1907) · Nature · doi:10.1038/075487a0 · Cited by 18 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- J Lorenz Repeated averaging and bounded confidence – modeling, analysis and simulation of continuous opinion dynamics. PhD thesis (Universität Bremen, Bremen, Germany, 2007).
Cited by
Sorted by citations
Social Influence Bias: A Randomized Experiment
Muchnik, L., Aral, S., & Taylor, S. J. (2013) · Science · doi:10.1126/science.1240466 · Cited by 871 · see citing works
Using the Delphi expert consensus method in mental health research
Jorm, A. F. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415600891 · Cited by 765 · see citing works
Social Influence and the Collective Dynamics of Opinion Formation
Moussaïd, M., Kämmer, J. E., Analytis, P. P., & Neth, H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078433 · Cited by 407 · see citing works
Network dynamics of social influence in the wisdom of crowds
Becker, J., Brackbill, D., & Centola, D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1615978114 · Cited by 397 · see citing works
A solution to the single-question crowd wisdom problem
Prelec, D., Seung, H. S., & McCoy, J. (2017) · Nature · doi:10.1038/nature21054 · Cited by 314 · see citing works
Human–machine partnership with artificial intelligence for chest radiograph diagnosis
Patel, B. N., Rosenberg, L., Willcox, G., Baltaxe, D., Lyons, M., Irvin, J., Rajpurkar, P., Amrhein, T., Gupta, R., Halabi, S., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0189-7 · Cited by 254 · see citing works
How algorithmic popularity bias hinders or promotes quality
Ciampaglia, G. L., Nematzadeh, A., Menczer, F., & Flammini, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-34203-2 · Cited by 204 · see citing works
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, J. F., Jia, M. (., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., et al. (2020) · Psychological Bulletin · doi:10.1037/bul0000220 · Cited by 168 · see citing works
Reciprocity and Unveiling in Two-sided Reputation Systems: Evidence from an Experiment on Airbnb
Fradkin, A., Grewal, E., & Holtz, D. (2021) · Marketing Science · doi:10.1287/mksc.2021.1311 · Cited by 113 · see citing works
Optimal incentives for collective intelligence
Mann, R. P., & Helbing, D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618722114 · Cited by 112 · see citing works
Self-citationLeveraging position bias to improve peer recommendation
Lerman, K., & Hogg, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098914 · Cited by 96 · see citing works
Is the true “wisdom of the crowd” to copy successful individuals?
King, A. J., Cheng, L., Starke, S. D., & Myatt, J. P. (2012) · Biology Letters · doi:10.1098/rsbl.2011.0795 · Cited by 61 · see citing works
Does My Rebuttal Matter? Insights from a Major NLP Conference
Gao, Y., Eger, S., Kuznetsov, I., Gurevych, I., & Miyao, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1129 · Cited by 48 · see citing works
Van Oerle, S., Lievens, A., & Mahr, D. (2018) · Psychology & Marketing · doi:10.1002/mar.21111 · Cited by 43 · see citing works
Prediction of robust scientific facts from literature
Belikov, A. V., Rzhetsky, A., & Evans, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00474-8 · Cited by 30 · see citing works
Quantifying machine influence over human forecasters
Abeliuk, A., Benjamin, D. M., Morstatter, F., & Galstyan, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72690-4 · Cited by 18 · see citing works
Excess reciprocity distorts reputation in online social networks
Livan, G., Caccioli, F., & Aste, T. (2017) · Scientific Reports · doi:10.1038/s41598-017-03481-7 · Cited by 20 · see citing works
Social cues influence perception of others' pain
Zhang, L., Wager, T. D., & Koban, L. (2024) · European Journal of Pain · doi:10.1002/ejp.2225 · Cited by 7 · see citing works
Multi-agent discussion gains less when dissent is withheld
Mansuri, C. S., Wang, X., Li, M., Acton, B., Eckardt, R., Patel, D., & Kojaku, S. (2026) · arXiv · doi:10.48550/arxiv.2609.38324
“As members see one another’s views, the majority exerts pressure to conform (Asch, 1956), and the group converges on a shared answer without becoming more accurate (Lorenz et al., 2011).”
- Authors
- Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D.
- Affiliations
- ETH Zurich (CH)
- Santa Fe Institute (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2011-05-16
- Volume
- 108
- Issue
- 22
- Pages
- 9020-9025
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC3107299
- OpenAlex
- W1993865115
- Semantic Scholar
- 8108f3c5332ea5aedaf3230c4388d9f50db66e41
- Cited by
- 1,017
- References
- 37
- Citations by year
- 2026: 59 · 2025: 57 · 2024: 61 · 2023: 74 · 2022: 72 · 2021: 78 · 2020: 80 · 2019: 83 · 2018: 71 · 2017: 98 · 2016: 55 · 2015: 65 · 2014: 63 · 2013: 48 · 2012: 39
- FWCI
- 28.37
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- wisdom of crowds, social influence, collective judgment, estimation accuracy, opinion diversity, convergence of estimates
- MeSH terms
- Commerce, Game Theory, Humans, Intelligence, Judgment, Politics, Regression Analysis, Research Design, Social Behavior, Switzerland, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review