Threatening communication: a critical re-analysis and a revised meta-analytic test of fear appeal theory

May combine two claims.

Nine studies were ultimately included: seven provided effect sizes for the main effects of threat and efficacy, their interaction, and simple effects, while two provided only main effects.

“Thus, in the end, nine studies were included, seven of which provided effect sizes for the main effects of threat and efficacy, the interaction between those two, and the resulting simple effects; and two of which only provided effect sizes for the main effects (see Table 1).”

May combine two claims.

Study #43 used a very weak behavioural measure: high school students reported willingness to take roundworm medication, with willingness measured immediately after the manipulation.

“The only study with a very weak behavioural measure was study #43, where high school students indicated their willingness to take medication for roundworm, with the understanding that willingness would mean they would indeed be taking the medication. This willingness was measured immediately after the manipulation (i.e., using a follow-up of 0 weeks).”

May combine two claims.

The Q statistics for the threat main effect and for threat under low efficacy were close to significance, with Q(8)=14.66, p=0.066 and Q(6)=10.66, p=0.099.

“Both corresponding Qs almost achieved significance [Q(8) = 14.66, p = 0.066 and Q(6) = 10.66, p = 0.099].”

May combine two claims.

Efficacy had no effect when threat was low (d=0.07, p=0.689) but did have an effect when threat was high (d=0.71, p<0.0001).

“and there was no effect of efficacy when threat was low (d = 0.07, p = 0.689), but there was an effect when threat was high (d = 0.71, p <0.0001).”

May combine two claims.

A number of included studies did not report data that allowed effect size computation; when authors were contacted, the data had been lost in all cases where they replied, with two exceptions.

“While extracting data, it became clear that a number of included studies did not report data that allowed effect size computation (e.g., only significant statistics from multivariate analyses of variance were reported).”

• Also stated in: § Data extraction · plain

  “In all cases where authors replied, the data had been lost, with two exceptions.”

May combine two claims.

Among 35 exploratory moderation analyses, the only moderator reaching p<0.05 was low quality of the behavioural measure, which predicted the threat main effect (p=0.0065) and the effect of threat under high efficacy (p=0.0171).

“The only moderator that emerged was the quality of the behavioural measure, specifically, low quality (this was the only moderator to reach p <0.05: specifically, p = 0.0065 in the prediction of the threat main effect, and p = 0.0171 in the prediction of the effect of threat under high efficacy).”

• AssumptionRests on an assumption: The quality ratings used for the behavioural measure and the two manipulations validly capture the quality of those study features.

May combine two claims.

The distribution of correlations between effect size and sample size and methodological quality measures was symmetrical, giving no evidence of publication bias.

“This distribution appeared symmetrical, providing no evidence of publication bias”

May combine two claims.

Among 10 publication-bias tests, only the test for the efficacy main effect was significant (z=2.07, p=0.039), and only in the random-effects model that included study #43.

“Of these 10 significance tests, only the test for the main effect of efficacy was significant (z = 2.07, p = 0.039), and only for the random effects model including study #43.”

May combine two claims.

Threat had no effect when efficacy was low (d=-0.31, p=0.061) but did have an effect when efficacy was high (d=0.31, p=0.036).

“There was no effect of threat when efficacy was low (d = −0.31, p = 0.061), but there was an effect when efficacy was high (d = 0.31, p = 0.036)”

Probably not a substantial claim: may not pass S3 (load-bearing).

The database queries yielded only three additional studies.

“as these queries yielded only three additional studies (Grala & McCauley, 1976; Muthusamy, Levine, & Weber, 2009; Ordoñana, González-Javier, Espín-López, & Gómez-Amor, 2009)”

May combine two claims.

After excluding study #43, heterogeneity fell substantially, with at most 29% of variance explained by between-study differences (only for the efficacy main effect), and no effect showed heterogeneity approaching significance.

“Heterogeneity was considerably reduced, with at most 29% of the heterogeneity explained by between-study differences (only for the main effect of efficacy). Heterogeneity did not approach significance for any effect.”

May combine two claims.

The fixed-effects model excluding study #43 produced the same outcomes as the random-effects model excluding study #43, and the authors based their conclusions on those outcomes.

“The outcomes were identical to the random effects model excluding study #43, and these are the outcomes that we will base our conclusions on.”

May combine two claims.

The effect of threat under low efficacy was negative and borderline significant.

“Interestingly, the effect of threat under low efficacy is borderline significant and negative”

• Also stated in: § Discussion · hedged

  “A somewhat worrying finding of this meta-analysis was that under low efficacy, the effect of threat was negative and almost significant.”

May combine two claims.

In the meta-analysis of the six studies that met the inclusion criteria, threat and efficacy interacted significantly: threat affected behaviour only when efficacy was high (d = 0.31), and efficacy affected behaviour only when threat was high (d = 0.71).

“Meta-analysis of the six studies that satisfied the inclusion criteria clearly showed a significant interaction between threat and efficacy, such that threat only had an effect under high efficacy (d = 0.31), and efficacy only had an effect under high threat (d = 0.71).”

• Also stated in: § Abstract · boosted

  “Proper tests of fear appeal theory yielded the theoretically hypothesised interaction effect.”

Classified as probably not a substantial claim.

The efficacy manipulations in the included studies contained at least a paragraph of text.

“The efficacy manipulations reported in the studies included in this meta-analysis entailed at least a paragraph of text”

May combine two claims.

The results matched theoretical predictions: threat and efficacy interact on behaviour, with threat affecting behaviour only when efficacy is high and efficacy affecting behaviour only when threat is high.

“The results are in line with the theoretical predictions. Threat and efficacy interact in their effects on behaviour, such that threat only has an effect if efficacy is high, and efficacy only has an effect is threat is high.”

Classified as probably not a substantial claim. May combine two claims.

When threatening information is given to people who derive self-esteem from a risk behaviour, their intention to engage in that risk behaviour decreases while their actual risk behaviour increases.

“it has been shown that when threatening information is presented to persons deriving self-esteem from a risk behaviour, their intention to engage in the risk behaviour decreases, but they simultaneously engage in more risk behaviour (Ben-Ari, Florian, & Mikulincer, 2000).”

• cites Ben-Ari, Florian, & Mikulincer, 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The variation in designs, measures, analyses, and the scarcity of reported information that allowed straightforward effect size computation necessitated examination of all original texts by one author (GJP).

“However, the variation in designs, used measures and analyses, and the scarcity of reported information that allowed straightforward computation of effect sizes, eventually necessitated examination of all original texts by one of the authors (GJP).”

May combine two claims.

Null findings have a low chance of being submitted and, if submitted, a low chance of being published.

“Thus, when a study does not reject the null hypothesis, it has a low probability of being submitted, and even if it is submitted, it has a low probability of being published.”

• cites Greenwald, 1975Not checked yet

  Not checked in the free run.

May combine two claims.

There is some evidence of publication bias for the efficacy main effect, which may mean that the estimate is biased and the true effect is likely smaller.

“Thus, there is some evidence of publication bias regarding the main effect of efficacy, which may mean that this estimate is biased (i.e., the true effect size is likely lower).”

May combine two claims.

Even if those hypotheses were wrong, the current meta-analysis resolves the controversy about the outcomes of earlier fear appeal meta-analyses.

“However, regardless of whether these hypotheses were correct, the current meta-analysis does resolve the controversy regarding the outcomes of previous fear appeal meta-analyses.”

Classified as probably not a substantial claim.

Whether a person changes behaviour or reacts defensively depends on moderators including efficacy and self-esteem.

“Whether one engages in behaviour change or defensive reactions depends on a number of moderators, one of which is again efficacy. Another moderator is self-esteem”

• cites Goldenberg & Arndt, 2008Not checked yet

  Not checked in the free run.

• cites Hansen, Winzeler, & Topolinski, 2010Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Requests for information or registrations for sessions were not accepted as behavioural outcomes because such behaviours are predicted by different determinants from target behaviours.

“Requests for information or registrations for sessions, albeit technically behavioural measures, were not accepted, as such behaviours are predicted by very different determinants from target behaviours (e.g., whereas self-efficacy for requesting information about quitting smoking can be assumed to be very high, self-efficacy for actually quitting is likely lower).”

May combine two claims.

The earlier decision to treat study #43 as having a valid behavioural outcome was mistaken, and the study should have been excluded.

“In retrospect, therefore, the judgement call that this study used a valid behavioural outcome measure appears to have been erroneous, and the study should not have been included.”

Classified as probably not a substantial claim.

Fear appeal theory predicts that fear appeals work only when both efficacy and threat are sufficiently high.

“Theory predicts that fear appeals exclusively work when both efficacy and threat are sufficiently high.”

• cites Witte, 1992Not checked yet

  Not checked in the free run.

• cites de Hoog et al., 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors followed the QUOROM guidelines as far as they could be applied to a psychological experiment context.

“In the description of the Methods section, we followed the QUOROM guidelines (Moher et al., 2000) as much as possible (where these could be translated to a psychological experiment context, rather than a clinical intervention context).”

• cites Moher et al., 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

For studies included from the first query, data were extracted through independent coding by RR and GK using a coding form.

“For the studies that were included on the basis of the first query, data were extracted through independent coding by RR and GK, using a coding form (available at http://fearappeals.com/meta-analysis/).”

May combine two claims.

When a study finds an effect of threatening communication on intention but does not measure behaviour, the fear appeal may be unrelated to, or may even reduce, the desirable behaviour.

“Therefore, when a study shows an effect of threatening communication on intention, but fails to measure behaviour, it is possible that the fear appeal is not associated to, or even decreases, the desirable behaviour.”

Classified as probably not a substantial claim.

Albarracín et al. (2005) found that no threat-inducing argument had any positive behavioural effect.

“One meta-analysis compared different behaviour change methods (or techniques) and found that ‘no threat-inducing argument had any positive behavioural effect whatsoever’ (Albarracín et al., 2005, p. 882).”

• cites Albarracín et al., 2005Not checked yet

  Not checked in the free run.

May combine two claims.

Earl and Albarracín (2007) found that inducing fear is not an effective way to promote HIV-relevant learning or condom use, either immediately or later.

“Similarly, as another meta-analysis into condom use, where long-term effects of fear appeals were studied, ‘[…] clearly shows, inducing fear is not an effective way to promote HIV-relevant learning or condom use either immediately following the intervention or later on’ (Earl & Albarracín, 2007, p. 504).”

• cites Earl & Albarracín, 2007Not checked yet

  Not checked in the free run.

May combine two claims.

Because the interaction was significant, the main effects of threat (d=0.11, p=0.215) and efficacy (d=0.49, p<0.0001) cannot be meaningfully interpreted.

“which means that the effect size and significance levels for the main effects of threat (d = 0.11, p = 0.215) and efficacy (d = 0.49, p <0.0001) have no meaning (i.e., there exists no effect of threat, as the effect depends on the value of efficacy, and vice versa).”

May combine two claims.

Study #43 differed from the other studies because the target behaviour was extremely easy to perform, with medication provided, and because willingness was assessed immediately after the manipulation.

“Thus, study #43 differed from the other studies in two important respects: first, the influenced behaviour was a behaviour that was extremely easy to perform, with medication simply provided to the participants (i.e., it was likely that baseline efficacy is high); and second, behaviour (i.e., willingness) was measured immediately after the manipulation.”

Classified as probably not a substantial claim. May combine two claims.

Because the review tests the EPPM, which concerns threat to individuals themselves, the behaviour and threat had to concern participants directly.

“In addition, because the review is a test of the EPPM which deals with threat to individuals themselves, the behaviour and threat had to concern the participants themselves directly.”

• cites Witte, 1992Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Manipulation quality was coded independently by GK and RR from extracts of the relevant methods fragments.

“Manipulation quality was coded independently by GK and RR on the basis of extracts of the relevant fragments from the methods section.”

May combine two claims.

Baseline levels of efficacy and threat vary between individuals and across populations for a given behaviour.

“Base levels of efficacy and threat, however, vary between individuals.”

• Also stated in: § Fear appeal theory and significant outcome bias (SOB) · plain

  “Of course, these baseline efficacy and threat values also differ per population.”

• cites Rimal & Juon, 2010Not checked yet

  Not checked in the free run.

• cites Rimal & Real, 2003Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In acknowledgement of concerns about limited disclosure of analytic decisions, the authors made everything used for their analyses available online.

“Note that in acknowledgement of the concerns of Simmons, Nelson, and Simonsohn (2011), who explain how limited disclosure of the decisions made in the analysis process can endanger the integrity of science, we have made everything we used for our analyses available through http://fearappeals.com/meta-analysis/.”

• cites Simmons, Nelson, and Simonsohn (2011)Not checked yet

  Not checked in the free run.

May combine two claims.

A review found that threat components (severity and susceptibility) significantly predict intention but are not associated with subsequent behaviour.

“a review that showed that although threat components (severity, susceptibility) significantly predict intention, these same components are not associated to subsequent behaviour (Milne, Sheeran, & Orbell, 2000).”

• cites Milne, Sheeran, & Orbell, 2000Not checked yet

  Not checked in the free run.

Not yet classified.

The terror management health model postulates that activating death-related cognitions triggers attempts to remove them from consciousness, leading either to behaviour change to reduce the actual threat or to defensive reactions that reduce the salience of the threatening cognitions.

“This model postulates that activation of death-related cognitions triggers attempts to jettison these cognitions from consciousness: behaviour change to reduce the actual threat and/or defensive reactions to reduce the salience of the threatening cognitions.”

• cites Goldenberg & Arndt, 2008Not checked yet

  Not checked in the free run.

• cites Hansen, Winzeler, & Topolinski, 2010Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Intention typically predicts about one-third of behaviour.

“intention typically predicts around one-third of behaviour; Cooke & Sheeran, 2004”

• cites Cooke & Sheeran, 2004Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The reasoned action approach predicts that a threatening communication raises perceived threat, which raises attitude, which raises intention, but does not raise behaviour unless self-efficacy is high.

“The reasoned action approach predicts that a threatening communication increases perceived threat, which increases attitude, which increases intention, which does not increase behaviour, unless self-efficacy is high (Fishbein & Ajzen, 2010, p. 64).”

• cites Fishbein & Ajzen, 2010Not checked yet

  Not checked in the free run.

May combine two claims.

Studies that inspired strong recommendations for threatening information, such as Hammond et al. (2004) and Borland et al. (2009), lacked experimental designs and therefore cannot support causal conclusions.

“These studies, for example, had no experimental designs, and therefore, allow no causal conclusions (Wilkinson et al., 1999).”

• cites Wilkinson et al., 1999Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The meta-analysis signals that recommendations to widely apply threatening communications should stop, because few studies could have supported their use and existing studies do not support wide application.

“The current meta-analysis strongly signals that these recommendations should cease as soon as possible, because (1) there are very few studies that could theoretically have supported the use of threatening communications; and (2) those studies that do exist do not support the wide application of threatening communications.”

May combine two claims.

Warning labels on cigarette packs seem ill-advised and may increase smoking among smokers who derive self-esteem from being a smoker.

“In conclusion, warning labels on packs of cigarettes seem ill-advised. They may in fact increase smoking among smokers who derive self-esteem from their identity as a smoker.”

Probably not a substantial claim: may not pass S2 (checkable).

Whatever the scenario, the conclusion remains that threatening communication should be avoided.

“Note that regardless of which scenario is true, the conclusion remains the same: threatening communication is better avoided.”

May combine two claims.

Although threatening communications may increase recognition, they either promote health-defeating behaviour or have no effect.

“The current metaanalysis clearly indicates, however, this although such communications may increase recognition, they either enhance health-defeating behaviour or have no effect.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The small number of studies remains a shortcoming, and assessing publication bias with only six studies is very difficult.

“Still, this paucity of research remains a shortcoming of the current meta-analysis, also because it is very difficult to assess the extent of publication bias with only six studies.”

May combine two claims.

Threatening communication works only when baseline efficacy is high for the behaviour-population combination or when the intervention includes a potent efficacy-enhancing element.

“Threatening communication only works when either the baseline efficacy is high in the relevant behaviour-population combination, or when the intervention also includes a potent efficacy-enhancing element.”

May combine two claims.

Simple recommendations at best enhance response efficacy, but without high self-efficacy threatening communication will have no effect or backfire.

“Mere recommendations at best enhance response efficacy, but without high self-efficacy, threatening communication will have no effect, or worse, backfire.”

May combine two claims.

Latour et al. (1996) found higher threat positively associated with behaviour, but the behaviour was buying a stun gun, a behaviour for which most people likely have high efficacy.

“For example, Latour, Snipes, and Bliss (1996) report the results on an experiment where higher threat was positively associated to behaviour. However, this behaviour was the purchase of a stun gun, which participants were informed was for sale in the area for $60–$80: an excellent example of a behaviour for which most people will have a high efficacy.”

• cites Latour, Snipes, and Bliss (1996)Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: Buying a stun gun is a behaviour for which most people have high efficacy.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

If Type 1 errors occurred, threats are ineffective; if Type 2 errors occurred, threats are potentially harmful.

“After all, if we made Type 1 errors, threats are ineffective; and if we made Type 2 errors, threats are potentially harmful.”

Classified as probably not a substantial claim. May combine two claims.

The significant efficacy main effect and the strong effect of efficacy under high threat suggest that focusing on efficacy is the better option.

“in fact, the significant main effect of efficacy and the strong effect of efficacy under high threat that were found in the current metaanalysis suggest that a focus on efficacy is a better bet in any case”

May combine two claims.

The inclusion criteria also caused the weakness that only eight studies entered the main-effects meta-analysis and only six entered the interaction and simple-effects meta-analysis after excluding study #43.

“Unfortunately, these inclusion criteria also caused the weakness of this meta-analysis: only eight studies were included in the meta-analysis of the main effects, and only six studies in the meta-analysis of the interaction effect and the simple effects (excluding study #43).”

Classified as probably not a substantial claim. May combine two claims.

Given the minimal or negative effects expected from threatening communication, evidence- and theory-based communications on cigarette pack labels have promising potential.

“Given the minimal, or even negative, effects we can expect from threatening communication, the potential of evidence- and theory-based communications on cigarette pack labels is promising.”

May combine two claims.

Threatening communication is at best ineffective and at worst causes health-defeating behaviour, unless the intervention effectively enhances response efficacy and especially self-efficacy.

“Instead, they indicate that using threatening communication is at best ineffective, and at worst causes health-defeating behaviour, unless the intervention contains an element that effectively enhances response efficacy and (especially) self-efficacy.”

Classified as probably not a substantial claim.

The terror management health model proposes that threatening information can lead people to engage in health-defeating behaviour.

“This is in line with the terror management health model (Goldenberg & Arndt, 2008), which suggests that threatening information can cause people to engage in health-defeating behaviour.”

• cites Goldenberg & Arndt, 2008Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The main strength of the meta-analysis is its stringent inclusion criteria.

“The main strength of the current meta-analysis is formed by the stringent inclusion criteria.”