CoChat
Sign Up
Cited by 113Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsMobile Learning in Education

The Comparative Effectiveness of Mobile Phone Interventions in Improving Health Outcomes: Meta-Analytic Review

Open access
Yang, Qinghua, Van Stee, Stephanie K·2019·JMIR mHealth and uHealth
DOI 10.2196/11244PMC PMC6468337

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: As mobile technology continues expanding, researchers have been using mobile phones to conduct health interventions (mobile health-mHealth-interventions). The multiple features of mobile phones offer great opportunities to disseminate large-scale, cost-efficient, and tailored messages to participants. However, the interventions to date have shown mixed results, with a large variance of effect sizes (Cohen d=-0.62 to 1.65). OBJECTIVE: The study aimed to generate cumulative knowledge that informs mHealth intervention research. The aims were twofold: (1) to calculate an overall effect magnitude for mHealth interventions compared with alternative interventions or conditions, and (2) to analyze potential moderators of mHealth interventions' comparative efficacy. METHODS: Comprehensive searches of the Communication & Mass Media Complete, PsycINFO, Web of Knowledge, Academic Search Premier, PubMed and MEDLINE databases were conducted to identify potentially eligible studies in peer-reviewed journals, conference proceedings, and dissertations and theses. Search queries were formulated using a combination of search terms: "intervention" (Title or Abstract) AND "health" (Title or Abstract) AND "*phone*" OR "black-berr*" (OR mHealth OR "application*" OR app* OR mobile OR cellular OR "short messag*" OR palm* OR iPhone* OR MP3* OR MP4* OR iPod*) (Title or Abstract). Cohen d was computed as the basic unit of analysis, and the variance-weighted analysis was implemented to compute the overall effect size under a random-effects model. Analysis of variance-like and meta-regression models were conducted to analyze categorical and continuous moderators, respectively. RESULTS: The search resulted in 3424 potential studies, the abstracts (and full text, as necessary) of which were reviewed for relevance. Studies were screened in multiple stages using explicit inclusion and exclusion criteria, and citations were evaluated for inclusion of qualified studies. A total of 64 studies were included in the current meta-analysis. Results showed that mHealth interventions are relatively more effective than comparison interventions or conditions, with a small but significant overall weighted effect size (Cohen d=0.31). In addition, the effects of interventions are moderated by theoretical paradigm, 3 engagement types (ie, changing personal environment, reinforcement tracking, social presentation), mobile use type, intervention channel, and length of follow-up. CONCLUSIONS: To the best of our knowledge, this is the most comprehensive meta-analysis to date that examined the overall effectiveness of mHealth interventions across health topics and is the first study that statistically tested moderators. Our findings not only shed light on intervention design using mobile phones, but also provide new directions for research in health communication and promotion using new media. Future research scholarship is needed to examine the effectiveness of mHealth interventions across various health issues, especially those that have not yet been investigated (eg, substance use, sexual health), engaging participants using social features on mobile phones, and designing tailored mHealth interventions for diverse subpopulations to maximize effects.

AI snapshot

Written from the paper’s full text.

TL;DR

This meta-analysis of 64 studies found that mobile health (mHealth) interventions were significantly more effective than comparison conditions, with a small overall weighted effect size (Cohen d=0.31). Effects were moderated by theoretical paradigm, engagement types, mobile use type, intervention channel, and length of follow-up.

Key findings

  • Across 64 studies, 142 effect sizes, and 10,296 participants, mHealth interventions had a small but significant overall weighted effect size versus comparison conditions: Cohen d=0.31 (95% CI 0.25–0.36, P<.001), with large between-study heterogeneity (I²=71.87%).
  • Interventions based on combined cognitive and behavioral theories had the highest weighted mean effect (d=0.45), significantly higher than no theory (d=0.28), behavioral theory only (d=0.23), or cognitive theory only (d=0.20).
  • Engagement features mattered: changing personal environment (d=0.76 vs d=0.29 without) and reinforcement tracking (d=0.43 vs d=0.24 without) were associated with larger effects, while social presentation/announcement was associated with smaller effects (d=0.04 vs d=0.33 without).
  • Combining SMS and mobile apps produced a larger effect (d=0.59) than SMS only (d=0.30) or apps only (d=0.22); mobile phone plus another media type (d=0.39) outperformed mobile phone only (d=0.19) or mobile phone plus face-to-face communication (d=0.18). Health topic, control group design, participant age, gender, health condition, and intervention length were not significant moderators.

Methods

  • Comprehensive searches of Communication & Mass Media Complete, PsycINFO, Web of Knowledge, Academic Search Premier, PubMed, and MEDLINE identified studies in peer-reviewed journals, conference proceedings, and dissertations/theses published through December 31, 2017; 64 studies met inclusion criteria, yielding 142 effect sizes and 10,296 participants.
  • Cohen d was the basic unit of analysis; variance-weighted analysis used a random-effects model after significant heterogeneity (Qtotal(df=141)=467.01, P<.001).
  • Categorical moderators were tested with ANOVA-like mixed-effects models and continuous moderators with meta-regression; publication bias was assessed via funnel plot, Rosenthal’s Fail-safe N, and Duval and Tweedie’s Trim and Fill.

Limitations

  • The paper states that several specific categories had small sample sizes: no included studies focused on tobacco use, sexual health, or HIV/AIDS; only 4 studies focused on health-related quality of life, and social presentation and social referencing engagement types were each applied in only 4 studies, limiting reliability and increasing chance differences.
  • The paper states that including conference papers and unpublished work made the study vulnerable to lower-quality studies, though only 2 included studies contributing 5 effect sizes were dissertations and all others were peer-reviewed journal articles.
  • The paper states that the meta-analysis is susceptible to the “apples and oranges” critique regarding comparability, and some specific media categories were grouped into larger categories for moderator analyses because of small sample sizes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/11244

Cites

108 of 108 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 40 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Sampling error variance accounted for only 18.91% of total variance, suggesting the presence of a moderator.

    “Sampling error variance (Se 2=.0135) only accounted for 18.91% of the total variance (S2=.0714), suggesting the presence of a moderator.”
    Quote in the paperSubstantial claim
  • Findingown data

    Theoretical paradigm significantly moderated the effects of mHealth interventions (Qbetween(3)=11.01, P=.01).

    “Under MEM, theoretical paradigm was significant as a moderator (Qbetween (df=3)=11.01, P=.01).”
    Quote in the paperSubstantial claim
  • Findingown data

    There was large between-study heterogeneity (I2=71.87%; Birge ratio=3.31).

    “I2 , an index representing the ratio of true heterogeneity to total variance across observed ESs, is 71.87%, indicating large between-study variance [102]. Similarly, Birge ratio, another index to quantify the magnitude of heterogeneity (computed as Q /df=467.01/141=3.31), is larger than 1 (the ratio when all the variance comes from sampling error), indicating large between-study heterogeneity.”
    Quote in the paperSubstantial claim
  • Findingown data

    The effect sizes across included studies were not homogeneous (Qtotal(141)=467.01, P<.001).

    “Estimated under the FEM, the Q statistic was significant (Qtotal (df=141)=467.01, P<.001), indicating that the ESs were not homogeneous, and mean ES was estimated under the REM using Restricted Maximum Likelihood Estimation method.”
    Quote in the paperSubstantial claim
  • Findingnone given

    This study is the most comprehensive meta-analysis to date that examined the overall effectiveness of mobile phone interventions across health topics.

    “To the best of our knowledge, this is the most comprehensive meta-analysis to date that examined the overall effectiveness of mobile phone interventions across health topics”
    Quote in the paperSubstantial claim
  • May combine two claims. 13
  • Classified as probably not a substantial claim. 10

  • May combine two claims.

    Findingown data

    mHealth interventions are relatively more effective than comparison interventions or conditions, with a small but significant overall weighted effect size (Cohen d=0.31, 95% CI 0.25-0.36).

    “Results showed that mHealth interventions are relatively more effective than comparison interventions or conditions, with a small but significant overall weighted effect size (Cohen d=0.31).”
    • Also stated in: § Overall Analysis · plain
      “Under the REM, the sample weighted mean for standardized mean difference was 0.31 (95% CIs; 0.25, 0.36), which is a small ES [101], but statistically significant (P<.001).”
    • Also stated in: § Overall Effects · hedged
      “Findings from this meta-analysis indicated that mHealth interventions are significantly more effective than comparison conditions at improving health outcomes (d=0.31; 95% CIs 0.25, 0. 36)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The weighted mean effect size was significant immediately after the intervention (d=0.27) and increased by 0.015 for each additional week of follow-up (P<.001).

    “The weighted mean ES was significant immediately after the intervention (d=0.27, P<.001), and increased by .015 for each additional week of follow-up (P<.001).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    mHealth interventions combining SMS and mobile apps had a weighted mean effect size (d=0.59) significantly higher than those using only SMS (d=0.30) or mobile apps (d=0.22).

    “Post hoc analysis indicated that the weighted mean ES of interventions combining SMS and mobile apps (d=0.59; SE=.12; 95% CIs 0.36, 0.83; P<.001) was significantly higher than the ESs of those using only SMS (d=0.30; SE=0.04; 95% CIs 0.23, 0.38; P<.001) or mobile apps (d=0.22; SE=.03; 95% CIs 0.16, 0.29; P<.001; ZSMS =3.37, ZApp =4.16).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    mHealth interventions combining mobile phone with another type of media had a weighted mean effect size (d=0.39) significantly higher than those using mobile phone only (d=0.19) or mobile phone combined with face-to-face communication (d=0.18).

    “Pairwise comparison indicated that the weighted mean ES of interventions combining mobile phone with another type of media (d=0.39; SE=.05 95% CIs 0.30, 0.49; P<.001) was significantly higher than that of interventions using mobile phone only (d=0.19; SE=.04; 95% CIs 0.12, 0.26; P<.001; z=2.63) or interventions combining mobile phone with face-to-face communication (d=0.18; SE=.04; K=14; 95% CIs 0”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    mHealth interventions based on cognitive and behavioral theories combined had a significantly higher weighted mean effect size than interventions not indicating a theory (d=0.28), behavioral theory (d=0.23), or cognitive theory only (d=0.20).

    “and was significantly higher than interventions not indicating a theory (d=0.28, P<.001) or interventions applying behavioral (d=0.23, P<.001) both at .05 level or cognitive theory only (d=0.20, P<.001) at .01 level.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among health topics, mHealth interventions on physical activity had the lowest weighted mean effect size (d=0.24) and interventions on nutrition and weight status had the highest (d=0.36), but these were not significantly different.

    “with mHealth interventions on physical activity showing the lowest weighted mean ES (d=0.24; SE=0.06; 95% CIs 0.11, 0.36; P<.001) and interventions on nutrition and weight status showing the highest ES (d=0.36; SE=.08; 95% CIs 0.21, 0.52; P<.001), which however are not significantly different from each other.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Among the eight types of engagement, changing personal environment, reinforcement tracking, and social presentation or announcement were significant moderators.

    “Among the 8 types of engagement proposed by Sama et al [18], only changing personal environment (Qbetween (df=1)=9.44, P=.002), reinforcement tracking (Qbetween (df=1)=10.24, P=.001), and social presentation or announcement (Qbetween (df=1)=6.42, P=.01) were significant moderators.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    All other included studies appeared in peer-reviewed journal articles.

    “all other included studies appeared in peer-reviewed journal articles.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    None of the studies meeting inclusion criteria focused on reducing substance use, promoting sexual health, or preventing HIV/AIDS.

    “none of the studies meeting inclusion criteria focused on reducing substance use, promoting sexual health, or preventing HIV/AIDS.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Previous mHealth interventions have yielded conflicting results and inconsistent effect sizes, ranging from Cohen d of −0.62 to 1.65.

    “Despite the promises of mHealth interventions, previous mHealth interventions have yielded conflicting results and inconsistent effect sizes (ESs), ranging from Cohen d of −0.62 [7] to 1.65 [8].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] source

      Not checked in the free run.

    • cites [8] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    A meta-analysis on health interventions using social networking sites reported that studies using a true control condition without giving any intervention had a significantly higher weighted mean effect size than studies giving an alternative intervention to the control group.

    “Specifically, a meta-analysis on health interventions using social networking sites reported that studies using a true control condition without giving any intervention had a significantly higher weighted mean ES than studies giving an alternative intervention to the control group [33].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    According to channel complementary theory, people use multiple sources, which serve different niches and present unique information, to acquire information in certain health topics.

    “According to channel complementary theory [103], people use multiple sources, which serve different niches and present unique information, to acquire information in certain health topics.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [103] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Social presentation or announcement may serve as a distraction to participants.

    “A possible explanation for this is that social presentation or announcement may serve as a distraction to participants.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [106] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    According to the Elaboration Likelihood Model, personally relevant messages are more likely to increase personal involvement and trigger central route message processing, which would achieve stable persuasive effects over time.

    “According to the Elaboration Likelihood Model [111], personally relevant messages are more likely to increase personal involvement and trigger central route message processing, which would achieve stable persuasive effects over time.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [111] Petty, RE, 1981, Attitudes and Persuasion: Classic and Contemporary Approaches

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study started from a comprehensive literature search.

    “although this study started from a comprehensive literature search and included more studies than previous reviews ([3,4,10]”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The convergence of cognitive and behavioral theories has observed success in CBT.

    “The convergence of cognitive and behavioral theories has observed success in CBT [110] and deserves further investigation by health”
    • Also stated in: § Theoretical and Practical Implications · plain
      “The convergence of cognitive and behavioral theories has observed success in CBT [110] and deserves further investigation by health communication researchers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [110] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Tailored communication is effective in promoting health behavior change and health outcomes.

    “given that tailored communication is effective in promoting health behavior change and health outcomes [31]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The finding of relative effectiveness of mHealth interventions is consistent with previous research and extends current research by examining effects across health contexts.

    “Our finding related to the relative effectiveness of mHealth interventions shows not only consistency with previous research but extends current research by examining the effects across health contexts.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The lower relative effects of mHealth interventions with a social presentation or announcement feature may be due largely to the one study with this feature that contributed the largest negative effect size.

    “Alternatively, the lower relative effects of mHealth interventions with a social presentation or announcement feature may be due largely to the 1 study with this feature that contributed the largest negative ES in the meta-analysis [7].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Previous meta-analyses documented that the use of theory to guide intervention development does not significantly moderate effect sizes.

    “However, which theoretical paradigm works best in mHealth interventions remains unclear, especially when previous meta-analyses documented that the use of theory to guide intervention development does not significantly moderate ESs [1].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Previous syntheses either focused on a specific health issue or were constrained by small sample sizes and a lack of moderator analyses, leaving what factors account for variance in effect sizes unanswered.

    “However, previous syntheses either focused on a specific health issue [9,10] or were constrained by small sample sizes and a lack of moderator analyses [3,4], leaving what factors account for variance in ESs unanswered.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [9] source

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

    • cites [3] source

      Not checked in the free run.

    • cites [4] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Due to the relatively small sample size in some categories, such as the variety of specific media that were used in combination with mobile phones in interventions, they were grouped in a larger category for moderator analyses.

    “Due to the relatively small sample size in some categories, such as the variety of specific media that were used in combination with mobile phones in interventions, they were grouped in a larger category for moderator analyses.”
    Quote in the paperNot a substantial claim
22 of 22

  • The authors state this as an assumption.

    Theory-based mHealth interventions may be more efficacious than their counterparts.

    “Therefore, it is plausible to assume that theory-based mHealth interventions may be more efficacious than their counterparts.”

  • The paper relies on this without support.

    SMS is easier to implement than apps.

    “SMS is easier to implement, whereas apps could afford multimedia interactive features”

  • The paper relies on this without support.

    Apps can afford multimedia interactive features.

    “SMS is easier to implement, whereas apps could afford multimedia interactive features”

  • The paper relies on this without support.

    Combining SMS and app features would complement each other to maximize user engagement and health outcomes.

    “such features would complement each other to potentially maximize user engagement and health outcomes.”

  • The paper relies on this without support.

    Mobile phones are easily accessible and cost efficient.

    “due to the easy accessibility and cost efficiency of mobile phones”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 1.51reported p = .13 · recomputed p = .131

    “(z=1.51, p = .13”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Search yield differs between abstract and methods

    Minor

    The abstract says the search resulted in 3424 potential studies, while the Methods section says 3506 studies were retrieved from the databases. The text does not explain the difference between these search counts.

  • Conflicting lists of quality-of-life studies

    Minor

    The Study Description and Limitations sections each identify four studies on health-related quality of life, but they give different sets of references: [73,86] appear only in the first list, and [72,112] only in the second.

2 of 2

Cites

Flagged first, then by citations

1 of the 109 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. ShawRDuke University20122019-02-02A mobile health intervention to sustain recent weight loss (Doctoral dissertation, Duke University) https://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/5866/Shaw_duke_0066D_11609.pdf?sequence=1
  2. 10.1089/tmj.1008.0099 · DOI
  3. 10.1177/10919817400200403 · DOI
  4. Bandura, A, 1986, Social Foundations of Thought and Action: A Social Cognitive Theory
  5. Sutton, AJ, 2009, The Handbook of Research Synthesis and Meta-Analysi, 435
  6. 10.2196/mheatlh.3535 · DOI

Cited by

Sorted by citations

Related

Authors
Yang, Q., & Van Stee, S. K.
Affiliations
Texas Christian University (US)
Corresponding author
Yang, Qinghua
ORCID
Venue
JMIR mHealth and uHealth
Publisher
JMIR Publications Inc.
Published
2019-04-03
Volume
7
Issue
4
Pages
e11244
ISSN
2291-5222 (ISSN-L)
DOI
10.2196/11244
PMC
PMC6468337
OpenAlex
W2904311307
Semantic Scholar
064d099339ae919483a3ab1dc225b470b5416252
Cited by
113
References
109
Citations by year
2026: 12 · 2025: 11 · 2024: 15 · 2023: 17 · 2022: 16 · 2021: 23 · 2020: 15 · 2019: 4
FWCI
1.91
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • IRL - University of Missouri, St. Louis (University of Missouri–St. Louis) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Physical Sciences › Computer Science › Information Systems › Mobile Learning in Education
Keywords
mobile health intervention, mobile phone intervention, comparative effectiveness, health outcomes, intervention moderators, theoretical paradigms, follow-up duration, health behavior change, digital health interventions, effect size heterogeneity, meta-regression, random effects models, mobile health apps, short message service, reinforcement
MeSH terms
Humans, Technology Assessment, Biomedical, Outcome Assessment, Health Care, Telemedicine, Mobile Applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review