Selecting and describing control conditions in mobile health randomized controlled trials: a proposed typology

Goldberg, Simon B.; Sun, Shufang; Carlbring, Per; Torous, John

doi:10.1038/s41746-023-00923-7

Cited by 75Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsStatistical Methods in Clinical Trials

Selecting and describing control conditions in mobile health randomized controlled trials: a proposed typology

Open access

Goldberg, Simon B., Sun, Shufang, Carlbring, Per, Torous, John·2023·npj Digital Medicine

DOI 10.1038/s41746-023-00923-7PMC PMC10541862

Open this paper in CoChat — free account Open PDF

Abstract

Hundreds of randomized controlled trials (RCTs) have tested the efficacy of mobile health (mHealth) tools for a wide range of mental and behavioral health outcomes. These RCTs have used a variety of control condition types which dramatically influence the scientific inferences that can be drawn from a given study. Unfortunately, nomenclature across mHealth RCTs is inconsistent and meta-analyses commonly combine control conditions that differ in potentially important ways. We propose a typology of control condition types in mHealth RCTs. We define 11 control condition types, discuss key dimensions on which they differ, provide a decision tree for selecting and identifying types, and describe the scientific inferences each comparison allows. We propose a five-tier comparison strength gradation along with four simplified categorization schemes. Lastly, we discuss unresolved definitional, ethical, and meta-analytic issues related to the categorization of control conditions in mHealth RCTs.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper proposes a typology for selecting and describing control conditions in mobile health randomized controlled trials, defining 11 control condition types and a five-tier comparison-strength scheme. It also offers a decision tree and four simplified coding schemes for trialists and meta-analysts.

Key findings

Defines 11 control condition types: No Treatment, Placebo-Minimal, Placebo-Active, TAU-Minimal, TAU-Active, mHealth Minimal Active, Dismantling Design, Additive Design, mHealth Comparative Efficacy, non-mHealth Other Active, and non-mHealth EBT.
Proposes a five-tier comparison strength gradation from Class V (No Treatment) to Class I (e.g., mHealth Comparative Efficacy, non-mHealth EBT, and Dismantling/Additive designs for components).
Offers four simplified categorization schemes: five-category, four-category, two-category passive vs. active, and two-category therapeutic vs. non-therapeutic.
Argues that meta-analyses commonly combine control conditions that differ in important ways, and that control condition type influences the scientific inferences and effect sizes that can be drawn from mHealth RCTs.

Methods

Conceptual/framework paper; no empirical sample or primary data analysis is reported.
Builds on prior guidelines for behavioral intervention control conditions and defines five dimensions of differentiation: intended to be therapeutic, intensity, intensity match, masking/expectancy match, and received by both groups.
Provides a decision tree, tables of examples, scientific inferences, comparison strength, and alternative coding schemes; the main analysis is narrative synthesis and framework development.

Limitations

Paper-stated: the framework may help guide control-group selection but cannot definitively identify the ideal control for any particular study; its value must be proven in utility, and boundaries between categories are fuzzy (e.g., intensity/expectancy match, TAU-Minimal vs. TAU-Active).
Paper-stated: unresolved definitional, ethical, and meta-analytic issues remain; future work should include a Delphi panel, measures to delineate similar control conditions, and coding of the existing mHealth RCT literature.
Evident from the text: the typology is not empirically validated or tested for reliability, and no data are presented showing that the proposed categories predict between-group effect sizes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-023-00923-7

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 47 cited sources checked · 0 supported · 0 flagged

Interpretationreasoning
Ambiguity defining and categorizing control conditions impacts attempts to synthesize trends across mHealth RCTs.
“Ambiguity defining and categorizing control conditions impacts attempts to synthesize trends across mHealth RCTs.”
Quote in the paperSubstantial claim
Interpretationreasoning
Control conditions that are not intended to be therapeutic have an upper bound on the comparison strength they provide.
“Control conditions that are not intended to be therapeutic have an upper bound on the comparison strength they provide.”
Quote in the paperNot a substantial claim
Premisecitation
There are no established guidelines for designing and categorizing control conditions for mHealth RCTs.
“However, there are no established guidelines for designing and categorizing control conditions for mHealth RCTs15,16”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites [15] Lutz, J., Offidani, E., Taraboanta, L., Lakhan, S. E. & Campellone, T. R. Appropriate controls for digital therapeutic clinical trials: a narrative review of control conditions in clinical trials of digital therapeutics (DTx) deploying psychosocial, cognitive, or behavioral content. Front. Digit. Health 4, 823977 (2022). source
  Not checked in the free run.
- cites [16] Blease, C. Out of control: how to design digital placebos. Curr. Treat. Options Psychiatry 1–10 https://doi.org/10.1007/S40501-023-00290-W (2023). source
  Not checked in the free run.
Interpretationcitation
Non-mHealth EBT Active provides the highest comparison strength.
“We define a two-level distinction based on whether the non-mHealth control is a frontline evidence-based treatment (non-mHealth EBT Active), providing the highest comparison strength38, or another active control (non-mHealth Other Active) that is intended to be therapeutic but is not a frontline EBT.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [38] Goodheart, C. D. et al. Evidence-based practice in psychology. Am. Psychol. 61, 271–285 (2006). source
  Not checked in the free run.
Interpretationnone given
mHealth RCTs have used a variety of control condition types that dramatically influence the scientific inferences that can be drawn from a given study.
“These RCTs have used a variety of control condition types which dramatically influence the scientific inferences that can be drawn from a given study.”
Quote in the paperSubstantial claim

Premisecitation
The same term "control" can be used across RCTs to refer to quite different control conditions.
“Currently, the same term—“control”—can be used across RCTs to refer to quite different control conditions18.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [18] Wampold, B. E. et al. Evidence-based treatments for depression and anxiety versus treatment-as-usual: a meta-analysis of direct comparisons. Clin. Psychol. Rev. 31, 1304–1312 (2011). source
  Not checked in the free run.
Interpretationcitation
There is no single control condition that is appropriate for every study at every stage of the research process.
“There is no single control condition that is appropriate for every study at every stage of the research process11,12,39.”
Quote in the paperSubstantial claim0 of 3 sources checked
- cites [11] Mohr, D. C. et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother. Psychosom. 78, 275–284 (2009). source
  Not checked in the free run.
- cites [12] Freedland, K. E. et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. J. Clin. Epidemiol. 110, 74–81 (2019). source
  Not checked in the free run.
- cites [39] Onken, L. S., Carroll, K. M., Shoham, V., Cuthbert, B. N. & Riddle, M. Reenvisioning clinical science: unifying the discipline to improve the public health. Clin. Psychol. Sci. 2, 22–34 (2014). source
  Not checked in the free run.
Premisecitation
TAU can be quite minimal or fairly extensive.
“For example, TAU can be quite minimal (e.g., permission to pursue treatment outside of the study, referrals to non-study-related providers)19 or fairly extensive (e.g., receipt of pharmacotherapy and/or psychotherapy)20–22.”
Quote in the paperSubstantial claim0 of 4 sources checked
- cites [19] Gao, M. et al. Targeting anxiety to improve sleep disturbance: a randomized clinical trial of app-based mindfulness training. Psychosom. Med. 84, 632–642 (2022). source
  Not checked in the free run.
- cites [20] Naughton, F. et al. Randomized controlled trial to assess the short-term effectiveness of tailored web- and text-based facilitation of smoking cessation in primary care (iQuit in Practice). Addiction 109, 1184–1193 (2014). source
  Not checked in the free run.
- cites [21] Lee, S. B. & Valerius, J. mHealth interventions to promote anti-retroviral adherence in HIV: narrative review. JMIR Mhealth Uhealth 8, e14739 (2020). source
  Not checked in the free run.
- cites [22] Keeler, J. L. et al. App-based food-specific inhibitory control training as an adjunct to treatment as usual in binge-type eating disorders: a feasibility trial. Appetite 168, 105788 (2022). source
  Not checked in the free run.
Premisereasoning
Extant guidelines for behavioral interventions do not necessarily map neatly onto the mHealth literature where intervention and control conditions vary widely in form, intensity, and delivery setting.
“Extant guidelines for behavioral interventions do not necessarily map neatly onto the mHealth literature where intervention and control conditions vary widely in form, intensity, and delivery setting.”
Quote in the paperSubstantial claim
Interpretationcitation
If control conditions are interpreted as a homogeneous group, interventions compared with stronger control conditions may be perceived as less effective.
“If control conditions are interpreted as a homogeneous group, interventions compared with stronger control conditions may be perceived as less effective18,26.”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites [18] Wampold, B. E. et al. Evidence-based treatments for depression and anxiety versus treatment-as-usual: a meta-analysis of direct comparisons. Clin. Psychol. Rev. 31, 1304–1312 (2011). source
  Not checked in the free run.
- cites [26] Goldberg, S. B., Riordan, K. M., Sun, S. & Davidson, R. J. The empirical status of mindfulness-based interventions: a systematic review of 44 meta-analyses of randomized controlled trials. Perspect. Psychol. Sci. 17, 174569162096877 (2022). source
  Not checked in the free run.
Premisecitation
As control conditions can dramatically impact effect sizes observed in RCTs of psychological interventions generally and mHealth RCTs specifically, considering control conditions to be a single category is typically not defensible.
“As control conditions can dramatically impact effect sizes observed in RCTs of psychological interventions generally58,59 and mHealth RCTs specifically44, considering control conditions to be a single category is typically not defensible.”
Quote in the paperSubstantial claim0 of 3 sources checked
- cites [58] Goldberg, S. B. et al. Mindfulness-based interventions for psychiatric disorders: a systematic review and meta-analysis. Clin. Psychol. Rev. 59, 52–60 (2018). source
  Not checked in the free run.
- cites [59] Mohr, D. C. et al. Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression. Transl. Behav. Med. 4, 407–423 (2014). source
  Not checked in the free run.
- cites [44] Firth, J. et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 16, 287–298 (2017). source
  Not checked in the free run.
Interpretationcitation
The choice of control condition heavily impacts the comparison strength within a study.
“The choice of control condition heavily impacts the comparison strength within a study12.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [12] Freedland, K. E. et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. J. Clin. Epidemiol. 110, 74–81 (2019). source
  Not checked in the free run.
Interpretationreasoning
Class IV comparison strength can be derived from either Placebo-Minimal or TAU-Minimal.
“Class IV comparison strength can be derived from either Placebo-Minimal or TAU-Minimal”
Quote in the paperSubstantial claim
Premisecitation
Engagement with mHealth interventions can be highly variable.
“As noted, engagement with mHealth interventions can be highly variable29,30.”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites [29] Ng, M. M., Firth, J., Minen, M. & Torous, J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr. Serv. 70, 538–544 (2019). source
  Not checked in the free run.
- cites [30] Lipschitz, J. M. et al. Digital mental health interventions for depression: scoping review of user engagement. J. Med. Internet Res. 24, e39204 (2022). source
  Not checked in the free run.
Interpretationreasoning
Placebo-Active control provides a stronger comparison than Placebo-Minimal in its ability to match on a larger number of non-specific factors.
“The Placebo-Active control provides a stronger comparison than the Placebo-Minimal in its ability to match on a larger number of non-specific factors (e.g., expectancy).”
Quote in the paperSubstantial claim
Interpretationreasoning
Class III comparison strength can be derived from Placebo-Active, TAU-Active, or mHealth Minimal Active controls.
“Class III comparison strength can be derived from Placebo-Active, TAU-Active, or mHealth Minimal Active controls.”
Quote in the paperSubstantial claim
Interpretationreasoning
TAU-Active sets a higher bar for demonstrating effects of mHealth than TAU-Minimal.
“Likewise, TAU-Active sets a higher bar for demonstrating the effects of mHealth than TAU-Minimal”
Quote in the paperSubstantial claim
Interpretationreasoning
Highest comparison strength (Class I) for mHealth intervention as a whole requires comparisons with either a matched mHealth Comparative Efficacy control or a non-mHealth EBT control.
“The highest comparison strength (Class I) for the mHealth intervention as a whole requires comparisons with either a matched mHealth Comparative Efficacy control or a non-mHealth EBT control.”
Quote in the paperSubstantial claim

Classified as probably not a substantial claim. 46
May combine two claims. 15

Probably not a substantial claim: may not pass S4 (based).
Findingnone given
Nomenclature across mHealth RCTs is inconsistent.
“Unfortunately, nomenclature across mHealth RCTs is inconsistent”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Findingnone given
Meta-analyses commonly combine control conditions that differ in potentially important ways.
“meta-analyses commonly combine control conditions that differ in potentially important ways.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Premisenone given
The proposed typology considers five dimensions: intended to be therapeutic, intensity, intensity match, masking/expectancy match, and received by both groups.
“Our proposed typology considers five dimensions (Table 1): intended to be therapeutic, intensity, intensity match, masking/expectancy match, and received by both groups.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
The comparison strength increases as intensity increases among non-therapeutic controls.
“The comparison strength increases as intensity increases.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
Control condition intensity is an important consideration, particularly in mHealth context where many focal interventions are self-guided and of potentially modest intensity.
“Control condition intensity is an important second consideration, particularly in the mHealth context where many focal interventions are self-guided and of potentially modest intensity.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Premisenone given
Non-therapeutic control conditions are further defined based on intensity: none (No Treatment), low (Placebo-Minimal), and high (Placebo-Active).
“Non-therapeutic control conditions are further defined based on intensity: none (No Treatment), low (Placebo-Minimal), and high (Placebo-Active).”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim. May combine two claims.
Premisereasoning
Whether the treatment condition is masked and expectancy is likely matched between focal and control conditions is a consideration.
“A fourth consideration is whether the treatment condition is masked and in such a way that expectancy is likely to be matched between focal and control conditions.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Premisereasoning
Whether intensity is matched between focal intervention and control condition is a consideration.
“A third consideration is whether the intensity is matched between the focal intervention and the control condition.”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationcitation
Dismantling Design and Additive Design provide a strong comparison for the specific ingredients being tested, although alone they cannot be used to evaluate the effects of an intervention as a whole.
“These designs provide a strong comparison for the specific ingredients being tested, although alone they cannot be used to evaluate the effects of an intervention as a whole.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [34] Bell, E. C., Marcus, D. K. & Goodlad, J. K. Are the parts as good as the whole? A meta-analysis of component treatment studies. J. Consult. Clin. Psychol. 81, 722–736 (2013). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Premisenone given
Whether the control condition is received by both focal intervention and control groups is a consideration.
“A fifth consideration is whether the control condition is received by both focal intervention and control groups.”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationcitation
Meta-analyses commonly combine substantially different control conditions, which complicates interpretation of results.
“Meta-analyses commonly combine substantially different control conditions, which complicates interpretation of results24.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [24] Bendtsen, M., McCambridge, J., Åsberg, K. & Bendtsen, P. Text messaging interventions for reducing alcohol consumption among risky drinkers: systematic review and meta-analysis. Addiction 116, 1021–1033 (2021). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Premisereasoning
TAU can vary in intensity, so this category is further graded into TAU-Minimal and TAU-Active.
“However, TAU can vary in intensity, so this category is further graded into TAU-Minimal and TAU-Active.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
Development of psychotherapy placebos remains challenging.
“the development of psychotherapy placebos remains a challenging (and some might say impossible)9,33 task.”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites [9] Wampold, B. E. & Imel, Z. E. The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work 2nd edn (Routledge, 2015). source
  Not checked in the free run.
- cites [33] Wampold, B. E., Frost, N. D. & Yulish, N. E. Placebo effects in psychotherapy: a flawed concept and a contorted history. Psychol. Conscious. Theory Res. Pract. 3, 108–120 (2016).
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation
There is a need for a consistently applied typology of control conditions as mHealth moves toward wider adoption within health systems.
“As mHealth moves toward wider adoption within health systems3, there is a need for a consistently applied typology of control conditions.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [3] FDA. FDA clears mobile medical app to help those with opioid use disorder stay in recovery programs. https://www.fda.gov/news-events/press-announcements/fda-clears-mobile-medical-app-help-those-opioid-use-disorder-stay-recovery-programs (2022).
  Not checked in the free run.
May combine two claims.
Interpretationcitation
Studies with non-therapeutic control conditions can establish the impact of non-specific factors but cannot provide the highest comparison strength.
“Studies with non-therapeutic control conditions are still scientifically valuable as they can establish the impact of non-specific factors12, but they cannot provide the highest comparison strength.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [12] Freedland, K. E. et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. J. Clin. Epidemiol. 110, 74–81 (2019). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation
Clarifying the universe of mHealth control condition types can help highlight the scientific questions each type can reasonably address and comparison strength each type provides.
“Clarifying the universe of mHealth control condition types can help highlight the scientific questions each type can reasonably address and comparison strength each type provides12.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [12] Freedland, K. E. et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. J. Clin. Epidemiol. 110, 74–81 (2019). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning
Several other aspects of modern mHealth RCTs do not fit neatly into the 11-category scheme.
“Several other aspects of modern mHealth RCTs do not fit neatly into the 11-category scheme.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation
The ethical use of digital health tools remains an active area of investigation, with frameworks emerging but none yet consistently adopted.
“The ethical use of digital health tools remains an active area of investigation, with frameworks emerging but none yet consistently adopted55,56.”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites [55] Nebeker, C., Bartlett Ellis, R. J. & Torous, J. Development of a decision-making checklist tool to support technology selection in digital health research. Transl. Behav. Med. 10, 1004–1015 (2020). source
  Not checked in the free run.
- cites [56] Shen, F. X. et al. An ethics checklist for digital health research in psychiatry: viewpoint. J. Med. Internet Res. 24, e31146 (2022). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone given
Expectancy may or may not actually be matched across conditions in mHealth RCTs.
“Similarly, expectancy may or may not actually be matched across conditions.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning
The framework proposed here may help guide the selection of a control group, but it cannot definitively identify the ideal one for any particular study.
“The framework we propose here may help guide the selection of a control group, but it cannot definitively identify the ideal one for any particular study.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
Although focal and control interventions may both in theory be equally intensive, participants may not engage with them both equally.
“Although focal and control interventions may both in theory be equally intensive, participants may not engage with them both equally.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
Highest classes of comparison strength (Class II and I) require therapeutic control conditions that may be matched in intensity and non-specific factors including expectancy.
“The highest classes of comparison strength (Class II and I) require therapeutic control conditions that may be matched in intensity and non-specific factors including expectancy.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationnone given
Dismantling Design and Additive Design controls provide very strong comparisons (Class I), but only for the components being subtracted or added and not for the intervention as a whole.
“Dismantling Design and Additive Design controls provide very strong comparisons (Class I), but only for the components being subtracted or added and not for the intervention as a whole.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation
As a general rule, clinical trials should only be run when there is the expectation that the treatment being studied may be beneficial (i.e., equipoise).
“As a general rule, clinical trials should only be run when there is the expectation that the treatment being studied may be beneficial (i.e., equipoise)54.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [54] Freedman, B. Equipoise and the ethics of clinical research. N. Engl. J. Med. 317, 427–431 (1987). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Interpretationreasoning
The contribution of study-related human support in mHealth RCTs makes categorization challenging.
“The contribution of study-related human support in mHealth RCTs also makes categorization challenging.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning
Scientific questions related to comparisons with other mHealth interventions are best addressed using mHealth Minimal Active controls or mHealth Comparative Efficacy controls.
“Scientific questions related to comparisons with other mHealth interventions are best addressed using mHealth Minimal Active controls or mHealth Comparative Efficacy controls.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Interpretationreasoning
The therapeutic vs. non-therapeutic two-category scheme requires collapsing across a theoretically wide range of non-therapeutic and therapeutic controls.
“However, it requires collapsing across a theoretically wide range of non-therapeutic (i.e., No Treatment, Placebo-Minimal, Placebo-Active) and therapeutic controls.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
Clinical trialists may reasonably disagree about how a particular control condition should be categorized.
“Moreover, clinical trialists may reasonably disagree about how a particular control condition should be categorized.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationreasoning
The boundary between TAU-Minimal and TAU-Active may often be fuzzy in contexts where TAU is heterogeneous.
“The boundary between TAU-Minimal and TAU-Active may often be fuzzy in contexts where TAU is heterogeneous (i.e., patients receive differing amounts and/or type of treatment).”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation
At later stages of research, stronger comparisons are warranted.
“At later stages of research, stronger comparisons are warranted13.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [13] Gold, S. M. et al. Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. Lancet Psychiatry 4, 725–732 (2017). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Premisenone given
The 11-category scheme is designed to characterize important differences between control conditions in mHealth RCTs.
“The 11-category scheme is designed to characterize important differences between control conditions in mHealth RCTs.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning
In contexts where trust is limited between community members and academic research teams, providing an active control may be necessary for building trust and completing recruitment and enrollment goals for an RCT.
“In contexts where trust is limited between community members and academic research teams, providing an active control may be necessary for building trust and completing recruitment and enrollment goals for an RCT.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation
Four-category scheme distinguishes between therapeutic and non-therapeutic controls, defined as non-specific and specific active controls, while combining minimal treatment controls into a single group.
“This scheme also distinguishes between therapeutic and non-therapeutic controls, defined as non-specific and specific active controls42,43, while combining minimal treatment controls into a single group.”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites [42] Goldberg, S. B. et al. Mindfulness-based interventions for psychiatric disorders: a systematic review and meta-analysis. Clin. Psychol. Rev. https://doi.org/10.1016/j.cpr.2017.10.011 (2017). source
  Not checked in the free run.
- cites [43] Galante, J. et al. Mindfulness-based programmes for mental health promotion in adults in nonclinical settings: a systematic review and meta-analysis of randomised controlled trials. PLoS Med. 18, 1–40 (2021). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation
Dismantling Designs may have removed all theoretically therapeutic ingredients and therefore be better categorized as Placebo-Active.
“Dismantling Designs, although categorized as intended to be therapeutic in our scheme, may have removed all theoretically therapeutic ingredients34 and therefore be better categorized as Placebo-Active.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [34] Bell, E. C., Marcus, D. K. & Goodlad, J. K. Are the parts as good as the whole? A meta-analysis of component treatment studies. J. Consult. Clin. Psychol. 81, 722–736 (2013). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Interpretationreasoning
The passive vs. active two-category scheme ignores whether a control condition is intended to be therapeutic.
“A downside of this scheme is it ignores whether a control is intended to be therapeutic.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation
In early stages of research, No Treatment controls may be very appropriate to evaluate intervention effects beyond passage of time and related confounds.
“In early stages of research39,40, No Treatment controls may be very appropriate in order to evaluate intervention effects beyond the passage of time and related confounds (e.g., regression to the mean)41.”
Quote in the paperNot a substantial claim0 of 3 sources checked
- cites [39] Onken, L. S., Carroll, K. M., Shoham, V., Cuthbert, B. N. & Riddle, M. Reenvisioning clinical science: unifying the discipline to improve the public health. Clin. Psychol. Sci. 2, 22–34 (2014). source
  Not checked in the free run.
- cites [40] Rounsaville, B. J., Carroll, K. M. & Onken, L. S. A stage model of behavioral therapies research: getting started and moving on from stage I. Clin. Psychol. Sci. Pract. 8, 133–142 (2001). source
  Not checked in the free run.
- cites [41] Bland, J. M. & Altman, D. G. Statistic notes: regression towards the mean. BMJ 308, 1499 (1994). source
  Not checked in the free run.
Classified as probably not a substantial claim.
Interpretationcitation
Clinical trialists are faced with balancing controlling threats to internal validity with statistical power.
“clinical trialists are faced with balancing controlling threats to internal validity (where stronger comparisons provide more confidence that observed differences between intervention and control groups are due to the intervention itself) with statistical power (where stronger comparisons are expected to produce smaller between-group differences that require larger sample sizes for detection).”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites [11] Mohr, D. C. et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother. Psychosom. 78, 275–284 (2009). source
  Not checked in the free run.
- cites [13] Gold, S. M. et al. Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. Lancet Psychiatry 4, 725–732 (2017). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
Four control conditions can be used to evaluate the effects of mHealth in the context of non-mHealth interventions.
“Four control conditions can be used to evaluate the effects of mHealth in the context of non-mHealth interventions.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationreasoning
There may be advantages to having simpler schemes (e.g., for meta-analysis).
“there may be advantages to having simpler schemes (e.g., for meta-analysis).”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning
TAU-Active is challenging to include in four-category scheme.
“One control condition that is challenging to include in this scheme is TAU-Active.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationcitation
As consensus is reached regarding the categorization of control conditions in mHealth RCTs, this information can be used by regulators, researchers, industry stakeholders, and patients and clinicians.
“As consensus is reached regarding the categorization of control conditions in mHealth RCTs, this information can be used by bodies charged with regulating these technologies (e.g., FDA)60, researchers seeking to adequately power mHealth RCTs, industry stakeholders involved in investing and marketing, and ultimately patients and clinicians deciding when to use a given mHealth tool.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [60] Sim, I. Mobile devices and health. N. Engl. J. Med. 381, 956–968 (2019). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
The passive vs. active two-category scheme for control conditions is used within the meta-analytic literature.
“it is used within the meta-analytic literature44”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [44] Firth, J. et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 16, 287–298 (2017). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
Selecting a control condition has ethical implications around access to care, scientific rigor, and bias.
“Selecting a control condition has ethical implications around access to care, scientific rigor, and bias.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone given
In theory, between-group effect sizes should become smaller as the comparison strength increases from Class V to Class I.
“In theory, between-group effect sizes should become smaller as the comparison strength increases from Class V to Class I.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone given
One issue not addressed by the proposed scheme is the ethical dimension of study design.
“One issue not addressed by our scheme, but of paramount importance, is the ethical dimension of study design.”
Quote in the paperNot a substantial claim
May combine two claims.
Premisecitation
Studies commonly involve interaction between participants and study staff, although it can be hard to evaluate the extent and nature of this interaction from published RCTs.
“Studies commonly involve interaction between participants and study staff, although it can be hard to evaluate the extent and nature of this interaction from published RCTs49.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [49] Meyer, A., Wisniewski, H. & Torous, J. Coaching to support mental health apps: exploratory narrative review. JMIR Hum. Factors 9, e28301 (2022). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation
As a program of research matures, increasingly rigorous and theoretically therapeutic control conditions may be warranted, both for ethical and scientific reasons.
“As a program of research matures (e.g., moves through the stages of intervention development)11,39,40, increasingly rigorous and theoretically therapeutic control conditions may be warranted, both for ethical (e.g., beneficence and non-maleficence)53 and scientific reasons (i.e., determining how a novel treatment compares with established treatments).”
Quote in the paperNot a substantial claim0 of 3 sources checked
- cites [11] Mohr, D. C. et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother. Psychosom. 78, 275–284 (2009). source
  Not checked in the free run.
- cites [39] Onken, L. S., Carroll, K. M., Shoham, V., Cuthbert, B. N. & Riddle, M. Reenvisioning clinical science: unifying the discipline to improve the public health. Clin. Psychol. Sci. 2, 22–34 (2014). source
  Not checked in the free run.
- cites [40] Rounsaville, B. J., Carroll, K. M. & Onken, L. S. A stage model of behavioral therapies research: getting started and moving on from stage I. Clin. Psychol. Sci. Pract. 8, 133–142 (2001). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation
Particularly in under-resourced settings and when working with vulnerable populations, No Treatment control conditions may be inappropriate as they may involve withholding interventions with known therapeutic value.
“Particularly in under-resourced settings and when working with vulnerable populations, No Treatment control conditions may be inappropriate as they may involve withholding interventions with known therapeutic value50,51.”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites [50] Boruch, R. F., Victor, T. & Cecil, J. S. Resolving ethical and legal problems in randomized experiments. Crime & Delinquency 46, 330–353 (2016). source
  Not checked in the free run.
- cites [51] Pavone, I. R. Legal responses to placebo-controlled trials in developing countries. Global Bioethics 27, 76–90 (2016). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Premisenone given
Five-category scheme collapses across levels of placebo controls, TAU controls, mHealth controls, and non-mHealth controls.
“A five-category scheme collapses across levels of placebo controls, TAU controls, mHealth controls, and non-mHealth controls.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationcitation
Adaptive designs and factorial designs in mHealth RCTs are challenging to categorize.
“Adaptive designs in which the treatment that is delivered changes over time (e.g., sequential multiple assignment randomized controlled trial [SMART])48 and factorial designs that evaluate multiple aspects of an mHealth intervention at once are challenging to categorize.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [48] Collins, L. M., Murphy, S. A. & Strecher, V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART). New methods for more potent eHealth interventions. Am. J. Prev. Med. 32, S112–S118 (2007). source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationreasoning
New meta-analyses sensitive to the impact of control groups may provide more accurate effect size estimates and results of prior meta-analyses may need to be reexamined in light of these considerations.
“New meta-analyses sensitive to the impact of controls groups may provide more accurate effect size estimates and results of prior meta-analyses may need to be reexamined in light of these considerations.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationnone given
Placebo-Minimal controls for some non-specific factors while Placebo-Active can theoretically control for all non-specific factors, particularly when time and expectancy are matched.
“Placebo-Minimal controls for some non-specific factors (e.g., minimal amounts of time, attention, or expectancy) while a Placebo-Active can theoretically control for all non-specific factors, particularly when time and expectancy are matched.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
Non-EBT non-mHealth controls provide strong comparisons (Class II), but weaker comparisons than frontline, non-mHealth EBTs.
“Non-EBT non-mHealth controls provide strong comparisons (Class II), but weaker comparisons than frontline, non-mHealth EBTs.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Interpretationreasoning
Five-category scheme ignores variations in intensity and evidence base which may influence scientific inferences and effect sizes.
“Although this scheme can help describe the types of comparisons, it ignores variations in intensity and evidence base which may influence scientific inferences and effect sizes.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Interpretationnone given
No Treatment controls provide the lowest tier (Class V).
“No Treatment controls provide the lowest tier (Class V).”
Quote in the paperSubstantial claim

55 of 55

The paper relies on this without support.
The 11 control condition types are comprehensive and capable of differentiating between the wide variety of control conditions in mHealth literature.
“We intend this typology to be comprehensive and capable of differentiating between the wide variety of control conditions appearing in the mHealth literature.”
The paper relies on this without support.
The five-tier comparison strength gradation validly orders control conditions by strength.
“Table 2 displays a five-tier scheme for grading the comparison strength provided by the 11 control conditions.”
The paper relies on this without support.
The five dimensions chosen are the relevant dimensions for differentiating control conditions in mHealth RCTs.
“Our proposed typology considers five dimensions (Table 1): intended to be therapeutic, intensity, intensity match, masking/expectancy match, and received by both groups.”

3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Ethical questions concerning blinding and control conditions in digital mental health research are still mostly unexamined.
“For example, it is clear that digital mental health studies should include digital control groups 106 , but ethical issues around blinding and control conditions remain largely unexplored.”
From Ethical challenges in contemporary psychiatry: an overview and an appraisal of possible strategies and research needs (2024) · cites this paper as [106]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

Additive Design is marked as not received by both groups despite its example
Minor
Table 1 marks the Additive Design control as not received by both groups, but its example gives both groups the Weight Watchers digital program; the focal group receives an additional app.
Placebo-Minimal is said to be received by both groups, but Table 1 says otherwise
Minor
The comparison-strength discussion says both Class IV control conditions are received by both groups. Table 1 lists Placebo-Minimal as received by both groups only "No/maybe," not yes.
TAU-Active is called active in the prose but unclear in Table 3
Minor
The prose says every condition other than No Treatment and TAU-Minimal is active, which includes TAU-Active. Table 3 instead assigns TAU-Active "Unclear" in the active-versus-non-active scheme.

3 of 3

Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478 · see citing works
Equipoise and the Ethics of Clinical Research
Freedman, B. (1987) · New England Journal of Medicine · doi:10.1056/nejm198707163170304 · Cited by 2,339 · see citing works
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237 · see citing works
The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work
Wampold, B. E. (2015) · doi:10.4324/9780203582015 · Cited by 1,773 · see citing works
The Effectiveness of Mobile-Health Technology-Based Health Behaviour Change or Disease Management Interventions for Health Care Consumers: A Systematic Review
Free, C., Phillips, G., Galli, L., Watson, L., Felix, L., Edwards, P., Patel, V., & Haines, A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001362 · Cited by 1,766 · see citing works
Mindfulness-based stress reduction for healthy individuals: A meta-analysis
Khoury, B., Sharma, M., Rush, S. E., & Fournier, C. (2015) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2015.03.009 · Cited by 1,617 · see citing works
Randomised controlled trials – the gold standard for effectiveness research: Study design: randomised controlled trials
Hariton, E., & Locascio, J. J. (2018) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15199 · Cited by 1,604 · see citing works
Mindfulness-based interventions for psychiatric disorders: A systematic review and meta-analysis
Goldberg, S. B., Tucker, R. P., Greene, P. A., Davidson, R. J., Wampold, B. E., Kearney, D. J., & Simpson, T. L. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.10.011 · Cited by 1,145 · see citing works
Self-citation
The efficacy of smartphone‐based mental health interventions for depressive symptoms: a meta‐analysis of randomized controlled trials
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097 · see citing works
Self-citation
A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, "all must have prizes."
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H.-N. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.122.3.203 · Cited by 1,091 · see citing works
The Multiphase Optimization Strategy (MOST) and the Sequential Multiple Assignment Randomized Trial (SMART)
Collins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · see citing works
The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials
Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 886 · see citing works
Self-citation
Reenvisioning Clinical Science: Unifying the Discipline to Improve the Public Health
Onken, L. S., Carroll, K. M., Shoham, V., Cuthbert, B. N., & Riddle, M. (2014) · Clinical Psychological Science · doi:10.1177/2167702613497932 · Cited by 872 · see citing works
A stage model of behavioral therapies research: getting started and moving on from stage I.
Rounsaville, B. J., Carroll, K. M., & Onken, L. S. (2001) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.8.2.133 · Cited by 846 · see citing works
The Selection and Design of Control Conditions for Randomized Controlled Trials of Psychological Interventions
Mohr, D. C., Spring, B., Freedland, K. E., Beckner, V., Arean, P., Hollon, S. D., Ockene, J., & Kaplan, R. (2009) · Psychotherapy and Psychosomatics · doi:10.1159/000228248 · Cited by 565 · see citing works
Statistic Notes: Regression towards the mean
Bland, J. M., & Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6942.1499 · Cited by 547 · see citing works
The Empirical Status of Mindfulness-Based Interventions: A Systematic Review of 44 Meta-Analyses of Randomized Controlled Trials
Goldberg, S. B., Riordan, K. M., Sun, S., & Davidson, R. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620968771 · Cited by 520 · see citing works
Self-citation
Mobile Devices and Health
Sim, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1806949 · Cited by 491 · see citing works
Putting the ‘app’ in Happiness: A Randomised Controlled Trial of a Smartphone-Based Mindfulness Intervention to Enhance Wellbeing
Howells, A., Ivtzan, I., & Eiroa-Orosa, F. J. (2016) · Journal of Happiness Studies · doi:10.1007/s10902-014-9589-1 · Cited by 407 · see citing works
The Use and Effectiveness of Mobile Apps for Depression: Results From a Fully Remote Clinical Trial
Arean, P. A., Hallgren, K. A., Jordan, J. T., Gazzaley, A., Atkins, D. C., Heagerty, P. J., & Anguera, J. A. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6482 · Cited by 376 · see citing works
Mindfulness training reduces loneliness and increases social contact in a randomized controlled trial
Lindsay, E. K., Young, S., Brown, K. W., Smyth, J. M., & Creswell, J. D. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1813588116 · Cited by 344 · see citing works
Mindfulness-based programmes for mental health promotion in adults in nonclinical settings: A systematic review and meta-analysis of randomised controlled trials
Galante, J., Friedrich, C., Dawson, A. F., Modrego-Alarcón, M., Gebbing, P., Delgado-Suárez, I., Gupta, R., Dean, L., Dalgleish, T., White, I. R., et al. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003481 · Cited by 319 · see citing works
User engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293 · see citing works
Self-citation
Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials
Goldberg, S. B., Lam, S. U., Simonsson, O., Torous, J., & Sun, S. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000002 · Cited by 237 · see citing works
Self-citation
Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework
Gold, S. M., Enck, P., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D. C., & Otte, C. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30153-0 · Cited by 236 · see citing works
Engagement in mobile phone app for self-monitoring of emotional wellbeing predicts changes in mental health: MoodPrism
Bakker, D., & Rickard, N. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.016 · Cited by 231 · see citing works
Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: A randomized controlled trial
Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Zou, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., McEvoy, P. M., et al. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.09.003 · Cited by 230 · see citing works
The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel
Freedland, K. E., King, A. C., Ambrosius, W. T., Mayo-Wilson, E., Mohr, D. C., Czajkowski, S. M., Thabane, L., Collins, L. M., Rebok, G. W., Treweek, S. P., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.011 · Cited by 218 · see citing works
The digital placebo effect: mobile mental health meets clinical psychiatry
Torous, J., & Firth, J. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00565-9 · Cited by 200 · see citing works
Self-citation
Are the parts as good as the whole? A meta-analysis of component treatment studies.
Bell, E. C., Marcus, D. K., & Goodlad, J. K. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033004 · Cited by 192 · see citing works
Smartphone Cognitive Behavioral Therapy as an Adjunct to Pharmacotherapy for Refractory Depression: Randomized Controlled Trial
Mantani, A., Kato, T., Furukawa, T. A., Horikoshi, M., Imai, H., Hiroe, T., Chino, B., Funayama, T., Yonemoto, N., Zhou, Q., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8602 · Cited by 192 · see citing works
Effectiveness of Self-guided App-Based Virtual Reality Cognitive Behavior Therapy for Acrophobia: A Randomized Clinical Trial
Donker, T., Cornelisz, I., van Klaveren, C., van Straten, A., Carlbring, P., Cuijpers, P., & van Gelder, J.-L. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0219 · Cited by 187 · see citing works
Self-citation
Attention Bias Modification Training Via Smartphone to Reduce Social Anxiety: A Randomized, Controlled Multi-Session Experiment
Enock, P. M., Hofmann, S. G., & McNally, R. J. (2014) · Cognitive Therapy and Research · doi:10.1007/s10608-014-9606-z · Cited by 186 · see citing works
A Systematic Review and Meta-analysis of Physical Exercise Prehabilitation in Major Abdominal Surgery (PROSPERO 2017 CRD42017080366)
Heger, P., Probst, P., Wiskemann, J., Steindorf, K., Diener, M. K., & Mihaljević, A. L. (2019) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04287-w · Cited by 179 · see citing works
Coached Mobile App Platform for the Treatment of Depression and Anxiety Among Primary Care Patients: A Randomized Clinical Trial
Graham, A. K., Greene, C. J., Kwasny, M. J., Kaiser, S. M., Lieponis, P., Powell, T., & Mohr, D. C. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1011 · Cited by 162 · see citing works
Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression
Mohr, D. C., Ho, J., Hart, T. L., Baron, K. G., Berendsen, M., Beckner, V., Cai, X., Cuijpers, P., Spring, B., Kinsinger, S. W., et al. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0262-3 · Cited by 139 · see citing works
Evidence-based treatments for depression and anxiety versus treatment-as-usual: A meta-analysis of direct comparisons
Wampold, B. E., Budge, S. L., Laska, K. M., Del Re, A. C., Baardseth, T. P., Flűckiger, C., Minami, T., Kivlighan, D. M., & Gunn, W. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.07.012 · Cited by 134 · see citing works
Consensus statement on the problem of terminology in psychological interventions using the internet or digital components
Smoktunowicz, E., Barak, A., Andersson, G., Baños, R., Berger, T., Botella, C., Dear, B. F., Donker, T., Ebert, D. D., Hadjistavropoulos, H. D., et al. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100331 · Cited by 108 · see citing works
Self-citation
Digital Mental Health Interventions for Depression: Scoping Review of User Engagement
Lipschitz, J. M., Van Boxtel, R., Torous, J., Firth, J., Lebovitz, J. G., Burdick, K. E., & Hogan, T. P. (2022) · Journal of Medical Internet Research · doi:10.2196/39204 · Cited by 105 · see citing works
Self-citation
Development of a decision-making checklist tool to support technology selection in digital health research
Nebeker, C., Bartlett Ellis, R. J., & Torous, J. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz074 · Cited by 99 · see citing works
Self-citation
Randomized controlled trial to assess the short‐term effectiveness of tailored web‐ and text‐based facilitation of smoking cessation in primary care (i<scp>Q</scp>uit in <scp>P</scp>ractice)
Naughton, F., Jamison, J., Boase, S., Sloan, M., Gilbert, H., Prevost, A. T., Mason, D., Smith, S., Brimicombe, J., Evans, R., et al. (2014) · Addiction · doi:10.1111/add.12556 · Cited by 93 · see citing works
Appropriate controls for digital therapeutic clinical trials: A narrative review of control conditions in clinical trials of digital therapeutics (DTx) deploying psychosocial, cognitive, or behavioral content
Lutz, J., Offidani, E., Taraboanta, L., Lakhan, S. E., & Campellone, T. R. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.823977 · Cited by 89 · see citing works
The FDA's Drug Review Process: Ensuring Drugs are Safe and Effective
Meadows, M. (2002) · PsycEXTRA Dataset · doi:10.1037/e542562006-004 · Cited by 85 · see citing works
Post-9/11 Veterans and Their Partners Improve Mental Health Outcomes with a Self-directed Mobile and Web-based Wellness Training Program: A Randomized Controlled Trial
Kahn, J. R., Collinge, W., & Soltysik, R. C. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5800 · Cited by 74 · see citing works
Text messaging interventions for reducing alcohol consumption among risky drinkers: systematic review and meta‐analysis
Bendtsen, M., McCambridge, J., Åsberg, K., & Bendtsen, P. (2021) · Addiction · doi:10.1111/add.15294 · Cited by 71 · see citing works
Do you believe it? verbal suggestions influence the clinical and neural effects of escitalopram in social anxiety disorder: a randomized trial
Faria, V., Gingnell, M., Hoppe, J. M., Hjorth, O., Alaie, I., Frick, A., Hultberg, S., Wahlstedt, K., Engman, J., Månsson, K. N. T., et al. (2017) · EBioMedicine · doi:10.1016/j.ebiom.2017.09.031 · Cited by 70 · see citing works
Self-citation
eHealth to Enhance Treatment Adherence Among Youth Living with HIV
Mulawa, M. I., LeGrand, S., & Hightow‐Weidman, L. (2018) · Current HIV/AIDS Reports · doi:10.1007/s11904-018-0407-y · Cited by 70 · see citing works
Individualized Texting for Adherence Building (iTAB): Improving Antiretroviral Dose Timing Among HIV-Infected Persons with Co-occurring Bipolar Disorder
Moore, D. J., Poquette, A., Casaletto, K. B., Gouaux, B. D., Montoya, J. L., Posada, C., Rooney, A. S., Badiee, J., Deutsch, R., Letendre, S. L., et al. (2014) · AIDS and Behavior · doi:10.1007/s10461-014-0971-0 · Cited by 68 · see citing works
Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps
Lagan, S., D'Mello, R., Vaidyam, A., Bilden, R., & Torous, J. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13306 · Cited by 65 · see citing works
App-based food-specific inhibitory control training as an adjunct to treatment as usual in binge-type eating disorders: A feasibility trial
Keeler, J. L., Chami, R., Cardi, V., Hodsoll, J., Bonin, E., MacDonald, P., Treasure, J., & Lawrence, N. (2022) · Appetite · doi:10.1016/j.appet.2021.105788 · Cited by 43 · see citing works
An Ethics Checklist for Digital Health Research in Psychiatry: Viewpoint
Shen, F. X., Silverman, B. C., Monette, P., Kimble, S., Rauch, S. L., & Baker, J. T. (2022) · Journal of Medical Internet Research · doi:10.2196/31146 · Cited by 43 · see citing works
Resolving Ethical and Legal Problems in Randomized Experiments
Boruch, R. F., Victor, T., & Cecil, J. S. (2000) · Crime & Delinquency · doi:10.1177/0011128700046003005 · Cited by 40 · see citing works
A Smartphone-Based Intervention as an Adjunct to Standard-of-Care Treatment for Schizophrenia: Randomized Controlled Trial
Ghaemi, S. N., Sverdlov, O., van Dam, J., Campellone, T. R., & Gerwien, R. W. (2021) · JMIR Formative Research · doi:10.2196/29154 · Cited by 36 · see citing works
Which adolescents are well-suited to app-based mindfulness training? A randomized clinical trial and data-driven approach for personalized recommendations.
Webb, C. A., Swords, C. M., Lawrence, H. R., & Hilt, L. M. (2022) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000763 · Cited by 36 · see citing works
mHealth Interventions to Promote Anti-Retroviral Adherence in HIV: Narrative Review
Lee, S. B., & Valerius, J. (2020) · JMIR mHealth and uHealth · doi:10.2196/14739 · Cited by 34 · see citing works
Coaching to Support Mental Health Apps: Exploratory Narrative Review
Meyer, A., Wisniewski, H., & Torous, J. (2022) · JMIR Human Factors · doi:10.2196/28301 · Cited by 34 · see citing works
Self-citation
Purpose-guided trial design in health-related behavioral intervention research.
Freedland, K. E. (2020) · Health Psychology · doi:10.1037/hea0000867 · Cited by 32 · see citing works
Targeting Anxiety to Improve Sleep Disturbance: A Randomized Clinical Trial of App-Based Mindfulness Training.
Gao, M., Roy, A., Deluty, A., Sharkey, K. M., Hoge, E. A., Liu, T., & Brewer, J. A. (2022) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001083 · Cited by 24 · see citing works
Out of Control: How to Design Digital Placebos
Blease, C. (2023) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-023-00290-w · Cited by 15 · see citing works
Legal responses to placebo-controlled trials in developing countries
Pavone, I. R. (2016) · Global Bioethics · doi:10.1080/11287462.2016.1192979 · Cited by 4 · see citing works
Let’s decide what would be convincing, conduct randomized trials with rigorous comparison conditions, and report tests of moderation and publication bias in meta-analyses
Goldberg, S. B., Torous, J. B., & Sun, S. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000127 · Cited by 2 · see citing works
Self-citation

Flagged first, then by citations

Research works printed but not linked: 2

Wampold, B. E., Frost, N. D. & Yulish, N. E. Placebo effects in psychotherapy: a flawed concept and a contorted history. Psychol. Conscious. Theory Res. Pract. 3, 108–120 (2016).
Goldberg, S. B. et al. Mindfulness-based interventions for psychiatric disorders: a systematic review and meta-analysis. Clin. Psychol. Rev. 59, 52–60 (2018). · DOI · Paper page

Sorted by citations

The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., Hassan, L., Hua, Y., Milton, A., & Firth, J. (2025) · World Psychiatry · doi:10.1002/wps.21299 · Cited by 323 · see citing works
Self-citation
Digital interventions in mental health: An overview and future perspectives
Löchner, J., Carlbring, P., Schuller, B. W., Torous, J. B., & Sander, L. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100824 · Cited by 170 · see citing works
Self-citation
Standardising the role of a digital navigator in behavioural health: a systematic review
Perret, S., Alon, N., Carpenter-Song, E. A., Myrick, K. J., Thompson, K., Li, S., Sharma, K., & Torous, J. B. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00152-8 · Cited by 95 · see citing works
Self-citation
Ethical challenges in contemporary psychiatry: an overview and an appraisal of possible strategies and research needs
Galderisi, S., Appelbaum, P. S., Gill, N., Gooding, P., Herrman, H., Melillo, A., Myrick, K., Pathare, S., Savage, M., Szmukler, G., et al. (2024) · World Psychiatry · doi:10.1002/wps.21230 · Cited by 76 · see citing works
Self-citation
Mobile phone interventions to improve health outcomes among patients with chronic diseases: an umbrella review and evidence synthesis from 34 meta-analyses
Sun, S., Simonsson, O., McGarvey, S. T., Torous, J. B., & Goldberg, S. B. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00119-5 · Cited by 57 · see citing works
Self-citation
Effectiveness of Online and Remote Interventions for Mental Health in Children, Adolescents, and Young Adults After the Onset of the COVID-19 Pandemic: Systematic Review and Meta-Analysis
Fischer-Grote, L., Fössing, V., Aigner, M., Fehrmann, E., & Boeckle, M. (2024) · JMIR Mental Health · doi:10.2196/46637 · Cited by 42 · see citing works
“Some of the included studies had wait list or no-treatment control groups [19,56,59,62,65,66,68], which can be considered as a control condition with low comparison strength [44].”
BackgroundInfluential
A Digital Health Behavior Intervention to Prevent Childhood Obesity
Heerman, W. J., Rothman, R. L., Sanders, L., Schildcrout, J. S., Flower, K. B., Delamater, A. M., Kay, M. C., Wood, C. T., Gross, R. S., Bian, A., et al. (2024) · JAMA · doi:10.1001/jama.2024.22362 · Cited by 41 · see citing works
Recommendations for Successful Development and Implementation of Digital Health Technology Tools
Loo, R. T. J., Nasta, F., Macchi, M., Baudot, A., Burstein, F., Bove, R. M., Greve, M., Fröhlich, H., Khalid, S., Küderle, A., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/56747 · Cited by 28 · see citing works
Self-citation
Resilience-based interventions in the public sector workplace: a systematic review
Hollaar, M. H. L., Kemmere, B., Kocken, P., & Denktaş, S. (2025) · BMC Public Health · doi:10.1186/s12889-024-21177-2 · Cited by 36 · see citing works
Guided and unguided internet-delivered psychodynamic therapy for social anxiety disorder: A randomized controlled trial
Mechler, J., Lindqvist, K., Magnusson, K., Ringström, A., Krafman, J. D., Alvinzi, P., Kassius, L., Sowa, J., Andersson, G., & Carlbring, P. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00063-0 · Cited by 20 · see citing works
Self-citation
Assessing generative artificial intelligence for mental health
Torous, J. B., & Topol, E. J. (2025) · The Lancet · doi:10.1016/s0140-6736(25)01237-1 · Cited by 19 · see citing works
Self-citation
Transdiagnostic-focused apps for depression and anxiety: a meta-analysis
Linardon, J., Anderson, C., Messer, M., Liu, C., & Torous, J. B. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01860-3 · Cited by 18 · see citing works
Self-citation
Efficacy of mental health smartphone apps on stress levels: a meta-analysis of randomised controlled trials
Linardon, J., Firth, J., Torous, J. B., Messer, M., & Fuller‐Tyszkiewicz, M. (2024) · Health Psychology Review · doi:10.1080/17437199.2024.2379784 · Cited by 19 · see citing works
Self-citation
Methods for evaluating the efficacy and effectiveness of direct-to-consumer mobile health apps: a scoping review
Prentice, C., Peven, K., Zhaunova, L., Nayak, V., Radović, T., Klepchukova, A., POTTS, H. W. W., & Ponzo, S. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00092-x · Cited by 17 · see citing works
Mental Health Impacts of Self‐Help Interventions for the Treatment and Prevention of Eating Disorders. A Meta‐Analysis
Linardon, J., Jarman, H. K., Liu, C., Anderson, C., McClure, Z., & Messer, M. (2025) · International Journal of Eating Disorders · doi:10.1002/eat.24405 · Cited by 18 · see citing works
The role of self-efficacy in internet-based interventions for mental health: A systematic review and meta-analysis
Behr, S., Garcia, L. M., Lucas, J., Kohlhase, E., Puetz, M., Boettcher, J., Schaeuffele, C., & Knaevelsrud, C. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100821 · Cited by 16 · see citing works
Effects of a single session low-threshold digital intervention for procrastination behaviors among university students (Focus): Findings from a randomized controlled trial
Åsberg, K., Löf, M., & Bendtsen, M. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100741 · Cited by 13 · see citing works
The effectiveness of smartphone app-based interventions for insomnia and sleep disturbances: A meta-analysis of randomized controlled trials
Linardon, J., Anderson, C., McClure, Z., Liu, C., & Messer, M. (2024) · Sleep Medicine · doi:10.1016/j.sleep.2024.08.025 · Cited by 13 · see citing works
Methodological quality in randomised clinical trials of mental health apps: systematic review and longitudinal analysis
Linardon, J., Xie, Q., Swords, C. M., Torous, J. B., Sun, S., & Goldberg, S. B. (2025) · BMJ Mental Health · doi:10.1136/bmjment-2025-301595 · Cited by 12 · see citing works
Self-citation
Treating comorbid insomnia in patients enrolled in therapist-assisted transdiagnostic internet-delivered cognitive behaviour therapy for anxiety and depression: A randomized controlled trial
Edmonds, M., Peynenburg, V. A., Kaldo, V., Jernelöv, S., Titov, N., Dear, B. F., & Hadjistavropoulos, H. D. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100729 · Cited by 10 · see citing works
Effects of a digital intervention on physical activity in adults: A randomized controlled trial in a large-scale sample
Fichtner, U. A., Tinsel, I., Sehlbrede, M., Maiwald, P., Bischoff, M., Metzner, G., Schlett, C., Brame, J., Kohl, J., König, D., et al. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100762 · Cited by 10 · see citing works
Comparative efficacy of PsyPills and OCAT mobile psychological interventions in reducing depressive, anxiety and stress symptoms: A blinded randomized clinical trial
Sîrbu, V., David, D., Sánchez-López, Á., & Blanco, I. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.10.052 · Cited by 11 · see citing works
Efficacy and risks of artificial intelligence chatbots for anxiety and depression: a narrative review of recent clinical studies
Bodner, R., Lim, K., Schneider, R. A., & Torous, J. B. (2025) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000001048 · Cited by 11 · see citing works
Self-citation
Integrating digital and in-person therapy for PTSD: feasibility and acceptability of blended trauma-focused cognitive therapy in routine care
Lundin, J., Jansson‐Fröjmark, M., Gustafsson-Björverud, L., Grey, N., Santoft, F., Ehlers, A., Carlbring, P., Lundgren, T., Bragesjö, M., & Salomonsson, S. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1447651 · Cited by 9 · see citing works
Self-citation
Artificial intelligence as a virtual coach in a cognitive behavioural intervention for perfectionism in young people: A randomised feasibility trial
Johnson, C., Egan, S. J., Carlbring, P., Shafran, R., & Wade, T. D. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100795 · Cited by 9 · see citing works
Self-citation
Methodological Challenges in Randomized Controlled Trials of mHealth Interventions: Cross-Sectional Survey Study and Consensus-Based Recommendations
López‐Alcalde, J., Wieland, L. S., Yan, Y., Barth, J., Khami, M. R., Shivalli, S., Lokker, C., Kaur, H., Macharia, P. M., Yun, S., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/53187 · Cited by 8 · see citing works
Therapist-guided, Internet-delivered cognitive behaviour therapy for adolescents with body dysmorphic disorder: A feasibility trial with long-term follow-up
Rautio, D., Andrén, P., Gumpert, M., Jolstedt, M., Jassi, A. D., Krebs, G., Jansson‐Fröjmark, M., Lundgren, T., Serlachius, E., Mataix‐Cols, D., et al. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100688 · Cited by 9 · see citing works
Efficacy of a smartphone-based Cognitive Bias Modification program for emotion regulation: A randomized-controlled crossover trial
Dietel, F. A., Rupprecht, R., Seriyo, A. M., Post, M., Sudhoff, B., Reichart, J. S., Berking, M., & Buhlmann, U. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100719 · Cited by 9 · see citing works
Effectiveness of a Mobile Phone-Delivered Multiple Health Behavior Change Intervention (LIFE4YOUth) in Adolescents: Randomized Controlled Trial
Seiterö, A., Henriksson, P., Thomas, K., Henriksson, H., Löf, M., Bendtsen, M., & Müssener, U. (2025) · Journal of Medical Internet Research · doi:10.2196/69425 · Cited by 8 · see citing works
Exploring reactivity effects of self-monitoring prolonged grief reactions in daily life: A randomized waitlist-controlled trial using experience sampling methodology
Franzen, M., & Lenferink, L. I. M. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100877 · Cited by 7 · see citing works
Effectiveness of interactive digital health services in non-communicable diseases: An umbrella review and evidence synthesis from 26 meta-analyses
Laukka, E., Jansson, M. M., Suonnansalo, P., Ojanperä, R., Härkönen, H., Lakoma, S., & Torkki, P. M. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105277 · Cited by 7 · see citing works
Smartphone-based self-help virtual reality exposure therapy for college students’ social anxiety: a randomized controlled study
Kan, C., Wang, Y., Hu, R., Chen, K. G., & Zhang, Y. (2025) · Virtual Reality · doi:10.1007/s10055-025-01195-0 · Cited by 6 · see citing works
A mobile-based, single-session intervention to empower parents of adolescents hospitalised for non-suicidal self-injury: A mixed-methods randomised controlled trial
He, J., Shen, Y., Chen, Z., Tian, Y., Hou, Y., Li, Y., & Ou, J. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100822 · Cited by 5 · see citing works
Effectiveness of a guided internet-based intervention in reducing procrastination among university students – a randomized controlled trial
Amarnath, A., Ozmen, S., van Klaveren, C., van Straten, A., Pei, J., de Wit, L., Raabe, R. E., Cuijpers, P., & Struijs, S. Y. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100878 · Cited by 6 · see citing works
Anna vs. Judith: A randomized comparison of AI-delivered psychodynamic and cognitive behavioral therapies for social anxiety disorder
Hlynsson, J. I., Mechler, J., Lindqvist, K., Andersson, G., & Carlbring, P. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100960 · Cited by 6 · see citing works
Self-citation
What to do when the unwanted happens? Negative event management in studies on internet- and mobile-based interventions for youths and adults with two case reports
Schulte, C. E., Sextl-Plötz, T., Baumeister, H., Titzler, I., Sander, L., Sachser, C., Steubl, L., & Zarski, A. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100710 · Cited by 5 · see citing works
Comparative efficacy of digital health interventions for depression and anxiety symptoms in adolescents and young adults: a systematic review and bayesian network meta-analysis
Wang, H., Yao, J., Jia, S., Chen, W., Guan, Y., Liu, J., Liao, M., Hu, E., & Zhang, Q. (2026) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-026-01042-3 · Cited by 4 · see citing works
Feasibility trial of an unguided ultra-brief online psychological intervention within an online mental health clinic: The “things you do” intervention
Sapkota, R. P., Gretton, A., Wilhelms, A., Bisby, M. A., Dear, B. F., Titov, N., & Hadjistavropoulos, H. D. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100852 · Cited by 4 · see citing works
Treating perfectionism via the Internet: a randomized controlled trial comparing cognitive behavior therapy to unified protocol
Rozental, A., Shafran, R., Johansson, F., Forsström, D., Jovicic, F., Gelberg, O., Molin, K., Carlbring, P., Andersson, G., & Buhrman, M. (2024) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2024.2327339 · Cited by 4 · see citing works
Self-citation
Should digital health trials be placebo controlled?
Druart, L., & Blease, C. (2025) · JME Practical Bioethics · doi:10.1136/jmepb-2025-000027 · Cited by 3 · see citing works
Enhancing the Scalability and Impact of the Motherly App for Postnatal Depression
Xie, Q., & Chen, W. (2025) · Psychotherapy and Psychosomatics · doi:10.1159/000543542 · Cited by 3 · see citing works
Looking beyond depression: meta-analysis of digital interventions for depression on eleven secondary mental health and wellbeing outcomes
Linardon, J., Anderson, C., Messer, M., McClure, Z., & Liu, C. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116657 · Cited by 4 · see citing works
Effectiveness of an online text-based stress management program for employees who work in micro- and small-sized enterprises: A randomized controlled trial
Sasaki, N., Ogawa, S., Sawada, U., Shimazu, T., Powell, B. J., Takeno, H., Tsutsumi, A., & Imamura, K. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100754 · Cited by 4 · see citing works
Crowdsourcing integrated into a digital mental health platform for anxiety and depression: A pilot randomized controlled trial
Kaveladze, B. T., Shkel, J., Le, S., Marcotte, V., Rushton, K., Nguyen, T., & Schueller, S. M. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100774 · Cited by 3 · see citing works
Feasibility of internet-based multimodal emotion recognition training in adolescents with and without autism: A pilot study
Olsson, N. C., Björkman, J. N., Lackell, R., Bergens, O., Fischer, H., Döllinger, L., Bergström, J., Carlbring, P., & Laukka, P. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100861 · Cited by 3 · see citing works
Self-citation
Multidisciplinary research priorities for artificial intelligence in mental health: a call to action
Linardon, J., Firth, J., Carvalho, A. F., Bockting, C., Christensen, H. M., Dieckmann, L. H. J., Griffiths, S., Gupta, S., Hegerl, U., Li, K., et al. (2026) · The Lancet Psychiatry · doi:10.1016/s2215-0366(26)00127-6 · Cited by 2 · see citing works
Self-citation
Smartphone intervention apps for schizophrenia: A review of the academic literature and app stores
Xia, W., Hau, C., Burns, J., Ryan, S., Firth, J., Linardon, J., & Torous, J. B. (2025) · Schizophrenia Research · doi:10.1016/j.schres.2025.09.007 · Cited by 1 · see citing works
Self-citation
Unfinished business in virtual reality: Development and preliminary evaluation of an empty chair intervention grounded in emotion-focused therapy
Halamová, J., Ottingerová, L., Haladová, Z. B., & Greenberg, L. S. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100885 · Cited by 1 · see citing works
Meta Self-Efficacy Internet Intervention to Support Occupational Health in Young Employees: Protocol for Co-Creation and a Randomized Controlled Trial
Maciejewski, J., Cieślak, R., Carlbring, P., & Smoktunowicz, E. (2025) · JMIR Research Protocols · doi:10.2196/85082 · Cited by 1 · see citing works
Self-citation
Time to take the Declaration of Helsinki seriously? A systematic review of comparison conditions in clinical trials on internet-delivered cognitive behavioral therapy for insomnia
Grolig, L., Brückner, H., Heimes, H., Buntrock, C., Hempeler, C., Spiegelhalder, K., & Lehr, D. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102224 · Cited by 1 · see citing works
Popular Online Content as a Treatment-as-Usual Control in Digital Mental Health Intervention Trials: Secondary Analysis of Two Online Randomized Controlled Trials With Repeated Measures
Kaveladze, B. T., Schueller, S. M., & Mohr, D. C. (2026) · JMIR Mental Health · doi:10.2196/83707 · Cited by 1 · see citing works
Efficacy and safety of a mobile app–based, flexible-schedule digital therapeutic intervention versus psychoeducation for panic disorder: An open-label randomized controlled trial
Kang, D. H., Hwang, H., Kim, D. S., 김혜리, Kwak, S., Kim, C., & Choi, S. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100977 · Cited by 1 · see citing works
Internet-delivered unguided psychodynamic vs. cognitive behavior therapy for anxiety and depression symptoms: A large, full factorial randomized controlled trial on treatment length and peer support
Lindqvist, K., Mechler, J., Hlynsson, J. I., Andersson, G., & Carlbring, P. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100980 · Cited by 1 · see citing works
Self-citation
Meta Self-Efficacy Intervention to Improve Work Self-Efficacy and Occupational Well-Being in Young Employees: A Study Protocol for Internet Intervention Co-Creation and a Randomized Controlled TrialPreprint
Maciejewski, J., Cieślak, R., Carlbring, P., & Smoktunowicz, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5272395 · Cited by 1 · see citing works
Self-citation
Intervention effects and mechanisms of online single session therapy on test anxiety in university students: A randomized controlled trial
Chen, L., Yan, J., Keong, L. T., & Zhang, Y. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100839
Efficacy of Internet-based cognitive behavioral therapy for reducing perfectionistic strivings in the Republic of Korea: A randomized controlled trial
Oh, S., Hee, J. Y., Joo, J., Lee, J. H., Lee, Y., Lee, H. J., Yoon, D. U., & Lee, J. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100851
A Smartphone App (PRIMI) to Promote Healthy Diet, Physical Activity, and Health Literacy After Childbirth Among Migrant Women: Protocol for a Randomized Controlled Trial
Seiterö, A., Shirvanifar, M., Leksell, M. J., Rydfjord, M., Al-saedi, B., Ahmed, A. S., Bahar, T. G., Berglind, D., Henriksson, H., Ahlqvist, V. H., et al. (2025) · JMIR Research Protocols · doi:10.2196/79277
Neural, Behavioral, and Clinical Outcomes of a Voice-based AI Coach for Depression and Anxiety: A Phase 2 Randomized TrialPreprint
Kannampallil, T., Ajilore, O., Smyth, J. M., Barve, A., Ronneberg, C. R., Lv, N., Kumar, V., Garcia, C., Aborisade, G., Wittels, N. E., et al. (2025) · medRxiv · doi:10.64898/2025.12.22.25342792
The effectiveness of digital interventions for enhancing empathy in adults: a meta-analysis of randomized controlled trials
Williams, C. Y., Xie, Q., Muratoglu, E., & Goldberg, S. B. (2026) · Current Psychology · doi:10.1007/s12144-025-08981-8 · Cited by 1 · see citing works
Self-citation
Tailored internet-delivered cognitive behavior therapy for breast cancer patients: a randomized controlled trial with an active control group
Andersson, G., Käll, A., Enström, M., Jonsson, L., Ehnsiö, K., Nilsson-Ihrfelt, E., Carlbring, P., & Fjällskog, M.-L. (2026) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2026.2635464
Self-citation
Bridging the mental health care gap for international students via digital interventions: conceptual foundations and a five-stage model for culturally sensitive design
Xie, Q., Zhu, Y., Lin, T., Yin, Z., & Goldberg, S. B. (2026) · Counselling Psychology Quarterly · doi:10.1080/09515070.2026.2643867 · Cited by 1 · see citing works
Self-citation
Investigating Placebos and Controls Used in Large Language Model–Based Chatbot Intervention Trials: Protocol for a Methodological Review
Druart, L., Faria, V., Annoni, M., Torous, J. B., Pontén, M., & Blease, C. (2026) · JMIR Research Protocols · doi:10.2196/90507
Self-citation
Game, Set, and Match: A Scoping Review of Matching Characteristics for Control and Intervention Groups in Adaptive Behavioral Interventions for Physical Activity or Healthy Eating Designs for Populations with Overweight and Obesity
Whitney, J., Suhy, V. A., Pauley, A. M., Shao, D., Khan, O., Moore, A. M., Chow, S., Rivera, D. E., & Downs, D. S. (2026) · Behavioral Medicine · doi:10.1080/08964289.2026.2641440
Magnitude and Moderators of Depressive Symptom Improvement in Digital Placebo Arms: Systematic Review and Meta-Analysis
Hosono, T., Tsutsumi, R., Niwa, Y., & Kondoh, M. (2026) · Journal of Medical Internet Research · doi:10.2196/preprints.100426
Cognitive control training as an add-on treatment for late-life depression: A multicenter randomized controlled trial
Zwalmen, Y. V., Vrijsen, J. N., Meuleman, B., Vanderhasselt, M., Naarding, P., Collard, R. M., Beek, M. H. C. T. V., Oostelbos, P., Bolier, L., van Driel, D. T. J. W., et al. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100951
Augmented-Reality Cognitive Behavior Therapy for Fear of Contamination in Obsessive-Compulsive Disorder: A Protocol for a Multicenter Randomized Controlled Trial (Preprint)Preprint
Skoko, A., Hlynsson, J. I., Čihařová, M., Berger, T., Carlbring, P., Krieger, T., Bergström, J., Daele, T. V., van Klaveren, C., Cornelisz, I., et al. (2026) · doi:10.2196/preprints.103262
Self-citation
The impact of a national transition of treatment platform on the acceptance of internet-delivered cognitive behavioral therapy: A longitudinal study among Norwegian therapists
Nævdal, R., Vis, C., Teig, I. L., Beurs, D. P. D., & Kenter, R. M. F. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100965
Facing fear through play: A smartphone-based proof-of-concept intervention using inverted augmented reality
Trapp, W., Röder, S., & Wimmer, F. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100966
Emotion regulation and social information processing as targets of a guided selective prevention app for emerging adults with a history of adverse childhood experiences: A randomised controlled trial
Stallmann, L., Jacinto, S., Vetsch, N., Stolz, T., Löchner, J., Marmet, S., Berger, T., & Brodbeck, J. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100987
Counterfactual Auditing of EHR-Based Predictors: A Framework for Identifying Spurious Clinical Associations
Ramirez, C., Torres, E., Ortega, P., & Mendes, S. (2023) · Journal of Health Informatics and Digital Systems · doi:10.68159/z254779038
A smartphone-based serious game to improve mental health and medication adherence in adults with depression: A randomised controlled trial
Brehmer, J., Berg, M., Astudillo, G., Lindegaard, T., Find, A. E., Carlbring, P., Ludvigsson, M., & Andersson, G. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2026.122494
Self-citation
No easy fix for adherence in internet interventions: two randomized studies targeting adherence self-efficacy
Maciejewski, J., Lesnierowska, M., Carlbring, P., & Smoktunowicz, E. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.101006
Self-citation
Augmented-Reality Cognitive Behavior Therapy for Contamination-Related Obsessive-Compulsive Symptoms: A Protocol for a Multicenter Randomized Controlled Trial (Preprint)
Skoko, A., Hlynsson, J. I., Čihařová, M., Berger, T., Carlbring, P., Krieger, T., Bergström, J., Daele, T. V., van Klaveren, C., Cornelisz, I., et al. (2026) · JMIR Research Protocols · doi:10.2196/103262
Self-citation
Impacts of the iCanCope Digital Pain Self-Management Program Among Adolescents and Young Adults With Chronic Pain: A Randomized Controlled Trial
Lalloo, C., Nishat, F., Palermo, T., Montgomery, L. S., Campbell, F. A., Baerg, K., Chorney, J. M. L., Dick, B. D., Renna, T. D., Mukhida, K., et al. (2026) · Canadian Journal of Pain · doi:10.1080/24740527.2026.2717605
Comparing a body-oriented and a cognitive online intervention for adults with depressive symptoms: A three-arm randomized controlled trial
Michalak, J., Gemander, M., Heidenreich, T., Kvetnaya, T., Puntke, T., & Schultze, M. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.101016

Is there a right control condition in mHealth trials? A critical view on pain medicine
Priebe, J. A., & Toelle, T. R. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0184-z · Cited by 16 · see citing works
Mobile Health Interventions and RCTs: Structured Taxonomy and Research Framework
Yang, A., Singh, N., & Varshney, U. (2022) · Journal of Medical Systems · doi:10.1007/s10916-022-01856-6 · Cited by 5 · see citing works
Beyond the Randomized Controlled Trial: A Review of Alternatives in mHealth Clinical Trial Methods
Pham, Q., Wiljer, D., & Cafazzo, J. A. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.5720 · Cited by 162 · see citing works
Randomised controlled trials as a method of evaluating mobile health interventions
Dick, S., O’Connor, Y., & Heavin, C. (2017) · Arrow@dit (Dublin Institute of Technology) · Cited by 1 · see citing works
How to operationalize regular treatment? Methodological challenges for investigating efficacy of mhealth solutions (Preprint)Preprint
Priebe, J. A., & Toelle, T. R. (2018) · doi:10.2196/preprints.13232

Authors: Goldberg, S. B., Sun, S., Carlbring, P., & Torous, J.
Affiliations: University of Wisconsin–Madison (US)
Brown University (US)
Stockholm University (SE)
Beth Israel Deaconess Medical Center (US)
Corresponding author: Goldberg, Simon B.
ORCID: Goldberg, Simon B.
Sun, Shufang
Carlbring, Per
Torous, John
Venue: npj Digital Medicine
Publisher: Springer Science and Business Media LLC
Published: 2023-09-30
Volume: 6
Issue: 1
Pages: 181-181
ISSN: 2398-6352 (ISSN-L)
DOI: 10.1038/s41746-023-00923-7
PMC: PMC10541862
OpenAlex: W4387210055
Semantic Scholar: 1b592f0cfee3267744b8f01f31918fae324a99b5
Cited by: 75
References: 61
Citations by year: 2026: 22 · 2025: 30 · 2024: 19 · 2023: 4
FWCI: 3.79
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0 (tdm)
creativecommons.org/licenses/by/4.0 (vor)
Locations: npj Digital Medicine (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
PsyArXiv (OSF Preprints) (submittedVersion)
Funders: U.S. Department of Health & Human Services | NIH | National Center for Complementary and Integrative Health (K23AT010879, K23AT011173)
Hope for Depression Research Foundation (Defeating Depression Award)
Topics: Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: mobile health, control conditions, randomized controlled trials, mental health interventions, behavioral health, meta-analysis comparability, decision tree, research ethics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Selecting and describing control conditions in mobile health randomized controlled trials: a proposed typology

Abstract

AI snapshot

TL;DR

Key findings

Methods

Limitations

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks