CoChat
Sign Up
Cited by 297Digital Mental Health InterventionsTelemedicine and Telehealth ImplementationOnline and Blended Learning

Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study

Open access
van der Vaart, Rosalie, Witting, Marjon, Riper, Heleen, Kooistra, Lisa, Bohlmeijer, Ernst T, van Gemert-Pijnen, Lisette JEWC·2014·BMC Psychiatry
DOI 10.1186/s12888-014-0355-zPMC PMC4271498
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Blending online modules into face-to-face therapy offers perspectives to enhance patient self-management and to increase the (cost-)effectiveness of therapy, while still providing the support patients need. The aim of this study was to outline optimal usage of blended care for depression, according to patients and therapists. METHODS: A Delphi method was used to find consensus on suitable blended protocols (content, sequence and ratio). Phase 1 was an explorative phase, conducted in two rounds of online questionnaires, in which patients' and therapists' preferences and opinions about online psychotherapy were surveyed. In phase 2, data from phase 1 was used in face-to-face interviews with therapists to investigate how blended therapy protocols could be set up and what essential preconditions would be. RESULTS: Twelve therapists and nine patients completed the surveys. Blended therapy was positively perceived among all respondents, especially to enhance the self-management of patients. According to most respondents, practical therapy components (assignments, diaries and psycho-education) may be provided via online modules, while process-related components (introduction, evaluation and discussing thoughts and feelings), should be supported face-to-face. The preferred blend of online and face-to-face sessions differs between therapists and patients; most therapists prefer 75% face-to-face sessions, most patients 50 to 60%. The interviews showed that tailoring treatment to individual patients is essential in secondary mental health care, due to the complexity of their problems. The amount and ratio of online modules needs to be adjusted according to the patient's problems, skills and characteristics. Therapists themselves should also develop skills to integrate online and face-to-face sessions. CONCLUSIONS: Blending online and face-to-face sessions in an integrated depression therapy is viewed as a positive innovation by patients and therapists. Following a standard blended protocol, however, would be difficult in secondary mental health care. A database of online modules could provide flexibility to tailor treatment to individual patients, which asks motivation and skills of both patients and therapists. Further research is necessary to determine the (cost-)effectiveness of blended care, but this study provides starting points and preconditions to blend online and face-to-face sessions and create a treatment combining the best of both worlds.

AI snapshot

Written from the paper’s full text.

TL;DR

This Delphi study found that patients and therapists viewed blending online modules into face-to-face depression therapy positively, especially for self-management, but preferred different online/face-to-face ratios and emphasized tailoring rather than standard protocols. It provides preconditions for secondary mental health care but calls for further research on effectiveness.

Key findings

  • Blended therapy was positively perceived. Self-management benefits included always having access to therapy content (92% therapists, 100% patients), encouraging patient responsibility (83%, 100%), blending therapy into home/private situation (92%, 88%), completing sessions in own time (92%, 80%) and pace (83%, 80%), and less travel (67%, 80%). Main drawbacks were that it is not suitable for every patient (92% therapists, 88% patients) and interpretation problems due to lack of non-verbal communication (75%, 80%).
  • Content preferences: treatment introduction was 100% face-to-face for both groups; treatment evaluation was face-to-face for 67% of therapists and 88% of patients; assignments online for 75% of therapists and 63% of patients; psycho-education online for 100% and 75%; diaries online for 100% and 88%; asking questions about assignments face-to-face for 83% and 75%; expressing thoughts/feelings on assignments face-to-face for 67% and 88%; diary content face-to-face for 83% and 75%; reminders online for 75% and 88%.
  • Preferred ratio differed: most therapists chose 75% face-to-face/25% online (33%, 4/12), while patients most commonly chose 50% face-to-face/50% online (38%, 3/8) or 40% face-to-face/60% online (38%, 3/8), i.e., 50–60% online.
  • Phase 2 therapists (n=9) said there is no “typical patient” in secondary mental health care and standard blended protocols were not feasible; they emphasized flexible online modules, tailoring to patient characteristics, meta-communication, online intake, technology innovations, therapist training, and management support.

Methods

  • A Delphi method was used: Phase 1 consisted of two rounds of online questionnaires with patients and therapists; Phase 2 consisted of individual interviews with therapists. Participants were recruited from three mental health care institutions in different regions of the Netherlands.
  • Twelve therapists (8 female, 4 male; mean age 45, range 28–60) completed both survey rounds; nine patients (3 female, 6 male; mean age 37, range 23–56) completed round 1, and eight patients completed round 2. Nine therapists were interviewed in Phase 2.
  • Phase 1 data were analyzed by quantifying item scores and calculating percentages; Phase 2 interviews were transcribed verbatim and coded by two independent researchers into themes and subthemes until consensus was reached.

Limitations

  • Stated: possible self-selection bias toward more positive attitudes; small participant numbers; the sample may not represent the diverse secondary mental health care depression population; no patients participated in Phase 2; findings are specific to secondary care and cannot simply be translated to primary care.
  • Stated/evident: many participants had no prior experience with online therapy, so benefits, drawbacks, and preferences were mostly perceived; the focus was largely on text-based online therapy, possibly limiting awareness of other technological possibilities.
  • Evident: therapists could not describe a typical patient or a standard blended protocol, limiting consensus on protocolized blended care; the small qualitative sample limits generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12888-014-0355-z

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 49 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    Developing standard blended protocols with a pre-set plan for routing and content of every session was not feasible in the interviews.

    “Therefore, developing standard blended protocols with a pre-set plan for routing and content of every session was not feasible in the interviews”
    Quote in the paperSubstantial claim
  • Findingown data

    Most patients prefer 50 to 60% of sessions online.

    “most patients 50 to 60%”
    • Also stated in: § Phase 1 · hedged
      “Patients are, overall, a little less conservative and would like to perform half or slightly more of the therapy sessions in an online environment (50 to 60%).”
    Quote in the paperSubstantial claim
  • Findingown data

    Preferences of patients and therapists were highly similar on suitable content for online and face-to-face sessions.

    “Preferences of patients and therapists were highly similar on these matters.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Practical therapy components (assignments, diaries and psycho-education) may be provided via online modules according to most respondents.

    “According to most respondents, practical therapy components (assignments, diaries and psycho-education) may be provided via online modules”
    • Also stated in: § Phase 1 · hedged
      “Practical components of treatment, such as assignments, psycho-education and diaries could be performed online according to most respondents.”
    Quote in the paperSubstantial claim
  • Findingown data

    Tailoring treatment to individual patients is essential in secondary mental health care, due to the complexity of their problems.

    “The interviews showed that tailoring treatment to individual patients is essential in secondary mental health care, due to the complexity of their problems.”
    • Also stated in: § Phase 2 · boosted
      “Based on these reactions, it became clear that there is a need for flexible, non-standardized, scenarios for blended therapies, aimed at individual patients.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 45
  • May combine two claims. 23
  • The quote only partly states this claim. 1

  • Classified as probably not a substantial claim.

    Findingown data

    Patients in secondary mental health care suffering from major depression have severe and complicated problems.

    “All therapists emphasized that patients in secondary mental health care, suffering from major depression, have severe and complicated problems.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twelve therapists and nine patients completed the surveys.

    “Twelve therapists and nine patients completed the surveys.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Findingown data

    Innovations in technology should be used to encourage patients to use the program, support usability, and increase discipline to keep using the program.

    “In order to encourage patients to use the program, to support usability, and to increase the discipline to keep using the program, innovations in technology should be used.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    With a large database, online components would not be fixed and protocolled, but therapists could shape and fill in online sessions themselves according to patients' needs and abilities.

    “This way, the online components would not be fixed and protocolled, but therapists could shape and fill in the online sessions themselves, according to the patients’ needs and abilities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Patients in secondary mental health care often suffer from personality disorders as well and experience problems in several areas of life.

    “Patients often suffer from personality disorders as well, and they often experience problems in several areas of life (personal relationships, work, or financially).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    All interviewees answered that a typical patient does not exist in secondary mental health care.

    “all interviewees answered that a typical patient does not exist in secondary mental health care”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Online feedback, reminders and stimuli such as games and collecting points might encourage patients' motivation and discipline to stay in therapy and fulfill assignments.

    “Online feedback, reminders and stimuli (such as games, collecting points) might encourage the motivation and discipline of patients to stay in therapy and to fulfill the assignments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    According to therapists, animations, videos and expressive examples could help patients understand the theory and assignments without reading large amounts of text.

    “According to the therapists, animations, videos and expressive examples could help patients understand the theory and assignments, without reading large amounts of texts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Opinions varied on whether online sessions could weaken patient-therapist bonding.

    “opinions varied with respect to the statement that the use of online sessions could weaken the patient-therapist bonding.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Many practical parts of therapy could be performed in an online environment, while discussing thoughts, feelings and difficulties regarding these practical parts should still be done face-to-face.

    “Our results show that many practical parts of the therapy could be performed in an online environment, while discussing thoughts, feelings and difficulties regarding these practical parts should still be done face-to-face.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both patients and therapists are positive about online and blended therapy and expect more benefits than drawbacks.

    “Our results show that both patients and therapists are positive about online and blended therapy and expect more benefits than drawbacks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    A perceived drawback of online sessions is that certain problems or matters could become more difficult to discuss, due to a lack of non-verbal communication or because patients need intensive support in dealing with their problems.

    “A perceived drawback of online sessions is that certain problems or matters could become more difficult to discuss, due to a lack in non-verbal communication or because patients need intensive support in dealing with their problems.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    It is not just patients and technology that must be suited for online therapy; the intentions and motivations of therapists and management weigh very heavily in the success of blended care.

    “Lastly, it is not just the patients and the technology that must be suited for online therapy; the intentions and motivations from therapists and management weigh very heavy in the succeeding of blended care.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Meta-communication about the form of therapy, how the content will be presented, and for what reason will help explain the rationale of the therapy.

    “Meta-communication about the form of therapy, how the content will be presented, and for what reason, will help to explain the rationale of the therapy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    All interviewees answered that protocols cannot be followed exactly as they are provided.

    “and that protocols cannot be followed exactly as they are provided.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Many respondents thought blended therapy could make therapy more transparent because patients can follow the structure and sequence of sessions.

    “According to many respondents, it could make the therapy more transparent because patients can exactly follow the structure of the therapy and the sequence of the sessions.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    According to therapists, taking patients' characteristics into account is essential to match online therapy with the individual patient.

    “According to the therapists, taking patients’ characteristics into account is essential in order to match the online therapy with the individual patient.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Findingown data

    To show how useful online tools can be, patients can start filling in intake forms right away and start their treatment.

    “To show how useful online tools can be, patients can start filling in intake forms right away and start their treatment.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Therapists stress that the online program should be flexible enough to fit a patient's specific needs.

    “Therapists stress that patients’ characteristics should be carefully taken into account in the choice for blended care, and the online program should be flexible enough to fit a patient’s specific needs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In every part of treatment, patients need extensive guidance in translating the theory and assignments of therapy into their own life and in becoming activated in a way that matches their abilities.

    “In every part of the treatment, patients need extensive guidance in translating the theory and assignments of the therapy into their own life and to become activated in a way that matches their abilities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Therapists predominantly consider the use of face-to-face sessions crucial for creating commitment, activating and motivating patients, monitoring patients, and keeping up the pace of treatment.

    “Predominantly, the use of face-to-face sessions is crucial; to create commitment, to activate and motivate patients, to monitor patients and to keep up the pace of the treatment.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingcitation

    Therapists stress that patients' characteristics should be carefully taken into account in the choice for blended care.

    “Therapists stress that patients’ characteristics should be carefully taken into account in the choice for blended care, and the online program should be flexible enough to fit a patient’s specific needs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Therapists would highly appreciate a large database of available online assignments.

    “Also, the therapists would highly appreciate a large database of available online assignments.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    The largest drawback agreed on was that interpretation problems could occur due to lack of non-verbal communication.

    “and that problems with interpretation could occur due to a lack of non-verbal communication.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Blended therapy was positively perceived among all respondents, especially to enhance self-management of patients.

    “Blended therapy was positively perceived among all respondents, especially to enhance the self-management of patients.”
    • Also stated in: § Phase 1 · plain
      “It can be seen that online sessions are positively perceived, especially to enhance the self-management of patients.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    This study particularly focused on secondary care.

    “Lastly, it should be noted that this study particularly focused on secondary care.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    It proved difficult to recruit participants for such an explorative study, since not many therapists have experience with online treatment, and therapists do not receive extra time to take part in research from their management.

    “It proved to be difficult to recruit participants for such an explorative study, since not many therapists have experience with online treatment, and therapists do not receive extra time to take part in research from their management.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Adding online sessions or modules to regular face-to-face psychotherapy can increase adherence and effectiveness but also increases costs and workload among professionals.

    “While recent studies have shown that adding online sessions or modules to regular face-to-face psychotherapy can increase the adherence and effectiveness of a patients’ treatment, it also increases costs and workload among professionals [17-19].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [17] Meglic M, Furlan M, Kuzmanic M, Kozel D, Baraga D, Kuhar I, Kosir B, Iljaz R, Novak Sarotar B, Dernovsek MZ, Marusic A, Eysenbach G, Brodnik A: Feasibility of an eHealth service to support collaborative depression care: results of a pilot study. J Med Internet Res. 2010, 12: e63-10.2196/jmir.1510. source

      Not checked in the free run.

    • cites [18] Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M: Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res. 2009, 11: e15-10.2196/jmir.1151. source

      Not checked in the free run.

    • cites [19] Robertson L, Smith M, Castle D, Tannenbaum D: Using the Internet to enhance the treatment of depression. Australas Psychiatry. 2006, 14: 413-417. 10.1111/j.1440-1665.2006.02315.x. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Many patients with depression do not receive appropriate treatment because psychotherapy is time-consuming and costly and mental health care institutions often lack capacity.

    “While effective psychotherapy for depression is available, many patients do not receive appropriate treatment since it is time-consuming and costly, and mental health care institutions often do not have the capacity to provide everyone with the care they need [2,3].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [2] Donohue JM, Pincus HA: Reducing the societal burden of depression: a review of economic costs, quality of care and effects of treatment. Pharmacoeconomics. 2007, 25: 7-24. 10.2165/00019053-200725010-00003. source

      Not checked in the free run.

    • cites [3] Richards D, Richardson T: Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012, 32: 329-342. 10.1016/j.cpr.2012.02.004. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Adopting a combination of online and face-to-face therapy, in which online sessions replace or substitute some sessions with a health professional, could offer patients the best of both worlds.

    “To overcome these drawbacks, adopting a combination of online and face-to-face therapy, in which online sessions replace or substitute some (parts) of the sessions with a health professional, could offer patients the best of both worlds [13,20].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Gega L, Marks I, Mataix-Cols D: Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004, 60: 147-157. 10.1002/jclp.10241. source

      Not checked in the free run.

    • cites [20] Postel M, Witting M, Van Gemert-Pijnen L: Blended care in mental health care [Blended behandeling in de geestelijke gezondheidszorg]. Dir Ther. 2013, 33: 210-221.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Problems with adherence to online programs and translation of therapy content into daily life can be difficult for patients.

    “Nevertheless, problems with adherence to the online program and translation of therapy content into daily life can be difficult for patients [8-10].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [8] Gerhards SA, Abma TA, Arntz A, de Graaf LE, Evers SM, Huibers MJH, Widdershoven GA: Improving adherence and effectiveness of computerised cognitive behavioural therapy without support for depression: a qualitative study on patient experiences. J Affect Disord. 2011, 129: 117-125. 10.1016/j.jad.2010.09.012. source

      Not checked in the free run.

    • cites [9] Marks IM, Kenwright M, McDONOUGH M, Whittaker M, Mataix-Cols D: Saving clinicians’ time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med. 2004, 34: 9-18. 10.1017/S003329170300878X. source

      Not checked in the free run.

    • cites [10] Sharry J, Davidson R, McLoughlin O, Doherty G: A service-based evaluation of a therapist-supported online cognitive behavioral therapy program for depression. J Med Internet Res. 2013, 15: e121-10.2196/jmir.2248. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Offering some form of support or guidance during online treatment increases its effectiveness and is associated with higher levels of completion.

    “Most systematic reviews found that offering some form of support or guidance during the online treatment increases its effectiveness and is associated with higher levels of completion [3,6,11].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [3] Richards D, Richardson T: Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012, 32: 329-342. 10.1016/j.cpr.2012.02.004. source

      Not checked in the free run.

    • cites [6] Andersson G, Cuijpers P: Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009, 38: 196-205. 10.1080/16506070903318960. source

      Not checked in the free run.

    • cites [11] Newman MG, Szkodny LE, Llera SJ, Przeworski A: A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy?. Clin Psychol Rev. 2011, 31: 89-103. 10.1016/j.cpr.2010.09.008. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Personal feedback and support are perceived as positive among patients, to optimally use the program and keep them motivated.

    “Studies on the experiences of patients with online therapy also show that personal feedback and support are perceived as positive among patients, to optimally use the program and to keep them motivated [12-14].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [12] Beattie A, Shaw A, Kaur S, Kessler D: Primary-care patients’ expectations and experiences of online cognitive behavioural therapy for depression: a qualitative study. Health Expect. 2009, 12: 45-59. 10.1111/j.1369-7625.2008.00531.x. source

      Not checked in the free run.

    • cites [13] Gega L, Marks I, Mataix-Cols D: Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004, 60: 147-157. 10.1002/jclp.10241. source

      Not checked in the free run.

    • cites [14] Bendelin N, Hesser H, Dahl J, Carlbring P, Nelson KZ, Andersson G: Experiences of guided Internet-based cognitive-behavioural treatment for depression: a qualitative study. BMC Psychiatry. 2011, 11: 107-10.1186/1471-244X-11-107. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The optimal level of support, the nature of support, and essential preconditions to make blended care acceptable and cost-effective have not been studied.

    “The optimal level of support, the nature of the support, and essential preconditions to make blended care acceptable and cost-effective have not been studied as well [11,10,22].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [11] Newman MG, Szkodny LE, Llera SJ, Przeworski A: A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy?. Clin Psychol Rev. 2011, 31: 89-103. 10.1016/j.cpr.2010.09.008. source

      Not checked in the free run.

    • cites [10] Sharry J, Davidson R, McLoughlin O, Doherty G: A service-based evaluation of a therapist-supported online cognitive behavioral therapy program for depression. J Med Internet Res. 2013, 15: e121-10.2196/jmir.2248. source

      Not checked in the free run.

    • cites [22] Kooistra LC, Wiersma JE, Ruwaard JJ, Van Oppen PC, Smit F, Lokkerbol J, Cuijpers P, Riper H: Blended vs. face-to-face cognitive behavioral treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiatry. 2014, 14: 290-10.1186/s12888-014-0290-z. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    There is a need for flexible, non-standardized scenarios for blended therapies aimed at individual patients.

    “Based on these reactions, it became clear that there is a need for flexible, non-standardized, scenarios for blended therapies, aimed at individual patients.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Blended therapy has the potential to bring about increased intensity and continuity of therapy by combining a face-to-face session with online modules within one week, which could possibly decrease the length of therapy in weeks.

    “Blended therapy has the potential to bring about an increased intensity and continuity of therapy, by combining a face-to-face session with online modules within one week, which could possibly decrease the length of the therapy in weeks.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The amount and ratio of online modules needs to be adjusted according to the patient's problems, skills and characteristics.

    “The amount and ratio of online modules needs to be adjusted according to the patient’s problems, skills and characteristics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The CeHRes-roadmap uses a bottom-up process to incorporate the context of a particular health care setting and the values of all stakeholders and end-users to improve implementation and impact of eHealth technologies.

    “This roadmap uses a bottom-up process to incorporate the context of a particular health care setting and the values of all stakeholders and end-users of an intervention, to improve the implementation and impact of eHealth technologies.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] Van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER: A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011, 13: e111-10.2196/jmir.1672. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Using other forms of media such as video or gamification elements could broaden the potential of online and blended therapy for many people.

    “Using other forms of media such as video, or gamification elements could broaden the potential of online and blended therapy for many people.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Categories were discussed between the two researchers until consensus about final categories was reached.

    “Subsequently, these categories were discussed between the two researchers, until consensus about the final categories was reached.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Of twelve therapists in round 1, nine agreed to participate in an interview.

    “Of the twelve therapists in round 1, nine agreed to participate in an interview.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The audiotapes of the interviews were transcribed verbatim and coded by two independent researchers.

    “the audiotapes of the interviews were transcribed verbatim and coded by two independent researchers.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    Using the first steps of an eHealth intervention development roadmap, this study offers a helpful starting point for the implementation of blended care for depression in secondary mental health care.

    “Using the first steps of an eHealth intervention development roadmap [23], this study offers a helpful starting point for the implementation of blended care for depression in secondary mental health care.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER: A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011, 13: e111-10.2196/jmir.1672. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Qualitative studies have previously shown essentials for adherence and effect of online therapy.

    “This agrees with what has been previously shown in qualitative studies on the essentials for adherence and effect of online therapy [13,8].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] Gega L, Marks I, Mataix-Cols D: Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004, 60: 147-157. 10.1002/jclp.10241. source

      Not checked in the free run.

    • cites [8] Gerhards SA, Abma TA, Arntz A, de Graaf LE, Evers SM, Huibers MJH, Widdershoven GA: Improving adherence and effectiveness of computerised cognitive behavioural therapy without support for depression: a qualitative study on patient experiences. J Affect Disord. 2011, 129: 117-125. 10.1016/j.jad.2010.09.012. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Involvement of end-users in the early stages of development is crucial to create commitment and determine requirements for design.

    “The involvement of end-users in the early stages of development is crucial to create commitment and to determine the requirements for design.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The first researcher examined the raw data again to ensure robustness and confirm all data reflected in coding.

    “Next, the first researcher (RV) examined the raw data again to ensure the robustness of the analytical process and to confirm that all the data was indeed reflected in the coding [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Patton M: Qualitative Research & Evaluation Methods. 1990, Sage, Newbury Park, CA, 2

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A database of online modules could provide flexibility to tailor treatment to individual patients.

    “A database of online modules could provide flexibility to tailor treatment to individual patients”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    An online intervention is not just a simple translation of a regular face-to-face therapy to an online environment.

    “An online intervention is not just a simple translation of a regular face-to-face therapy to an online environment [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Kelders SM, Pots WTM, Oskam MJ, Bohlmeijer ET, van Gemert-Pijnen JEWC: Development of a web-based intervention for the indicated prevention of depression. BMC Med Inform Decis Mak. 2013, 13: 26-10.1186/1472-6947-13-26. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In round 2, eight out of nine patients completed the survey.

    “In round 2, eight out of nine patients completed the survey”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationnone given

    In the intake period and for practical assignments, online tools could be used in a low key manner, from which the database of online modules can be built up.

    “In the intake period and for practical assignments online tools could be used in a low key manner, from which the database of online modules can be built up.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    This could increase knowledge on how to tailor blended care to individual patients, related to the severity of their symptoms and their characteristics.

    “This could increase our knowledge on how to tailor blended care to individual patients, related to the severity of their symptoms and their characteristics [41,42].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [41] Carlbring P, Maurin L, Törngren C, Linna E, Eriksson T, Sparthan E, Strååt M, Marquez von Hage C, Bergman-Nordgren L, Andersson G: Individually-tailored, Internet-based treatment for anxiety disorders: a randomized controlled trial. Behav Res Ther. 2011, 49: 18-24. 10.1016/j.brat.2010.10.002. source

      Not checked in the free run.

    • cites [42] Johansson R, Sjöberg E, Sjögren M, Johnsson E, Carlbring P, Andersson T, Rousseau A, Andersson G: Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One. 2012, 7: e36905-10.1371/journal.pone.0036905. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationnone given

    It is possible that the patients and therapists who participated in the study have a more positive attitude to online therapy than non-responders (self-selection bias).

    “It is possible that the patients and therapists that participated in our study have a more positive attitude to online therapy than non-responders (self-selection bias).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Although this study adds new insights to the body of literature, its results should be interpreted with care.

    “Although this study adds new insights to the body of literature, our results should be interpreted with care.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    Concurrent with the roadmap, thorough development from the start of an online intervention is inevitable for proper use and effects of eHealth in general, and also of blended care.

    “Nevertheless, concurrent with the roadmap [23], thorough development from the start of an online intervention is inevitable for proper use and effects of eHealth in general, and also of blended care.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER: A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011, 13: e111-10.2196/jmir.1672. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    When both patients and therapists become more adept at online therapy and experience the possibilities of technology, blending online sessions into face-to-face therapy could create a treatment using the best of both worlds.

    “When both patients and therapists become more adepted to online therapy and experience the possibilities of technology, blending online sessions into face-to-face therapy could create a treatment using the best of both worlds.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In a study by Wright et al., patients were seen every week by a therapist, but half of the sessions were done independently by the patient behind a computer at the mental health care institution.

    “In a study by Wright et al. [21], patients were seen every week by a therapist, but half of the sessions were done independently by the patient behind a computer, at the mental health care institution.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, Otto MW: Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005, 162: 1158-1164. 10.1176/appi.ajp.162.6.1158. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Log files will provide further insight into the suitability among groups and in the effectiveness of blended therapy from the perspective of what actually happens during therapy.

    “This will provide further insight into the suitability among groups and in the effectiveness of blended therapy from a different perspective, namely from what actually happens during the therapy [40].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Kelders SM, Bohlmeijer ET, Van Gemert-Pijnen JE: Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med Internet Res. 2013, 15: e172-10.2196/jmir.2258. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    This study provides a first exploration of how the two modes of delivery could be blended optimally, integrated as equal parts of treatment.

    “Our study provides a first exploration of how the two modes of delivery could be blended optimally, integrated as equal parts of treatment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A recent study by Høifødt et al. investigated the effect of a guided web-based intervention with face-to-face support, with positive results on depression and anxiety symptoms.

    “A recent study by Høifødt et al. [35] investigated the effect of a guided web-based intervention with face-to-face support, with positive results on depression and anxiety symptoms.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, Kolstrup N: The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res. 2013, 15: e153-10.2196/jmir.2714. source

      Not checked in the free run.

59 of 59

  • The paper relies on this without support.

    Patients receiving blended therapy would have actual duration of therapy stay equal or become shorter.

    “while the actual duration of therapy stays equal or could become shorter.”

  • The paper relies on this without support.

    The preferences and opinions of the included patients and therapists are representative of patients and therapists in secondary mental health care.

    “The aim of this study was to outline optimal usage of blended care for depression, according to patients and therapists.”

  • The paper relies on this without support.

    A Delphi consensus among a small sample of patients and therapists can identify optimal usage of blended care.

    “A Delphi method was used [24], which aimed to find consensus on suited protocols, concerning content, sequence and ratio to blend online and face-to-face therapy for depression.”

  • The paper relies on this without support.

    Online therapy is still scarce in clinical practice.

    “Because online therapy is still scarce in clinical practice many therapists and patients are probably not aware of the large amount of other possibilities that technology can offer.”

  • The paper relies on this without support.

    Therapists have experience with regular protocols for depression therapy among a broad range of patients.

    “This choice was made because therapists have experience with regular protocols for depression therapy among a broad range of patients.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • EA and MS created the interview guide with the help and synthesis of interview guidelines from a number of studies that qualitatively examined user experiences in applications developed in the field of digital health.

    “EA and MS created the interview guide with the help and synthesis of interview guidelines from a number of studies that qualitatively examined user experiences in applications developed in the field of digital health [45-49].”

    From Barriers to and Facilitators of a Blended Cognitive Behavioral Therapy Program for Depression and Anxiety Based on Experiences of University Students: Qualitative Interview Study (2023) · cites this paper as [49]

    The Methods (Phase 2 and Data-analysis; page not provided) describe an interview scheme and qualitative analysis of therapists’ views on blended online and face-to-face care. The interviews addressed how care could be designed and used, rather than users’ experiences with a developed digital-health application. The source can therefore serve as an example of an interview-based digital-health study, but not fully of the kind of user-experience study described. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The perceived benefits of this bCBT program are in line with many of the explored expectations of patients receiving standard CBT but who have not yet had experience with blended formats.

    “Interestingly, our study showed that the perceived benefits of this bCBT program are in line with many of the explored expectations of patients receiving standard CBT but who have not yet had an experience with blended formats [38,49].”

    From Barriers to and Facilitators of a Blended Cognitive Behavioral Therapy Program for Depression and Anxiety Based on Experiences of University Students: Qualitative Interview Study (2023) · cites this paper as [49]

    The Results (Phase 1; page unavailable) report that patients anticipated benefits of online sessions and that eight of nine had no experience with online treatment. The Discussion (page unavailable) likewise describes expected benefits of blended therapy. The paper does not establish that these patients were receiving standard CBT, and one patient was already following online therapy. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Reaching consensus can take time and can be complex, meaning Delphi studies can require a lot of time from researchers and participants, which is not always available in eHealth development processes.

    “Third, reaching consensus can take time and can be complex. This means that Delphi studies can require a lot of time from researchers and participants, which is not always available in eHealth development processes”

    From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [101]

  • It can be challenging to recruit participants for a Delphi study because, for example, there are not many experts in a specific or new field, which can be especially relevant when studying novel applications of an eHealth technology.

    “Second, it can be challenging to recruit participants for a Delphi study because, for example, there are not many experts in a specific or new field, which can be especially relevant when studying novel applications of an eHealth technology”

    From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [101]

  • A recent Delphi study suggested that blended treatment is positively perceived by patients and therapists.

    “Further, a recent Delphi study suggested that blended treatment is positively perceived by patients and therapists (Van der Vaart et al., 2014).”

    From Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care (2016) · cites this paper as Van der Vaart et al., 2014

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The abstract calls a 33% preference the choice of most therapists

    Minor

    The abstract says most therapists prefer 75% face-to-face sessions, but Table 3 shows that only four of twelve therapists (33%) chose that ratio.

  • Six of eight patients is reported as 80%

    Minor

    The table reports 80% for six of eight patients, but 6/8 is 75%. The same 80 (6) discrepancy appears in four Table 1 rows.

    Reported 80 · recomputed 75

  • Not all therapists indicated a need for face-to-face contact

    Minor

    The text attributes this view to all therapists and patients, but Table 1 records agreement from only ten of twelve therapists; two left the item blank.

3 of 3

Cites

Flagged first, then by citations

1 of the 43 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Van der Vaart R, Van Gemert-Pijnen L, Bohlmeijer E: Eindverslag Ontwikkeling en Evaluatie van Gepaste Blended Behandeling bij Depressie [Final Report Development and Evaluation of Suitable Blended Care for Depression]. 2013, University of Twente, Enschede
  2. Semlianoi B, Wentzel J: Co-creatie met zorgverleners: Blended Care instrument [co-creation with health care professionals: Blended Care instrument] Presentation 4th NSRII (Netherlands’ Society for Research on Internet Interventions) symposium ‘Designing Persuasive Technology to Improve eHealth Interventions’. June 20th 2014, Enschede: University of Twente.

Cited by

Sorted by citations

Related

Authors
van der Vaart, R., Witting, M., Riper, H., Kooistra, L., Bohlmeijer, E. T., & van Gemert-Pijnen, L. J.
Affiliations
  • Saxion (NL)
  • Vrije Universiteit Amsterdam (NL)
  • University of Twente (NL)
ORCID
Venue
BMC Psychiatry
Publisher
Springer Science and Business Media LLC
Published
2014-12
Volume
14
Issue
1
Pages
355-355
ISSN
1471-244X (ISSN-L)
DOI
10.1186/s12888-014-0355-z
PMC
PMC4271498
OpenAlex
W2117200319
Cited by
273
References
43
Citations by year
2026: 13 · 2025: 16 · 2024: 13 · 2023: 24 · 2022: 28 · 2021: 44 · 2020: 46 · 2019: 14 · 2018: 26 · 2017: 24 · 2016: 19 · 2015: 6
FWCI
25.32
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Psychiatry (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
  • University of Twente Research Information (publishedVersion, cc-by)
  • Leiden Repository (Leiden University) (submittedVersion)
  • Pure Amsterdam UMC (submittedVersion)
  • VU Research Portal (submittedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • University of Twente Research Information (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
Funders
Innovatiefonds Zorgverzekeraars
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
  • Social Sciences › Education › Online and Blended Learning
Keywords
blended care, depression, online therapy, face-to-face therapy, online modules, therapist preferences, patient preferences, self-management, psychoeducation, secondary mental health care, treatment tailoring, therapist skills
MeSH terms
Adult, Attitude of Health Personnel, Attitude to Health, Clinical Protocols, Cost-Benefit Analysis, Delphi Technique, Depressive Disorder, Female, Health Personnel, Humans, Male, Middle Aged, Motivation, Personal Satisfaction, Psychotherapy, Surveys and Questionnaires, Self Care, Patient Satisfaction, Remote Consultation, Internet, Consensus, Young Adult, Patient Preference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review