A Web-Based Mental Health Platform for Individuals Seeking Specialized Mental Health Care Services: Multicenter Pragmatic Randomized Controlled Trial
Open accessPublished version of a preprint
The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Abstract
Background: Web-based self-directed mental health applications are rapidly emerging to address health service gaps and unmet needs for information and support. Objective: The aim of this study was to determine if a multicomponent, moderated Web-based mental health application could benefit individuals with mental health symptoms severe enough to warrant specialized mental health care. Methods: A multicenter, pragmatic randomized controlled trial was conducted across several outpatient mental health programs affiliated with 3 hospital programs in Ontario, Canada. Individuals referred to or receiving treatment, aged 16 years or older, with access to the internet and an email address, and having the ability to navigate a Web-based mental health application were eligible. A total of 812 participants were randomized 2:1 to receive immediate (immediate treatment group, ITG) or delayed (delayed treatment group, DTG) access for 3 months to the Big White Wall (BWW), a multicomponent Web-based mental health intervention based in the United Kingdom and New Zealand. The primary outcome was the total score on the Recovery Assessment Scale, revised (RAS-r) which measures mental health recovery. Secondary outcomes were total scores on the Patient Health Questionnaire-9 item (PHQ-9), the Generalized Anxiety Disorder Questionnaire-7 item (GAD-7), the EuroQOL 5-dimension quality of life questionnaire (EQ-5D-5L), and the Community Integration Questionnaire. An exploratory analysis examined the association between actual BWW use (categorized into quartiles) and outcomes among study completers. Results: Intervention participants achieved small, statistically significant increases in adjusted RAS-r score (4.97 points, 95% CI 2.90 to 7.05), and decreases in PHQ-9 score (−1.83 points, 95% CI −2.85 to −0.82) and GAD-7 score (−1.55 points, 95% CI −2.42 to −0.70). Follow-up was achieved for 55% (446/812) at 3 months, 48% (260/542) of ITG participants and 69% (186/270) of DTG participants. Only 58% (312/542) of ITG participants logged on more than once. Some higher BWW user groups had significantly greater improvements in PHQ-9 and GAD-7 relative to the lowest use group. Conclusions: The Web-based application may be beneficial; however, many participants did not engage in an ongoing way. This has implications for patient selection and engagement as well as delivery and funding structures for similar Web-based interventions. Trial Registration: ClinicalTrials.gov NCT02896894; https://clinicaltrials.gov/ct2/show/NCT02896894 (Archived by WebCite at http://www.webcitation.org/78LIpnuRO)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the last 10 years, several large implementation studies of DMHIs have shown that the majority of patients offered these interventions do not engage at the recommended frequency or complete the full course of treatment [4-6].”
From Digital Mental Health Interventions for Depression: Scoping Review of User Engagement (2022) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similarly, concerns related to sustained engagement with web-based psychiatric interventions have been reported in routine-care implementation studies [4-6,17].”
From Digital Mental Health Interventions for Depression: Scoping Review of User Engagement (2022) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · cited 1×
“All data were entered into a REDCap database [19].”
Psychometric properties of the credibility/expectancy questionnaire
Devilly, G. J., & Borkovec, T. D. (2000) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/s0005-7916(00)00012-4 · Cited by 2,295 · cited 1×
“Item 4 from the Credibility and Expectancy Questionnaire [17] was adapted for this study, “By the end of the [BWW access period], how much improvement in your symptoms do you think will occur?””
Monitoring Depression Treatment Outcomes With the Patient Health Questionnaire-9
L??we, B., Un??tzer, J., Callahan, C. M., Perkins, A. J., & Kroenke, K. (2004) · Medical Care · doi:10.1097/00005650-200412000-00006 · Cited by 1,354 · cited 2×
“These statically significant findings are limited by high, differential drop out between treatment groups and overall, and were below minimal clinically important differences for these outcome measures (eg, a difference of 1.8 on the PHQ-9 where a clinically important change is 5 points) [22].”
InfluentialEysenbach, G., Powell, J., Englesakis, M., Rizo, C., & Stern, A. (2004) · BMJ · doi:10.1136/bmj.328.7449.1166 · Cited by 1,268
Whiteford, H. A., Ferrari, A. J., Degenhardt, L., Feigin, V., & Vos, T. (2015) · PLOS ONE · doi:10.1371/journal.pone.0116820 · Cited by 1,175 · cited 2×
“In addition, mental and substance use disorders are emerging as a leading cause of disability, accounting for nearly 10% of global disability-adjusted life years [1].”
Adherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 3×
“More specifically, other studies of Web-based mental health programs have also reported disproportionate dropout in the intervention group [10].”
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097 · cited 1×
“…apps and Web-based treatment programs for common mental disorders have demonstrated small-to-moderate treatment effect sizes for symptom reduction [6-8], although these interventions are commonly recommended as standalone or preventative treatment options for those with milder symptoms, where…”
Barriers to mental health treatment: results from the National Comorbidity Survey Replication
Mojtabai, R., Olfson, M., Sampson, N. A., Jin, R., Druss, B., Wang, P. S., Wells, K. B., Pincus, H. A., & Kessler, R. C. (2011) · Psychological Medicine · doi:10.1017/s0033291710002291 · Cited by 951
A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies
Donkin, L., Christensen, H., Naismith, S. L., Neal, B., Hickie, I. B., & Glozier, N. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1772 · Cited by 876 · cited 2×
“A systematic review examining the relationship between e-therapy adherence and outcomes reported that studies targeting depression did not find a significant impact of total number of logins on outcomes in contrast to studies examining physical health outcomes like weight management or smoking [9].”
Michie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865 · cited 2×
“Compared with a modular program, it is more difficult to define an adequate dose for multicomponent interventions such as the BWW [13].”
Cited by
Lipschitz, J. M., Pike, C. K., Hogan, T. P., Murphy, S. A., & Burdick, K. E. (2023) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-023-00297-3 · Cited by 225
Digital Mental Health Interventions for Depression: Scoping Review of User Engagement
Lipschitz, J. M., Van Boxtel, R., Torous, J., Firth, J., Lebovitz, J. G., Burdick, K. E., & Hogan, T. P. (2022) · Journal of Medical Internet Research · doi:10.2196/39204 · Cited by 105
“Similarly, concerns related to sustained engagement with web-based psychiatric interventions have been reported in routine-care implementation studies [4-6 , 17].”
Egmose, C. H., Poulsen, C. H., Hjorthøj, C. R., Mundy, S. S., Hellström, L. C., Nielsen, M. N., Korsbek, L., Rasmussen, K. S., & Eplov, L. F. (2023) · Psychiatric Services · doi:10.1176/appi.ps.202100138 · Cited by 71
“Four trials evaluating “ peers online ” comprised unmoderated and moderated online mutual support groups (80 – 87).”
BackgroundResultInfluentialSmit, D., Vrijsen, J. N., Groeneweg, B. F., Vellinga‐Dings, A., Peelen, J., & Spijker, J. (2021) · Journal of Medical Internet Research · doi:10.2196/25917 · Cited by 66
“Recent RCTs on MHISGs indicate that high user engagement quantified in terms of the number of posts [29] or login frequencies [31] is relevant for attaining health gains [29,31].”
Budhwani, S., Fujioka, J. K., Chu, C., Baranek, H., Pus, L., Wasserman, L., Vigod, S. N., Martin, D., Agarwal, P., & Mukerji, G. (2021) · JMIR Formative Research · doi:10.2196/30280 · Cited by 31
“Similar opportunities from and barriers to adoption have also been noted in mental health contexts [15-18].”
BackgroundInfluentialSelf-citationCumyn, A., Barton, A., Dault, R., Safa, N., Cloutier, A., & Éthier, J. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00647-x · Cited by 28
Kolaas, K., Berman, A. H., Hedman‐Lagerlöf, E., Lindsäter, E., Hybelius, J., & Axelsson, E. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-075796 · Cited by 30
Marinova, N., Rogers, T., & MacBeth, A. M. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.05.058 · Cited by 20
Morriss, R. K., Kaylor‐Hughes, C., Rawsthorne, M., Coulson, N., Simpson, S., Guo, B., James, M., Lathe, J., Moran, P., Tata, L. J., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/23487 · Cited by 21
“Better recruitment and retention rates were obtained in another trial of BWW in mental health service users involving more human support [10].”
Gordon, D., Hensel, J. M., Bouck, Z., Desveaux, L., Soobiah, C., Saragosa, M., Jeffs, L. P., Bhatia, S., & Shaw, J. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03391-z · Cited by 16
Self-citationREVIEW OF INNOVATIVE APPROACHES TO MENTAL HEALTH TELETHERAPY: ACCESS AND EFFECTIVENESS
Odugbose, T., Adegoke, B. O., & Adeyemi, C. (2024) · International Medical Science Research Journal · doi:10.51594/imsrj.v4i4.1022 · Cited by 19
Smit, D., Vrijsen, J. N., Broekman, T. G., Groeneweg, B. F., & Spijker, J. (2022) · JMIR Formative Research · doi:10.2196/39912 · Cited by 15
“In particular, for online PSIs, a high level of user engagement is considered a crucial factor for recovery [24,30,31].”
BackgroundLo, B., Durocher, K., Charow, R., Kimball, S., Phạm, Q., Sockalingam, S., Wiljer, D., & Strudwick, G. (2025) · Journal of Medical Internet Research · doi:10.2196/67820 · Cited by 11
“…playbook, among others, has led to the uptake of many digital mental health tools across health systems, including Big White Wall (now known as Togetherall) in Canada and the United Kingdom [12-14] and the Digital Opportunities for Outcomes in Recovery Services program in the United States [15-19].”
Hamilton, D., Taylor, C., & Maben, J. (2023) · Perspectives on Medical Education · doi:10.5334/pme.930 · Cited by 10
Wanniarachchi, V. U., Greenhalgh, C., & Warren, J. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100884 · Cited by 14
“Notably, some studies have reported low rates of sustained engagement and intervention completion when conducted in real-world settings (Fleming et al., 2018; Hensel et al., 2019; Lattie et al., 2016).”
Syarifa, D. F. P., Moeljono, A. A. K., Hidayat, W. N., Shafelbilyunazra, A., Abednego, V. K., & Prasetya, D. D. (2024) · International Conference on Electrical and Information Technology (IEIT) · doi:10.1109/ieit64341.2024.10763282 · Cited by 8
“An additional inconsistency is relevant to patient selection and engagement, as well as the delivery and financing models of web-based interventions that are very similar in scope [20].”
Desveaux, L., Budhwani, S., Stamenova, V., Bhattacharyya, O. K., Shaw, J., & Bhatia, R. S. (2021) · Journal of Medical Internet Research · doi:10.2196/25797 · Cited by 7
“Small but statistically significant benefits in mental health recovery, anxiety, and depression symptoms were observed among participants who received immediate access [34].”
InfluentialSelf-citationBert, F., Moro, G. L., Giacomini, G., Minutiello, E., & Siliquini, R. (2023) · Health Promotion International · doi:10.1093/heapro/daad006 · Cited by 5
Transforming College Counseling: An In-House Solution for Student Mental Wellness
Dalvi, A., Kapur, P., Ashtekar, R., Gorugantu, A., & Siddavatam, I. N. A. (2023) · International Conference for Innovation in Technology (INOCON) · doi:10.1109/inocon57975.2023.10101364 · Cited by 5
“A randomized controlled trial was conducted to determine whether a particular web-based mental health program for college students would be effective [4].”
Intervenções on-line em saúde mental em tempos de COVID-19: Revisão sistemática
Secchi, Á., Dullius, W. R., Garcez, L., & Scortegagna, S. A. (2021) · Revista Brasileira de Psicoterapia · doi:10.5935/2318-0404.20210013 · Cited by 7
Cumyn, A., Barton, A., Dault, R., Safa, N., Cloutier, A., & Éthier, J. (2021) · doi:10.21203/rs.3.rs-137973/v1 · Cited by 3
Pelgrim, T. A. D., Schaars, K. K., Moldovan, R., Deen, M., & van Westrhenen, R. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2026.121250 · Cited by 4
Automation of the psychological care process in university students using machine learning
Arroni, J. G., Tresierra, C. A., & Burga-Durango, D. (2023) · IEEE International Conference on Advanced Learning Technologies on Education & Research (ICALTER) · doi:10.1109/icalter61411.2023.10372924 · Cited by 2
“In [7] Big White Wall (BWW) was implemented, a web-based application focused on users suffering from some mental illness.”
MethodLipschitz, J. M., Boxtel, R. V., Torous, J. B., Firth, J., Lebovitz, J. G., Burdick, K. E., & Hogan, T. P. (2022) · doi:10.2196/preprints.39204
Smit, D., Vrijsen, J. N., Broekman, T. G., Groeneweg, B. F., & Spijker, J. (2022) · doi:10.2196/preprints.39912
Casoy, F., Cuyler, R. N., & Fishkind, A. B. (2022) · Springer eBooks · doi:10.1007/978-3-031-10239-4_54
Desveaux, L., Budhwani, S., Stamenova, V., Bhattacharyya, O. K., Shaw, J., & Bhatia, R. S. (2023) · doi:10.32920/24191463
Self-citationDesveaux, L., Budhwani, S., Stamenova, V., Bhattacharyya, O. K., Shaw, J., & Bhatia, R. S. (2023) · doi:10.32920/24191463.v1
Self-citationUso de tecnologia no acompanhamento da saúde mental do trabalhador: uma revisão de escopo
Barbosa, M. D. S. A., do Nascimento, K. C., de Lima, T. N. B., Costa, W. T., Santos, A. K. T. D., Souza, M. E., Neves, A. B. S., Lira, R., Silva, K. N. F., da Silva, F. P., et al. (2023) · Contribuciones a las Ciencias Sociales · doi:10.55905/revconv.16n.10-058
Ribanszki, R., Oziel, R., Fonseca, J. A. S., & POTTS, H. W. W. (2025) · doi:10.2196/preprints.73500
Diop, M., Lionel Perrier, A., Haon, B., Rochaix, L., Behaghel, L., Dupont, J. K., Morelle, M., Elias, M., Esposito, L., Legendre, C. M., et al. (2025) · The European Journal of Health Economics · doi:10.1007/s10198-025-01828-w
Cumyn, A., Barton, A., Dault, R., Safa, N., Cloutier, A., & Éthier, J. (2021) · Figshare · doi:10.6084/m9.figshare.14880651
Cumyn, A., Barton, A., Dault, R., Safa, N., Cloutier, A., & Éthier, J. (2021) · Figshare · doi:10.6084/m9.figshare.14880648
Empowering Mental Health Support in Panama: A Web Application
Singh, J., Chong, K., & Bonilla-Morales, B. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16761-3_5
Rainbow, C., Khanna, R., & Green, R. (2026) · Australian Psychologist · doi:10.1080/00050067.2026.2650144
“…including (1) self-guided internet or mobile app-based programmes that provide social connection and psychoeducation (e.g., de Almeida et al., 2018; Hensel et al., 2019), and (2) guided one-to-one or group supports delivered via telehealth mechanisms that provide tailored coaching and goal-setting…”
InfluentialTelehealth Treatments of Common Psychological Disorders: An Overview and Meta-analysis
Gerdes, R., Roberts, R., Jackson, T., Premji, R., Deibert, D., Choi, C., Steer, K. J. D., Bani‐Fatemi, A., Dennett, L., Lytvyak, E., et al. (2026) · Journal of Occupational Rehabilitation · doi:10.1007/s10926-026-10406-7
Madrid‐Cagigal, A., Ryan, L., Kealy, C., McNulty, J., Mulvenna, M. D., Byrne, M., Barry, M. M., & Donohoe, G. J. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2675626 · Cited by 1
Lengier, M., Zych, S., Powęska, N., Świstak, S., Pastuszak, F. C., Małyszek, P., Świderska, M., Buziak, J., Papińska, J., & Pielacha, J. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.2(50).2026.5736
Charles, A., Rennick‐Egglestone, S., Fortuna, K. L., Coulson, N., Møeller, S. B., Geeleher, M., Harrison, J., Holden, M. K., Slade, M., & Ng, F. (2026) · Journal of Medical Internet Research · doi:10.2196/99367
Metadata
- Authors
- Hensel, J. M., Shaw, J., Ivers, N. M., Desveaux, L., Vigod, S. N., Cohen, A., Onabajo, N., Agarwal, P., Mukerji, G., Yang, R., Nguyen, M., Bouck, Z., Wong, I., Jeffs, L., Jamieson, T., & Bhatia, R. S.
- Affiliations
- University of Toronto (CA)
- Women’s College Hospital Research and Innovation Institute (CA)
- St. Michael's Hospital (CA)
- Corresponding author
- Hensel, Jennifer M
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-06-04
- Volume
- 21
- Issue
- 6
- Pages
- e10838
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6684216
- OpenAlex
- W2941120290
- Cited by
- 39
- References
- 31
- Citations by year
- 2026: 7 · 2025: 4 · 2024: 3 · 2023: 9 · 2022: 6 · 2021: 9 · 2020: 1
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- mental health recovery, depression, anxiety, quality of life, social reintegration, treatment delay, online intervention, access to mental health care, user engagement, Recovery Assessment Scale, PHQ-9, GAD-7, EQ-5D-5L, Community Integration Questionnaire, Ontario, New Zealand
- MeSH terms
- Adult, Chromonar, Female, Humans, Male, Mental Health, Mental Health Services, Quality of Life, Surveys and Questionnaires, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.