Examining an App-Based Mental Health Self-Care Program, IntelliCare for College Students: Single-Arm Pilot Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Background: In recent years, there has been an increase in symptoms of depression, anxiety, and other mental illnesses in college student populations alongside a steady rise in the demand for counseling services. Digital mental health programs, such as those delivered through mobile apps, can add to the array of available services but must be tested for usability and acceptability before implementation. Objective: This study aims to examine how students used IntelliCare for College Students over an 8-week period to examine the preliminary associations between app use and psychosocial targets and to gather user feedback about usability issues that need to be remedied before a larger implementation study. Methods: IntelliCare for College Students is an app-based platform that provides symptom assessments with personalized feedback, information about campus resources, lessons on mental health and wellness topics, and access to the suite of interactive skill–focused IntelliCare apps. A total of 20 students were recruited to participate in an 8-week study. To test for a broad range of potential users, we recruited a mixed sample of students with elevated symptoms of depression or anxiety and students without elevated symptoms. Participants completed psychosocial questionnaires at baseline, week 4, and week 8. Participants also completed user feedback interviews at weeks 4 and 8 in which they provided feedback on their experience using the app and suggestions for changes they would like to be made to the app. Results: Of the 20 students who downloaded the app, 19 completed the study, indicating a high rate of retention. Over the study period, participants completed an average of 5.85 (SD 2.1; range 1-8) symptom assessments. Significant improvements were observed in the Anxiety Literacy Questionnaire scores (Z=−2.006; P=.045) and in the frequency with which participants used both cognitive (Z=−2.091; P=.04) and behavioral (Z=−2.249; P=.03) coping skills. In the feedback interviews, we identified a high degree of usability with minor bugs in the app software, which were quickly fixed. Furthermore, in feedback interviews, we identified that users found the app to be convenient and appreciated the ability to use the program in short bursts of time. Conclusions: The findings indicate that the IntelliCare for College Students program was perceived as largely usable and engaging. Although the program demonstrated usability and preliminary benefits to students, further testing is needed to determine its clinical utility among college students. Trial Registration: ClinicalTrials.gov NCT04035577; https://clinicaltrials.gov/ct2/show/NCT04035577
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The most common assignment revolved around using the app once daily. This could be one prompt, one session, or one exercise daily, usually taking about ten minutes to complete”
From Analysis of Mobile App-Based Mental Health Solutions for College Students: A Rapid Review (2023) · cites this paper as [16]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Two studies found no effective results but indications of substantial potential developments in the future [16,24].”
From Analysis of Mobile App-Based Mental Health Solutions for College Students: A Rapid Review (2023) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
The PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898 · cited 6×
“To ensure a range of symptoms of depression and anxiety, we attempted to recruit a balanced sample of participants who had higher symptoms (defined as a baseline PHQ-9 or GAD-7 score ≥10, indicating at least moderate symptoms) and those who had lower symptoms (defined as baseline PHQ-9 and GAD-7 scores both 10) from e…”
MethodInfluentialThe PHQ-8 as a measure of current depression in the general population
Kroenke, K., Strine, T. W., Spitzer, R. L., Williams, J. B. W., Berry, J. T., & Mokdad, A. H. (2009) · Journal of Affective Disorders · doi:10.1016/j.jad.2008.06.026 · Cited by 5,121 · cited 7×
“The PHQ-8 was selected as the symptom check tool because the IntelliCare for College Students app was designed as a self-guided resource that, following this initial study, would be made freely available to all interested students at these universities.”
MethodInfluentialGulliver, A., Griffiths, K. M., & Christensen, H. (2010) · BMC Psychiatry · doi:10.1186/1471-244x-10-113 · Cited by 3,292
Adolescent Onset of the Gender Difference in Lifetime Rates of Major Depression: A Theoretical Model
Cyranowski, J. M., Frank, E., Young, E., & Shear, M. K. (2000) · Archives of General Psychiatry · doi:10.1001/archpsyc.57.1.21 · Cited by 1,276
Help-Seeking and Access to Mental Health Care in a University Student Population
Eisenberg, D., Golberstein, E., & Gollust, S. E. (2007) · Medical Care · doi:10.1097/mlr.0b013e31803bb4c1 · Cited by 1,131 · cited 4×
“The KBSS includes 5 questions designed to measure knowledge of campus mental health services (eg, “What have you heard from other students about the quality of mental health and psychological counseling services on your campus?”).”
MethodInfluentialBeyond the five-user assumption: benefits of increased sample sizes in usability testing
Faulkner, L. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195514 · Cited by 1,053 · cited 1×
“previous data, suggesting that 20 users can find 95% of usability problems [14].”
MethodObjective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965 · cited 3×
“Finally, because participants had committed to an 8-week study period, usage rates may have been higher than would have been in a more naturalistic study design [22].”
BackgroundInfluentialLipson, S. K., Lattie, E. G., & Eisenberg, D. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800332 · Cited by 947
Self-citationLattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843 · cited 1×
“A systematic review found that the majority of digital mental health interventions for college students were effective in producing changes in psychological outcomes, such as depression and anxiety, yet few of the tested programs were publicly available and fewer were delivered via apps [3].”
BackgroundSelf-citationGriffiths, K. M., Christensen, H., Jorm, A. F., Evans, K., & Groves, C. (2004) · British Journal of Psychiatry · doi:10.1192/bjp.185.4.342 · Cited by 789
Woolson, R. F. (2008) · Wiley Encyclopedia of Clinical Trials · doi:10.1002/9780471462422.eoct979 · Cited by 763
Duffy, M. E., Twenge, J. M., & Joiner, T. E. (2019) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2019.04.033 · Cited by 579 · cited 1×
“Over the last decade, the number of college students experiencing symptoms of depression and anxiety has increased [1].”
BackgroundICBT in routine care: A descriptive analysis of successful clinics in five countries
Titov, N., Dear, B., Nielssen, O., Staples, L., Hadjistavropoulos, H., Nugent, M., Adlam, K., Nordgreen, T., Bruvik, K. H., Hovland, A., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.07.006 · Cited by 353
Wiley Encyclopedia of Clinical Trials
(2007) · Wiley eBooks · doi:10.1002/9780471462422 · Cited by 341
IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety
Mohr, D. C., Tomasino, K. N., Lattie, E. G., Palac, H. L., Kwasny, M. J., Weingardt, K., Karr, C. J., Kaiser, S. M., Rossom, R. C., Bardsley, L. R., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6645 · Cited by 301
Self-citationGulliver, A., Griffiths, K. M., Christensen, H., Mackinnon, A., Calear, A. L., Parsons, A., Bennett, K., Batterham, P. J., & Stanimirovic, R. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1864 · Cited by 243 · cited 7×
“Although this study was not powered to detect differences between the participants with higher symptoms and lower symptoms, it appeared that the statistically significant improvements observed in the ALQ and the frequency subscales of the CB-RSS were driven by participants in the higher symptom group.”
MethodInfluentialUsing evidence-based internet interventions to reduce health disparities worldwide.
Muñoz, R. F. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1463 · Cited by 242 · cited 1×
“Thus, the cost is less dependent on the number of individuals accessing the intervention [9].”
BackgroundColes, M. E., & Coleman, S. L. (2010) · Depression and Anxiety · doi:10.1002/da.20620 · Cited by 204
Removing obstacles to eliminating racial and ethnic disparities in behavioral health care
Alegría, M., Alvarez, K., Ishikawa, R. Z., DiMarzio, K., & McPeck, S. (2016) · Health Affairs · doi:10.1377/hlthaff.2016.0029 · Cited by 201
Hundt, N. E., Mignogna, J., Underhill, C., & Cully, J. A. (2012) · Behavior Therapy · doi:10.1016/j.beth.2012.10.001 · Cited by 167
Graham, A. K., Greene, C. J., Kwasny, M. J., Kaiser, S. M., Lieponis, P., Powell, T., & Mohr, D. C. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1011 · Cited by 162
Self-citationMohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116 · cited 3×
“The IntelliCare app platform has demonstrated efficacy in reducing symptoms of depression and anxiety in general adult populations [6-8].”
BackgroundMethodResultSelf-citationAligning students and counseling centers on student mental health needs and treatment resources.
Cohen, K. A., Graham, A. K., & Lattie, E. G. (2022) · Journal of American College Health · doi:10.1080/07448481.2020.1762611 · Cited by 110 · cited 1×
“The adaptation of IntelliCare to form the IntelliCare for College Students program was guided by a series of user-centered design activities with college students and counseling center staff members [10,11].”
MethodSelf-citationMental health apps in a college setting: openness, usage, and attitudes
Kern, A., Hong, V., Song, J., Lipson, S. K., & Eisenberg, D. (2018) · mHealth · doi:10.21037/mhealth.2018.06.01 · Cited by 95
Designing Mental Health Technologies that Support the Social Ecosystem of College Students
Lattie, E., Kornfield, R. F., Ringland, K. E., Zhang, R.-W., Winquist, N., & Reddy, M. C. (2020) · International Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376362 · Cited by 88
Self-citationKornfield, R., Zhang, R., Nicholas, J., Schueller, S. M., Cambo, S. A., Mohr, D. C., & Reddy, M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376309 · Cited by 64
Self-citationHeron, K. E., Romano, K. A., & Braitman, A. L. (2019) · mHealth · doi:10.21037/mhealth.2019.01.01 · Cited by 37
Miner, A. S., Schueller, S. M., Lattie, E. G., & Mohr, D. C. (2015) · Psychiatry Research · doi:10.1016/j.psychres.2015.10.033 · Cited by 17
Self-citation
Cited by
Thieme, A., Hanratty, M., Lyons, M., Palacios, J., Marques, R. F., Morrison, C., & Doherty, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3564752 · Cited by 171
Oliveira, C., Pereira, A., Vagos, P., Nóbrega, C. A., Gonçalves, J., & Afonso, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.647606 · Cited by 150
Xu, W., Liang, H., Baghaei, N., Ma, X., Yu, K., Meng, X., & Wen, S. (2021) · JMIR Serious Games · doi:10.2196/29330 · Cited by 93
Uptake and effectiveness of a self-guided mobile app platform for college student mental health
Lattie, E. G., Cohen, K. A., Hersch, E., Williams, K. D. A., Kruzan, K. P., MacIver, C., Hermes, J., Maddi, K. L., Kwasny, M., & Mohr, D. C. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100493 · Cited by 89
Self-citationGlobal youth perspectives on digital health promotion: a scoping review
Ferretti, A., Hubbs, S., & Vayena, E. (2023) · BMC Digital Health · doi:10.1186/s44247-023-00025-0 · Cited by 76
Metadata
- Authors
- Lattie, E., Cohen, K. A., Winquist, N., & Mohr, D. C.
- Affiliations
- Northwestern University (US)
- Corresponding author
- Lattie, Emily
- Venue
- JMIR Mental Health
- Publisher
- JMIR Publications Inc.
- Published
- 2020-10-10
- Volume
- 7
- Issue
- 10
- Pages
- e21075
- ISSN
- 2368-7959 (ISSN-L)
- PMC
- PMC7585772
- OpenAlex
- W3092178332
- Cited by
- 50
- References
- 30
- Citations by year
- 2026: 4 · 2025: 14 · 2024: 8 · 2023: 10 · 2022: 7 · 2021: 7
- FWCI
- 5.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Mental Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health (UL1TR001422)
- National Institute of Mental Health (K08MH112878)
- National Center for Advancing Translational Sciences
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- mental health apps, university students, depression, anxiety, mental health self-care, app usability, user feedback, coping strategies, symptom assessment, user retention, digital mental health interventions, university counseling services, user engagement, program acceptability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.