Evaluation Methods for Assessing Users’ Psychological Experiences of Web-Based Psychosocial Interventions: A Systematic Review
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The use of Web-based interventions to deliver mental health and behavior change programs is increasingly popular. They are cost-effective, accessible, and generally effective. Often these interventions concern psychologically sensitive and challenging issues, such as depression or anxiety. The process by which a person receives and experiences therapy is important to understanding therapeutic process and outcomes. While the experience of the patient or client in traditional face-to-face therapy has been evaluated in a number of ways, there appeared to be a gap in the evaluation of patient experiences of therapeutic interventions delivered online. Evaluation of Web-based artifacts has focused either on evaluation of experience from a computer Web-design perspective through usability testing or on evaluation of treatment effectiveness. Neither of these methods focuses on the psychological experience of the person while engaged in the therapeutic process. OBJECTIVE: This study aimed to investigate what methods, if any, have been used to evaluate the in situ psychological experience of users of Web-based self-help psychosocial interventions. METHODS: A systematic literature review was undertaken of interdisciplinary databases with a focus on health and computer sciences. Studies that met a predetermined search protocol were included. RESULTS: Among 21 studies identified that examined psychological experience of the user, only 1 study collected user experience in situ. The most common method of understanding users' experience was through semistructured interviews conducted posttreatment or questionnaires administrated at the end of an intervention session. The questionnaires were usually based on standardized tools used to assess user experience with traditional face-to-face treatment. CONCLUSIONS: There is a lack of methods specified in the literature to evaluate the interface between Web-based mental health or behavior change artifacts and users. Main limitations in the research were the nascency of the topic and cross-disciplinary nature of the field. There is a need to develop and deliver methods of understanding users' psychological experiences while using an intervention.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355
The qualitative interview in IS research: Examining the craft
Myers, M. D., & Newman, M. (2007) · Information and Organization · doi:10.1016/j.infoandorg.2006.11.001 · Cited by 1,941
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442
Barak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2008) · Journal of Technology in Human Services · doi:10.1080/15228830802094429 · Cited by 1,283 · cited 2×
“…which have information, health assessment screening, and treatment and are targeted to specific issues; (3) Internet-based therapeutic software programs such as gaming and virtual worlds; (4) mobile apps; and (5) other activities such as support discussion groups, blogs, and podcasts [3,12,13].”
Adherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 1×
“Second, the psychological experience of the person may be important to determining usage and adherence to an intervention with some reports of dropout rates exceeding 98% from open-access websites dealing with psychosocial issues [25].”
What Is eHealth (3): A Systematic Review of Published Definitions
Oh, H., Jadad, A., Rizo, C., Enkin, M., Powell, J., & Pagliari, C. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e1 · Cited by 1,015
Defining Internet-Supported Therapeutic Interventions
Barak, A., Klein, B., & Proudfoot, J. G. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9130-7 · Cited by 731 · cited 1×
“…which have information, health assessment screening, and treatment and are targeted to specific issues; (3) Internet-based therapeutic software programs such as gaming and virtual worlds; (4) mobile apps; and (5) other activities such as support discussion groups, blogs, and podcasts [3,12,13].”
Why Do Evaluations of eHealth Programs Fail? An Alternative Set of Guiding Principles
Greenhalgh, T., & Russell, J. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000360 · Cited by 273 · cited 1×
“However, all of these evaluation approaches must bear in mind the “emergent” nature of eHealth interventions and the degree to which the objectives and nature of the evaluation of the intervention may differ as prototypes are developed and evaluators gain experience [57].”
Using evidence-based internet interventions to reduce health disparities worldwide.
Muñoz, R. F. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1463 · Cited by 242 · cited 1×
“Within health care, the growth of technology means that eHealth is seen as a means of providing targeted treatment to a wide client base [2,3], and self-help Web-based interventions can also contribute to the reduction of disparities in health care internationally [4,5].”
Health Disparities, Communication Inequalities, and eHealth
Viswanath, K., & Kreuter, M. W. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.02.012 · Cited by 238 · cited 1×
“Within health care, the growth of technology means that eHealth is seen as a means of providing targeted treatment to a wide client base [2,3], and self-help Web-based interventions can also contribute to the reduction of disparities in health care internationally [4,5].”
Cited by
Søgaard Neilsen, A., & Wilson, R. L. (2019) · International Journal of Mental Health Nursing · doi:10.1111/inm.12527 · Cited by 71
“…for support, collaborative care and treatment are utilized increasingly and successfully for mental health advantages (Schueller et al., 2017, Feather et al., 2016, Andersson et al., 2014). r p A wide range of information is conveniently available to the general public and can be accessed…”
BackgroundDarley, A., Coughlan, B., & Furlong, E. (2021) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2021.102030 · Cited by 36
Radomski, A. D., Bagnell, A., Curtis, S., Hartling, L., & Newton, A. S. (2020) · JMIR Mental Health · doi:10.2196/15795 · Cited by 30
“For iCBT programs designed and delivered to adolescents with anxiety, program evaluations should aim to understand how iCBT is experienced by adolescents to further ensure its relevance, use, and impact as a self-help treatment [63-66].”
Significant events in an Internet‐delivered (Space from Depression) intervention for depression
Richards, D., Dowling, M., O'Brien, E., Viganò, N., & Timulak, L. (2018) · Counselling and Psychotherapy Research · doi:10.1002/capr.12142 · Cited by 13
Martínez-Borba, V., Suso-Ribera, C., & Osma, J. (2019) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-25872-6_14 · Cited by 6
Metadata
- Authors
- Feather, J. S., Howson, M., Ritchie, L., Carter, P. D., Parry, D. T., & Koziol-McLain, J.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2016-06-30
- Volume
- 18
- Issue
- 6
- Pages
- e181
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2466957612
- Cited by
- 28
- References
- 27
- FWCI
- 0.86
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.