Development of a web-based intervention for the indicated prevention of depression
Open accessAbstract
BACKGROUND: To reduce the large public health burden of the high prevalence of depression, preventive interventions targeted at people at risk are essential and can be cost-effective. Web-based interventions are able to provide this care, but there is no agreement on how to best develop these applications and often the technology is seen as a given. This seems to be one of the main reasons that web-based interventions do not reach their full potential. The current study describes the development of a web-based intervention for the indicated prevention of depression, employing the CeHRes (Center for eHealth Research and Disease Management) roadmap. The goals are to create a user-friendly application which fits the values of the stakeholders and to evaluate the process of development. METHODS: The employed methods are a literature scan and discussion in the contextual inquiry; interviews, rapid prototyping and a requirement session in the value specification stage; and user-based usability evaluation, expert-based usability inspection and a requirement session in the design stage. RESULTS: The contextual inquiry indicated that there is a need for easily accessible interventions for the indicated prevention of depression and web-based interventions are seen as potentially meeting this need. The value specification stage yielded expected needs of potential participants, comments on the usefulness of the proposed features and comments on two proposed designs of the web-based intervention. The design stage yielded valuable comments on the system, content and service of the web-based intervention. CONCLUSIONS: Overall, we found that by developing the technology, we successfully (re)designed the system, content and service of the web-based intervention to match the values of stakeholders. This study has shown the importance of a structured development process of a web-based intervention for the indicated prevention of depression because: (1) it allows the development team to clarify the needs that have to be met for the intervention to be of use to the target audience; and (2) it yields feedback on the design of the application that is broader than color and buttons, but encompasses comments on the quality of the service that the application offers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The CeHRes Roadmap has been successfully used to develop effective digital interventions.
“The framework has been successfully used to develop effective digital interventions (Kelders et al., 2013; Kip et al., 2019).”
From Strength Back – A qualitative study on the co-creation of a positive psychology digital health intervention for spinal surgery patients (2023) · cites this paper as Kelders et al., 2013
The Background says the study used the CeHRes roadmap to develop a web-based intervention. But the Discussion > Limitations explicitly says the intervention’s effectiveness was not assessed, so this paper does not support calling it effective. (LLM, conf=0.90)
Experimental — accuracy not yet measuredAn online intervention is not just a simple translation of a regular face-to-face therapy to an online environment.
“An online intervention is not just a simple translation of a regular face-to-face therapy to an online environment [27].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
The DeLone and McLean Model of Information Systems Success: A Ten-Year Update
(2003) · Journal of Management Information Systems · doi:10.1080/07421222.2003.11045748 · Cited by 11,791 · cited 3× · see citing works
“These comments were analyzed using a coding scheme following the work of DeLone and McLean [33] and Van Gemert-Pijnen [13] which distinguishes between system quality, content quality and service quality.”
BackgroundMethod(2008) · Medical Entomology and Zoology · Cited by 7,477 · see citing works
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1× · see citing works
“Intervention Mapping [29] and the RE-AIM framework [30,31]) lie in the specific focus of the development of the technology and the fit between technology and the content; and in the attention for implementation during the whole process of development.”
BackgroundBusiness Model Generation : A Handbook for Visionaries, Game Changers, and Challengers
Osterwalder, A., & Pigneur, Y. (2010) · Journal of International Crisis and Risk Communication Research · Cited by 6,870 · see citing works
(1999) · Cited by 5,934 · see citing works
(2005) · Medical Entomology and Zoology · Cited by 3,197 · see citing works
Wherever you go, there you are: Mindfulness meditation in everyday life
Kwee, M. G. T. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(95)90133-7 · Cited by 2,900 · see citing works
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · cited 3× · see citing works
“[16,19,57]) and therefore seems appropriate for this specific web-based intervention.”
BackgroundWebb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1376 · Cited by 2,517 · cited 1× · see citing works
“Furthermore, there is indicative evidence for the effectiveness of using additional text messages to communicate with participants [40].”
BackgroundEysenbach, G. (2001) · Journal of Medical Internet Research · doi:10.2196/jmir.3.2.e20 · Cited by 2,449 · cited 1× · see citing works
“Advantages of web-based interventions can be seen not only in the broader reach, but also in increasing convenience for the users, the opportunity to provide information in an interactive and timely manner and cost-effectiveness [7-9].”
Designing for usability---key principles and what designers think
Gould, J. D., & Lewis, C. (1983) · Communications of the ACM · doi:10.1145/3166.3170 · Cited by 2,069 · see citing works
Planning health promotion programs an intervention mapping approach
Bartholomew, L. K., Parcel, G. S., Kok, G., & Gottlieb, N. (2006) · Cited by 2,038 · see citing works
Planning Health Promotion Programs: An Intervention Mapping Approach
Hicking-Woodison, L. (2017) · Nursing Standard · doi:10.7748/ns.32.6.33.s38 · Cited by 1,843 · see citing works
Persuasive Systems Design: Key Issues, Process Model, and System Features
Oinas-Kukkonen, H., & Harjumaa, M. (2009) · Communications of the Association for Information Systems · doi:10.17705/1cais.02428 · Cited by 1,663 · cited 2× · see citing works
“Persuasive technology refers to the capacity of technology to influence behavior and is used in eHealth research to understand the role of technology in changing behavior [24,25].”
BackgroundThe Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442 · cited 1× · see citing works
“In many cases the effects are less than expected and the implementation of these interventions in regular care is lacking [5,12-14].”
BackgroundInternet-Based and Other Computerized Psychological Treatments for Adult Depression: A Meta-Analysis
Andersson, G., & Cuijpers, P. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070903318960 · Cited by 1,368 · cited 1× · see citing works
“Our literature scan for features that might enhance the effect of a web-based intervention found strong indications that providing support is essential for an intervention to be effective [38,39].”
BackgroundAndrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013196 · Cited by 1,311 · cited 1× · see citing works
“treatment and indicated prevention of mental health complaints have shown that these interventions, on average, are effective in reducing the severity of mental health complaints [10,11].”
BackgroundPersuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · cited 1× · see citing works
“participants not following the intervention protocol, is one of the issues behind the lacking effect [15-17].”
BackgroundSelf-citationBarak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2008) · Journal of Technology in Human Services · doi:10.1080/15228830802094429 · Cited by 1,283 · cited 1× · see citing works
“treatment and indicated prevention of mental health complaints have shown that these interventions, on average, are effective in reducing the severity of mental health complaints [10,11].”
BackgroundAdherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 1× · see citing works
“participants not following the intervention protocol, is one of the issues behind the lacking effect [15-17].”
BackgroundA Holistic Framework to Improve the Uptake and Impact of eHealth Technologies
van Gemert-Pijnen, J. E., Nijland, N., van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1672 · Cited by 1,126 · cited 7× · see citing works
“Important proposed reasons are: issues with the usability of the application [16]; issues regarding the attunement of the goals of the technology with the aims of the participants [13,19]; and implementation issues as the lack of clarity in the costs-benefit structure and integrating the technology in usual care and d…”
BackgroundMethodInfluentialSelf-citationMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · cited 1× · see citing works
“Our literature scan for features that might enhance the effect of a web-based intervention found strong indications that providing support is essential for an intervention to be effective [38,39].”
BackgroundCollins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · cited 1× · see citing works
“To assess the effectiveness of the intervention, it is necessary not only to look at the effectiveness of the intervention as a whole, but also to investigate the active ingredients of the intervention [58].”
BackgroundUser involvement: A review of the benefits and challenges
Kujala, S. (2003) · Behaviour & Information Technology · doi:10.1080/01449290301782 · Cited by 943 · cited 1× · see citing works
“Human centered design advocates the systematic, continuous consultation of potential users during the whole design process [26] and has been shown to have a positive effect especially on user satisfaction and on fitting to user needs [27].”
BackgroundA Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies
Donkin, L., Christensen, H., Naismith, S. L., Neal, B., Hickie, I. B., & Glozier, N. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1772 · Cited by 876 · cited 1× · see citing works
“Studies have shown that better adherence is associated with better (clinical) outcomes of an intervention (see for a systematic review [18]).”
Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P., & Thorogood, M. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e10 · Cited by 820 · cited 2× · see citing works
“For example, web-based interventions can decrease the stigma associated with a (mental) health condition by providing a certain degree of anonymity [6,7].”
BackgroundKey principles for user-centred systems design
Gulliksen, J., Göransson, B., Boivie, I., Blomkvist, S., Persson, J., & Cajander, Å. (2003) · Behaviour and Information Technology · doi:10.1080/01449290310001624329 · Cited by 765 · cited 1× · see citing works
“The interview scheme was based upon eHealth and Human Centered Design literature [13,32,51,52] and consisted of three parts.”
MethodJaspers, M. W. M. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.10.002 · Cited by 649 · cited 7× · see citing works
“However, the number of participants in all methods match the recommended numbers (see [32,55,56]).”
BackgroundMethodInfluentialAndrews, G., Issakidis, C., Sanderson, K., Corry, J., & Lapsley, H. (2004) · British Journal of Psychiatry · doi:10.1192/bjp.184.6.526 · Cited by 560 · cited 1× · see citing works
“To reduce the large public health burden of the high prevalence of depression, preventive interventions targeted at people at risk (indicated prevention) are essential and can be cost-effective [1,2].”
BackgroundRapid contextual design a how-to guide to key techniques for user-centered design
Holtzblatt, K., Wendell, J. B., & Wood, S. (2005) · Cited by 505 · see citing works
Estimating the number of subjects needed for a thinking aloud test
Nielsen, J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1065 · Cited by 476 · cited 2× · see citing works
“However, the number of participants in all methods match the recommended numbers (see [32,55,56]).”
BackgroundMethodPreventing the Onset of Depressive Disorders: A Meta-Analytic Review of Psychological Interventions
Cuijpers, P., van Straten, A., Smit, F., Mihalopoulos, C., & Beekman, A. (2008) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2008.07091422 · Cited by 456 · cited 2× · see citing works
“To reduce the large public health burden of the high prevalence of depression, preventive interventions targeted at people at risk (indicated prevention) are essential and can be cost-effective [1,2].”
BackgroundFry, J. P., & Neff, R. A. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1138 · Cited by 447 · cited 1× · see citing works
“This is strengthened by the results of a systematic review that shows that periodic prompts can enhance the effectiveness of web-based interventions [41].”
BackgroundPetrou, A. D. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10328 · Cited by 387 · see citing works
Hollon, S. D., Muñoz, R. F., Barlow, D. H., Beardslee, W. R., Bell, C. C., Bernal, G., Clarke, G. N., Franciosi, L. P., Kazdin, A. E., Kohn, L. P., et al. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01384-7 · Cited by 327 · cited 1× · see citing works
“The literature scan revealed a supposed need for easily accessible interventions for the indicated prevention of depression [2,3,5,34-37].”
BackgroundInvestigating evaluation frameworks for health information systems
Yusof, M. M., Papazafeiropoulou, A., Paul, R. J., & Stergioulas, L. K. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.08.004 · Cited by 304 · cited 1× · see citing works
“interventions do not reach their full potential in terms of adherence and outcomes [13,22,23].”
BackgroundDesign and Evaluation in eHealth: Challenges and Implications for an Interdisciplinary Field
Pagliari, C. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e15 · Cited by 275 · cited 2× · see citing works
“Web-based applications are developed at a startling rate, but there is no scientifically underpinned agreement on how to best develop these applications [22].”
BackgroundCost Effectiveness of Internet Interventions: Review and Recommendations
Tate, D. F., Finkelstein, E. A., Khavjou, O., & Gustafson, A. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9131-6 · Cited by 261 · cited 1× · see citing works
“Advantages of web-based interventions can be seen not only in the broader reach, but also in increasing convenience for the users, the opportunity to provide information in an interactive and timely manner and cost-effectiveness [7-9].”
BackgroundAcceptance and commitment: Implications for prevention science
Biglan, A., Hayes, S. C., & Pistorello, J. (2008) · Prevention Science · doi:10.1007/s11121-008-0099-4 · Cited by 240 · cited 1× · see citing works
“This intervention is based on acceptance and commitment therapy (ACT; [44]) and mindfulness [45,46], and targets experiential avoidance that can be considered as a generic risk factor for mental illnesses [47].”
Bohlmeijer, E. T., Fledderus, M., Rokx, T. A. J. J., & Pieterse, M. E. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.10.003 · Cited by 232 · cited 1× · see citing works
“Several studies show the effectiveness of this intervention as indicated prevention for depression for a similar target group as is intended for the web-based intervention [48-50].”
BackgroundSelf-citationeHealth Evaluation and Dissemination Research
Glasgow, R. E. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.023 · Cited by 220 · see citing works
Fledderus, M., Bohlmeijer, E. T., Pieterse, M. E., & Schreurs, K. M. G. (2012) · Psychological Medicine · doi:10.1017/s0033291711001206 · Cited by 209 · cited 3× · see citing works
“[49] showed that the ACT-based intervention as a self-help programme with both minimal and extensive email support was effective for people with mild to moderate depressive symptomatology.”
BackgroundSelf-citationKukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.09.002 · Cited by 206 · cited 1× · see citing works
“The interview scheme was based upon eHealth and Human Centered Design literature [13,32,51,52] and consisted of three parts.”
MethodFledderus, M., Bohlmeijer, E. T., Smit, F., & Westerhof, G. J. (2010) · American Journal of Public Health · doi:10.2105/ajph.2010.196196 · Cited by 197 · cited 2× · see citing works
“[50] showed that ‘Living to the full’ as a group course is effective in substantially reducing depressive symptomatology and improving positive mental health in adults.”
BackgroundSelf-citationWhy Business Modeling is Crucial in the Development of eHealth Technologies
van Limburg, M., van Gemert-Pijnen, J. E., Nijland, N., Ossebaard, H. C., Hendrix, R. M., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1674 · Cited by 179 · cited 2× · see citing works
“Important proposed reasons are: issues with the usability of the application [16]; issues regarding the attunement of the goals of the technology with the aims of the participants [13,19]; and implementation issues as the lack of clarity in the costs-benefit structure and integrating the technology in usual care and d…”
BackgroundKelders, S. M., Van Gemert-Pijnen, J. E. W. C., Werkman, A., Nijland, N., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1624 · Cited by 156 · cited 2× · see citing works
“[16,19,57]) and therefore seems appropriate for this specific web-based intervention.”
BackgroundSelf-citationBauer, S., Golkaramnay, V., & Kordy, H. (2004) · Psychotherapeut · doi:10.1007/s00278-004-0403-0 · Cited by 144 · see citing works
Kinzie, M. B. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m0822 · Cited by 136 · see citing works
Postel, M. G., de Haan, H. A., ter Huurne, E. D., Becker, E. S., & de Jong, C. A. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1642 · Cited by 117 · cited 1× · see citing works
“Developing and implementing webbased interventions provide an opportunity to overcome this challenge by tackling the reasons for the low participation rates [3-5].”
BackgroundCuijpers, P., van Straten, A., Warmerdam, L., & van Rooy, M. J. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-181 · Cited by 106 · see citing works
Diaries in clinical psychology and psychotherapy: a selective review
Thiele, C., Laireiter, A., & Baumann, U. (2002) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.302 · Cited by 88 · cited 1× · see citing works
“Lastly, diaries for self-monitoring have been shown to be successful in a variety of mental health conditions [42] and have been advocated as a useful persuasive technology feature [24,25].”
BackgroundChristensen, H. M., Reynolds, J., & Griffiths, K. M. (2011) · Early Intervention in Psychiatry · doi:10.1111/j.1751-7893.2010.00242.x · Cited by 70 · cited 1× · see citing works
“In many cases the effects are less than expected and the implementation of these interventions in regular care is lacking [5,12-14].”
BackgroundPatterns of Use of an Automated Interactive Personalized Coaching Program for Smoking Cessation
Balmford, J., Borland, R., & Benda, P. J. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.1016 · Cited by 61 · cited 1× · see citing works
“[16,19,57]) and therefore seems appropriate for this specific web-based intervention.”
BackgroundPreventing depression in high-risk groups
Beekman, A. T. F., Smit, F., Stek, M. L., Reynolds, C. F., & Cuijpers, P. J. (2009) · Current Opinion in Psychiatry · doi:10.1097/yco.0b013e328333e17f · Cited by 57 · cited 2× · see citing works
“The literature scan revealed a supposed need for easily accessible interventions for the indicated prevention of depression [2,3,5,34-37].”
BackgroundNijland, N. (2010) · doi:10.3990/1.9789036531337 · Cited by 45 · cited 1× · see citing works
“Important proposed reasons are: issues with the usability of the application [16]; issues regarding the attunement of the goals of the technology with the aims of the participants [13,19]; and implementation issues as the lack of clarity in the costs-benefit structure and integrating the technology in usual care and d…”
BackgroundInternet-Based Depression Prevention over the Life Course: A Call for Behavioral Vaccines
Voorhees, B. W. V., Mahoney, N., Mazo, R., Barrera, A. Z., Siemer, C. P., Gladstone, T. R. G., & Muñoz, R. F. (2011) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2010.11.002 · Cited by 42 · cited 2× · see citing works
“The literature scan revealed a supposed need for easily accessible interventions for the indicated prevention of depression [2,3,5,34-37].”
BackgroundCharacteristics of problem drinkers in e-therapy versus face-to-face treatment
Postel, M. G., de Haan, H. A., ter Huurne, E. D., Becker, E. S., & de Jong, C. A. J. (2011) · The American Journal of Drug and Alcohol Abuse · doi:10.3109/00952990.2011.600388 · Cited by 19 · cited 1× · see citing works
“For example, web-based interventions can decrease the stigma associated with a (mental) health condition by providing a certain degree of anonymity [6,7].”
BackgroundPhm, V. B., van den Berg, M., Tariq, L., Rt, H., Schoemaker, C., Aj, S., de Wit, G. A., Vtv, & Pzo (2008) · Rivm Repository (Netherlands National Institute for Public Health and the Environment) · Cited by 5 · see citing works
Flagged first, then by citations
1 of the 60 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Fogg BJ Persuasive technology: using computers to change what we think and do 2003 Boston: Morgan Kaufmann Publishers
Cited by
Sorted by citations
van der Vaart, R., Witting, M., Riper, H., Kooistra, L., Bohlmeijer, E. T., & van Gemert-Pijnen, L. J. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0355-z · Cited by 273 · see citing works
“An online intervention is not just a simple translation of a regular face-toface therapy to an online environment [27].”
BackgroundSelf-citationTavares, J., & Oliveira, T. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5069 · Cited by 204 · see citing works
“…important when designing or redeploying an EHR portal to make it easy and simple to use, and we therefore suggest that a pilot application of the platform be tested by the potential users so that improvements can be made during the development stage to increase the platform’s acceptance [71,72].”
Biesheuvel-Leliefeld, K. E. M., Kok, G. D., Bockting, C. L. H., Cuijpers, P., Hollon, S. D., van Marwijk, H. W. J., & Smit, F. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.12.016 · Cited by 175 · see citing works
“Possible ways of offering psychological interventions are over the Internet (Hollandare et al., 2011; Kelders et al., 2013), by a nurse (Bosmans et al., 2012), by self-help (Biesheuvel-Leliefeld et al., 2012) or by low intensity psychological interventions (Rodgers et al., 2012).”
BackgroundInfluentialUsability in telemedicine systems—A literature survey
Klaassen, B., van Beijnum, B.-J. F., & Hermens, H. J. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.06.004 · Cited by 152 · see citing works
Pots, W. T. M., Fledderus, M., Meulenbeek, P. A. M., ten Klooster, P. M., Schreurs, K. M. G., & Bohlmeijer, E. T. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.146068 · Cited by 133 · see citing works
Self-citationKelders, S. M., Bohlmeijer, E. T., Pots, W. T. M., & van Gemert-Pijnen, J. E. W. C. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.014 · Cited by 105 · see citing works
Self-citationFeasibility of an eHealth application “OncoKompas” to improve personalized survivorship cancer care
Duman-Lubberding, S., van Uden-Kraan, C. F., Jansen, F., Witte, B. I., van der Velden, L. A., Lacko, M., Cuijpers, P., Leemans, C. R., & Verdonck-de Leeuw, I. M. (2016) · Supportive Care in Cancer · doi:10.1007/s00520-015-3004-2 · Cited by 112 · see citing works
Moore, G., Wilding, H., Gray, K., & Castle, D. (2019) · Journal of Participatory Medicine · doi:10.2196/11474 · Cited by 95 · see citing works
“…iterative development process, prototype, think aloud, usability testing, cognitive walkthrough CeHRes Roadmap, Human Centered Design Kelders 2013 [46] Ethics approval, interviews (HSU), interviews (HSP), thematic analysis, content creation (HSU), iterative development process, prototype,…”
MethodExploring mental health providers' interest in using web and mobile-based tools in their practices
Schueller, S. M., Washburn, J. J., & Price, M. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.06.004 · Cited by 98 · see citing works
Kelders, S. M., Bohlmeijer, E. T., & Van Gemert-Pijnen, J. E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2258 · Cited by 88 · see citing works
“The intervention was developed employing methods from the CeHRes Roadmap for eHealth development [10] and this process is described elsewhere [45].”
Self-citationElectronic Health Record Portal Adoption: a cross country analysis
Tavares, J., & Oliveira, T. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0482-9 · Cited by 88 · see citing works
“When redeploying or designing an EHR portal, we should thus strive to make it easy and simple for the healthcare consumers to use [21, 90, 91].”
BackgroundMethodElectronic Health Record Portals adoption: Empirical model based on UTAUT2
Tavares, J., Goulão, A. P. B. A., & Oliveira, T. (2017) · Informatics for Health and Social Care · doi:10.1080/17538157.2017.1363759 · Cited by 83 · see citing works
“It is very important when designing or redeploying an EHR portal to make it easy and simple to use, and we therefore suggest that a pilot application should be tested by the potential users of the platform so that improvements can be made in the development stage to increase the acceptance of the platform.(55,56) Soci…”
Methods for Human-Centered eHealth Development: Narrative Scoping Review
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2022) · Journal of Medical Internet Research · doi:10.2196/31858 · Cited by 67 · see citing works
“[15,21,24,26,28-31,33,34,40,44,45,47,49,50,52,54-56, 58,61,67,69,70,74,76,78,79,81,83,84,88,89,91,96-129,178] 21 (30) 32 (46) 41 (59) Interview (n=70) [15,24,26,29,32-34,46, 49”
Self-citationSøgaard Neilsen, A., & Wilson, R. L. (2019) · International Journal of Mental Health Nursing · doi:10.1111/inm.12527 · Cited by 71 · see citing works
Methods for Human-Centered eHealth Development: Narrative Scoping Review (Preprint)
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2021) · doi:10.2196/preprints.31858 · Cited by 67 · see citing works
Self-citationDuman-Lubberding, S., van Uden‐Kraan, C. F., Peek, N. B., Cuijpers, P. J., Leemans, C. R., & Leeuw, I. M. V. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4870 · Cited by 53 · see citing works
“Initial codes for the CWs were generated focusing on system quality (ease of use), content quality (usefulness and relevance), and service quality (the process of care provided) [15,19,20].”
Cnossen, I. C., van Uden‐Kraan, C. F., Eerenstein, S. E. J., Jansen, F., Lissenberg-Witte, B. I., Lacko, M., Hardillo, J. A., Honings, J., Halmos, G. B., Goedhart-Schwandt, N. L. Q., et al. (2015) · Supportive Care in Cancer · doi:10.1007/s00520-015-2896-1 · Cited by 50 · see citing works
“To ensure that ITwC would adequately meet the needs of the end-users, a participatory design approach was followed, with stakeholders’ involvement in the development process [16, 17].”
MethodPots, W. T. M., Trompetter, H. R., Schreurs, K. M. G., & Bohlmeijer, E. T. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0841-6 · Cited by 46 · see citing works
“[38].”
Self-citationPelletier, J., Rowe, M., François, N., Bordeleau, J., & LUPIEN, S. J. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-78 · Cited by 48 · see citing works
“Redesigning an app to match the expectations and values of psychiatric patients involved more than feedback on the app’s colors and buttons: encompassed comments on the quality of the service and how this app could be integrated into service delivery and organizational processes [31,32].”
BackgroundKip, H., Kelders, S. M., Bouman, Y. H. A., & van Gemert-Pijnen, L. J. E. W. C. (2019) · Journal of Medical Internet Research · doi:10.2196/12972 · Cited by 40 · see citing works
“This might result in a tool kit that researchers can use to choose and operationalize methods.”
InfluentialSelf-citationPilkington, P. D., Milne, L. C., Cairns, K., & Whelan, T. A. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0721-0 · Cited by 41 · see citing works
“A more dynamic and engaging format, such as a website, may be more effective in eliciting behavior change [67].”
BackgroundKip, H., Jong, N. B.-D., van Gemert‐Pijnen, J. E. W. C., & Kelders, S. M. (2024) · Journal of Medical Internet Research · doi:10.2196/59601 · Cited by 41 · see citing works
“…of frameworks like the Roadmap mostly requires sharing experiences with and lessons learned from its application, which again emphasizes the importance of systematically reporting and reflecting on development, implementation, and evaluation processes, in line with reporting guidelines [13,14,31].”
Self-citationPerski, O., Baretta, D., Blandford, A., West, R., & Michie, S. (2018) · DIGITAL HEALTH · doi:10.1177/2055207618785841 · Cited by 31 · see citing works
“The design of health apps is often driven by the possibility of using technology, and not because the target group has expressed a need for such technology.(14) The terms ‘co-design’ and ‘user-centred design’ are used to denote design processes in which potential users influence whether and how a design takes shape.”
BackgroundDeSmet, A., Bourdeaudhuij, I. D., CHASTIN, S. F. M., Crombez, G., Maddison, R., & Cardon, G. (2019) · JMIR mhealth and uhealth · doi:10.2196/15707 · Cited by 33 · see citing works
“User-centered design, in which user preferences for BCTs and other intervention features are taken into account, can help increase user engagement [20-22].”
BackgroundCnossen, I. C., van Uden‐Kraan, C. F., Eerenstein, S. E. J., Rinkel, R. N. P. M., Aalders, I. J., van den Berg, K., de Goede, C. J. T., van Stijgeren, A. J., Cruijff-Bijl, Y., de Bree, R., et al. (2015) · Folia Phoniatrica et Logopaedica · doi:10.1159/000441251 · Cited by 34 · see citing works
Health Behavior Change Support Systems as a research discipline; A viewpoint
Kelders, S. M., Oinas-Kukkonen, H., Oörni, A., & van Gemert-Pijnen, J. E. W. C. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.06.022 · Cited by 32 · see citing works
Self-citationMaenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C. A., Hove, G. V., & Compernolle, S. (2023) · Research Involvement and Engagement · doi:10.1186/s40900-023-00420-x · Cited by 33 · see citing works
Fledderus, M., Schreurs, K. M., Bohlmeijer, E. T., & Vollenbroek-Hutten, M. M. (2015) · JMIR Human Factors · doi:10.2196/humanfactors.3302 · Cited by 30 · see citing works
“This roadmap has been successfully used to develop an online, ACT, Web-based intervention for depression [22] and chronic pain [23].”
Self-citationKingston, D., Janes-Kelley, S., Tyrrell, J., Clark, L., Hamza, D., Holmes, P., Parkes, C., Moyo, N., McDonald, S., & Austin, M.-P. (2015) · JMIR Research Protocols · doi:10.2196/resprot.4037 · Cited by 42 · see citing works
“Web-based CBT is clinically and cost-effective [59-62], accessible [59], and recommended as a primary therapy for mild and moderate depression [24,63].”
Tiburcio, M., Lara, M. A., Abrego, A. A., Fernández-Torres, M., Martínez-Vélez, N. A., & Sánchez, A. (2016) · JMIR Mental Health · doi:10.2196/mental.6001 · Cited by 26 · see citing works
“Likewise, a multidisciplinary approach considering different points of view leads to more satisfactory results [52].”
BackgroundKoneska, E., Appelbe, D., Williamson, P. R., & Dodd, S. R. (2020) · Journal of Medical Internet Research · doi:10.2196/15474 · Cited by 26 · see citing works
“Unlike drug interventions, the adherence to which can be summarized using uncomplicated measures of treatment intake (eg, initiation, completion, and persistence [21]), web-based interventions often involve multiple features [40,41], engagement with which may be more complex to record.”
BackgroundInvolvement as a Working Mechanism for Persuasive Technology
Kelders, S. M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20306-5_1 · Cited by 25 · see citing works
“These components were chosen based on research into the design of this intervention (these were the features that deemed to be important to the target group to keep using the intervention [11]).”
MethodInfluentialSelf-citationAl‐Itejawi, H. H., van Uden‐Kraan, C. F., Vis, A. N., Nieuwenhuijzen, J. A., Hofstee, M. J., van Moorselaar, R. J. A., & Verdonck‐de Leeuw, I. M. (2016) · Journal of Clinical Nursing · doi:10.1111/jocn.13120 · Cited by 25 · see citing works
“The two coders independently selected citations using a coding scheme concerning system quality (ease of use), content quality (usefulness and relevance) and service quality (the process of care provided) (DeLone & McLean 2003, Kelders et al. 2013) and coded these into categories.”
MethodFaber, J. S., Al-Dhahir, I., Kraal, J. J., Breeman, L. D., van den Berg-Emons, R. J. G., Reijnders, T., van Dijk, S., Janssen, V. R., Kraaijenhagen, R. A., Visch, V., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/48461 · Cited by 22 · see citing works
“Themes were coded through open coding and thereafter categorized through axial coding within 3 predetermined categories: service, system, and content, as provided in Kelders et al [64].”
‘This Is Your Life!’: The Design of a Positive Psychology Intervention Using Metaphor to Motivate
Ludden, G. D. S., Kelders, S. M., & Snippert, B. H. J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07127-5_16 · Cited by 20 · see citing works
Self-citationHouwen, T., Vugts, M. A. P., Lansink, K. W. W., Theeuwes, H. P., Neequaye, N., Beerekamp, M. S. H., Joosen, M. C. W., & de Jongh, M. A. C. (2022) · JMIR Human Factors · doi:10.2196/35342 · Cited by 15 · see citing works
“Previous studies have addressed a wide range of eHealth initiatives such as the development of a web-based intervention for depression [37], a computerized clinical decision support system for patients with type 2 diabetes [36], an mHealth intervention for patients with prediabetes [33], and a digital training tool to…”
BackgroundLambert, J. D., Greaves, C. J., Farrand, P., Haase, A. M., & Taylor, A. H. (2017) · Mental Health and Physical Activity · doi:10.1016/j.mhpa.2017.10.003 · Cited by 14 · see citing works
“The CeHReS roadmap also has an explicit focus on the fit between the technology and content, which the other frameworks do not offer (Kelders et al., 2013).”
BackgroundMethodInfluentialSkjøth, M. M., Hansen, H. P., Draborg, E., Pedersen, C. D., Lamont, R., & Jørgensen, J. S. (2015) · JMIR Research Protocols · doi:10.2196/resprot.4291 · Cited by 15 · see citing works
“Other studies concerning the development of eHealth tools recognized the importance of a structured process to develop a successful intervention and used a theoretical framework for the process [48,50,56,57].”
BackgroundMethodA Systematic Review and Meta-Analysis of Third-Wave Online Interventions for Depression
Sierra, M. A., Ruiz, F. J., & Flórez, C. L. (2018) · Revista Latinoamericana de Psicología · doi:10.14349/rlp.2018.v50.n2.6 · Cited by 15 · see citing works
MyStrengths, a strengths-focused mobile health tool: participatory design and development
Jessen, S., Mirkovic, J., & Nes, L. S. (2020) · JMIR Formative Research · doi:10.2196/18049 · Cited by 14 · see citing works
“This third strategy also supported the important task of communicating and creating a deeper understanding of the users’ needs and requirements within the project group [99].”
BackgroundBrown, M., Lord, E., & John, A. M. (2023) · JMIR Formative Research · doi:10.2196/39913 · Cited by 13 · see citing works
“However, other studies reported similar sample sizes [75-77].”
ResultPilkington, P. D., Rominov, H., Milne, L. C., Giallo, R., & Whelan, T. A. (2016) · Advances in Mental Health · doi:10.1080/18387357.2016.1173517 · Cited by 11 · see citing works
“System quality refers to the extent to which the website is user-friendly, including the visual layout and placement of menus and buttons (Kelders et al., 2013).”
BackgroundMethodInfluentialDeveloping a Guided Web App for Postpartum Depression Symptoms: User-Centered Design Approach
Franco, P., Olhaberry, M., Muzard, A., Harismendy, Á., & Kelders, S. M. (2024) · JMIR Formative Research · doi:10.2196/56319 · Cited by 11 · see citing works
Self-citationE-Health individual adoption - empirical model based on UTAUT 2.
Goulão, A. P. B. A. (2014) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · Cited by 8 · see citing works
A Web-Based Positive Psychology App for Patients With Bipolar Disorder: Development Study
Geerling, B., Kelders, S. M., Stevens, A. W. M. M., Kupka, R., & Bohlmeijer, E. T. (2022) · JMIR Formative Research · doi:10.2196/39476 · Cited by 7 · see citing works
“The development of new interventions benefits from user involvement in all stages to meet the target group’s needs [89].”
BackgroundSelf-citationAl-Itejawi, H. H. M., van Uden‐Kraan, C. F., van de Ven, P. M., Coupé, V. M. H., Vis, A. N., Nieuwenhuijzen, J. A., van Moorselaar, R. J. A., & Leeuw, I. M. V. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015154 · Cited by 7 · see citing works
Brown, M., Hooper, N., Eslambolchilar, P., & John, A. M. (2020) · JMIR Formative Research · doi:10.2196/22507 · Cited by 6 · see citing works
“Kelders et al [16] developed an intervention to prevent depression.”
Van Der Horst, A. Y., Bohlmeijer, E. T., Schreurs, K. M. G., & Kelders, S. M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1117357 · Cited by 4 · see citing works
“The framework has been successfully used to develop effective digital interventions (Kelders et al., 2013; Kip et al., 2019).”
MethodSelf-citationNobakht, Z., Rassafiani, M., & Hosseini, S. A. (2018) · Europe PMC (PubMed Central) · Cited by 5 · see citing works
E-mental health: state of the art
Thewissen, V., & Günther, N. (2015) · Tijdschrift voor Psychotherapie · doi:10.1007/s12485-015-0102-z · Cited by 4 · see citing works
Design and Evaluation of Online and Mobile Applications for Stress Management and Healthy Eating
Kaipainen, K. (2014) · Tampere University Institutional Repository (Tampere University) · Cited by 4 · see citing works
Vernmark, K. (2017) · doi:10.3384/diss.diva-142389 · Cited by 3 · see citing works
Lemay, J., Roberge, J., Provencher, V., Tremblay, A., Panahi, S., Jacob, R., Brunelle, L., Saintonge, G., & Drapeau, V. (2025) · JMIR Formative Research · doi:10.2196/66582 · Cited by 2 · see citing works
“The iterative design process within a multidisciplinary team helps create evidence-based and targeted content for users, though it may extend development timelines [40].”
Orr, M., Isaacs, J. Y., Ilie, A., Chambers, C. T., Smith, I. M., & Corkum, P. V. (2023) · Journal of Educational and Developmental Psychology · doi:10.5539/jedp.v13n2p95 · Cited by 2 · see citing works
Intensifying innovation adoption in educational eHealth
Rissanen, M. (2014) · Science education international · Cited by 2 · see citing works
Kelly, M., Fullen, B. M., Martin, D., Bradley, C., O’Mahony, B., & McVeigh, J. G. (2023) · HRB Open Research · doi:10.12688/hrbopenres.13611.2 · Cited by 2 · see citing works
Kelly, M., Fullen, B. M., Martin, D., Bradley, C., O’Mahony, B., & McVeigh, J. G. (2022) · HRB Open Research · doi:10.12688/hrbopenres.13611.1 · Cited by 3 · see citing works
Kip, H. (2021) · doi:10.3990/1.9789036551311 · Cited by 1 · see citing works
Therapie op afstand, werkt dat?
Fledderus, M., & Pots, W. T. M. (2013) · Nederlands Tijdschrift voor Evidence Based Practice · doi:10.1007/s12468-013-0031-1
Self-citationGeerling, B., Kelders, S. M., Stevens, A. W. M. M., Kupka, R., & Bohlmeijer, E. T. (2022) · doi:10.2196/preprints.39476
Self-citationOrr, M., Belliveau, J., Chambers, C. T., Smith, I. M., & Corkum, P. V. (2023) · Journal of Educational and Developmental Psychology · doi:10.5539/jedp.v13n2p78
Orr, M., Ilie, A., Chambers, C. T., Smith, I. M., & Corkum, P. V. (2023) · Journal of Educational and Developmental Psychology · doi:10.5539/jedp.v14n1p1
Lemay, J., Roberge, J., Provencher, V., Tremblay, A., Panahi, S., Jacob, R., Brunelle, L., Saintonge, G., & Drapeau, V. (2024) · doi:10.2196/preprints.66582
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619180
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619192
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619177
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619186
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619183
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619195
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619189
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619171
Maenhout, L., Verloigne, M., Cairns, D., Cardon, G., Crombez, G., Melville, C., Hove, G. V., & Compernolle, S. (2023) · Figshare · doi:10.6084/m9.figshare.22619174
Critical success factors and risks for PACS implementation across Welsh health boardsPreprint
Cooke, G., & Pennington, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9345189/v1
Tönis, K. J. M., Drossaert, C. H. C., ten Klooster, P. M., Segers, J., Keren, N., Staal, W., & Bohlmeijer, E. T. (2026) · Research in Neurodiversity · doi:10.1016/j.rin.2026.100050
Self-citationTavares, I. M., Dawson, S. J., Fitzpatrick, E. T., Jang, G. E., Stephenson, K. R., & Rosen, N. O. (2026) · Pilot and Feasibility Studies · doi:10.1186/s40814-026-01878-8
“…0) and first content (Module 1) modules, we followed best practices for participant usability and engagement in online prevention programs [22, 23], including those in the transition to parenthood [18], such as short video segments (and limited text) to illustrate key concepts and interactive…”
van Lotringen, C. M., Kelders, S. M., Westerhof, G. J., & Noordzij, M. L. (2026) · Digital Health · doi:10.1177/20552076261483463
Self-citation
- Authors
- Kelders, S. M., Pots, W. T., Oskam, M. J., Bohlmeijer, E. T., & van Gemert-Pijnen, J. E.
- Affiliations
- University of Twente (NL)
- Corresponding author
- Kelders, Saskia M
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2013-02-20
- Volume
- 13
- Issue
- 1
- Pages
- 26-26
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC3598782
- OpenAlex
- W1967789340
- Cited by
- 80
- References
- 60
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 3 · 2023: 18 · 2022: 4 · 2021: 3 · 2020: 3 · 2019: 2 · 2018: 5 · 2017: 6 · 2016: 9 · 2015: 11 · 2014: 7 · 2013: 4
- FWCI
- 15.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Twente Research Information (publishedVersion, cc-by)
- University of Twente Research Information (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- online intervention, depression prevention, indicated prevention, e-health, intervention development, stakeholder value, usability evaluation, contextual inquiry, rapid prototyping, user-centered design
- MeSH terms
- Adult, Depression, Female, Humans, Male, Middle Aged, Program Development, Telemedicine, Needs Assessment, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review