Seven lessons for interdisciplinary research on interactive digital health interventions
Open accessAbstract
Research and development for interactive digital health interventions requires multi-disciplinary expertise in identifying user needs, and developing and evaluating each intervention. Two of the central areas of expertise required are Health (broadly defined) and Human–Computer Interaction. Although these share some research methods and values, they traditionally have deep differences that can catch people unawares, and make interdisciplinary collaborations challenging, resulting in sub-optimal project outcomes. The most widely discussed is the contrast between formative evaluation (emphasised in Human–Computer Interaction) and summative evaluation (emphasised in Health research). However, the differences extend well beyond this, from the nature of accepted evidence to the culture of reporting. In this paper, we present and discuss seven lessons that we have learned about the contrasting cultures, values, assumptions and practices of Health and Human–Computer Interaction. The lessons are structured according to a research lifecycle, from establishing the state of the art for a given digital intervention, moving through the various (iterative) stages of development, evaluation and deployment, through to reporting research results. Although our focus is on enabling people from different disciplinary backgrounds to work together with better mutual understanding, we also highlight ways in which future research in this interdisciplinary space could be better supported.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent research into digital health intervention development highlights the importance of an iterative, multidisciplinary approach in which the end user is placed at the heart of the system and the technology is grounded in user wants and needs.
“Recent research into digital health intervention development highlights the importance of an iterative, multidisciplinary approach in which the end user is placed at the heart of the system and the technology is grounded in user wants and needs [45-47].”
To ensure tools are convenient and useful in the context of day-to-day life, it is necessary to involve end-users across the design and development process.
“To ensure tools are convenient and useful in the context of day-to-day life, it is necessary to involve end-users across the design and development process (Mohr et al., 2017; Blandford et al., 2018; Borghouts et al., 2021; Poole, 2013).”
From A text messaging intervention to support the mental health of young adults: User engagement and feedback from a field trial of an intervention prototype (2023) · cites this paper as Blandford et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697 · see citing works
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Co-creation and the new landscapes of design
Sanders, E. B.-N., & Stappers, P. J. (2008) · CoDesign · doi:10.1080/15710880701875068 · Cited by 5,561 · see citing works
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,074 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Nielsen, J. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260531 · Cited by 2,759 · see citing works
Contextual Design: Defining Customer-Centered Systems
Beyer, H. R., & Holtzblatt, K. (1997) · Cited by 2,748 · see citing works
Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,053 · see citing works
Eysenbach, G., & CONSORT-EHEALTH Group (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1923 · Cited by 1,863 · cited 1× · see citing works
“For example, standards developed for reporting of interactive DHIs 13 tend to focus on RCTs and not on alternative evaluation designs or formative research.”
BackgroundEvidence-Based Medicine: How to Practice and Teach EBM
Satya-Murti, S. (2000) · JAMA · doi:10.1001/jama.284.18.2382 · Cited by 1,659 · see citing works
Plans and Situated Actions: The Problem of Human Machine Communication.
Woolgar, S., & Suchman, L. A. (1989) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2073874 · Cited by 1,600 · see citing works
Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4055 · Cited by 1,442 · see citing works
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · see citing works
Perski, O., Blandford, A., West, R., & Michie, S. (2017) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0453-1 · Cited by 1,193 · see citing works
Self-citationDourish, P. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124855 · Cited by 860 · see citing works
Evaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847 · see citing works
Self-citationEvidence-based policy: the promise of ‘realist synthesis’
Pawson, R. (2002) · Evaluation · doi:10.1177/135638902401462448 · Cited by 729 · see citing works
The Delphi Method: Techniques and Applications.
Moore, P. G., Lingstone, H. A., & Turoff, M. (1977) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344913 · Cited by 622 · see citing works
A Hermeneutic Approach for Conducting Literature Reviews and Literature Searches
Boell, S. K., & Ćećez-Kecmanović, D. (2014) · Communications of the Association for Information Systems · doi:10.17705/1cais.03412 · Cited by 615 · see citing works
Author Sequence and Credit for Contributions in Multiauthored Publications
Tscharntke, T., Hochberg, M., Rand, T. A., Resh, V. H., & Krauß, J. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050018 · Cited by 580 · see citing works
Nahum-Shani, I., Hekler, E. B., & Spruijt-Metz, D. (2015) · Health Psychology · doi:10.1037/hea0000306 · Cited by 568 · see citing works
A strategy for optimizing and evaluating behavioral interventions
Collins, L. M., Murphy, S. A., Nair, V. N., & Strecher, V. J. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3001_8 · Cited by 492 · see citing works
An implementation research agenda
Eccles, M. P., Armstrong, D., Baker, R., Cleary, K., Davies, H., Davies, S., Glasziou, P., Ilott, I., Kinmonth, A.-L., Leng, G., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-18 · Cited by 481 · see citing works
How to evaluate technologies for health behavior change in HCI research
Klasnja, P., Consolvo, S., & Pratt, W. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979396 · Cited by 414 · see citing works
Getting around the task-artifact cycle
Carroll, J. M., & Rosson, M. B. (1992) · ACM Transactions on Information Systems · doi:10.1145/146802.146834 · Cited by 391 · see citing works
An open letter toThe BMJeditors on qualitative research
Greenhalgh, T., Annandale, E. C., Ashcroft, R. E., Barlow, J. G., Black, N. A., Bleakley, A., Boaden, R., Braithwaite, J., Britten, N., Carnevale, F. A., et al. (2016) · BMJ · doi:10.1136/bmj.i563 · Cited by 378 · see citing works
Design and Evaluation in eHealth: Challenges and Implications for an Interdisciplinary Field
Pagliari, C. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e15 · Cited by 275 · see citing works
Self-Care Technologies in HCI: Trends, Tensions, and Opportunities
Nunes, F., Verdezoto, N., Fitzpatrick, G., Kyng, M., Grönvall, E., & Storni, C. (2015) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2803173 · Cited by 258 · see citing works
Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · see citing works
Attitudes Toward the Ethics of Research Using Social Media: A Systematic Review
Golder, S., Ahmed, S., Norman, G., & Booth, A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7082 · Cited by 214 · see citing works
Understanding heart failure; explaining telehealth—a hermeneutic systematic review
Greenhalgh, T., A’Court, C., & Shaw, S. (2017) · BMC Cardiovascular Disorders · doi:10.1186/s12872-017-0594-2 · Cited by 141 · see citing works
How to develop web-based decision support interventions for patients: A process map
Elwyn, G. J., Kreuwel, I., Durand, M., Sivell, S., Joseph‐Williams, N., Evans, R. M., & Edwards, A. (2010) · Patient Education and Counseling · doi:10.1016/j.pec.2010.04.034 · Cited by 141 · see citing works
Qualitative HCI Research: Going Behind the Scenes
BLANDFORD, A. E., Furniss, D. J., & Makri, S. (2016) · Synthesis lectures on human-centered informatics · doi:10.2200/s00706ed1v01y201602hci034 · Cited by 89 · see citing works
Self-citationCheong, Y. S., Yong, H., & Borland, R. (2007) · Nicotine & Tobacco Research · doi:10.1080/14622200701484961 · Cited by 88 · see citing works
Mazurek, J. A., & Jessup, M. L. (2015) · Cardiac Electrophysiology Clinics · doi:10.1016/j.ccep.2015.08.001 · Cited by 77 · see citing works
Pyramids are guides not rules: the evolution of the evidence pyramid
Shaneyfelt, T. M. (2016) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2016-110498 · Cited by 54 · see citing works
Drawing on human factors engineering to evaluate the effectiveness of health information technology
Cresswell, K. M., BLANDFORD, A. E., & Sheikh, A. (2017) · Journal of the Royal Society of Medicine · doi:10.1177/0141076817712252 · Cited by 28 · see citing works
Self-citationWhat happens to data gathered by health and wellness apps?
Armstrong, S. (2016) · BMJ · doi:10.1136/bmj.i3406 · Cited by 24 · see citing works
Unbounding the interaction design problem
Smith, W., Wadley, G., Webber, S. E., Ploderer, B., & Lederman, R. (2014) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/2686612.2686672 · Cited by 21 · see citing works
Snake oil, hustlers and hambones: the American medicine show
Parascandola, J. (2000) · Choice Reviews Online · doi:10.5860/choice.38-2183 · Cited by 20 · see citing works
DRAPER, S. W., & Norman, D. A. (1986) · CRC Press eBooks · doi:10.1201/b15703-01 · Cited by 9 · see citing works
Flagged first, then by citations
1 of the 42 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [24]Norman DA and Draper SW (eds). User centered system design: New perspectives on human–computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1986, pp.1–2.
- [25]Rogers Y Sharp H Preece J Interaction design: Beyond human-computer interaction, New York: John Wiley & Sons, 2011.
- [29]Greenhalgh T Annandale E Ashcroft R et al. An open letter to The BMJ editors on qualitative research. Brit Med J 2016; 352: i563. 26865572
Cited by
Sorted by citations
Sanches, P., Janson, A., Karpashevich, P., Nadal, C., Qu, C., Roquet, C. D., Umair, M., Windlin, C., Doherty, G. J., Höök, K., et al. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300475 · Cited by 227 · see citing works
Bergin, A. D., Vallejos, E. P., Davies, E. B., Daley, D., Ford, T., Harold, G., Hetrick, S., Kidner, M., Long, Y., Merry, S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00339-7 · Cited by 157 · see citing works
HCI for health and wellbeing: Challenges and opportunities
Blandford, A. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.06.007 · Cited by 152 · see citing works
Self-citationThe Development and Use of Chatbots in Public Health: Scoping Review
Wilson, L., & Mărăşoiu, M. (2022) · JMIR Human Factors · doi:10.2196/35882 · Cited by 135 · see citing works
Suijkerbuijk, S., Nap, H. H., Cornelisse, L., IJsselsteijn, W. A., de Kort, Y. A. W., & Minkman, M. M. N. (2019) · Journal of Alzheimer s Disease · doi:10.3233/jad-190050 · Cited by 116 · see citing works
Duffy, A., Christie, G. J., & Moreno, S. (2022) · JMIR Human Factors · doi:10.2196/35693 · Cited by 114 · see citing works
Cornet, V. P., Toscos, T., Bolchini, D. P., Ghahari, R. R., Ahmed, R., Daley, C. N., Mirro, M. J., & Holden, R. J. (2020) · JMIR mhealth and uhealth · doi:10.2196/17703 · Cited by 98 · see citing works
Zając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 76 · see citing works
Impact of mobile health and medical applications on clinical practice in gastroenterology
Kernebeck, S., Busse, T. S., Böttcher, M. D., Weitz, J., Ehlers, J. P., & Bork, U. (2020) · World Journal of Gastroenterology · doi:10.3748/wjg.v26.i29.4182 · Cited by 70 · see citing works
Tadas, S., & Coyle, D. (2020) · Journal of Medical Internet Research · doi:10.2196/18025 · Cited by 63 · see citing works
From Digital Health to Digital Well-being: Systematic Scoping Review
Smits, M., Kim, C. M., van Goor, H., & Ludden, G. (2022) · Journal of Medical Internet Research · doi:10.2196/33787 · Cited by 53 · see citing works
Alberts, N. M., Badawy, S. M., Hodges, J., Estepp, J. H., Nwosu, C., Khan, H., Smeltzer, M. P., Homayouni, R., Norell, S., Klesges, L. M., et al. (2020) · JMIR mhealth and uhealth · doi:10.2196/14884 · Cited by 60 · see citing works
Damen, I., Brombacher, H., Lallemand, C., Brankaert, R., Brombacher, A., van Wesemael, P. J. V., & Vos, S. B. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17020499 · Cited by 59 · see citing works
Li, Z., Lu, F., Wu, J., Bao, R., Rao, Y., Yang, Y. J., & Wang, H. (2024) · Journal of Medical Internet Research · doi:10.2196/56556 · Cited by 52 · see citing works
Farič, N., Smith, L., Hon, A., POTTS, H. W. W., Newby, K. V., Steptoe, A., & Fisher, A. (2020) · Journal of Medical Internet Research · doi:10.2196/18161 · Cited by 49 · see citing works
Wisk, L. E., Nelson, E. B., Magane, K. M., & Weitzman, E. R. (2019) · Journal of Diabetes Science and Technology · doi:10.1177/1932296819839503 · Cited by 42 · see citing works
Abdelaal, Y., Aupetit, M. J. M., Baggag, A., & Al‐Thani, D. (2024) · Journal of Medical Internet Research · doi:10.2196/53863 · Cited by 45 · see citing works
Andersen, T. O., Nunes, F., Wilcox, L. G., Coiera, E. W., & Rogers, Y. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589961 · Cited by 44 · see citing works
Prinjha, S., Ricci-Cabello, I., Newhouse, N., & Farmer, A. (2020) · JMIR mHealth and uHealth · doi:10.2196/15789 · Cited by 34 · see citing works
“Recent research into digital health intervention development highlights the importance of an iterative, multidisciplinary approach in which the end user is placed at the heart of the system and the technology is grounded in user wants and needs [45-47].”
BackgroundSelf-citationAyobi, A., Stawarz, K., Katz, D., Marshall, P., Yamagata, T., Santos‐Rodríguez, R., Flach, P. A., & O’Kane, A. A. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479601 · Cited by 40 · see citing works
Berry, N., Machin, M., Ainsworth, J. D., Berry, K., Edge, D., Haddock, G., Lewis, S. W., Morris, R., & Bucci, S. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.602861 · Cited by 38 · see citing works
Dimensions of ecological validity for usability evaluations in clinical settings
van Berkel, N., Clarkson, M. J., Xiao, G., Dursun, E., Allam, M., Davidson, B. R., & BLANDFORD, A. E. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103553 · Cited by 37 · see citing works
Self-citationDhinagaran, D. A., Martinengo, L., Ho, M. R., Joty, S., Kowatsch, T., Atun, R. A., & Car, L. T. (2022) · JMIR mhealth and uhealth · doi:10.2196/38740 · Cited by 36 · see citing works
Design Considerations for Supporting Mindfulness in Virtual Reality
Kelly, R., Seabrook, E., Foley, F., Thomas, N., Nedeljkovic, M., & Wadley, G. (2022) · Frontiers in Virtual Reality · doi:10.3389/frvir.2021.672556 · Cited by 36 · see citing works
Heneghan, M. B., Hussain, T., Barrera, L., Cai, S. W., Haugen, M., Morgan, E., Rossoff, J., Weinstein, J., Hijiya, N., Cella, D., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/24893 · Cited by 31 · see citing works
Bartlett, Y. K., Kenning, C., Crosland, J., Newhouse, N., Miles, L. M., Williams, V., McSharry, J., Locock, L., Farmer, A. A., & French, D. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06663-2 · Cited by 33 · see citing works
Self-citationKornfield, R., Stamatis, C. A., Bhattacharjee, A., Pang, B., Nguyen, T., Williams, J. J., Kumar, H., Popowski, S., Beltzer, M., Karr, C. J., et al. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100667 · Cited by 33 · see citing works
“To ensure tools are convenient and useful in the context of day-to-day life, it is necessary to involve end-users across the design and development process (Mohr et al., 2017; Blandford et al., 2018; Borghouts et al., 2021; Poole, 2013).”
MethodDoherty, K., Barry, M., Belisario, J. S. M., Morrison, C. P., Car, J., & Doherty, G. J. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.102373 · Cited by 33 · see citing works
An Evaluation Service for Digital Public Health Interventions: User-Centered Design Approach
Karpathakis, K., Libow, G., POTTS, H. W. W., Dixon, S., Greaves, F., & Murray, E. (2021) · Journal of Medical Internet Research · doi:10.2196/28356 · Cited by 31 · see citing works
Self-citationBabbage, C. M., Jackson, G. M., Davies, E. B., & Nixon, E. (2022) · JMIR Mental Health · doi:10.2196/25716 · Cited by 31 · see citing works
HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design
Slovák, P., & Munson, S. A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642624 · Cited by 30 · see citing works
Using Computer Games to Support Mental Health Interventions: Naturalistic Deployment Study
van der Meulen, H., McCashin, D., O’Reilly, G., & Coyle, D. (2019) · JMIR Mental Health · doi:10.2196/12430 · Cited by 29 · see citing works
Feroz, A. S., Afzal, N., & Seto, E. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056130 · Cited by 27 · see citing works
Alarm Management in Intensive Care: Qualitative Triangulation Study
Mosch, L. K., Sümer, M., Flint, A. R., Feufel, M. A., Balzer, F., Mörike, F., & Poncette, A.-S. (2024) · JMIR Human Factors · doi:10.2196/55571 · Cited by 26 · see citing works
Birk, M. V., Wadley, G., Abeele, V. V., Mandryk, R. L., & Torous, J. B. (2019) · interactions · doi:10.1145/3328483 · Cited by 25 · see citing works
Patient-centered Design Grounded in User and Clinical Realities: Towards Valid Digital Health
Cornet, V. P., Daley, C., Bolchini, D., Toscos, T., Mirro, M. J., & Holden, R. J. (2019) · Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care · doi:10.1177/2327857919081023 · Cited by 25 · see citing works
Stawarz, K., Preist, C., Tallon, D., Wiles, N., Kessler, D., Turner, K., Shafran, R., & Coyle, D. (2020) · JMIR Mental Health · doi:10.2196/15972 · Cited by 24 · see citing works
McCashin, D., Coyle, D., & O’Reilly, G. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100489 · Cited by 22 · see citing works
Jardine, J., Earley, C., Richards, D., Timulák, L., Palacios, J. E., Duffy, D. J., Tierney, K., & Doherty, G. J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376254 · Cited by 22 · see citing works
Mackintosh, N., Gong, Q. (., Hadjiconstantinou, M., & Verdezoto, N. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2021.114299 · Cited by 22 · see citing works
Lin, Y., Lemos, M., & Neuschaefer‐Rube, C. (2021) · JMIR Medical Education · doi:10.2196/30873 · Cited by 24 · see citing works
Agapie, E., Haldar, S., & Poblete, S. G. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555610 · Cited by 21 · see citing works
Nordgreen, T., Rabbi, F., Tørresen, J., Skar, Y. S., Guribye, F., İnal, Y., Flobak, E., Wake, J. D., Mukhiya, S. K., Aminifar, A., et al. (2021) · Journal of Enabling Technologies · doi:10.1108/jet-03-2021-0013 · Cited by 24 · see citing works
Kornfield, R., Mohr, D. C., Ranney, R. M., Lattie, E. G., Meyerhoff, J., Williams, J. J., & Reddy, M. C. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512946 · Cited by 19 · see citing works
Veldmeijer, L., Terlouw, G., Veer, J. V. ‘., & Kuipers, D. A. (2022) · Design for Health · doi:10.1080/24735132.2022.2059997 · Cited by 19 · see citing works
Knox, L., McDermott, C. J., & Hobson, E. (2022) · Journal of Medical Engineering & Technology · doi:10.1080/03091902.2022.2040625 · Cited by 19 · see citing works
Benton, J. S., Cotterill, S., Hawkes, R. E., Miles, L. M., & French, D. (2022) · Journal of Medical Internet Research · doi:10.2196/39483 · Cited by 17 · see citing works
Siering, L., Ludden, G., Mader, A., & van Rees, H. (2019) · Journal of Personalized Medicine · doi:10.3390/jpm9020027 · Cited by 20 · see citing works
Flobak, E., Nordby, E. S., Guribye, F., Kenter, R. M. F., Nordgreen, T., & Lundervold, A. J. (2021) · JMIR Mental Health · doi:10.2196/30292 · Cited by 18 · see citing works
Smits, M., Ludden, G., Verbeek, P., & van Goor, H. (2022) · Interactive Journal of Medical Research · doi:10.2196/39323 · Cited by 17 · see citing works
Alrizq, M., Solangi, S. A., Alghamdi, A., Nizamani, M. A., Memon, M. A., & Hamdi, M. (2021) · Computer Systems Science and Engineering · doi:10.32604/csse.2022.018800 · Cited by 19 · see citing works
Prentice, C., Peven, K., Zhaunova, L., Nayak, V., Radović, T., Klepchukova, A., POTTS, H. W. W., & Ponzo, S. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00092-x · Cited by 17 · see citing works
Pusa, S., Dorell, Å., Erlingsson, C., Antonsson, H., Brännström, M., & Sundin, K. (2019) · Journal of Clinical Nursing · doi:10.1111/jocn.14745 · Cited by 12 · see citing works
Behavior Change Apps for Gestational Diabetes Management: Exploring Desirable Features
Kytö, M., Strömberg, L., Tuomonen, H., Ruonala, A., Koivusalo, S. B., & Jacucci, G. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1987678 · Cited by 16 · see citing works
Interdisciplinary Tensions When Developing Digital Interventions Supporting Individuals With ADHD
Cibrian, F. L., Monteiro, E. M., Schuck, S. E. B., Nelson, M. A., Hayes, G. R., & Lakes, K. D. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.876039 · Cited by 15 · see citing works
Kolopaking, L. M., Wahyono, E. B., Irmayani, N. R., Habibullah, H., & Erwinsyah, R. G. (2022) · International Journal of Sustainable Development and Planning · doi:10.18280/ijsdp.170713 · Cited by 15 · see citing works
Aldaweesh, S., AlAteeq, D. A., Kleek, M. G. V., & Shadbolt, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642642 · Cited by 15 · see citing works
Kechagias, E. P., Papadopoulos, G. A., & Rokai, I. (2024) · Administrative Sciences · doi:10.3390/admsci14060131 · Cited by 15 · see citing works
Smartphones within Psychological Science
Ellis, D. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108671408 · Cited by 14 · see citing works
Davis, K. S., Anderson, M., Yang, C., Choukas‐Bradley, S., Bell, B. T., & Slovák, P. (2024) · Journal of Adolescent Research · doi:10.1177/07435584241291163 · Cited by 17 · see citing works
User-Centered Design of a Mobile App to Support Peer Recovery in a Clinical Setting
Pater, J. A., Phelan, C., Cornet, V. P., Ahmed, R., Colletta, S., Hess, E. P., Kerrigan, C., & Toscos, T. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449186 · Cited by 13 · see citing works
Smits, M., Ludden, G., Verbeek, P., & van Goor, H. (2022) · Journal of Responsible Innovation · doi:10.1080/23299460.2022.2076986 · Cited by 14 · see citing works
Njoku, C., Green, S. A., Sathyamoorthy, G., Patel, N., & POTTS, H. W. W. (2023) · Digital Health · doi:10.1177/20552076231173303 · Cited by 14 · see citing works
Rudroff, T., Rainio, O., & Klén, R. (2024) · Brain Sciences · doi:10.3390/brainsci14080831 · Cited by 14 · see citing works
Xie, L. F., Housni, A., Nakhla, M. M., Cianci, R., Leroux, C., Costa, D. D., & Brazeau, A. (2023) · JMIR Diabetes · doi:10.2196/42564 · Cited by 14 · see citing works
Merolli, M., Hinman, R. S., Lawford, B. J., Choo, D., & Gray, K. M. (2021) · JMIR Research Protocols · doi:10.2196/25177 · Cited by 13 · see citing works
Agapie, E., Karkar, R., Aung, T., Burgess, E. R., Chinguwa, M. J., Graham, A. K., Klasnja, P., Lyon, A. R., McCall, T., Munson, S. A., et al. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636298 · Cited by 12 · see citing works
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 14 · see citing works
Disability Interactions: Creating Inclusive Innovations
Holloway, C., & Barbareschi, G. (2021) · Synthesis Lectures on Human-Centered Informatics · doi:10.2200/s01141ed1v01y202111hci053 · Cited by 15 · see citing works
Healthcare Infrastructures in Ecuador
Carlo, L., Carpio-Arias, T. V., Verdezoto, N. X., Eslambolchilar, P., Cruz, E., Malo, F., & Espinosa, D. (2020) · International Conference on Information and Communication Technologies and Development (ICTD) · doi:10.1145/3392561.3397578 · Cited by 13 · see citing works
McLaughlin, M. J., Duff, J. S., McKenzie, T., Campbell, E. M., Sutherland, R., Wiggers, J. H., & Wolfenden, L. (2021) · JMIR Pediatrics and Parenting · doi:10.2196/26690 · Cited by 10 · see citing works
Evaluation of Patient-Facing Mobile Apps to Support Physiotherapy Care: Systematic Review
Merolli, M., Francis, J., Vallance, P., Bennell, K. L., Malliaras, P., & Hinman, R. S. (2024) · JMIR mhealth and uhealth · doi:10.2196/55003 · Cited by 11 · see citing works
Lessons from working with researchers and practitioners in healthcare
BLANDFORD, A. E. (2018) · interactions · doi:10.1145/3292023 · Cited by 11 · see citing works
Self-citationRobinson, H., Appelbe, D., Dodd, S. R., Flowers, S., Johnson, S., Jones, S. H., Mateus, C., Mezes, B., Murray, E., Rainford, N., et al. (2020) · JMIR Mental Health · doi:10.2196/15878 · Cited by 11 · see citing works
Self-citationLiverpool, S., & Edbrooke-Childs, J. H. (2021) · JMIR Formative Research · doi:10.2196/25235 · Cited by 9 · see citing works
Mehta, P., Moore, S. L., Bull, S. S., & Kwan, B. M. (2021) · Digital Health · doi:10.1177/20552076211019877 · Cited by 11 · see citing works
MacPherson, M., Cranston, K. D., Johnston, C., Locke, S., & Jung, M. E. (2021) · JMIR Formative Research · doi:10.2196/28163 · Cited by 10 · see citing works
Cannata, D., Breil, S. M., Lepri, B., Back, M. D., & O’Hora, D. P. (2022) · Technology Mind and Behavior · doi:10.1037/tmb0000054 · Cited by 10 · see citing works
Yoo, S., Ramalhinho, J., Dowrick, T., Somasundaram, M., Gurusamy, K. S., Davidson, B. R., Clarkson, M. J., & BLANDFORD, A. E. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.01.008 · Cited by 9 · see citing works
Self-citationGreenwell, K., Ainsworth, B., Bruton, A., Murray, E., Russell, D., Thomas, M., & Yardley, L. (2021) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-021-00248-6 · Cited by 8 · see citing works
Self-citationSun, Y., Tenesa, A., & Vines, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713567 · Cited by 8 · see citing works
Behavior Change Interventions Combating Online Misinformation: A Scoping Review
Konstantinou, L., & Karapanos, E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713127 · Cited by 10 · see citing works
Stassen, G., Grieben, C., Froböse, I., & Schaller, A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17072180 · Cited by 10 · see citing works
Chen, N.-J., Chang, C.-H., Huang, C., Lin, F.-H., Lu, L.-T., Liu, K.-Y., Lai, C.-L., Lin, C.-Y., Hou, Y., & Guo, J. (2024) · Journal of Medical Internet Research · doi:10.2196/68072 · Cited by 7 · see citing works
Architecting Utopias: How AI in Healthcare Envisions Societal Ideals and Human Flourishing
Wieczorek, C., Biggs, H. R., Thiruvenkatanathan, K. P., & Bardzell, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713118 · Cited by 8 · see citing works
Hovingh, J. W., van Duin, C. E. , Kuipers, D. A., van Rood, Y. R., Ludden, G., Hanssen, D. J. C., & Rosmalen, J. G. M. (2025) · JMIR Human Factors · doi:10.2196/76172 · Cited by 7 · see citing works
Aryeetey, R., Lasisi, O., Hromi‐Fiedler, A. J., Carroll, G. J., Pérez‐Escamilla, R., & Harding, K. L. (2020) · Digital Health · doi:10.1177/2055207620909291 · Cited by 7 · see citing works
Manning, J. B., BLANDFORD, A. E., & Edbrooke-Childs, J. H. (2021) · JMIR Formative Research · doi:10.2196/32312 · Cited by 10 · see citing works
Self-citationDigital Health in Schools: A Systematic Review
França, C., Santos, F., Martins, F., Lopes, H., de Gouveia, B. R. F. O., Gonçalves, F., Campos, P., Marques, A., Ihle, A., Gonçalves, T., et al. (2022) · Sustainability · doi:10.3390/su142113848 · Cited by 10 · see citing works
Human Computer Interaction Design for Mobile Devices Based on a Smart Healthcare ArchitecturePreprint
Liu, P., Fels, S., West, N., & Görges, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.03541 · Cited by 11 · see citing works
Human-Centered Design and Digital Transformation of Mental Health Services
Fleming, W. J., Coutts, A. P., Pochard, D., Trivedi, D. P., & Sanderson, K. A. (2025) · JMIR Human Factors · doi:10.2196/66040 · Cited by 7 · see citing works
Evaluation Methods in Clinical Health Technologies: A Systematic Review
Mohammadzadeh, N., Katigari, M. R., Hosseini, R., & Pahlevanynejad, S. (2023) · Iranian Journal of Public Health · doi:10.18502/ijph.v52i5.12708 · Cited by 5 · see citing works
Karparvar, Z., Mirzabeigi, M., & Salimi, G. (2023) · Aslib Journal of Information Management · doi:10.1108/ajim-06-2022-0287 · Cited by 6 · see citing works
Cooney, E., Toomey, E. C., Ryan, K., Meade, O., & McSharry, J. (2024) · British Journal of Health Psychology · doi:10.1111/bjhp.12741 · Cited by 5 · see citing works
BLANDFORD, A. E., Bindman, J., Bradbury, K. J., Cooper, B., Costanza, E., Edwards, S. J., Hamilton, F. L., Heightman, M., Hurst, J. R., Hylton, H., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3711839 · Cited by 5 · see citing works
Self-citationAnand-Kumar, V., Schreier, M., Bakshi, N., Messiha, K., Longworth, G. R., An, Q., Agnello, D. M., CHASTIN, S. F. M., & Lippke, S. (2025) · Public Health · doi:10.1016/j.puhe.2025.105847 · Cited by 5 · see citing works
Meyer, J., Ratz, T., Pauls, A., Hellmers, S., Boll, S., Fudickar, S. J. F., Hein, A., Bauer, J. M., Koppelin, F., Lippke, S., et al. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.832922 · Cited by 5 · see citing works
Karpathakis, K., Pencheon, E., & Cushnan, D. (2024) · JMIR AI · doi:10.2196/51168 · Cited by 5 · see citing works
Armitage, B. T., POTTS, H. W. W., Irwin, M. R., & Fisher, A. (2024) · JMIR Formative Research · doi:10.2196/39554 · Cited by 4 · see citing works
Hao, C., Uusitalo, S., Figueroa, C. A., Smit, Q. T. S., Strange, M., Chang, W.-T., Ribeiro, M. I., Nana, V. K., Tielman, M. L., & de Boer, M. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1544185 · Cited by 5 · see citing works
- Authors
- Blandford, A., Gibbs, J., Newhouse, N., Perski, O., Singh, A., & Murray, E.
- Affiliations
- University College London (GB)
- The Royal Free Hospital (GB)
- Corresponding author
- Blandford, Ann
- Venue
- DIGITAL HEALTH
- Publisher
- SAGE Publications
- Published
- 2018-01
- Volume
- 4
- Pages
- 2055207618770325-2055207618770325
- ISSN
- 2055-2076 (ISSN-L)
- PMC
- PMC6016567
- OpenAlex
- W2800091779
- Cited by
- 209
- References
- 42
- Citations by year
- 2026: 36 · 2025: 30 · 2024: 28 · 2023: 14 · 2022: 24 · 2021: 24 · 2020: 32 · 2019: 17 · 2018: 4
- FWCI
- 27.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Digital Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Engineering and Physical Sciences Research Council (EP/G004560/1)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Interdisciplinary Research and Collaboration
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- interdisciplinary research, human-computer interaction, health research, formative assessment, summative assessment, research culture, research reporting, research lifecycle, user needs, interdisciplinary collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
