The DOI matches its Crossref record; 69 of the 72 references it cites were found in a registry or an index.
The Pace of Technologic Change
Abstract
This paper addresses the rapid pace of change in the technologies that support digital interventions; the complexity of the health problems they aim to address; and the adaptation of scientific methods to accommodate the volume, velocity, and variety of data and interventions possible from these technologies. Information, communication, and computing technologies are now part of every societal domain and support essentially every facet of human activity. Ubiquitous computing, a vision articulated fewer than 30 years ago, has now arrived. Simultaneously, there is a global crisis in health through the combination of lifestyle and age-related chronic disease and multiple comorbidities. Computationally intensive health behavior interventions may be one of the most powerful methods to reduce the consequences of this crisis, but new methods are needed for health research and practice, and evidence is needed to support their widespread use. The challenges are many, including a reluctance to abandon timeworn theories and models of health behavior-and health interventions more broadly-that emerged in an era of self-reported data; medical models of prevention, diagnosis, and treatment; and scientific methods grounded in sparse and expensive data. There are also many challenges inherent in demonstrating that newer approaches are, indeed, effective. Potential solutions may be found in leveraging methods of research that have been shown to be successful in other domains, particularly engineering. A more "agile science" may be needed that streamlines the methods through which elements of health interventions are shown to work or not, and to more rapidly deploy and iteratively improve those that do. There is much to do to advance the issues discussed in this paper, and the papers in this theme issue. It remains an open question whether interventions based in these new models and methods are, in fact, equally if not more efficacious as what is available currently. Economic analyses of these new approaches are needed because assumptions of net worth compared to other approaches are just that, assumptions. Human-centered design research is needed to ensure that users ultimately benefit. Finally, a translational research agenda will be needed, as the status quo will likely be resistant to change.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is in line with agile science, which states that eHealth development should be iterative and flexible, with short sprints to allow for constant changes to the process and products [14,183].”
From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [183]
In the citing paper
“Supervised learning predicts adherence and effects from early use patterns, which enables early intervention for users at risk [31,33,34].”
From Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data (2017) · cites this paper as [31]
In the citing paper
“More advanced and predictive information for pattern recognition within complex data sets can be obtained by applying machine-learning algorithms [31].”
From Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data (2017) · cites this paper as [31]
Citations
Cites
The Internet of Things: A survey
Atzori, L., Iera, A., & Morabito, G. (2010) · Computer Networks · doi:10.1016/j.comnet.2010.05.010 · Cited by 15,613
Lim, S. S., Vos, T., Flaxman, A. D., Danaei, G., Shibuya, K., Adair-Rohani, H., AlMazroa, M. A., Amann, M., Anderson, H. R., Andrews, K. G., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61766-8 · Cited by 11,805
Murray, C. J. L., Vos, T., Lozano, R., Naghavi, M., Flaxman, A. D., Michaud, C., Ezzati, M., Shibuya, K., Salomon, J. A., Abdalla, S., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61689-4 · Cited by 8,888
Human gut microbes associated with obesity
Ley, R. E., Turnbaugh, P. J., Klein, S., & Gordon, J. I. (2006) · Nature · doi:10.1038/4441022a · Cited by 8,785
Turnbaugh, P. J., Ley, R. E., Hamady, M., Fraser-Liggett, C. M., Knight, R., & Gordon, J. I. (2007) · Nature · doi:10.1038/nature06244 · Cited by 6,091
A New Initiative on Precision Medicine
Collins, F. S., & Varmus, H. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500523 · Cited by 5,202
On the Self-Regulation of Behavior
Wright, B. R. E., Carver, C. S., & Scheier, M. F. (2000) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2654424 · Cited by 3,327
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524
Bauer, U. E., Briss, P. A., Goodman, R. A., & Bowman, B. A. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60648-6 · Cited by 1,453
Exercise and associated dietary extremes impact on gut microbial diversity
Clarke, S. F., Murphy, E. F., O'Sullivan, O., Lucey, A. J., Humphreys, M., Hogan, A., Hayes, P., O'Reilly, M., Jeffery, I. B., Wood-Martin, R., et al. (2014) · Gut · doi:10.1136/gutjnl-2013-306541 · Cited by 1,439
Heron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438
Internet-Based and Other Computerized Psychological Treatments for Adult Depression: A Meta-Analysis
Andersson, G., & Cuijpers, P. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070903318960 · Cited by 1,368
Best care at lower cost: the path to continuously learning health care in America
Smith, M. A., Saunders, R., Stuckhardt, L., & McGinnis, J. M. (2014) · Choice Reviews Online · doi:10.5860/choice.51-3277 · Cited by 1,358
Jumpertz, R., Le, D. S., Turnbaugh, P. J., Trinidad, C., Bogardus, C., Gordon, J. I., & Krakoff, J. (2011) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.110.010132 · Cited by 1,232
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions
Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., Merchant, G. C., Naughton, F., & Blandford, A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.015 · Cited by 1,118
Self-citationMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097
Self-citationBayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · cited 1×
“The accumulation of knowledge includes both discovery and confirmation, and thus the Bayesian style of statistics may become more useful than frequentist or null hypothesis statistical testing approaches.(23,51) A Bayesian approach, by design, supports incremental model building via the use of “priors” knowledge.”
MethodMobile Health Technology Evaluation
Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., et al. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.017 · Cited by 900
Self-citationEvaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847
Self-citationMicrorandomized trials: An experimental design for developing just-in-time adaptive interventions.
Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015) · Health Psychology · doi:10.1037/hea0000305 · Cited by 671
Self-citationThe Chronic Care Model and Diabetes Management in US Primary Care Settings: A Systematic Review
Stellefson, M., Dipnarine, K., & Stopka, C. B. (2013) · Preventing Chronic Disease · doi:10.5888/pcd10.120180 · Cited by 591
The Learning Healthcare System: Workshop Summary
Olsen, L., Aisner, D. L., & McGinnis, J. M. (2013) · Cited by 588
Social physics: how good ideas spread--the lessons from a new science
Pentland, A. (2014) · Choice Reviews Online · doi:10.5860/choice.52-0805 · Cited by 573
Nahum-Shani, I., Hekler, E. B., & Spruijt-Metz, D. (2015) · Health Psychology · doi:10.1037/hea0000306 · Cited by 568
Self-citationPayne, H., Lister, C., West, J. H., & Bernhardt, J. M. (2015) · JMIR mhealth and uhealth · doi:10.2196/mhealth.3335 · Cited by 556
Celis-Morales, C. A., Perez-Bravo, F., Ibañez, L., Salas, C., Bailey, M. E. S., & Gill, J. M. R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036345 · Cited by 486
Tate, D. F., Jackvony, E. H., & Wing, R. R. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.15.1620 · Cited by 427
A systems-oriented multilevel framework for addressing obesity in the 21st century.
Huang, T. T. ‐., Drewnosksi, A., Kumanyika, S., & Glass, T. A. (2009) · PubMed Central · Cited by 385
Recent trends in chronic disease, impairment and disability among older adults in the United States
Hung, W. W., Ross, J. S., Boockvar, K. S., & Siu, A. L. (2011) · BMC Geriatrics · doi:10.1186/1471-2318-11-47 · Cited by 373
Leal, C., & Chaix, B. (2010) · Obesity Reviews · doi:10.1111/j.1467-789x.2010.00726.x · Cited by 363
Wicks, P., Vaughan, T. E., Massagli, M. P., & Heywood, J. (2011) · Nature Biotechnology · doi:10.1038/nbt.1837 · Cited by 362
Ellis, K., Kerr, J., Godbole, S. V., Lanckriet, G. R. G., Wing, D., & Marshall, S. J. (2014) · Physiological Measurement · doi:10.1088/0967-3334/35/11/2191 · Cited by 354
Laing, B. Y., Mangione, C. M., Tseng, C.-H., Leng, M., Vaisberg, E., Mahida, M., Bholat, M., Glazier, E., Morisky, D. E., & Bell, D. S. (2014) · Annals of Internal Medicine · doi:10.7326/m13-3005 · Cited by 353
Open mHealth Architecture: An Engine for Health Care Innovation
Estrin, D. L., & Sim, I. (2010) · Science · doi:10.1126/science.1196187 · Cited by 347
Self-citationA Content Analysis of Popular Smartphone Apps for Smoking Cessation
Abroms, L. C., Westmaas, J. L., Bontemps-Jones, J. E., Ramani, R., & Mellerson, J. L. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.07.008 · Cited by 325
A randomized controlled trial of an automated exercise coach for older adults
Bickmore, T. W., Silliman, R. A., Nelson, K., Cheng, D. M., Winter, M., Henault, L., & Paasche‐Orlow, M. K. (2013) · Journal of the American Geriatrics Society · doi:10.1111/jgs.12449 · Cited by 305 · cited 1×
“However, it is possible that the functions of human coaching, such as providing accountability and support,(72) may be automated through virtual conversational agents.(74,75) Although digital interventions have shown promise in pilot studies, current evaluation methods are undeveloped or perform poorly.”
BackgroundFurukawa, M. F., King, J., Patel, V., Hsiao, C., Adler‐Milstein, J., & Jha, A. K. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0445 · Cited by 282
Weight loss—there is an app for that! But does it adhere to evidence-informed practices?
Breton, E. R., Fuemmeler, B. F., & Abroms, L. C. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0076-5 · Cited by 276
Rabbi, M., Aung, M. H., Zhang, M., & Choudhury, T. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2805840 · Cited by 274
Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise
Riley, W. T., Glasgow, R. E., Etheredge, L., & Abernethy, A. P. (2013) · Clinical and Translational Medicine · doi:10.1186/2001-1326-2-10 · Cited by 267
Self-citationCowan, L. T., Van Wagenen, S. A., Brown, B. A., Hedin, R. J., Seino-Stephan, Y., Hall, P. C., & West, J. H. (2013) · Health Education & Behavior · doi:10.1177/1090198112452126 · Cited by 264
Mobile Health: Revolutionizing Healthcare Through Transdisciplinary Research
Kumar, S., Nilsen, W., Misha Pavel, C., & Srivastava, M. B. (2013) · Computer · doi:10.1109/mc.2012.392 · Cited by 243
Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216
Self-citationAgile science: creating useful products for behavior change in the real world
Hekler, E. B., Klasnja, P., Riley, W. T., Buman, M. P., Huberty, J., Rivera, D. E., & Martin, C. A. (2016) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0395-7 · Cited by 208
Self-citationAutomated interventions for multiple health behaviors using conversational agents
Bickmore, T. W., Schulman, D., & Sidner, C. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.05.011 · Cited by 206
Advancing Models and Theories for Digital Behavior Change Interventions
Hekler, E. B., Michie, S., Pavel, M., Rivera, D. E., Collins, L. M., Jimison, H. B., Garnett, C., Parral, S., & Spruijt-Metz, D. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.013 · Cited by 174
Self-citationContinuous Evaluation of Evolving Behavioral Intervention Technologies
Mohr, D. C., Cheung, K., Schueller, S. M., Brown, C. H., & Duan, N. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.06.006 · Cited by 171
Self-citationHypoglycemia Prediction Using Machine Learning Models for Patients With Type 2 Diabetes
Sudharsan, B., Peeples, M., & Shomali, M. E. (2014) · Journal of Diabetes Science and Technology · doi:10.1177/1932296814554260 · Cited by 162
Rivera, D. E., Pew, M. D., & Collins, L. M. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.10.020 · Cited by 151
Digital Medical Tools and Sensors
Topol, E. J., Steinhubl, S. R., & Torkamani, A. (2015) · JAMA · doi:10.1001/jama.2014.17125 · Cited by 146
Helander, E., Kaipainen, K., Korhonen, I., & Wansink, B. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3084 · Cited by 143
Ellis, K., Godbole, S., Marshall, S., Lanckriet, G., Staudenmayer, J., & Kerr, J. (2014) · Frontiers in Public Health · doi:10.3389/fpubh.2014.00036 · Cited by 123
Kleiboer, A. M., Donker, T., Seekles, W., van Straten, A., Riper, H. M., & Cuijpers, P. J. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.013 · Cited by 113
Self-citationEstrin, D. L. (2014) · Communications of the ACM · doi:10.1145/2580944 · Cited by 109
Self-citationAn Ecological Framework for Cancer Communication: Implications for Research
Patrick, K., Intille, S. S., & Zabinski, M. F. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.3.e23 · Cited by 98
Self-citationDiagnostic Verification of Temperature Forecasts
Murphy, A. H., Brown, B. G., & Chen, Y. (1989) · Weather and Forecasting · doi:10.1175/1520-0434(1989)004 · Cited by 97
MARKOV MODELING AND DISCRETE EVENT SIMULATION IN HEALTH CARE: A SYSTEMATIC COMPARISON
Standfield, L., Comans, T. A., & Scuffham, P. (2014) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462314000117 · Cited by 90
Project onward: an innovative e-health intervention for cancer survivors
Duffecy, J., Sanford, S., Wagner, L., Begale, M., Nawacki, E., & Mohr, D. C. (2013) · Psycho-Oncology · doi:10.1002/pon.3075 · Cited by 89
Self-citationDevelopment of a dynamic computational model of social cognitive theory
Riley, W. T., Martín, C. A., Rivera, D. E., Hekler, E. B., Adams, M. A., Buman, M. P., Pavel, M., & King, A. C. (2015) · Translational Behavioral Medicine · doi:10.1007/s13142-015-0356-6 · Cited by 88
Self-citationDesigning and undertaking a health economics study of digital health interventions
McNamee, P., Murray, E., Kelly, M. P., Bojke, L., Chilcott, J., Fischer, A., West, R., & Yardley, L. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.05.007 · Cited by 82
Riley, W. T., Serrano, K. J., Nilsen, W., & Atienza, A. A. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.024 · Cited by 76
Self-citationSisko, A. M., Keehan, S. P., Cuckler, G. A., Madison, A. J., Smith, S. D., Wolfe, C. J., Stone, D. A., Lizonitz, J. M., & Poisal, J. A. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0560 · Cited by 74
Just-in-time technology to encourage incremental, dietary behavior change.
Intille, S. S., Kukla, C., Farzanfar, R., & Bakr, W. (2003) · Europe PMC (PubMed Central) · Cited by 72
Innovative ICT Solutions to Improve Treatment Outcomes for Depression: The ICT4Depression Project
Warmerdam, L., Riper, H. M., Michel, K., Pepijn, V. D. V., Rocha, A., Mario, R. H., Tousset, E., Hugo, S., Andersson, G., & Cuijpers, P. J. (2012) · Studies in health technology and informatics · doi:10.3233/978-1-61499-121-2-339 · Cited by 54
Self-citationBroekhuizen, K., van Poppel, M. N. M., Koppes, L. L. J., Kindt, I., Brug, J., & van Mechelen, W. (2012) · PLoS ONE · doi:10.1371/journal.pone.0050032 · Cited by 50
A smart mirror to promote a healthy lifestyle
Colantonio, S., Coppini, G., Germanese, D., Giorgi, D., Magrini, M., Marraccini, P., Martinelli, M., Morales, M. A., Pascali, M. A., Raccichini, G., et al. (2015) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2015.06.008 · Cited by 47
Realizing Effective Behavioral Management of Health: The Metamorphosis of Behavioral Science Methods
Hekler, E. B., Klasnja, P., Traver, V., & Hendriks, M. (2013) · IEEE Pulse · doi:10.1109/mpul.2013.2271681 · Cited by 45
Self-citationMethodologies for optimizing behavioral interventions: introduction to special section
Riley, W. T., & Rivera, D. E. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0281-0 · Cited by 28
Self-citation
Research works printed but not linked: 3
- 2014, mHealth App Developer Economics
- 2013, Mobile Health Market Report 2013-2017
- 2013, Green paper on Citizen Science for Europe: towards a society of empowered citizens and enhanced research, Socientize Consort, 1
Cited by
Sorted by citations
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions
Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., Merchant, G. C., Naughton, F., & Blandford, A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.015 · Cited by 1,118
Self-citationMichie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865
“Thus, the methods by which we deploy, evaluate, and improve these DBCIs are moving from relatively data-poor, infrequent, and typically post-hoc assessments to methods that incorporate continuous measurement of the intervention effects in real time [3].”
BackgroundMethodInfluentialSelf-citationLewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588
Mohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432
Self-citationAttitudes towards digital treatment for depression: A European stakeholder survey
Topooco, N., Riper, H., Araya, R., Berking, M., Brunn, M., Chevreul, K., Cieslak, R., Ebert, D. D., Etchmendy, E., Herrero, R., et al. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.01.001 · Cited by 262
Self-citation
Metadata
- Authors
- Patrick, K., Hekler, E. B., Estrin, D., Mohr, D. C., Riper, H., Crane, D., Godino, J., & Riley, W. T.
- Affiliations
- University of California San Diego (US)
- Arizona State University (US)
- Cornell University (US)
- Northwestern University (US)
- Vrije Universiteit Amsterdam (NL)
- University College London (GB)
- National Institutes of Health (US)
- Corresponding author
- Patrick, Kevin
- ORCID
- Venue
- American Journal of Preventive Medicine
- Publisher
- Elsevier BV
- Published
- 2016-11
- Volume
- 51
- Issue
- 5
- Pages
- 816-824
- ISSN
- 0749-3797 (ISSN-L)
- OpenAlex
- W2530300749
- Cited by
- 169
- References
- 72
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 14 · 2023: 13 · 2022: 17 · 2021: 22 · 2020: 19 · 2019: 23 · 2018: 27 · 2017: 19 · 2016: 3
- FWCI
- 44.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Preventive Medicine (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- digital health interventions, health behavior change, ubiquitous computing, chronic disease, comorbidities, evidence-based assessment, human-centered design, translational research, economic evaluation, health behavior theory
- MeSH terms
- Health Promotion, Humans, Research Design, Telecommunications, Health Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
