The Pace of Technologic Change
Abstract
This paper addresses the rapid pace of change in the technologies that support digital interventions; the complexity of the health problems they aim to address; and the adaptation of scientific methods to accommodate the volume, velocity, and variety of data and interventions possible from these technologies. Information, communication, and computing technologies are now part of every societal domain and support essentially every facet of human activity. Ubiquitous computing, a vision articulated fewer than 30 years ago, has now arrived. Simultaneously, there is a global crisis in health through the combination of lifestyle and age-related chronic disease and multiple comorbidities. Computationally intensive health behavior interventions may be one of the most powerful methods to reduce the consequences of this crisis, but new methods are needed for health research and practice, and evidence is needed to support their widespread use. The challenges are many, including a reluctance to abandon timeworn theories and models of health behavior-and health interventions more broadly-that emerged in an era of self-reported data; medical models of prevention, diagnosis, and treatment; and scientific methods grounded in sparse and expensive data. There are also many challenges inherent in demonstrating that newer approaches are, indeed, effective. Potential solutions may be found in leveraging methods of research that have been shown to be successful in other domains, particularly engineering. A more "agile science" may be needed that streamlines the methods through which elements of health interventions are shown to work or not, and to more rapidly deploy and iteratively improve those that do. There is much to do to advance the issues discussed in this paper, and the papers in this theme issue. It remains an open question whether interventions based in these new models and methods are, in fact, equally if not more efficacious as what is available currently. Economic analyses of these new approaches are needed because assumptions of net worth compared to other approaches are just that, assumptions. Human-centered design research is needed to ensure that users ultimately benefit. Finally, a translational research agenda will be needed, as the status quo will likely be resistant to change.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Agile science states that eHealth development should be iterative and flexible, with short sprints to allow for constant changes to the process and products.
“This is in line with agile science, which states that eHealth development should be iterative and flexible, with short sprints to allow for constant changes to the process and products [14,183].”
From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [183]
Supervised learning predicts adherence and effects from early use patterns, which enables early intervention for users at risk.
“Supervised learning predicts adherence and effects from early use patterns, which enables early intervention for users at risk [31,33,34].”
From Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data (2017) · cites this paper as [31]
More advanced and predictive information for pattern recognition within complex data sets can be obtained by applying machine-learning algorithms.
“More advanced and predictive information for pattern recognition within complex data sets can be obtained by applying machine-learning algorithms [31].”
From Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data (2017) · cites this paper as [31]
Cites
The Internet of Things: A survey
Atzori, L., Iera, A., & Morabito, G. (2010) · Computer Networks · doi:10.1016/j.comnet.2010.05.010 · Cited by 15,613 · see citing works
Lim, S. S., Vos, T., Flaxman, A. D., Danaei, G., Shibuya, K., Adair-Rohani, H., AlMazroa, M. A., Amann, M., Anderson, H. R., Andrews, K. G., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61766-8 · Cited by 11,805 · see citing works
Murray, C. J. L., Vos, T., Lozano, R., Naghavi, M., Flaxman, A. D., Michaud, C., Ezzati, M., Shibuya, K., Salomon, J. A., Abdalla, S., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61689-4 · Cited by 8,888 · see citing works
Human gut microbes associated with obesity
Ley, R. E., Turnbaugh, P. J., Klein, S., & Gordon, J. I. (2006) · Nature · doi:10.1038/4441022a · Cited by 8,785 · see citing works
Turnbaugh, P. J., Ley, R. E., Hamady, M., Fraser-Liggett, C. M., Knight, R., & Gordon, J. I. (2007) · Nature · doi:10.1038/nature06244 · Cited by 6,091 · see citing works
A New Initiative on Precision Medicine
Collins, F. S., & Varmus, H. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500523 · Cited by 5,202 · see citing works
On the Self-Regulation of Behavior
Wright, B. R. E., Carver, C. S., & Scheier, M. F. (2000) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2654424 · Cited by 3,327 · see citing works
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · see citing works
Exercise and associated dietary extremes impact on gut microbial diversity
Clarke, S. F., Murphy, E. F., O'Sullivan, O., Lucey, A. J., Humphreys, M., Hogan, A., Hayes, P., O'Reilly, M., Jeffery, I. B., Wood-Martin, R., et al. (2014) · Gut · doi:10.1136/gutjnl-2013-306541 · Cited by 1,439 · see citing works
Heron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438 · see citing works
Bauer, U. E., Briss, P. A., Goodman, R. A., & Bowman, B. A. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60648-6 · Cited by 1,395 · see citing works
Internet-Based and Other Computerized Psychological Treatments for Adult Depression: A Meta-Analysis
Andersson, G., & Cuijpers, P. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070903318960 · Cited by 1,368 · see citing works
Best care at lower cost: the path to continuously learning health care in America
Smith, M. A., Saunders, R., Stuckhardt, L., & McGinnis, J. M. (2014) · Choice Reviews Online · doi:10.5860/choice.51-3277 · Cited by 1,356 · see citing works
Jumpertz, R., Le, D. S., Turnbaugh, P. J., Trinidad, C., Bogardus, C., Gordon, J. I., & Krakoff, J. (2011) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.110.010132 · Cited by 1,232 · see citing works
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136 · see citing works
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions
Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., Merchant, G. C., Naughton, F., & Blandford, A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.015 · Cited by 1,118 · see citing works
Self-citationMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · see citing works
Self-citationBayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · see citing works
Mobile Health Technology Evaluation
Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., et al. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.017 · Cited by 900 · see citing works
Self-citationEvaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847 · see citing works
Self-citationMicrorandomized trials: An experimental design for developing just-in-time adaptive interventions.
Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015) · Health Psychology · doi:10.1037/hea0000305 · Cited by 671 · see citing works
Self-citationThe Learning Healthcare System: Workshop Summary
Olsen, L., Aisner, D. L., & McGinnis, J. M. (2013) · Cited by 588 · see citing works
Social physics: how good ideas spread--the lessons from a new science
Pentland, A. (2014) · Choice Reviews Online · doi:10.5860/choice.52-0805 · Cited by 573 · see citing works
Nahum-Shani, I., Hekler, E. B., & Spruijt-Metz, D. (2015) · Health Psychology · doi:10.1037/hea0000306 · Cited by 568 · see citing works
Self-citationThe Chronic Care Model and Diabetes Management in US Primary Care Settings: A Systematic Review
Stellefson, M., Dipnarine, K., & Stopka, C. B. (2013) · Preventing Chronic Disease · doi:10.5888/pcd10.120180 · Cited by 541 · see citing works
Payne, H., Lister, C., West, J. H., & Bernhardt, J. M. (2015) · JMIR mhealth and uhealth · doi:10.2196/mhealth.3335 · Cited by 525 · see citing works
Celis-Morales, C. A., Perez-Bravo, F., Ibañez, L., Salas, C., Bailey, M. E. S., & Gill, J. M. R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036345 · Cited by 486 · see citing works
Tate, D. F., Jackvony, E. H., & Wing, R. R. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.15.1620 · Cited by 427 · see citing works
A systems-oriented multilevel framework for addressing obesity in the 21st century.
Huang, T. T. ‐., Drewnosksi, A., Kumanyika, S., & Glass, T. A. (2009) · PubMed Central · Cited by 385 · see citing works
Recent trends in chronic disease, impairment and disability among older adults in the United States
Hung, W. W., Ross, J. S., Boockvar, K. S., & Siu, A. L. (2011) · BMC Geriatrics · doi:10.1186/1471-2318-11-47 · Cited by 373 · see citing works
Leal, C., & Chaix, B. (2010) · Obesity Reviews · doi:10.1111/j.1467-789x.2010.00726.x · Cited by 363 · see citing works
Wicks, P., Vaughan, T. E., Massagli, M. P., & Heywood, J. (2011) · Nature Biotechnology · doi:10.1038/nbt.1837 · Cited by 362 · see citing works
Laing, B. Y., Mangione, C. M., Tseng, C.-H., Leng, M., Vaisberg, E., Mahida, M., Bholat, M., Glazier, E., Morisky, D. E., & Bell, D. S. (2014) · Annals of Internal Medicine · doi:10.7326/m13-3005 · Cited by 353 · see citing works
Open mHealth Architecture: An Engine for Health Care Innovation
Estrin, D. L., & Sim, I. (2010) · Science · doi:10.1126/science.1196187 · Cited by 347 · see citing works
Self-citationEllis, K., Kerr, J., Godbole, S. V., Lanckriet, G. R. G., Wing, D., & Marshall, S. J. (2014) · Physiological Measurement · doi:10.1088/0967-3334/35/11/2191 · Cited by 344 · see citing works
A Content Analysis of Popular Smartphone Apps for Smoking Cessation
Abroms, L. C., Westmaas, J. L., Bontemps-Jones, J. E., Ramani, R., & Mellerson, J. L. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.07.008 · Cited by 325 · see citing works
A randomized controlled trial of an automated exercise coach for older adults
Bickmore, T. W., Silliman, R. A., Nelson, K., Cheng, D. M., Winter, M., Henault, L., & Paasche‐Orlow, M. K. (2013) · Journal of the American Geriatrics Society · doi:10.1111/jgs.12449 · Cited by 305 · cited 1× · see citing works
“However, it is possible that the functions of human coaching, such as providing accountability and support,(72) may be automated through virtual conversational agents.(74,75) Although digital interventions have shown promise in pilot studies, current evaluation methods are undeveloped or perform poorly.”
BackgroundWeight loss—there is an app for that! But does it adhere to evidence-informed practices?
Breton, E. R., Fuemmeler, B. F., & Abroms, L. C. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0076-5 · Cited by 276 · see citing works
Furukawa, M. F., King, J., Patel, V., Hsiao, C., Adler‐Milstein, J., & Jha, A. K. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0445 · Cited by 273 · see citing works
Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise
Riley, W. T., Glasgow, R. E., Etheredge, L., & Abernethy, A. P. (2013) · Clinical and Translational Medicine · doi:10.1186/2001-1326-2-10 · Cited by 267 · see citing works
Self-citationCowan, L. T., Van Wagenen, S. A., Brown, B. A., Hedin, R. J., Seino-Stephan, Y., Hall, P. C., & West, J. H. (2013) · Health Education & Behavior · doi:10.1177/1090198112452126 · Cited by 264 · see citing works
Rabbi, M., Aung, M. H., Zhang, M., & Choudhury, T. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2805840 · Cited by 262 · see citing works
Mobile Health: Revolutionizing Healthcare Through Transdisciplinary Research
Kumar, S., Nilsen, W., Misha Pavel, C., & Srivastava, M. B. (2013) · Computer · doi:10.1109/mc.2012.392 · Cited by 243 · see citing works
Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · see citing works
Self-citationAgile science: creating useful products for behavior change in the real world
Hekler, E. B., Klasnja, P., Riley, W. T., Buman, M. P., Huberty, J., Rivera, D. E., & Martin, C. A. (2016) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0395-7 · Cited by 208 · see citing works
Self-citationAutomated interventions for multiple health behaviors using conversational agents
Bickmore, T. W., Schulman, D., & Sidner, C. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.05.011 · Cited by 206 · see citing works
Advancing Models and Theories for Digital Behavior Change Interventions
Hekler, E. B., Michie, S., Pavel, M., Rivera, D. E., Collins, L. M., Jimison, H. B., Garnett, C., Parral, S., & Spruijt-Metz, D. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.013 · Cited by 174 · see citing works
Self-citationContinuous Evaluation of Evolving Behavioral Intervention Technologies
Mohr, D. C., Cheung, K., Schueller, S. M., Brown, C. H., & Duan, N. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.06.006 · Cited by 169 · see citing works
Self-citationHypoglycemia Prediction Using Machine Learning Models for Patients With Type 2 Diabetes
Sudharsan, B., Peeples, M., & Shomali, M. E. (2014) · Journal of Diabetes Science and Technology · doi:10.1177/1932296814554260 · Cited by 162 · see citing works
Rivera, D. E., Pew, M. D., & Collins, L. M. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.10.020 · Cited by 151 · see citing works
Helander, E., Kaipainen, K., Korhonen, I., & Wansink, B. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3084 · Cited by 143 · see citing works
Digital Medical Tools and Sensors
Topol, E. J., Steinhubl, S. R., & Torkamani, A. (2015) · JAMA · doi:10.1001/jama.2014.17125 · Cited by 142 · see citing works
Ellis, K., Godbole, S., Marshall, S., Lanckriet, G., Staudenmayer, J., & Kerr, J. (2014) · Frontiers in Public Health · doi:10.3389/fpubh.2014.00036 · Cited by 123 · see citing works
Kleiboer, A. M., Donker, T., Seekles, W., van Straten, A., Riper, H. M., & Cuijpers, P. J. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.013 · Cited by 109 · see citing works
Self-citationEstrin, D. L. (2014) · Communications of the ACM · doi:10.1145/2580944 · Cited by 109 · see citing works
Self-citationAn Ecological Framework for Cancer Communication: Implications for Research
Patrick, K., Intille, S. S., & Zabinski, M. F. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.3.e23 · Cited by 98 · see citing works
Self-citationDiagnostic Verification of Temperature Forecasts
Murphy, A. H., Brown, B. G., & Chen, Y. (1989) · Weather and Forecasting · doi:10.1175/1520-0434(1989)004 · Cited by 97 · see citing works
MARKOV MODELING AND DISCRETE EVENT SIMULATION IN HEALTH CARE: A SYSTEMATIC COMPARISON
Standfield, L., Comans, T. A., & Scuffham, P. (2014) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462314000117 · Cited by 90 · see citing works
Project onward: an innovative e-health intervention for cancer survivors
Duffecy, J., Sanford, S., Wagner, L., Begale, M., Nawacki, E., & Mohr, D. C. (2013) · Psycho-Oncology · doi:10.1002/pon.3075 · Cited by 89 · see citing works
Self-citationDesigning and undertaking a health economics study of digital health interventions
McNamee, P., Murray, E., Kelly, M. P., Bojke, L., Chilcott, J., Fischer, A., West, R., & Yardley, L. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.05.007 · Cited by 82 · see citing works
Development of a dynamic computational model of social cognitive theory
Riley, W. T., Martín, C. A., Rivera, D. E., Hekler, E. B., Adams, M. A., Buman, M. P., Misha Pavel, C., & King, A. C. (2015) · Translational Behavioral Medicine · doi:10.1007/s13142-015-0356-6 · Cited by 78 · see citing works
Self-citationRiley, W. T., Serrano, K. J., Nilsen, W., & Atienza, A. A. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.024 · Cited by 76 · see citing works
Self-citationSisko, A. M., Keehan, S. P., Cuckler, G. A., Madison, A. J., Smith, S. D., Wolfe, C. J., Stone, D. A., Lizonitz, J. M., & Poisal, J. A. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0560 · Cited by 74 · see citing works
Just-in-time technology to encourage incremental, dietary behavior change.
Intille, S. S., Kukla, C., Farzanfar, R., & Bakr, W. (2003) · Europe PMC (PubMed Central) · Cited by 72 · see citing works
Innovative ICT Solutions to Improve Treatment Outcomes for Depression: The ICT4Depression Project
Warmerdam, L., Riper, H. M., Michel, K., Pepijn, V. D. V., Rocha, A., Mario, R. H., Tousset, E., Hugo, S., Andersson, G., & Cuijpers, P. J. (2012) · Studies in health technology and informatics · doi:10.3233/978-1-61499-121-2-339 · Cited by 54 · see citing works
Self-citationA smart mirror to promote a healthy lifestyle
Colantonio, S., Coppini, G., Germanese, D., Giorgi, D., Magrini, M., Marraccini, P., Martinelli, M., Morales, M. A., Pascali, M. A., Raccichini, G., et al. (2015) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2015.06.008 · Cited by 47 · see citing works
Broekhuizen, K., van Poppel, M. N. M., Koppes, L. L. J., Kindt, I., Brug, J., & van Mechelen, W. (2012) · PLoS ONE · doi:10.1371/journal.pone.0050032 · Cited by 46 · see citing works
Realizing Effective Behavioral Management of Health: The Metamorphosis of Behavioral Science Methods
Hekler, E. B., Klasnja, P., Traver, V., & Hendriks, M. (2013) · IEEE Pulse · doi:10.1109/mpul.2013.2271681 · Cited by 44 · see citing works
Self-citationMethodologies for optimizing behavioral interventions: introduction to special section
Riley, W. T., & Rivera, D. E. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0281-0 · Cited by 28 · see citing works
Self-citation
Flagged first, then by citations
3 of the 72 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions
Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., Merchant, G. C., Naughton, F., & Blandford, A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.015 · Cited by 1,118 · see citing works
Self-citationMichie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865 · see citing works
Self-citationLewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588 · see citing works
Mohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432 · see citing works
Self-citationAttitudes towards digital treatment for depression: A European stakeholder survey
Topooco, N., Riper, H., Araya, R., Berking, M., Brunn, M., Chevreul, K., Cieslak, R., Ebert, D. D., Etchmendy, E., Herrero, R., et al. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.01.001 · Cited by 262 · see citing works
Self-citationBaumel, A., Faber, K., Mathur, N., Kane, J. M., & Muench, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7270 · Cited by 206 · see citing works
“Agile science tries to answer some of these challenges by focusing on an adaptable and nimble scientific process to support the collective development and evaluation of interventions [5], while taking into account the rapid pace of change in the technologies that support digital interventions [6].”
Digital Interventions on Healthy Lifestyle Management: Systematic Review
Chatterjee, A., Prinz, A., Gerdes, M. W., & Martínez, S. (2021) · Journal of Medical Internet Research · doi:10.2196/26931 · Cited by 214 · see citing works
Advancing Models and Theories for Digital Behavior Change Interventions
Hekler, E. B., Michie, S., Pavel, M., Rivera, D. E., Collins, L. M., Jimison, H. B., Garnett, C., Parral, S., & Spruijt-Metz, D. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.013 · Cited by 174 · see citing works
Self-citationBagot, K. S., Matthews, S. A., Mason, M., Squeglia, L. M., Fowler, J., Gray, K., Herting, M., May, A., Colrain, I., Godino, J., et al. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2018.03.008 · Cited by 145 · see citing works
Self-citationPrecision Health: The Role of the Social and Behavioral Sciences in Advancing the Vision
Hekler, E. B., Tiro, J. A., Hunter, C. M., & Nebeker, C. (2020) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaa018 · Cited by 147 · see citing works
Self-citationTutorial for Using Control Systems Engineering to Optimize Adaptive Mobile Health Interventions
Hekler, E. B., Rivera, D. E., Martín, C. A., Phatak, S. S., Freigoun, M. T., Korinek, E. V., Klasnja, P., Adams, M. A., & Buman, M. P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8622 · Cited by 137 · see citing works
Self-citationCrane, D., Garnett, C., Michie, S., West, R., & Brown, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-22420-8 · Cited by 118 · see citing works
Self-citationPham, Q., Graham, G., Carrion, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2019) · JMIR mHealth and uHealth · doi:10.2196/11941 · Cited by 131 · see citing works
Duffy, A., Christie, G. J., & Moreno, S. (2022) · JMIR Human Factors · doi:10.2196/35693 · Cited by 114 · see citing works
Digital Health Apps in the Clinical Care of Inflammatory Bowel Disease: Scoping Review
Yin, A. L., Hachuel, D., Pollak, J. P., Scherl, E., & Estrin, D. L. (2019) · Journal of Medical Internet Research · doi:10.2196/14630 · Cited by 103 · see citing works
Self-citationKorinek, E. V., Phatak, S. S., Martín, C. A., Freigoun, M. T., Rivera, D. E., Adams, M. A., Klasnja, P., Buman, M. P., & Hekler, E. B. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-017-9878-3 · Cited by 97 · see citing works
Self-citationInnovative methods for observing and changing complex health behaviors: four propositions
Chevance, G., Perski, O., & Hekler, E. B. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibaa026 · Cited by 95 · see citing works
Self-citationNaslund, J. A., Aschbrenner, K. A., Kim, S. J., McHugo, G. J., Unützer, J., Bartels, S. J., & Marsch, L. A. (2017) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000246 · Cited by 94 · see citing works
eHealth in Treatment of Offenders in Forensic Mental Health: A Review of the Current State
Kip, H., Bouman, Y. H. A., Kelders, S. M., & van Gemert-Pijnen, L. J. E. W. C. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00042 · Cited by 89 · see citing works
Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data
Sieverink, F., Kelders, S., Poel, M., & van Gemert-Pijnen, L. (2017) · JMIR Research Protocols · doi:10.2196/resprot.6452 · Cited by 88 · see citing works
mHealth Interventions for Disadvantaged and Vulnerable People with Type 2 Diabetes
Mayberry, L. S., Lyles, C. R., Oldenburg, B. F., Osborn, C. Y., Parks, M., & Peek, M. E. (2019) · Current Diabetes Reports · doi:10.1007/s11892-019-1280-9 · Cited by 84 · see citing works
Five Crucial Challenges in Digital Health
Cummins, N., & Schuller, B. W. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.536203 · Cited by 84 · see citing works
The Potential of Game-Based Digital Biomarkers for Modeling Mental Health
Mandryk, R. L., & Birk, M. V. (2019) · JMIR Mental Health · doi:10.2196/13485 · Cited by 84 · see citing works
Methods for Human-Centered eHealth Development: Narrative Scoping Review
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2022) · Journal of Medical Internet Research · doi:10.2196/31858 · Cited by 67 · see citing works
Airola, E. (2021) · Journal of Medical Internet Research · doi:10.2196/23804 · Cited by 63 · see citing works
Edney, S. M., Bogomolova, S., Ryan, J. C., Olds, T., Sanders, I., & Maher, C. A. (2018) · Journal of Medical Internet Research · doi:10.2196/10911 · Cited by 69 · see citing works
Methods for Human-Centered eHealth Development: Narrative Scoping Review (Preprint)
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2021) · doi:10.2196/preprints.31858 · Cited by 67 · see citing works
mHealth Assessment and Intervention of Depression and Anxiety in Older Adults
Grossman, J. T., Frumkin, M., Rodebaugh, T., & Lenze, E. (2020) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000255 · Cited by 45 · see citing works
Dyadic Psychosocial eHealth Interventions: Systematic Scoping Review
Shaffer, K. M., Tigershtrom, A., Badr, H., Benvengo, S., Hernandez, M., & Ritterband, L. M. (2019) · Journal of Medical Internet Research · doi:10.2196/15509 · Cited by 63 · see citing works
Brigden, A., Anderson, E., Linney, C., Morris, R., Parslow, R., Serafimova, T., Smith, L., Briggs, E., Loades, M., & Crawley, E. (2020) · Journal of Medical Internet Research · doi:10.2196/16924 · Cited by 63 · see citing works
Damen, I., Brombacher, H., Lallemand, C., Brankaert, R., Brombacher, A., van Wesemael, P. J. V., & Vos, S. B. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17020499 · Cited by 59 · see citing works
Müller, A. M., BLANDFORD, A. E., & Yardley, L. (2017) · mHealth · doi:10.21037/mhealth.2017.08.05 · Cited by 58 · see citing works
Martínez, W., Threatt, A. L., Rosenbloom, S. T., Wallston, K. A., Hickson, G. B., & Elasy, T. A. (2018) · JMIR Human Factors · doi:10.2196/humanfactors.9569 · Cited by 56 · see citing works
Ruzek, J. I., & Yeager, C. M. (2017) · Global Mental Health · doi:10.1017/gmh.2017.11 · Cited by 54 · see citing works
Phatak, S. S., Freigoun, M. T., Martín, C. A., Rivera, D. E., Korinek, E. V., Adams, M. A., Buman, M. P., Klasnja, P., & Hekler, E. B. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.01.010 · Cited by 53 · see citing works
Self-citationKip, H., Kelders, S. M., Bouman, Y. H. A., & van Gemert-Pijnen, L. J. E. W. C. (2019) · Journal of Medical Internet Research · doi:10.2196/12972 · Cited by 40 · see citing works
Kruzan, K. P., Fitzsimmons‐Craft, E. E., Dobias, M., Schleider, J. L., & Pratap, A. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.09.081 · Cited by 42 · see citing works
Maxwell, H., O’Shea, M., Stronach, M. M., & Pearce, S. (2019) · Annals of Leisure Research · doi:10.1080/11745398.2019.1674677 · Cited by 43 · see citing works
Supporting medication adherence for adults with cystic fibrosis: a randomised feasibility study
Hind, D., Drabble, S. J., Arden, M. A., Mandefield, L., Waterhouse, S., Maguire, C., Cantrill, H., Robinson, L., Beever, D., Scott, A. J., et al. (2019) · BMC Pulmonary Medicine · doi:10.1186/s12890-019-0834-6 · Cited by 39 · see citing works
Kim, M., Patrick, K., Nebeker, C., Godino, J., Stein, S., Klasnja, P., Perski, O., Viglione, C., Coleman, A., & Hekler, E. B. (2024) · Journal of Medical Internet Research · doi:10.2196/49208 · Cited by 40 · see citing works
Self-citationKip, H., Jong, N. B.-D., van Gemert‐Pijnen, J. E. W. C., & Kelders, S. M. (2024) · Journal of Medical Internet Research · doi:10.2196/59601 · Cited by 41 · see citing works
Cliffe, B., Croker, A., Denne, M., & Stallard, P. N. (2019) · Child and Adolescent Mental Health · doi:10.1111/camh.12362 · Cited by 39 · see citing works
Smith, C. E., Lane, W., Miller, H. J., Kluver, D., Terveen, L., & Yarosh, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479561 · Cited by 37 · see citing works
Asif, M., & Gouqing, Z. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00134-3 · Cited by 37 · see citing works
RetractedHolistic development of eHealth technology
Kip, H., & van Gemert-Pijnen, L. (. E. W. C. ). (2018) · Routledge eBooks · doi:10.4324/9781315385907-7 · Cited by 36 · see citing works
Vilardaga, R., Rizo, J. L., Ries, R. K., Kientz, J. A., Ziedonis, D. M., Hernandez, K., & McClernon, F. J. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/iby097 · Cited by 35 · see citing works
Kip, H., Oberschmidt, K., & Bierbooms, J. J. P. A. (2020) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2020.1808914 · Cited by 32 · see citing works
Murray, E., Ross, J., Pal, K., Li, J., Dack, C., Stevenson, F. A., Sweeting, M. J., Parrott, S., Barnard, M., Yardley, L., et al. (2018) · Programme Grants for Applied Research · doi:10.3310/pgfar06050 · Cited by 30 · see citing works
Day, J. J., Freiberg, K., Hayes, A. J., & Homel, R. J. (2019) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-019-00282-4 · Cited by 31 · see citing works
Asbjørnsen, R. A., Hjelmesaeth, J. S., Smedsrød, M. L., Wentzel, J., Ollivier, M., Clark, M. M., van Gemert‐Pijnen, J. E. W. C., & Nes, L. S. (2022) · JMIR Human Factors · doi:10.2196/37372 · Cited by 30 · see citing works
Pham, Q., Graham, G., Lalloo, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2018) · JMIR mHealth and uHealth · doi:10.2196/11447 · Cited by 30 · see citing works
Schwartz, L. A., Psihogios, A. M., Henry‐Moss, D., Daniel, L. C., Hoeve, E. S. V., Velázquez‐Martin, B., Butler, E. S., Hobbie, W. L., Lunsford, N. B., Sabatino, S. A., et al. (2019) · Clinical Practice in Pediatric Psychology · doi:10.1037/cpp0000272 · Cited by 29 · see citing works
Changing Hearts and Minds in the Equestrian World One Behaviour at a Time
Wolframm, I., Douglas, J. E., & Pearson, G. (2023) · Animals · doi:10.3390/ani13040748 · Cited by 29 · see citing works
Conversational Agents Supporting Self-Management in People With a Chronic Disease: Systematic Review
Peerbolte, T. F., van Diggelen, R. J. A., van den Haak, P., Geurts, K., Evers, L. J. W., Bloem, B. R., de Vries, N. M., & van den Berg, S. W. (2025) · Journal of Medical Internet Research · doi:10.2196/72309 · Cited by 28 · see citing works
Jenkins, C. L., Imran, S., Mahmood, A., Bradbury, K. J., Murray, E., Stevenson, F. A., & Hamilton, F. L. (2021) · JMIR Research Protocols · doi:10.2196/32538 · Cited by 27 · see citing works
Goldstein, S. P., Zhang, F., Klasnja, P., Hoover, A., Wing, R. R., & Thomas, J. G. (2021) · JMIR Research Protocols · doi:10.2196/33568 · Cited by 26 · see citing works
Mann, D. M., Chokshi, S. K., & Kushniruk, A. W. (2018) · JMIR Human Factors · doi:10.2196/10721 · Cited by 26 · see citing works
Patient and Provider Cocreation of Mobile Texting Apps to Support Behavioral Health: Usability Study
Arevian, A. C., O’Hora, J., Rosser, J., Mango, J. D., Miklowitz, D. J., & Wells, K. B. (2020) · JMIR mhealth and uhealth · doi:10.2196/12655 · Cited by 26 · see citing works
Digital interventions for adolescent and young adult cancer survivors
Viola, A., Panigrahi, G., & Devine, K. A. (2019) · Current Opinion in Supportive and Palliative Care · doi:10.1097/spc.0000000000000480 · Cited by 25 · see citing works
Sobolev, M., Vitale, R., Wen, H., Kizer, J., Leeman, R. F., Pollak, J., Baumel, A., Vadhan, N. P., Estrin, D. L., & Muench, F. J. (2020) · JMIR mhealth and uhealth · doi:10.2196/25018 · Cited by 22 · see citing works
Self-citationPham, Q., Cafazzo, J. A., & Feifer, A. (2017) · JMIR Research Protocols · doi:10.2196/resprot.8051 · Cited by 22 · see citing works
Morton, E. E., Nicholas, J., Lapadat, L., O'Brien, H. L., Barnes, S. J., Poh, C., & Michalak, E. E. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.08.132 · Cited by 21 · see citing works
Benden, M. E., Mehta, R. K., Pickens, A., Harp, B., Smith, M. L., Towne, S. D., & Peres, S. C. (2021) · BMC Public Health · doi:10.1186/s12889-021-11975-3 · Cited by 21 · see citing works
Fallon, E. A., Driscoll, D., Smith, T. G., Richardson, K., & Portier, K. M. (2018) · Journal of Cancer Survivorship · doi:10.1007/s11764-018-0706-8 · Cited by 20 · see citing works
Toward a Common Agenda for the Public and Private Sectors to Advance Digital Health Communication
Abroms, L. C., Allegrante, J. P., Auld, M. E., Gold, R. S., Riley, W. T., & Smyser, J. (2019) · American Journal of Public Health · doi:10.2105/ajph.2018.304806 · Cited by 21 · see citing works
Self-citationSingh, A., Newhouse, N., Gibbs, J., BLANDFORD, A. E., Chen, Y., Briggs, P., Mentis, H. M., Sellen, K., & Bardram, J. E. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3049287 · Cited by 18 · see citing works
Leveraging Implementation Science in Human-Centred Design for Digital Health
Waddell, A., Seguin, J. P., Wu, L., Stragalinos, P., Wherton, J. P., Watterson, J., Prawira, C. O., Olivier, P. L., Manning, V., Lubman, D. I., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642161 · Cited by 22 · see citing works
Baretta, D., Sartori, F., Greco, A., D’Αddario, M., Melen, R., & Steca, P. (2019) · Advances in Human-Computer Interaction · doi:10.1155/2019/3068748 · Cited by 17 · see citing works
The Role of a Smart Health Ecosystem in Transforming the Management of Chronic Health Conditions
Nourse, R. A., Dingler, T., Kelly, J. T., Kwaśnicka, D., & Maddison, R. (2023) · Journal of Medical Internet Research · doi:10.2196/44265 · Cited by 17 · see citing works
Ford, K. L., Portz, J. D., Zhou, S., Gornail, S., Moore, S. L., Zhang, X., & Bull, S. S. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.616278 · Cited by 18 · see citing works
Willms, A., Rhodes, R. E., & Liu, S. (2023) · JMIR Formative Research · doi:10.2196/43823 · Cited by 16 · see citing works
Garnett, C. V., Michie, S., West, R., & Brown, J. (2019) · F1000Research · doi:10.12688/f1000research.17952.1 · Cited by 16 · see citing works
Shaping Blended Care: Adapting an Instrument to Support Therapists in Using eMental Health
Kip, H., Wentzel, J., & Kelders, S. M. (2020) · JMIR Mental Health · doi:10.2196/24245 · Cited by 16 · see citing works
The influence of automated praise on behavior and performance.
Motz, B. A., Canning, E. A., Green, D. J.-A., Mallon, M. G., & Quick, J. (2021) · Technology Mind and Behavior · doi:10.1037/tmb0000042 · Cited by 15 · see citing works
Schafer, M., Lachman, J. M., Gardner, F. E. M., Zinser, P., Calderón, F., Han, Q., Facciolà, C., & Clements, L. (2023) · BMC Public Health · doi:10.1186/s12889-023-16649-w · Cited by 17 · see citing works
Consolvo, S., Bentley, F. R., Hekler, E. B., & Phatak, S. S. (2017) · Synthesis lectures on mobile and pervasive computing · doi:10.2200/s00763ed1v01y201703mpc012 · Cited by 14 · see citing works
Self-citationGreene, K. L., Choi, H. J., Glenn, S. D., Ray, A. E., & Hecht, M. L. (2020) · Prevention Science · doi:10.1007/s11121-020-01181-9 · Cited by 15 · see citing works
Digital medicine: Emergence, definition, scope, and future
Zhang, S., Liao, R., Alpert, J. S., Kong, J., Spetzger, U., Milia, P., Thiriet, M., & Wortley, D. J. (2018) · Digital Medicine · doi:10.4103/digm.digm_9_18 · Cited by 14 · see citing works
Ryan, P. A., Papanek, P. E., Csuka, M. E., Brown, M. E., Hopkins, S. A., Lynch, S. B., Scheer, V. L., Schlidt, A. M., Yan, K., Simpson, P. M., et al. (2018) · Contemporary Clinical Trials · doi:10.1016/j.cct.2018.06.006 · Cited by 14 · see citing works
Taking an HPV vaccine research-tested intervention to scale in a clinical setting
Hopfer, S., Ray, A. E., Hecht, M. L., Miller-Day, M. A., BeLue, R., Zimet, G. D., Evans, W. D., & McKee, F. X. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx066 · Cited by 14 · see citing works
Kip, H., Kelders, S. M., & Van Gemert-Pijnen, L. J. E. W. C. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300365 · Cited by 11 · see citing works
Li, L., Yang, P., Duan, Y., Xie, J., Liu, M., Zhou, Y., Luo, X., Zhang, C., Li, Y., Wang, J., et al. (2024) · BMC Public Health · doi:10.1186/s12889-023-17567-7 · Cited by 13 · see citing works
Games for health research—past, present, and future
Baranowski, T. (2018) · Prävention und Gesundheitsförderung · doi:10.1007/s11553-018-0657-y · Cited by 13 · see citing works
Principles for Developing Innovative HIV Digital Health Interventions
Burrus, O., Gupta, C., Ortiz, A., Zulkiewicz, B. A., Furberg, R., Uhrig, J. D., Harshbarger, C. L., & Lewis, M. A. (2018) · Medical Care · doi:10.1097/mlr.0000000000000957 · Cited by 13 · see citing works
Ford, K. L., Moore, S. L., Zhou, S., Gore, M. O., Portz, J., Zhang, X., Zane, R. D., Wiler, J. L., & Bull, S. S. (2019) · mHealth · doi:10.21037/mhealth.2019.08.08 · Cited by 12 · see citing works
Direito, A., Rawstorn, J. C., Mair, J. L., Daryabeygi‐Khotbehsara, R., Maddison, R., & Tai, E. S. (2019) · Digital Health · doi:10.1177/2055207619890480 · Cited by 10 · see citing works
Nelson, L. A., Williamson, S., Nigg, A., & Martínez, W. (2020) · Current Diabetes Reports · doi:10.1007/s11892-020-01356-2 · Cited by 13 · see citing works
Advancing Digital Behavior Change Interventions by Exploring a Calendar-Based Suggestion System
Damen, I., van den Heuvel, R., Brankaert, R., & Vos, S. B. (2021) · European Conference on Cognitive Ergonomics · doi:10.1145/3452853.3452866 · Cited by 11 · see citing works
Digital technology for preventative health care in Myanmar
Hlaing, P. M., Nopparatjamjomras, T. R., & Nopparatjamjomras, S. (2018) · Digital Medicine · doi:10.4103/digm.digm_25_18 · Cited by 10 · see citing works
James, D. C. S., Harville, C., McQueen, D. S., & Facey, J. A. (2021) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2021.06.310 · Cited by 10 · see citing works
Phạm, Q., Shaw, J., Morita, P. P., Seto, E., Stinson, J., & Cafazzo, J. A. (2019) · Journal of Medical Internet Research · doi:10.2196/14849 · Cited by 10 · see citing works
Getting Research on Games for Health Funded
Baranowski, T., Blumberg, F. C., Gao, Z., Kato, P. M., Kok, G., Lu, A. S., Lyons, E., Morrill, B. A., Peng, W., Prins, P. J. M., et al. (2016) · Games for Health Journal · doi:10.1089/g4h.2016.0106 · Cited by 10 · see citing works
Digital Tools to Promote Healthy Eating for Working-Age Individuals: A Scoping Review
Pan, S., Ren, X., Vos, S. B., & Brombacher, A. (2021) · International Symposium of Chinese CHI (Chinese CHI) · doi:10.1145/3490355.3490356 · Cited by 11 · see citing works
ter Harmsel, J. F., Smulders, L. M., Noordzij, M. L., Swinkels, L. T. A., Goudriaan, A. E., Popma, A., & van der Pol, T. M. (2022) · JMIR Formative Research · doi:10.2196/40237 · Cited by 9 · see citing works
Digital Health Research Methods and Tools: Suggestions and Selected Resources for Researchers
Gray, K. M., & Gilbert, C. A. (2017) · Intelligent systems reference library · doi:10.1007/978-3-319-67513-8_2 · Cited by 9 · see citing works
Digital Innovation in Healthcare Entrepreneurship
Mohammed-Nasir, R., Oshikoya, K. A., & Oreagba, I. A. (2023) · Springer eBooks · doi:10.1007/978-981-19-6696-5_22 · Cited by 10 · see citing works
Klurfeld, D. M., Hekler, E. B., Nebeker, C., Patrick, K., & Khoo, C. S. (2018) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2018.06.013 · Cited by 7 · see citing works
Self-citationRetractedCommentary: Pediatric Digital Health Supported by the National Institutes of Health
Riley, W. T., Oh, A., Aklin, W. M., Sherrill, J. T., Wolff‐Hughes, D. L., Diana, A., Griffin, J. A., & Campo, R. (2018) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsy108 · Cited by 9 · see citing works
Self-citation“It depends:” a qualitative study on digital health academic-industry collaboration
Ford, K. L., Leiferman, J. A., Sobral, B., Bennett, J. K., Moore, S. L., & Bull, S. S. (2021) · mHealth · doi:10.21037/mhealth-20-140 · Cited by 8 · see citing works
- Authors
- Patrick, K., Hekler, E. B., Estrin, D., Mohr, D. C., Riper, H., Crane, D., Godino, J., & Riley, W. T.
- Affiliations
- University of California San Diego (US)
- Arizona State University (US)
- Cornell University (US)
- Northwestern University (US)
- Vrije Universiteit Amsterdam (NL)
- University College London (GB)
- National Institutes of Health (US)
- Corresponding author
- Patrick, Kevin
- ORCID
- Venue
- American Journal of Preventive Medicine
- Publisher
- Elsevier BV
- Published
- 2016-11
- Volume
- 51
- Issue
- 5
- Pages
- 816-824
- ISSN
- 0749-3797 (ISSN-L)
- OpenAlex
- W2530300749
- Cited by
- 169
- References
- 72
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 14 · 2023: 13 · 2022: 17 · 2021: 22 · 2020: 19 · 2019: 23 · 2018: 27 · 2017: 19 · 2016: 3
- FWCI
- 44.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Preventive Medicine (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- digital health interventions, health behavior change, ubiquitous computing, chronic disease, comorbidities, evidence-based assessment, human-centered design, translational research, economic evaluation, health behavior theory
- MeSH terms
- Health Promotion, Humans, Research Design, Telecommunications, Health Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review