Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise
Open accessAbstract
Our current health research enterprise is painstakingly slow and cumbersome, and its results seldom translate into practice. The slow pace of health research contributes to findings that are less relevant and potentially even obsolete. To produce more rapid, responsive, and relevant research, we propose approaches that increase relevance via greater stakeholder involvement, speed research via innovative designs, streamline review processes, and create and/or better leverage research infrastructure. Broad stakeholder input integrated throughout the research process can both increase relevance and facilitate study procedures. More flexible and rapid research designs should be considered before defaulting to the traditional two‐arm randomized controlled trial (RCT), but even traditional RCTs can be designed for more rapid findings. Review processes for grant applications, IRB protocols, and manuscript submissions can be better streamlined to minimize delays. Research infrastructures such as rapid learning systems and other health information technologies can be leveraged to rapidly evaluate new and existing treatments, and alleviate the extensive recruitment delays common in traditional research. These and other approaches are feasible but require a culture shift among the research community to value not only methodological rigor, but also the pace and relevance of research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When developing interventions delivered via digital technologies, the technologies often mutate at a faster rate than the scientific process.
“Further, when developing interventions delivered via digital technologies, there is an added issue that the technologies are often mutating at a faster rate than the scientific process [3].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A new dawn for citizen science
SILVERTOWN, J. W. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2009.03.017 · Cited by 2,199 · see citing works
The role and interpretation of pilot studies in clinical research
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2010.10.008 · Cited by 1,917 · cited 1× · see citing works
“This belabored research process typically begins with pilot trials that we posit have limited benefit, are used inappropriately to estimate effect size [14], and often prematurely concretize a less than optimal intervention.”
BackgroundStenting versus Aggressive Medical Therapy for Intracranial Arterial Stenosis
Chimowitz, M. I., Lynn, M., Derdeyn, C. P., Turan, T. N., Fiorella, D., Lane, B. F., Janis, L. S., Lutsep, H. L., Barnwell, S. L., Waters, M. F., et al. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1105335 · Cited by 1,901 · see citing works
Managing Clinical Knowledge for Health Care Improvement
Balas, E. A., & Boren, S. A. (2000) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1637943 · Cited by 1,479 · see citing works
Collins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · see citing works
The stepped wedge trial design: a systematic review
Brown, C. A., & Lilford, R. J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-54 · Cited by 907 · see citing works
The PhenX Toolkit: get the most from your measures
Hamilton, C. M., Strader, L. C., Pratt, J. G., Maiese, D., Hendershot, T., Kwok, R. K., Hammond, J. A., Huggins, W., Jackman, D., Pan, H., et al. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr193 · Cited by 889 · see citing works
The n-of-1 clinical trial: the ultimate strategy for individualizing medicine?
Lillie, E. O., Patay, B., Diamant, J., Issell, B., Topol, E. J., & Schork, N. J. (2011) · Personalized Medicine · doi:10.2217/pme.11.7 · Cited by 706 · see citing works
Making research relevant: if it is an evidence-based practice, where's the practice-based evidence?
Green, L. W. (2008) · Family Practice · doi:10.1093/fampra/cmn055 · Cited by 633 · see citing works
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.4.281 · Cited by 632 · see citing works
Hippisley‐Cox, J., & Coupland, C. A. C. (2010) · BMJ · doi:10.1136/bmj.c2197 · Cited by 511 · see citing works
A Rapid-Learning Health System
Etheredge, L. M. (2007) · Health Affairs · doi:10.1377/hlthaff.26.2.w107 · Cited by 345 · see citing works
Self-citationGlasgow, R. E., Magid, D. J., Beck, A., Ritzwoller, D., & Estabrooks, P. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000163645.41407.09 · Cited by 308 · see citing works
Self-citationAdvancing the science of mHealth
Nilsen, W., Kumar, S., Shar, A., Varoquiers, C., Wiley, T., Riley, W. T., Pavel, M., & Atienza, A. A. (2012) · Journal of Health Communication · doi:10.1080/10810730.2012.677394 · Cited by 226 · see citing works
Self-citationLinden, A., & Adams, J. L. (2011) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01504.x · Cited by 219 · see citing works
Zucker, D. R., Ruthazer, R., & Schmid, C. H. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.020 · Cited by 184 · see citing works
Evaluability Assessment to Improve Public Health Policies, Programs, and Practices
Leviton, L. C., Khan, L. K., Rog, D. J., Dawkins, N., & Cotton, D. (2010) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.012809.103625 · Cited by 180 · see citing works
Rivera, D. E., Pew, M. D., & Collins, L. M. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.10.020 · Cited by 151 · see citing works
Tannen, R. L., Weiner, M. G., & Xie, D. (2009) · BMJ · doi:10.1136/bmj.b81 · Cited by 145 · see citing works
Fiore, L. D., Brophy, M. T., Ferguson, R. E., DʼAvolio, L., Hermos, J. A., Lew, R., Doros, G. D., Conrad, C. H., O’Neil, J. A., Sabin, T. P., et al. (2011) · Clinical Trials · doi:10.1177/1740774511398368 · Cited by 100 · see citing works
A randomized experiment comparing random and cutoff-based assignment.
Shadish, W. R., Galindo, R. J., Wong, V. C., Steiner, P. M., & Cook, T. D. (2011) · Psychological Methods · doi:10.1037/a0023345 · Cited by 86 · see citing works
Solberg, L. I., Glasgow, R. E., Unützer, J., Jaeckels, N., Oftedahl, G., Beck, A. L., Maciosek, M. V., & Crain, A. L. (2010) · Medical Care · doi:10.1097/mlr.0b013e3181dbea62 · Cited by 58 · see citing works
Self-citationAgile methods for open source safety‐critical software
Gary, K. A., Enquobahrie, D. A., Ibáñez, L., et al. (2011) · Software Practice and Experience · doi:10.1002/spe.1075 · Cited by 58 · see citing works
Noonan, V. K., Cook, K. F., Bamer, A. M., Choi, S. W., Kim, J., & Amtmann, D. (2012) · Quality of Life Research · doi:10.1007/s11136-011-0040-3 · Cited by 54 · see citing works
A systematic review of validated methods for identifying pancreatitis using administrative data
Moores, K. G., Gilchrist, B., Carnahan, R. M., & Abrams, T. E. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2334 · Cited by 39 · see citing works
Finding the Minimal Intervention Needed for Sustained Mammography Adherence
Gierisch, J. M., DeFrank, J. T., Bowling, J. M., Rimer, B. K., Matuszewski, J., Farrell, D. W., & Skinner, C. S. (2010) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2010.05.020 · Cited by 32 · see citing works
Boudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2184791 · Cited by 17 · see citing works
Web-based dynamic Delphi: a new survey instrument
Yao, J., & Liu, W. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.666849 · Cited by 7 · see citing works
Singh, B. (2004) · Canadian Journal of Infectious Diseases and Medical Microbiology · doi:10.1155/2004/925872 · Cited by 5 · see citing works
Envisioning a Transformed Clinical Trials Enterprise in the United States
Institute of Medicine (2012) · National Academies Press eBooks · doi:10.17226/13345 · Cited by 5 · see citing works
What does the marriage of Open Access with online publication bring?
Gupta, K. C. (2004) · AIDS Research and Therapy · doi:10.1186/1742-6405-1-1 · Cited by 1 · see citing works
Finn, R. L., Donovan, A. P., Wadhwa, K., et al. (2014) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.49165 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Bender MW Mitwalli AH Van Kuiken SJ What’s holding back online medical data McKinsey Quarterly http://www.mckinseyquarterly.com
- Collins F A Vision for a National Patient-Centered Research Network http://www.pcori.org/events/national-workshop-to-advance-use-of-electronic-data/?type=past
- National Research Council Toward Precision Medicine: Building a New Knowledge Network for Biomedical Research and a New Taxonomy of Disease 2011 Washington DC: National Academies Press 22536618
- Sollano JA Delivering Value Through Personalized Medicine: An Industry Perspective http://healthforum.brandeis.edu/meetings/conference-pages/2012-24-july.html
Cited by
Sorted by citations
Gale, R. C., Wu, J., Erhardt, T., Bounthavong, M., Reardon, C. M., Damschroder, L. J., & Midboe, A. M. (2019) · Implementation Science · doi:10.1186/s13012-019-0853-y · Cited by 846 · see citing works
Nevedal, A. L., Reardon, C. M., Opra Widerquist, M. A., Jackson, G. L., Cutrona, S. L., White, B. S., & Damschroder, L. J. (2021) · Implementation Science · doi:10.1186/s13012-021-01111-5 · Cited by 708 · see citing works
Mohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432 · see citing works
Advancing the Aging and Technology Agenda in Gerontology
Schulz, R., Wahl, H.-W., Matthews, J. T., De Vito Dabbs, A., Beach, S. R., & Czaja, S. J. (2015) · The Gerontologist · doi:10.1093/geront/gnu071 · Cited by 449 · see citing works
Social media and suicide prevention: a systematic review
Robinson, J., Cox, G., Bailey, E., Hetrick, S., Rodrigues, M., Fisher, S., & Herrman, H. (2016) · Early Intervention in Psychiatry · doi:10.1111/eip.12229 · Cited by 336 · see citing works
Baker, T. B., Gustafson, D. H., & Shah, D. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2925 · Cited by 221 · see citing works
Agile science: creating useful products for behavior change in the real world
Hekler, E. B., Klasnja, P., Riley, W. T., Buman, M. P., Huberty, J., Rivera, D. E., & Martin, C. A. (2016) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0395-7 · Cited by 208 · see citing works
Self-citationMohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · see citing works
Self-citationThe Pace of Technologic Change
Patrick, K., Hekler, E. B., Estrin, D., Mohr, D. C., Riper, H., Crane, D., Godino, J., & Riley, W. T. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.05.001 · Cited by 169 · see citing works
Self-citationTreatment burden and treatment fatigue as barriers to health.
Heckman, B. W., Mathew, A. R., & Carpenter, M. J. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.004 · Cited by 179 · see citing works
Schueller, S. M., Tomasino, K. N., & Mohr, D. C. (2017) · Clinical Psychology: Science and Practice · doi:10.1037/h0101740 · Cited by 70 · see citing works
Using e-technologies in clinical trials
Rosa, C., Campbell, A. N. C., Miele, G. M., Brunner, M., & Winstanley, E. L. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.07.007 · Cited by 147 · see citing works
Commercial video games as therapy: a new research agenda to unlock the potential of a global pastime
Colder Carras, M., Van Rooij, A. J., Spruijt-Metz, D., Kvedar, J., Griffiths, M. D., Carabas, Y., & Labrique, A. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00300 · Cited by 147 · see citing works
Evaluating the Impact of Physical Activity Apps and Wearables: Interdisciplinary Review
McCallum, C., Rooksby, J., & Gray, C. M. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9054 · Cited by 126 · see citing works
The 5 R's: An Emerging Bold Standard for Conducting Relevant Research in a Changing World
Peek, C. J., Glasgow, R. E., Stange, K. C., Klesges, L. M., Purcell, E. P., & Kessler, R. S. (2014) · The Annals of Family Medicine · doi:10.1370/afm.1688 · Cited by 108 · see citing works
Self-citationKorpershoek, Y. J. G., Hermsen, S., Schoonhoven, L., Schuurmans, M. J., & Trappenburg, J. C. A. (2020) · Journal of Medical Internet Research · doi:10.2196/15449 · Cited by 78 · see citing works
Riley, W. T., Serrano, K. J., Nilsen, W., & Atienza, A. A. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.024 · Cited by 76 · see citing works
Self-citationSmartphone-Based Psychotherapeutic Micro-Interventions to Improve Mood in a Real-World Setting
Meinlschmidt, G., Lee, J.-H., Stalujanis, E., Belardi, A., Oh, M., Jung, E. K., Kim, H.-C., Alfano, J., Yoo, S.-S., & Tegethoff, M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01112 · Cited by 68 · see citing works
Distinguishing features in the assessment of mHealth apps
Tarricone, R., Petracca, F., Ciani, O., & Cucciniello, M. (2021) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2021.1891883 · Cited by 62 · see citing works
Effectiveness of a Multimodal Online Well-Being Intervention
Cobb, N. K., & Poirier, J. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.08.018 · Cited by 53 · see citing works
Baumel, A., & Schueller, S. M. (2016) · JMIR Mental Health · doi:10.2196/mental.5335 · Cited by 41 · see citing works
“This approach could also ensure that the resultant product or program can stay relevant to the current technological environment, which is a major challenge in this field [32].”
Tønning, M. L., Kessing, L. V., Bardram, J. E., & Faurholt-Jepsen, M. (2019) · Journal of Medical Internet Research · doi:10.2196/15362 · Cited by 40 · see citing works
“Other groups have previously suggested changes to the development process, to speed up the process from idea to publication [14,16,17].”
BackgroundAdapting to a Changing Tobacco Landscape
Mermelstein, R. J. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.04.015 · Cited by 29 · see citing works
Pham, Q., Graham, G., Lalloo, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2018) · JMIR mHealth and uHealth · doi:10.2196/11447 · Cited by 30 · see citing works
Pham, Q., Cafazzo, J. A., & Feifer, A. (2017) · JMIR Research Protocols · doi:10.2196/resprot.8051 · Cited by 22 · see citing works
Horwood, J., Pithara, C., Lorenc, A., Kesten, J. M., Murphy, M., Turner, A., Farr, M., Banks, J., Redwood, S., Lambert, H., et al. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.970333 · Cited by 14 · see citing works
Vilardaga, R. (2014) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2014.05.002 · Cited by 5 · see citing works
Vilardaga, R., & McClernon, F. J. (2021) · Addiction · doi:10.1111/add.15690 · Cited by 4 · see citing works
- Authors
- Riley, W. T., Glasgow, R. E., Etheredge, L., & Abernethy, A. P.
- Affiliations
- National Institutes of Health (US)
- George Washington University (US)
- Duke University (US)
- Corresponding author
- Riley, William T
- Venue
- Clinical and Translational Medicine
- Publisher
- Wiley
- Published
- 2013-05-10
- Volume
- 2
- Issue
- 1
- Pages
- 10-10
- ISSN
- 2001-1326 (ISSN-L)
- PMC
- PMC3658895
- OpenAlex
- W2154880887
- Cited by
- 267
- References
- 33
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 11 · 2023: 13 · 2022: 21 · 2021: 29 · 2020: 34 · 2019: 20 · 2018: 13 · 2017: 21 · 2016: 23 · 2015: 32 · 2014: 30 · 2013: 9
- FWCI
- 32.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Clinical and Translational Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Information Systems and Management › Interdisciplinary Research and Collaboration
- Keywords
- research relevance, stakeholder engagement, research design flexibility, randomized controlled trials, research infrastructure, health information technology, clinical translation, research culture change
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
