Can We Systematically Review Studies That Evaluate Complex Interventions?
Open accessAbstract
BACKGROUND TO THE DEBATE: The UK Medical Research Council defines complex interventions as those comprising "a number of separate elements which seem essential to the proper functioning of the interventions although the 'active ingredient' of the intervention that is effective is difficult to specify." A typical example is specialist care on a stroke unit, which involves a wide range of health professionals delivering a variety of treatments. Michelle Campbell and colleagues have argued that there are "specific difficulties in defining, developing, documenting, and reproducing complex interventions that are subject to more variation than a drug". These difficulties are one of the reasons why it is challenging for researchers to systematically review complex interventions and synthesize data from separate studies. This PLoS Medicine Debate considers the challenges facing systematic reviewers and suggests several ways of addressing them.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Interactions between intervention components and their effects on outcomes are not always linear or obvious, and they are influenced by several factors.
“Interactions between intervention components and their effects on outcomes are not always linear or obvious, and they are influenced by several factors [2].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [2]
The infrequent use of RCTs in EHR evaluations has been due to multiple concerns including ethics, difficulty to control for effect mediators, costs and time-span, impracticalities of randomizing parts of a hospital system, and difficulties in measuring effects across diverse outcomes.
“This has been due to multiple concerns including: the ethics of such approaches; difficulty to ‘control for’ potentially important effect mediators; associated costs and time-span [17,74]; impracticalities of randomizing parts of a hospital system; and difficulties in measuring the effects across a diverse array of outcomes [13,75,76].”
An additional 55 supplementary reviews provided context to the findings and aided in their interpretation.
“An additional 55 supplementary reviews provided context to the findings [68]–[122], aiding in their interpretation [123].”
From The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview (2011) · cites this paper as [123]
While the number of eHealth technologies in health care is growing, we still have insufficient understanding of how and why such interventions do or do not work.
“The paradox is that while the number of eHealth technologies in health care is growing, we still have insufficient understanding of how and why such interventions do or do not work [123].”
From The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview (2011) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Kirk, J. M. W., MacDonald, A., Lavender, P., Dean, J. M., & Rubin, G. (2017) · Biomedicine Hub · doi:10.1159/000480347 · Cited by 3,901 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · see citing works
Users' guides to the medical literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 2,970 · see citing works
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817 · see citing works
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791 · see citing works
Complex interventions: how “out of control” can a randomised controlled trial be?
Hawe, P., Shiell, A., & Riley, T. (2004) · BMJ · doi:10.1136/bmj.328.7455.1561 · Cited by 1,244 · see citing works
Interventions for providers to promote a patient-centred approach in clinical consultations
Dwamena, F., Holmes-Rovner, M., Gaulden, C. M., Jorgenson, S., Sadigh, G., Sikorskii, A., Lewin, S., Smith, R. C., Coffey, J., Olomu, A., et al. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003267.pub2 · Cited by 1,227 · see citing works
Self-citationUsers' guides to the medical literature : a manual for evidence-based clinical practice
Guyatt, G. H., Rennie, D., Meade, M. O., & Cook, D. J. (2002) · Cited by 1,196 · see citing works
Patient Adherence to Tuberculosis Treatment: A Systematic Review of Qualitative Research
Munro, S. A., Lewin, S. A., Smith, H. J., Engel, M. E., Fretheim, A., & Volmink, J. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040238 · Cited by 1,080 · see citing works
Self-citationPublication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898 · see citing works
Self-citationBeswick, A. D., Rees, K., Dieppe, P., Ayis, S., Gooberman-Hill, R., Horwood, J., & Ebrahim, S. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60342-6 · Cited by 780 · see citing works
Enhancing the quality and transparency of health research
Groves, T. (2008) · British medical journal · doi:10.1136/bmj.a718 · Cited by 727 · see citing works
Lewin, S., Glenton, C., & Oxman, A. D. (2009) · BMJ · doi:10.1136/bmj.b3496 · Cited by 678 · see citing works
Self-citationDirectly observed therapy for treating tuberculosis
Karumbi, J., & Garner, P. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003343.pub4 · Cited by 646 · see citing works
Grey literature in meta-analyses of randomized trials of health care interventions
Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000010.pub3 · Cited by 598 · cited 1× · see citing works
“Data may be unpublished and only accessed through policy documents, conference proceedings, or book chapters, but need to be obtained to minimise the effects of publication bias [30,31].”
BackgroundSelf-citationRealist synthesis - an introduction
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004) · UCL Discovery (University College London) · Cited by 570 · see citing works
Atkins, S. A., Lewin, S. A., Swart, T. M., & Volmink, J. A. (2007) · BMC Public Health · doi:10.1186/1471-2458-7-104 · Cited by 564 · see citing works
Self-citationInterventions for providers to promote a patient-centred approach in clinical consultations
Lewin, S., Skea, Z., Entwistle, V. A., Zwarenstein, M., & Dick, J. (2001) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003267 · Cited by 530 · cited 1× · see citing works
“For example, reaching a common understanding of patient-centred interventions was not easily achieved in a review of interventions intended to promote patient-centred care [32].”
BackgroundIntegrating qualitative research with trials in systematic reviews
Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., Brunton, G., & Kavanagh, J. (2004) · BMJ · doi:10.1136/bmj.328.7446.1010 · Cited by 515 · cited 2× · see citing works
“A systematic review of barriers and facilitators to healthy eating among children used qualitative evidence, unrelated to the trial data, to gain a better understanding of children’s perspectives [21].”
BackgroundAdmission avoidance hospital at home
Shepperd, S., Iliffe, S., Doll, H., Clarke, M. J., Kalra, L., & Gonçalves, D. C. (2016) · UCL Discovery (University College London) · doi:10.1002/14651858.cd007491.pub2/full · Cited by 419 · see citing works
Self-citationEvidence-based behavioral medicine: What is it and how do we achieve it?
Davidson, K. W., Goldstein, M. G., Kaplan, R. M., Kaufmann, P. G., Knatterud, G. L., Orleans, C. T., Spring, B. J., Trudeau, K. J., & Whitlock, E. P. (2003) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2603_01 · Cited by 407 · see citing works
Evidence-based policy: a realist perspective
(2005) · Routledge eBooks · doi:10.4324/9780203624289-9 · Cited by 384 · see citing works
What are the components of effective stroke unit care?
Langhorne, P. (2002) · Age and Ageing · doi:10.1093/ageing/31.5.365 · Cited by 380 · see citing works
Tobacco cessation interventions for young people
Grimshaw, G. M., & Stanton, A. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003289.pub4 · Cited by 359 · see citing works
On Theoretical Sociology: Five Essays, Old and New
Merton, R. Κ. (1967) · Cited by 348 · see citing works
Use of serial qualitative interviews to understand patients' evolving experiences and needs
Murray, S. A., Kendall, M., Carduff, E., Worth, A., Harris, F., Lloyd, A., Cavers, D., Grant, L., & Sheikh, A. (2009) · BMJ · doi:10.1136/bmj.b3702 · Cited by 309 · see citing works
Self-citationUse of research to inform public policymaking
Lavis, J. N., Posada, F. B., Haines, A., & Osei, E. (2004) · The Lancet · doi:10.1016/s0140-6736(04)17317-0 · Cited by 307 · see citing works
Clinical service organisation for heart failure
Takeda, A., Taylor, S. J., Taylor, R. S., Khan, F., Krum, H., & Underwood, M. R. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002752.pub3 · Cited by 300 · see citing works
Qualitative Research and Cochrane Reviews
Noyes, J., Popay, J., Pearson, A., Hannes, K., & Booth, A. (2008) · Wiley eBooks · doi:10.1002/9780470712184.ch20 · Cited by 291 · see citing works
Shepperd, S., Doll, H., Angus, R., Clarke, M. J., Iliffe, S., Kalra, L., Ricauda, N. A., Tibaldi, V. M., & Wilson, A. D. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081491 · Cited by 262 · see citing works
Self-citationRealist review to understand the efficacy of school feeding programmes
Greenhalgh, T., Kristjansson, E., & Robinson, V. (2007) · British medical journal · doi:10.1136/bmj.39359.525174.ad · Cited by 242 · see citing works
Interventions for improving patients' trust in doctors and groups of doctors
Rolfe, A., Cash‐Gibson, L., Car, J., Sheikh, A., & McKinstry, B. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004134.pub3 · Cited by 241 · see citing works
Self-citationSteptoe, A., Kerry, S., Rink, E., & Hilton, S. R. (2001) · American Journal of Public Health · doi:10.2105/ajph.91.2.265 · Cited by 203 · see citing works
Arai, L., Britten, N., Popay, J., Roberts, H., Petticrew, M., Rodgers, M., & Sowden, A. (2007) · Evidence & Policy: A Journal of Research, Debate and Practice · doi:10.1332/174426407781738029 · Cited by 177 · see citing works
Directly observed therapy for treating tuberculosis
Volmink, J. A., & Garner, P. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003343.pub3 · Cited by 177 · see citing works
New MRC guidance on evaluating complex interventions
Anderson, R. (2008) · BMJ · doi:10.1136/bmj.a1937 · Cited by 172 · see citing works
Introduction to the philosophy of science: cutting nature at its seams
(1998) · Choice Reviews Online · doi:10.5860/choice.35-3263 · Cited by 171 · see citing works
Occupational therapy for rheumatoid arthritis
Steultjens, E. M. J., Dekker, J., Bouter, L. M., van Schaardenburg, D., van Kuyk, M. H., & den Ende, E. C. V. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003114.pub2 · Cited by 163 · see citing works
Pinnock, H., Bawden, R., Proctor, S., Wolfe, S., Scullion, J., Price, D. B., & Sheikh, A. (2003) · BMJ · doi:10.1136/bmj.326.7387.477 · Cited by 159 · see citing works
Self-citationPeer support telephone calls for improving health
Dale, J. R., Caramlau, I. O., Lindenmeyer, A., & Williams, S. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006903.pub2 · Cited by 130 · see citing works
Directly observed therapy for treating tuberculosis
Volmink, J. A., & Garner, P. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003343.pub2 · Cited by 128 · see citing works
Developing a theory‐based taxonomy of methods for implementing change in practice
Leeman, J., Baernholdt, M., & Sandelowski, M. (2007) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2006.04207.x · Cited by 120 · see citing works
Cunliffe, A. L., Gladman, J. R. F., Husbands, S. L., Miller, P., Dewey, M. E., & Harwood, R. (2004) · Age and Ageing · doi:10.1093/ageing/afh076 · Cited by 113 · see citing works
Admission avoidance hospital at home
Shepperd, S., Doll, H., Angus, R., Clarke, M. J., Iliffe, S., Kalra, L., Ricauda, N. A., & Wilson, A. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007491 · Cited by 112 · see citing works
Self-citationInterventions for improving patients' trust in doctors and groups of doctors
McKinstry, B., Ashcroft, R. E., Car, J., Freeman, G. K., & Sheikh, A. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004134.pub2 · Cited by 84 · see citing works
Self-citationCar, J., Black, A. D., Anandan, C., Cresswell, K. M., Pagliari, C., Mckinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2008) · Cited by 66 · see citing works
Self-citationCase management: effects on professional practice and health care outcomes
Zwarenstein, M., Reeves, S., Straus, S. E., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002797.pub2 · Cited by 62 · see citing works
Self-citationPinnock, H., Adlem, L., Gaskin, S., Harris, J., Snellgrove, C., & Sheikh, A. (2007) · PubMed Central · Cited by 61 · see citing works
Self-citationGlenton, C., Underland, V., Kho, M. E., Pennick, V., & Oxman, A. D. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.12.011 · Cited by 51 · see citing works
Clinical service organisation for heart failure
Taylor, S. J., Bestall, J., Cotter, S., Falshaw, M., Hood, S. G., Parsons, S., Wood, L., & Underwood, M. R. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002752.pub2 · Cited by 47 · see citing works
Tobacco cessation interventions for young people
Grimshaw, G., Stanton, A., & Lancaster, T. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003289.pub3 · Cited by 46 · see citing works
Assessing the value of assertive outreach
Weaver, T. D., Tyrer, P. J., Ritchie, J., & Renton, A. M. (2003) · The British Journal of Psychiatry · doi:10.1192/bjp.183.5.437 · Cited by 44 · see citing works
Avery, A. J., Rodgers, S., Cantrill, J. A., Armstrong, S., Elliott, R. A., Howard, R., Kendrick, D., Morris, C. J., Murray, S. A., Prescott, R. J., et al. (2009) · Trials · doi:10.1186/1745-6215-10-28 · Cited by 44 · see citing works
Self-citationFuredy, J. J. (1990) · Canadian Psychology/Psychologie canadienne · doi:10.1037/h0078903 · Cited by 41 · see citing works
Phillips, C. O., Wright, S. M., Kern, D. E., Singa, R. M., Shepperd, S., & Rubin, H. R. (2004) · JAMA · doi:10.1001/jama.291.11.1358 · Cited by 21 · see citing works
Self-citationShojania, K. G., Ranji, S. R., McDonald, K. M., Grimshaw, J. M., Sundaram, V., Rushakoff, R. J., & Owens, D. K. (2006) · JAMA · doi:10.1001/jama.296.4.427 · Cited by 20 · see citing works
Continuity of care - a chameleon concept
Shepperd, S., & Richards, S. (2002) · Journal of Health Services Research & Policy · doi:10.1258/135581902760082427 · Cited by 9 · see citing works
Self-citationQuality Improvement Strategies for Type 2 Diabetes
Pignone, M. P., & DeWalt, D. A. (2006) · JAMA · doi:10.1001/jama.296.22.2680-b · Cited by 3 · see citing works
Cost-effectiveness of Breast Magnetic Resonance Imaging to Screen
Crystal, P. (2006) · JAMA · doi:10.1001/jama.296.22.2681-b · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [49]Pawson R 2002 Does Megan's Law work? A theory driven systematic review. ESRC UK Centre for Evidence Based Policy and Practice University of London
Cited by
Sorted by citations
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987 · see citing works
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442 · see citing works
Self-citationPsychological therapies for the management of chronic pain (excluding headache) in adults
Williams, A. C. D. C., Eccleston, C., & Morley, S. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007407.pub3 · Cited by 1,634 · see citing works
RAMESES publication standards: realist syntheses
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-21 · Cited by 1,374 · see citing works
Self-citationRenfrew, M. J., McFadden, A., Bastos, M. H., Campbell, J., Channon, A. A., Cheung, N. F., Silva, D. R. A. D., Downe, S., Kennedy, H. P., Malata, A., et al. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60789-3 · Cited by 1,297 · see citing works
Clarifying differences between review designs and methods
Gough, D., Thomas, J., & Oliver, S. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-28 · Cited by 1,063 · see citing works
The Lancet Commission on the future of care and clinical research in autism
Lord, C., Charman, T., Havdahl, A., Carbone, P., Anagnostou, E., Boyd, B., Carr, T., de Vries, P. J., Dissanayake, C., Divan, G., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)01541-5 · Cited by 924 · see citing works
Langford, R., Bonell, C. P., Jones, H. E., Pouliou, T., Murphy, S. M., Waters, E., Komro, K. A., Gibbs, L. F., Magnus, D., & Campbell, R. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008958.pub2 · Cited by 768 · see citing works
RAMESES publication standards: realist syntheses
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · Journal of Advanced Nursing · doi:10.1111/jan.12095 · Cited by 385 · see citing works
Self-citationUsing logic models to capture complexity in systematic reviews: Logic Models in Systematic Reviews
Anderson, L. M., Petticrew, M., Rehfuess, E., Armstrong, R., Ueffing, E., Baker, P., Francis, D., & Tugwell, P. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.32 · Cited by 372 · see citing works
Kitsiou, S., Paré, G., & Jaana, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4174 · Cited by 327 · see citing works
Kastner, M., Tricco, A. C., Soobiah, C., Lillie, E., Perrier, L., Horsley, T., Welch, V., Cogo, E., Antony, J., & Straus, S. E. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-114 · Cited by 345 · see citing works
Self-citationWu, Y., Yao, X., Vespasiani, G., Nicolucci, A., Dong, Y., Kwong, J., Li, L., Sun, X., Tian, H., & Li, S. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6522 · Cited by 283 · see citing works
Liu, J., Davidson, E., Bhopal, R., White, M., Johnson, M., Netto, G., Deverill, M., & Sheikh, A. (2012) · Health Technology Assessment · doi:10.3310/hta16440 · Cited by 237 · see citing works
Self-citationAgile science: creating useful products for behavior change in the real world
Hekler, E. B., Klasnja, P., Riley, W. T., Buman, M. P., Huberty, J., Rivera, D. E., & Martin, C. A. (2016) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0395-7 · Cited by 208 · see citing works
Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable?
Booth, A., & Carroll, C. (2015) · Health Information & Libraries Journal · doi:10.1111/hir.12108 · Cited by 196 · see citing works
Tobacco cessation interventions for young people.
Fanshawe, T. R., Halliwell, W., Lindson, N., Aveyard, P., Livingstone-Banks, J., & Hartmann-Boyce, J. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003289.pub6 · Cited by 196 · see citing works
Templier, M., & Paré, G. (2018) · European Journal of Information Systems · doi:10.1080/0960085x.2017.1398880 · Cited by 180 · see citing works
Campbell, M., Katikireddi, S. V., Sowden, A., & Thomson, H. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.08.019 · Cited by 169 · see citing works
Amplification with hearing aids for patients with tinnitus and co-existing hearing loss.
Hoare, D. J., Edmondson-Jones, M., Sereda, M., Akeroyd, M. A., & Hall, D. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010151.pub2 · Cited by 170 · see citing works
Sound therapy (using amplification devices and/or sound generators) for tinnitus.
Sereda, M., Xia, J., El Refaie, A., Hall, D. A., & Hoare, D. J. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013094.pub2 · Cited by 135 · see citing works
Douglas, P. S., & Hill, P. S. (2013) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0b013e31829cafa6 · Cited by 129 · see citing works
Levati, S., Campbell, P., Frost, R., Dougall, N., Wells, M., Donaldson, C., & Hagen, S. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0058-y · Cited by 101 · see citing works
Grant, S. P., Mayo-Wilson, E., Melendez-Torres, G. J., & Montgomery, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065442 · Cited by 111 · see citing works
McLean, S., Chandler, D., Nurmatov, U., Liu, J. L., Pagliari, C., Car, J., & Sheikh, A. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007717.pub2 · Cited by 119 · see citing works
Self-citationTobacco cessation interventions for young people
Stanton, A., & Grimshaw, G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003289.pub5 · Cited by 113 · see citing works
Fairbrother, P., Pinnock, H., Hanley, J., McCloughan, L., Sheikh, A., Pagliari, C., & McKinstry, B. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.04.003 · Cited by 92 · see citing works
Self-citationThe case for randomized controlled trials to assess the impact of clinical information systems
Liu, J. L. Y., & Wyatt, J. C. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.010306 · Cited by 66 · see citing works
Osborne, R. H., Batterham, R., & Livingston, J. (2011) · Nursing Clinics of North America · doi:10.1016/j.cnur.2011.05.010 · Cited by 62 · see citing works
Meta-Analysis of Complex Interventions
Tanner-Smith, E. E., & Grant, S. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014112 · Cited by 62 · see citing works
A Reality Checkpoint for Mobile Health: Three Challenges to Overcome
Editors, T. P. M. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001395 · Cited by 56 · see citing works
Takian, A., Petrakaki, D., Cornford, T., Sheikh, A., & Barber, N. (2012) · BMC Health Services Research · doi:10.1186/1472-6963-12-105 · Cited by 51 · see citing works
“Study designs include systematic reviews [13]; descriptive theory-based case studies [13-15]; ‘observational’; ‘quasi-experimental’ [11,16-18]; before-after, and, less frequently, randomized controlled trials (RCTs) [19].”
BackgroundMethodSelf-citation“Real-World” Practical Evaluation Strategies: A Review of Telehealth Evaluation
Agboola, S., Hale, T. M., Masters, C., Kvedar, J., & Jethwani, K. (2014) · JMIR Research Protocols · doi:10.2196/resprot.3459 · Cited by 49 · see citing works
“The third challenge to telehealth evaluation is that health programs are more complex and dynamic than other types of interventions [16,20].”
McCleary, N., Duncan, E. M., Stewart, F., & Francis, J. J. (2013) · Trials · doi:10.1186/1745-6215-14-146 · Cited by 52 · see citing works
“Agreed labels (drug names) are used for the same PI [18,36], making specification more straightforward.”
Liu, J. J., Wabnitz, C., Davidson, E., Bhopal, R. S., White, M., Johnson, M. R. D., Netto, G., & Sheikh, A. (2013) · Preventive Medicine · doi:10.1016/j.ypmed.2013.09.014 · Cited by 51 · see citing works
Self-citationWarner, G., Packer, T. L., Kervin, E., Sibbald, K., & Audulv, Å. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.07.002 · Cited by 38 · see citing works
Setting Housing Standards to Improve Global Health
Howden-Chapman, P., Roebbel, N., & Chisholm, E. (2017) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph14121542 · Cited by 33 · see citing works
Hoeft, T. J., Hinton, L., Liu, J., & Unützer, J. (2016) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2015.07.003 · Cited by 25 · see citing works
Möhler, R., Bartoszek, G., & Meyer, G. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-125 · Cited by 21 · see citing works
Cooper, A. M., Coffey, M., & Dugdill, L. (2014) · Pragmatic and Observational Research · doi:10.2147/por.s52287 · Cited by 8 · see citing works
“In conjunction with this, process evaluation can help those designing trials to understand how “the actions taken by the ‘human components’ of CHIs [complex health interventions] are influenced by the context in which the intervention takes place” 60 and any impact this may have on an intervention.”
- Authors
- Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., Wong, G., & Sheikh, A.
- Affiliations
- University of Oxford (GB)
- South African Medical Research Council (ZA)
- University of Toronto (CA)
- Trinity College Dublin (IE)
- Newcastle University (GB)
- University College London (GB)
- Maastricht University (NL)
- Corresponding author
- Shepperd, Sasha
- Wong, Geoff
- Sheikh, Aziz
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-08-11
- Volume
- 6
- Issue
- 8
- Pages
- e1000086
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC2717209
- OpenAlex
- W2145257154
- Cited by
- 408
- References
- 61
- Citations by year
- 2026: 8 · 2025: 6 · 2024: 3 · 2023: 3 · 2022: 7 · 2021: 11 · 2020: 22 · 2019: 17 · 2018: 19 · 2017: 26 · 2016: 21 · 2015: 35 · 2014: 36 · 2013: 115 · 2012: 33
- FWCI
- 32.25
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Newcastle University ePrints (Newcastle Univesity)
- Funders
- Cancer Research Institute
- National Cancer Research Institute
- Scottish Government
- National Institute for Health and Care Research (RSES/PDA/C2/017)
- Asthma and Lung UK
- University College London
- Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- complex interventions, systematic review methodology, intervention definition, intervention documentation, data synthesis, stroke unit care
- MeSH terms
- Clinical Trials as Topic, Humans, Research Design, Review Literature as Topic, Meta-Analysis as Topic, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
