Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR)
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. METHODS: We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. RESULTS: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. CONCLUSIONS: The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697
Self-citationGRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Self-citationHoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Self-citationMichie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629
Self-citationProcess evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204
Bellg, A. J., Treatment Fidelity Workgroup of the NIH Behavior Change Consortium, Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., et al. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.443 · Cited by 2,620 · cited 1×
“In particular, systematic reviews need to be able to describe complexity accurately and consistently across included primary studies in order to facilitate the grouping of studies for analysis, investigate causes of heterogeneity, and explore the relationship between intervention complexity and fidelity of implementat…”
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · cited 1×
“Other approaches have sought to differentiate complex, complicated and simple interventions [9, 10].”
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014) · Trials · doi:10.1186/1745-6215-15-267 · Cited by 738
Michie, S., Hyder, N., Walia, A., & West, R. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2010.11.016 · Cited by 459
Self-citationGlenton, C., Colvin, C., Carlsen, B., Swartz, A., Lewin, S., Noyes, J., & Rashidian, A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010414.pub2 · Cited by 454 · cited 2×
“This is shown in the example in Table 4 which draws on data from both a Cochrane review of effectiveness [26] and a Cochrane qualitative evidence synthesis [25].”
Self-citation
Cited by
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262
Self-citationPage, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916
Self-citationSkivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · Health Technology Assessment · doi:10.3310/hta25570 · Cited by 522
Defining the criteria for including studies and how they will be grouped for the synthesis
McKenzie, J. E., Brennan, S. E., Ryan, R. E., Thomson, H. J., Johnston, R. V., & Thomas, J. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch3 · Cited by 290
Self-citationNoyes, J., Booth, A., Lewin, S., Carlsen, B., Glenton, C., Colvin, C. J., Garside, R., Bohren, M. A., Rashidian, A., Wainwright, M., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0693-6 · Cited by 266
Self-citation
Metadata
- Authors
- Lewin, S., Hendry, M., Chandler, J., Oxman, A. D., Michie, S., Shepperd, S., Reeves, B. C., Tugwell, P., Hannes, K., Rehfuess, E. A., Welch, V., Mckenzie, J. E., Burford, B., Petkovic, J., Anderson, L. M., Harris, J., & Noyes, J.
- Venue
- BMC Medical Research Methodology
- Published
- 2017-04-26
- Volume
- 17
- Issue
- 1
- Pages
- 76-76
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2608508808
- Cited by
- 147
- References
- 16
- FWCI
- 3.96
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). https://app.cochat.ai/papers/10.1186/s12874-017-0349-x
- The paper
- Lewin, S., Hendry, M., Chandler, J., Oxman, A. D., Michie, S., Shepperd, S., Reeves, B. C., Tugwell, P., Hannes, K., Rehfuess, E. A., Welch, V., Mckenzie, J. E., Burford, B., Petkovic, J., Anderson, L. M., Harris, J., & Noyes, J. (2017). Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Medical Research Methodology. https://doi.org/10.1186/s12874-017-0349-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.