The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Innovative tools for quality assessment: integrated quality criteria for review of multiple study designs (ICROMS)
Open accessAbstract
Objectives With the aim to facilitate a more comprehensive review process in public health including patient safety, we established a tool that we have termed ICROMS (Integrated quality Criteria for the Review Of Multiple Study designs), which unifies, integrates and refines current quality criteria for a large range of study designs including qualitative research.Study design Review, pilot testing and expert consensus.Methods The tool is the result of an iterative four phase process over two years: 1) gathering of established criteria for assessing controlled, non-controlled and qualitative study designs; 2) pilot testing of a first version in two systematic reviews on behavioural change in infection prevention and control and in antibiotic prescribing; 3) further refinement and adding of additional study designs in the context of the European Centre for Disease Prevention and Control funded project 'Systematic review and evidence-based guidance on organisation of hospital infection control programmes' (SIGHT); 4) scrutiny by the pan-European expert panel of the SIGHT project, which had the objective of ensuring robustness of the systematic review.Results ICROMS includes established quality criteria for randomised studies, controlled before-and-after studies and interrupted time series, and incorporates criteria for non-controlled before-and-after studies, cohort studies and qualitative studies. The tool consists of two parts: 1) a list of quality criteria specific for each study design, as well as criteria applicable across all study designs by using a scoring system; 2) a 'decision matrix', which specifies the robustness of the study by identifying minimum requirements according to the study type and the relevance of the study to the review question. The decision matrix directly determines inclusion or exclusion of a study in the review. ICROMS was applied to a series of systematic reviews to test its feasibility and usefulness in the appraisal of multiple study designs. The tool was applicable across a wide range of study designs and outcome measures.Conclusion ICROMS is a comprehensive yet feasible appraisal of a large range of study designs to be included in systematic reviews addressing behaviour change studies in patient safety and public health. The tool is sufficiently flexible to be applied to a variety of other domains in health-related research. Beyond its application to systematic reviews, we envisage that ICROMS can have a positive effect on researchers to be more rigorous in their study design and more diligent in their reporting.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Study quality was assessed using the integrated criteria for the review of multiple study designs ICROMS.
“the integrated criteria for the review of multiple study designs ICROMS (Zingg et al., 2016) was used to assess study quality.”
From Global Prevalence and Mental Health Outcomes of Intimate Partner Violence Among Women: A Systematic Review and Meta-Analysis (2024) · cites this paper as Zingg et al., 2016
The review used the ICROMS criteria to assess study quality.
“Given the wide variety of study designs, we employed the integrated criteria for review of multiple study designs (ICROMS by Zingg et al., 2015) to assess quality.”
From eHealth interventions for family carers of people with long term illness: A promising approach? (2018) · cites this paper as Zingg et al., 2015
Recommendations were made mainly on two topics: whether different study designs should be included in a systematic review; and whether different study designs should be analysed together in a single meta-analysis.
“Recommendations were made mainly on 2 topics: whether different study designs should be included in a systematic review; [29, 34, 37–39, 49, 56, 65, 73, 75] and whether different study designs should be analysed together in a single meta-analysis [8, 34, 35, 37, 38, 42, 49, 51, 56, 61, 70, 75–78].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [73]
A variation of this recommendation was to exclude studies that are at high risk of bias.
“A variation of this recommendation was to exclude studies that are at high risk of bias [42, 44, 54, 73, 82].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [73]
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Vandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010-w1 · Cited by 3,543
Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00684.x · Cited by 3,437
Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,289
RAMESES publication standards: realist syntheses
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-21 · Cited by 1,374
RAMESES publication standards: meta-narrative reviews
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-20 · Cited by 610
Can We Systematically Review Studies That Evaluate Complex Interventions?
Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., Wong, G., & Sheikh, A. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000086 · Cited by 408
Flagged first, then by citations
Research works printed but not linked: 22
- Craig, 2008, Developing and evaluating complex interventions: the new Medical Research Council guidance, BMJ, 337, a1655, 10.1136/bmj.a1655 · DOI · Paper page
- Mays, 2005, Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field, J Health Serv Res Policy, 10, 6, 10.1258/1355819054308576 · DOI
- Dixon-Woods, 2005, Synthesising qualitative and quantitative evidence: a review of methods, J Health Serv Res Policy, 10, 45, 10.1258/1355819052801804 · DOI · Paper page
- Cochrane Effective Practice and Organisation of Care Review Group (EPOC). http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/datacollectionchecklist.pdf (accessed 03 November 2014).
- Brown, 2012, Reviewing methodologically disparate data: a practical guide to the patient safety research field, J Eval Clin Pract, 18, 172, 10.1111/j.1365-2753.2010.01519.x · DOI
- Dixon-Woods, 2011, Explaining Michigan: developing an ex post theory of a quality improvement program, Milbank Q, 89, 167, 10.1111/j.1468-0009.2011.00625.x · DOI · Paper page
- Gould, 2010, Interventions to improve hand hygiene compliance in patient care, Cochrane Database Syst Rev, 9, CD005186
- Harris, 2005, A systematic review of quasi-experimental designs in the fields of infection control and antibiotic resistance, Clin Infect Dis, 41, 77, 10.1086/430713 · DOI
- Edwards, 2012, Optimising infection prevention and control practice using behaviour change: a systematic review, Lancet Infect Dis, 12, 318, 10.1016/S1473-3099(11)70283-3 · DOI
- Charani, 2011, Behavior change strategies to influence antibiotic prescribing in acute care: a systematic review, Clin Infect Dis, 53, 651, 10.1093/cid/cir445 · DOI
- Cohen, 2008, Evaluative criteria for qualitative research in healthcare: controversies and recommendations, Ann Fam Med, 6, 331, 10.1370/afm.818 · DOI
- Perencevich, 2011, Infection prevention and comparative effectiveness research, JAMA, 305, 1482, 10.1001/jama.2011.450 · DOI
- Shojania, 2005, Evidence-based quality improvement: the state of the science, Health Aff (Millwood), 24, 138, 10.1377/hlthaff.24.1.138 · DOI
- Harris, 2004, The use and interpretation of quasi-experimental studies in infectious diseases, Clin Infect Dis, 38, 1586, 10.1086/420936 · DOI · Paper page
- Zingg, 2015, Hospital organisation, management, and structure for prevention of health-care-associated infection: a systematic review and expert consensus, Lancet Infect Dis, 15, 212, 10.1016/S1473-3099(14)70854-0 · DOI
- Lingard, 2008, Grounded theory, mixed methods and action research, BMJ, 337, a567, 10.1136/bmj.39602.690162.47 · DOI
- Kuper, 2008, An introduction to reading and appraising qualitative research, BMJ, 337, a288, 10.1136/bmj.a288 · DOI · Paper page
- Carter, 2007, Justifying knowledge, justifying method, taking action: epistemiologies, methodologies and methods in qualitative research, Qual Health Res, 17, 1316, 10.1177/1049732307306927 · DOI
- Kuper, 2008, Critically appraising qualitative research, BMJ, 337, a1035, 10.1136/bmj.a1035 · DOI · Paper page
- Grime, 2002, Cohort studies: marching towards outcomes, Lancet, 359, 341, 10.1016/S0140-6736(02)07500-1 · DOI
- Davidoff, 2008, Publication guidelines for quality improvement in health care: evolution of the SQUIRE project, Qual Saf Health Care, 17, i3, 10.1136/qshc.2008.029066 · DOI · Paper page
- Atkins, 2004, Grading quality of evidence and strength of recommendations, BMJ, 328, 1490, 10.1136/bmj.328.7454.1490 · DOI · Paper page
Cited by
Sorted by citations
White, S. J., Sin, J., Sweeney, A., Salisbury, T., Wahlich, C., Montesinos Guevara, C. M., Gillard, S., Brett, E., Allwright, L., Iqbal, N., et al. (2024) · Trauma, Violence, & Abuse · doi:10.1177/15248380231155529 · Cited by 465
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400
Njagi, P., Groot, W., Arsenijevic, J., Dyer, S., Mburu, G., & Kiarie, J. (2023) · Human Reproduction Open · doi:10.1093/hropen/hoad007 · Cited by 277
Mental health training programs for secondary school teachers: a systematic review
Anderson, M., Werner-Seidler, A., King, C., Gayed, A., Harvey, S. B., & O’Dea, B. (2019) · School Mental Health · doi:10.1007/s12310-018-9291-2 · Cited by 124
eHealth interventions for family carers of people with long term illness: A promising approach?
Sin, J., Henderson, C., Spain, D., Cornelius, V., Chen, T., & Gillard, S. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.01.008 · Cited by 96
Reeve, A., Tickle, A., & Moghaddam, N. (2018) · Mental Health Review Journal · doi:10.1108/mhrj-11-2017-0052 · Cited by 49
Fitzgerald, F. C., Zingg, W., Chimhini, G., Chimhuya, S., Wittmann, S., Brotherton, H., Olaru, I. D., Neal, S. R., Russell, N., da Silva, A. R. A., et al. (2022) · Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000003320 · Cited by 48
Self-citationDigitally-delivered cognitive-behavioural therapy for youth insomnia: A systematic review
Werner-Seidler, A., Johnston, L., & Christensen, H. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.01.007 · Cited by 36
O'Connell, J., Pote, H., & Shafran, R. (2021) · Early Intervention in Psychiatry · doi:10.1111/eip.12964 · Cited by 26
Metadata
- Authors
- Zingg, W., Castro-Sanchez, E., Secci, F. V., Edwards, R., Drumright, L. N., Sevdalis, N., & Holmes, A. H.
- Affiliations
- University Hospital of Geneva (CH)
- Imperial College London (GB)
- St Mary's Hospital (GB)
- Corresponding author
- Zingg, W.
- Venue
- Public Health
- Publisher
- Elsevier BV
- Published
- 2016-04
- Volume
- 133
- Pages
- 19-37
- ISSN
- 0033-3506 (ISSN-L)
- OpenAlex
- W2215491151
- Cited by
- 143
- References
- 8
- Citations by year
- 2026: 14 · 2025: 12 · 2024: 22 · 2023: 17 · 2022: 26 · 2021: 10 · 2020: 12 · 2019: 8 · 2018: 12 · 2017: 6 · 2016: 2
- FWCI
- 6.38
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Public Health (publishedVersion)
- PubMed (publishedVersion)
- Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
- Funders
- National Cattlemen's Beef Association
- National Institute for Health and Care Research (CDF-2011-04-017)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- quality standards, systematic review methodology, decision matrix, before-after study, interrupted time series, cohort studies, qualitative research, infection control, antibiotic prescribing, patient safety, behavior change
- MeSH terms
- Diffusion of Innovation, Humans, Public Health, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
