The DOI matches its Crossref record; 71 of the 74 references it cites were found in a registry or an index.
INTERRUPTED TIME SERIES DESIGNS IN HEALTH TECHNOLOGY ASSESSMENT: LESSONS FROM TWO SYSTEMATIC REVIEWS OF BEHAVIOR CHANGE STRATEGIES
Abstract
Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies). Methods: Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression. Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered “effective” in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences. Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We planned to assess all items included in Ramsay 2003, three of which are also included in RoB 1 (blinding of outcome assessors, reliable outcome measure, and incomplete data).”
From Adapting shift work schedules for sleep quality, sleep duration, and sleepiness in shift workers (2023) · cites this paper as Ramsay 2003
In the citing paper
“Ramsay 2003 describes a risk of bias assessment method for interrupted time series.”
From Adapting shift work schedules for sleep quality, sleep duration, and sleepiness in shift workers (2023) · cites this paper as Ramsay 2003
In the citing paper
“The quasi-experimental interrupted time series with nonequivalent groups, which tracks target data both before and after introducing a programmatic change to a group whose membership may vary over time, is appropriate for testing viability and impacts of new interventions delivered in field settings [88-90].”
From Family Support Protocol for Adolescent Internalizing Disorders: Protocol for a Pre-Post Quantitative Treatment Development Study (2024) · cites this paper as [90]
In the citing paper
“A common shortcoming in ITS analyses is a lack of statistical power (61).”
From Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research (2017) · cites this paper as [61]
Citations
Cites
(2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,923
(1963) · Medical Entomology and Zoology · Cited by 15,085
(1979) · Medical Entomology and Zoology · Cited by 11,252
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,010
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044
Self-citationBero, L. A., Grilli, R., Grimshaw, J. M., Harvey, E., Oxman, A. D., & Thomson, M. A. (1998) · BMJ · doi:10.1136/bmj.317.7156.465 · Cited by 2,396
Self-citationEffectiveness and Efficiency: Random Reflections on Health Services
(1973) · Public Health · doi:10.1016/s0033-3506(73)80082-4 · Cited by 1,912
Do Practice Guidelines Guide Practice?
Lomas, J., Anderson, G. M., Domnick-Pierre, K., Vayda, E., Enkin, M. W., & Hannah, W. J. (1989) · New England Journal of Medicine · doi:10.1056/nejm198911093211906 · Cited by 1,234
HASLOCK, I. (1989) · Rheumatology · doi:10.1093/rheumatology/28.3.185 · Cited by 841
Mass media interventions: effects on health services utilisation
Grilli, R., Ramsay, C., & Minozzi, S. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000389 · Cited by 533
Self-citation
Research works printed but not linked: 1
- Bero L , Grilli R , Grimshaw JM et al .2002 The Cochrane Effective Practice and Organisation of care Group (EPOC) Module.In: The Cochrane Library.Oxford:Update Software Issue 1.
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Conducting Interrupted Time-series Analysis for Single- and Multiple-group Comparisons
Linden, A. (2015) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1501500208 · Cited by 1,233
Printed educational materials: effects on professional practice and healthcare outcomes
Giguère, A., Légaré, F., Grimshaw, J., Turcotte, S., Fiander, M., Grudniewicz, A., Makosso-Kallyth, S., Wolf, F. M., Farmer, A. P., & Gagnon, M.-P. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004398.pub3 · Cited by 878
Self-citationInterventions for improving the adoption of shared decision making by healthcare professionals
Légaré, F., Ratté, S., Stacey, D., Kryworuchko, J., Gravel, K., Graham, I. D., & Turcotte, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006732.pub2 · Cited by 823
Interventions to improve hand hygiene compliance in patient care
Gould, D. J., Moralejo, D., Drey, N., Chudleigh, J. H., & Taljaard, M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005186.pub4 · Cited by 663
Metadata
- Authors
- Ramsay, C. R., Matowe, L., Grilli, R., Grimshaw, J. M., & Thomas, R. E.
- Affiliations
- University of Aberdeen (GB)
- Kuwait University (KW)
- Agenzia Sanitaria e Sociale Regionale (IT)
- University of Ottawa (CA)
- Venue
- International Journal of Technology Assessment in Health Care
- Publisher
- Cambridge University Press (CUP)
- Published
- 2003-12
- Volume
- 19
- Issue
- 4
- Pages
- 613-623
- ISSN
- 0266-4623 (ISSN-L)
- 1471-6348
- OpenAlex
- W2119566762
- Cited by
- 703
- References
- 74
- Citations by year
- 2026: 12 · 2025: 17 · 2024: 25 · 2023: 33 · 2022: 46 · 2021: 39 · 2020: 58 · 2019: 37 · 2018: 50 · 2017: 46 · 2016: 41 · 2015: 55 · 2014: 52 · 2013: 37 · 2012: 37
- FWCI
- 1.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Technology Assessment in Health Care (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- interrupted time series design, methodological quality, behaviour change interventions, mass media interventions, guideline dissemination, implementation strategies, underpowered studies, reanalysis, time series regression, health technology assessment
- MeSH terms
- Cross-Sectional Studies, Diffusion of Innovation, Health Services, Mass Media, Regression Analysis, Research Design, Technology Assessment, Biomedical, Time Factors, Practice Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: INTERRUPTED TIME SERIES DESIGNS IN HEALTH TECHNOLOGY ASSESSMENT: LESSONS FROM TWO SYSTEMATIC REVIEWS OF BEHAVIOR CHANGE STRATEGIES. https://app.cochat.ai/papers/10.1017/s0266462303000576
- The paper
- Ramsay, C. R., Matowe, L., Grilli, R., Grimshaw, J. M., & Thomas, R. E. (2003). INTERRUPTED TIME SERIES DESIGNS IN HEALTH TECHNOLOGY ASSESSMENT: LESSONS FROM TWO SYSTEMATIC REVIEWS OF BEHAVIOR CHANGE STRATEGIES. International Journal of Technology Assessment in Health Care. https://doi.org/10.1017/s0266462303000576
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.