Applying a propensity score-based weighting model to interrupted time series data: improving causal inference in programme evaluation: Weighted interrupted times series analysis
Abstract
Often, when conducting programme evaluations or studying the effects of policy changes, researchers may only have access to aggregated time series data, presented as observations spanning both the pre- and post-intervention periods. The most basic analytic model using these data requires only a single group and models the intervention effect using repeated measurements of the dependent variable. This model controls for regression to the mean and is likely to detect a treatment effect if it is sufficiently large. However, many potential sources of bias still remain. Adding one or more control groups to this model could strengthen causal inference if the groups are comparable on pre-intervention covariates and level and trend of the dependent variable. If this condition is not met, the validity of the study findings could be called into question. In this paper we describe a propensity score-based weighted regression model, which overcomes these limitations by weighting the control groups to represent the average outcome that the treatment group would have exhibited in the absence of the intervention. We illustrate this technique studying cigarette sales in California before and after the passage of Proposition 99 in California in 1989. While our results were similar to those of the Synthetic Control method, the weighting approach has the advantage of being technically less complicated, rooted in regression techniques familiar to most researchers, easy to implement using any basic statistical software, may accommodate any number of treatment units, and allows for greater flexibility in the choice of treatment effect estimators.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparisons of results from different methods are particularly useful in early applications of novel methods whose strengths and weaknesses are not fully understood.
“Such comparisons are particularly useful in early applications of novel methods whose strengths and weaknesses are not fully understood (49).”
From Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research (2017) · cites this paper as [49]
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · see citing works
A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity
White, H. (1980) · Econometrica · doi:10.2307/1912934 · Cited by 25,861 · see citing works
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836 · see citing works
(1963) · Medical Entomology and Zoology · Cited by 15,085 · see citing works
(2002) · Cited by 13,211 · see citing works
(1979) · Medical Entomology and Zoology · Cited by 11,252 · see citing works
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,091 · see citing works
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581 · see citing works
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513 · see citing works
The economic costs of conflict: a case study of the Basque Country
Abadie, A., & Gardeazabal, J. (2003) · American Economic Review · doi:10.1257/000282803321455188 · Cited by 4,517 · see citing works
Segmented regression analysis of interrupted time series studies in medication use research
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00430.x · Cited by 3,279 · see citing works
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011 · see citing works
Generalized linear models. 2nd ed.
McCullagh, P., & Nelder, J. (1993) · Journal of the American Statistical Association · doi:10.2307/2290358 · Cited by 1,581 · see citing works
Generalized Linear Models, 2nd Edn.
Cheek, P. J., McCullagh, P., & Nelder, J. A. (1990) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347392 · Cited by 1,517 · see citing works
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.403 · Cited by 1,317 · see citing works
Durbin, J. (1970) · Econometrica · doi:10.2307/1909547 · Cited by 1,200 · see citing works
Rubin, D. B. (2007) · Statistics in Medicine · doi:10.1002/sim.2739 · Cited by 1,047 · see citing works
Abadie, A., Diamond, A., & Hainmueller, J. (2007) · National Bureau of Economic Research · doi:10.3386/t0335 · Cited by 830 · see citing works
Zhang, F., Wagner, A. K., Soumerai, S. B., & Ross‐Degnan, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.08.007 · Cited by 247 · see citing works
From association to causation: some remarks on the history of statistics
Freedman, D. A. (1999) · Statistical Science · doi:10.1214/ss/1009212409 · Cited by 243 · see citing works
Rosenblum, M. (2011) · Springer series in statistics · doi:10.1007/978-1-4419-9782-1_9 · Cited by 185 · see citing works
Cross-sectional time-series experiments: Some suggested statistical analyses.
Simonton, D. K. (1977) · Psychological Bulletin · doi:10.1037/0033-2909.84.3.489 · Cited by 159 · see citing works
Evaluating Disease Management Program Effectiveness: An Introduction to Time-Series Analysis
Linden, A., Adams, J., & Roberts, N. E. (2003) · Disease Management · doi:10.1089/109350703322682559 · Cited by 67 · see citing works
Self-citationLinden, A., & Adams, J. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2009.01219.x · Cited by 65 · see citing works
Self-citationErratum and Discussion of Propensity-Score Reweighting
Nichols, A. (2008) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0800800405 · Cited by 52 · see citing works
Estimating the Effect of Regression to the Mean in Health Management Programs
Linden, A. (2007) · Disease Management & Health Outcomes · doi:10.2165/00115677-200715010-00002 · Cited by 42 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Interrupted time series regression for the evaluation of public health interventions: a tutorial
Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw098 · Cited by 3,049 · see citing works
Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044327 · Cited by 570 · see citing works
Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise
Riley, W. T., Glasgow, R. E., Etheredge, L., & Abernethy, A. P. (2013) · Clinical and Translational Medicine · doi:10.1186/2001-1326-2-10 · Cited by 267 · see citing works
Moffitt, S. L., Willse, C., Smith, K. B., & Cohen, D. K. (2023) · Perspectives on Politics · doi:10.1017/s1537592721002012 · Cited by 10 · see citing works
- Authors
- Linden, A., & Adams, J. L.
- Affiliations
- RAND Corporation (US)
- Corresponding author
- Linden, Ariel
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2010-10-25
- Volume
- 17
- Issue
- 6
- Pages
- 1231-1238
- ISSN
- 1356-1294 (ISSN-L)
- OpenAlex
- W1537003895
- Cited by
- 219
- References
- 26
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 19 · 2023: 28 · 2022: 21 · 2021: 29 · 2020: 26 · 2019: 14 · 2018: 17 · 2017: 12 · 2016: 15 · 2015: 4 · 2014: 2 · 2013: 4
- FWCI
- 1.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Evaluation in Clinical Practice (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- propensity score weighting, interrupted time series, program evaluation, causal inference, control group selection, regression to the mean, synthetic control method, cigarette sales, treatment effect estimation
- MeSH terms
- California, Data Interpretation, Statistical, Health Promotion, Humans, Research Design, Taxes, Reproducibility of Results, Bias, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
