Inferring causal impact using Bayesian structural time-series models
Open accessThe DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
An important problem in econometrics and marketing is to infer the causal impact that a designed market intervention has exerted on an outcome metric over time. This paper proposes to infer causal impact on the basis of a diffusion-regression state-space model that predicts the counterfactual market response in a synthetic control that would have occurred had no intervention taken place. In contrast to classical difference-in-differences schemes, state-space models make it possible to (i) infer the temporal evolution of attributable impact, (ii) incorporate empirical priors on the parameters in a fully Bayesian treatment, and (iii) flexibly accommodate multiple sources of variation, including local trends, seasonality and the time-varying influence of contemporaneous covariates. Using a Markov chain Monte Carlo algorithm for posterior inference, we illustrate the statistical properties of our approach on simulated data. We then demonstrate its practical utility by estimating the causal effect of an online advertising campaign on search-related site visits. We discuss the strengths and limitations of state-space models in enabling causal attribution in those settings where a randomised experiment is unavailable. The CausalImpact R package provides an implementation of our approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085
(2002) · Cited by 13,211
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
The economic costs of conflict: a case study of the Basque Country
Abadie, A., & Gardeazabal, J. (2003) · American Economic Review · doi:10.1257/000282803321455188 · Cited by 4,517
Variable Selection via Gibbs Sampling
George, E. I., & McCulloch, R. E. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476353 · Cited by 2,828
On making causal claims: A review and recommendations
Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010) · The Leadership Quarterly · doi:10.1016/j.leaqua.2010.10.010 · Cited by 2,441
Semiparametric Difference-in-Differences Estimators
Abadie, A. (2005) · The Review of Economic Studies · doi:10.1111/0034-6527.00321 · Cited by 2,286
Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania
Card, D., & Krueger, A. (1993) · National Bureau of Economic Research · doi:10.3386/w4509 · Cited by 2,063
On Gibbs sampling for state space models
CARTER, C. K., & KOHN, R. (1994) · Biometrika · doi:10.1093/biomet/81.3.541 · Cited by 1,917
Bayesian Forecasting and Dynamic Models
(1997) · Springer Series in Statistics · doi:10.1007/b98971 · Cited by 1,917
Natural and Quasi- Experiments in Economics
Meyer, B. (1994) · Journal of Business and Economic Statistics · doi:10.2307/1392369 · Cited by 1,792
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628
(2007) · Cited by 1,428
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Bayesian Forecasting and Dynamic Models
West, M., & Harrison, J. (1989) · Springer Series in Statistics · doi:10.1007/978-1-4757-9365-9 · Cited by 1,295
(1997) · Cited by 1,193
Counterfactuals and Causal Inference: Methods and Principles for Social Research
Morgan, S. L., & Winship, C. (2007) · doi:10.1017/cbo9780511804564 · Cited by 1,146
The Practice of Econometrics: Classic and Contemporary.
Nakamura, A., & Berndt, E. R. (1992) · Journal of the American Statistical Association · doi:10.2307/2290668 · Cited by 1,114
Natural and Quasi-Experiments in Economics
Meyer, B. D. (1995) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1995.10524589 · Cited by 1,100
Mixtures of g Priors for Bayesian Variable Selection
Liang, F., Paulo, R., Molina, G., Clyde, M. A., & Berger, J. O. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000001337 · Cited by 1,089
Identification and Inference in Nonlinear Difference-in-Differences Models
Athey, S., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00668.x · Cited by 1,084
Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs
Ashenfelter, O., & Card, D. (1985) · The Review of Economics and Statistics · doi:10.2307/1924810 · Cited by 1,023
THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA
Winship, C., & Morgan, S. L. (1999) · Annual Review of Sociology · doi:10.1146/annurev.soc.25.1.659 · Cited by 890
On assessing prior distributions and Bayesian regression analysis with g-prior distributions
Zellner, A. (1986) · Medical Entomology and Zoology · Cited by 886
Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs
Ashenfelter, O., & Card, D. (1984) · National Bureau of Economic Research · doi:10.3386/w1489 · Cited by 876
DATA AUGMENTATION AND DYNAMIC LINEAR MODELS
Frühwirth‐Schnatter, S. (1994) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.1994.tb00184.x · Cited by 808
Bayesian Forecasting and Dynamic Models (2nd edn)
Wheeler, F. P. (1998) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2600494 · Cited by 671
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
A simple and efficient simulation smoother for state space time series analysis
Durbin, J. (2002) · Biometrika · doi:10.1093/biomet/89.3.603 · Cited by 617
Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
Scott, J. G., & Berger, J. O. (2010) · The Annals of Statistics · doi:10.1214/10-aos792 · Cited by 573
The simulation smoother for time series models
DE JONG, P., & SHEPHARD, N. (1995) · Biometrika · doi:10.1093/biomet/82.2.339 · Cited by 566
Heckman, J. J., & Vytlacil, E. J. (2007) · Handbook of Econometrics · doi:10.1016/s1573-4412(07)06070-9 · Cited by 548
The Practical Implementation of Bayesian Model Selection
Chipman, H., George, E. I., & McCulloch, R. E. (2001) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215540964 · Cited by 416
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2002) · National Bureau of Economic Research · doi:10.3386/w8841 · Cited by 409
Counterfactuals and Causal Inference: Methods and Principles for Social Research
Hipp, J. R. (2008) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/009430610803700409 · Cited by 394
Predicting the present with Bayesian structural time series
Scott, S. L., & Varian, H. R. (2014) · International Journal of Mathematical Modelling and Numerical Optimisation · doi:10.1504/ijmmno.2014.059942 · Cited by 393
Self-citationHansen, C. B. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2006.07.011 · Cited by 389
Asymptotic properties of a robust variance matrix estimator for panel data when is large
Hansen, C. B. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2006.10.009 · Cited by 293
Heckman, J. J., & Vytlacil, E. (2007) · RePEc: Research Papers in Economics · Cited by 228
Marketing accountability: Linking marketing actions to financial results
Stewart, D. W. (2008) · Journal of Business Research · doi:10.1016/j.jbusres.2008.02.005 · Cited by 185
On Gibbs Sampling for State Space Models
Carter, C., & Kohn, R. (1994) · Biometrika · doi:10.2307/2337125 · Cited by 172
A review of causal inference for biomedical informatics
Kleinberg, S., & Hripcsak, G. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.07.001 · Cited by 171
Data augmentation for support vector machines
Polson, N. G., & Scott, S. L. (2011) · Bayesian Analysis · doi:10.1214/11-ba601 · Cited by 169
Self-citationSeggie, S. H., Çavuşgil, E., & Phelan, S. E. (2007) · Industrial Marketing Management · doi:10.1016/j.indmarman.2006.11.001 · Cited by 162
Bayesian Analysis of Latent Threshold Dynamic Models
Nakajima, J., & West, M. (2012) · Journal of Business and Economic Statistics · doi:10.1080/07350015.2012.747847 · Cited by 155
Ataman, M. B., Mela, C. F., & van Heerde, H. J. (2008) · Marketing Science · doi:10.1287/mksc.1080.0358 · Cited by 120
Evaluating online ad campaigns in a pipeline: causal models at scale
Chan, D. X., Ge, R., Gershony, O., Hesterberg, T., & Lambert, D. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835809 · Cited by 116
Lewis, R. A., Rao, J. M., & Reiley, D. H. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963431 · Cited by 105
Creating lift versus building the base: Current trends in marketing dynamics
Leeflang, P. S. H., Bijmolt, T. H. A., van Doorn, J., Hanssens, D. M., van Heerde, H. J., Verhoef, P. C., & Wieringa, J. E. (2009) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2008.06.006 · Cited by 91
Observing the Counterfactual? The Search for Political Experiments in Nature
Robinson, G., McNulty, J. E., & Krasno, J. S. (2009) · Political Analysis · doi:10.1093/pan/mpp011 · Cited by 87
Estimating the causal effects of marketing interventions using propensity score methodologyPreprint
Rubin, D. B., & Waterman, R. P. (2006) · Cited by 67
Ghosh, J., & Clyde, M. A. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10518 · Cited by 61
Determining the optimal return on investment for an advertising campaign
Danaher, P. J., & Rust, R. T. (1996) · European Journal of Operational Research · doi:10.1016/0377-2217(95)00319-3 · Cited by 60
Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology
Rubin, D. B., & Waterman, R. P. (2006) · Statistical Science · doi:10.1214/088342306000000259 · Cited by 59
Measuring Ad Effectiveness Using Geo Experiments
Vaver, J., & Koehler, J. (2011) · Cited by 52
Self-citationRubin, D. B. (2007) · Handbook of statistics · doi:10.1016/s0169-7161(07)27002-6 · Cited by 46
Variational Bayesian mixed-effects inference for classification studies
Brodersen, K. H., Daunizeau, J., Mathys, C. D., Chumbley, J. R., Buhmann, J. M., & Stephan, K. E. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.008 · Cited by 44
Self-citationProgram evaluation with high-dimensional data
Belloni, A., Chernozhukov, V., Fernández‐Val, I., & Hansen, C. (2017) · doi:10.1920/wp.cem.2015.5515 · Cited by 43
Estimating Autocorrelations in Fixed-Effects Models
Solon, G. (1984) · National Bureau of Economic Research · doi:10.3386/t0032 · Cited by 41
Identification and Inference in Nonlinear Difference-In-Differences Models
Athey, S. C., & Imbens, G. W. (2002) · National Bureau of Economic Research · doi:10.3386/t0280 · Cited by 41
Claveau, F. (2012) · Studies in History and Philosophy of Science Part C Studies in History and Philosophy of Biological and Biomedical Sciences · doi:10.1016/j.shpsc.2012.05.004 · Cited by 39
Economic Theory and Causal Inference
Hoover, K. D. (2012) · Elsevier eBooks · doi:10.1016/b978-0-444-51676-3.50004-x · Cited by 34
Program evaluation with high-dimensional data
Belloni, A., Chernozhukov, V., Fernández‐Val, I., & Hansen, C. (2017) · doi:10.1920/wp.cem.2013.7713 · Cited by 24
A new data mining approach to estimate causal effects of policy interventions
Camillo, F., & D’Attoma, I. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.05.072 · Cited by 17
Modeling Internet firm survival using Bayesian dynamic models with time-varying coefficients
Banerjee, S., Kauffman, R. J., & Wang, B. (2006) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2006.06.004 · Cited by 15
Causal Inference and Statistical Fallacies
Cox, D. R., & Wermuth, N. (2001) · Elsevier eBooks · doi:10.1016/b0-08-043076-7/00396-x · Cited by 5
Cited by
Big Data: New Tricks for Econometrics
Varian, H. R. (2014) · Journal of Economic Perspectives · doi:10.1257/jep.28.2.3 · Cited by 1,557
Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects
Abadie, A. (2021) · Journal of Economic Literature · doi:10.1257/jel.20191450 · Cited by 1,486
“They include Bai and Ng (2019); Brodersen et al. (2015); Gobillon and Magnac (2016); Gunsilius (2020); Kennedy-Shaffe, de Gruttola, and Lipsitch (2020); Viviano and Bradic (2019); and Xu (2017), among many others. Samartsidis et al. (2019) study the performance of the canonical synthetic control estimator and related…”
BackgroundMethodGeneralized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models
Xu, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.2 · Cited by 938
Matrix Completion Methods for Causal Panel Data Models
Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2018) · doi:10.3386/w25132 · Cited by 497
Mutation Testing Advances: An Analysis and Survey
(2019) · Advances in Computers · doi:10.1016/bs.adcom.2018.03.015 · Cited by 425
Metadata
- Authors
- Brodersen, K. H., Gallusser, F., Koehler, J., Remy, N., & Scott, S. L.
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2015-03-01
- Volume
- 9
- Issue
- 1
- Pages
- 247-274
- ISSN
- 1932-6157 (ISSN-L)
- OpenAlex
- W2078639378
- Cited by
- 961
- References
- 68
- FWCI
- 34.46
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.