Panel experiments and dynamic causal effects: A finite population perspective
Open accessThe DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
In panel experiments, we randomly assign units to different interventions, measuring their outcomes, and repeating the procedure in several periods. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For a rich class of dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution and derive its finite population limiting distribution as either the sample size or the duration of the experiment increases. We develop two methods for inference: a conservative test for weak null hypotheses and an exact randomization test for sharp null hypotheses. We further analyze the finite population probability limit of linear fixed effects estimators. These commonly‐used estimators do not recover a causally interpretable estimand if there are dynamic causal effects and serial correlation in the assignments, highlighting the value of our proposed estimator.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We note that, as described in Bojinov, Rambachan and Shephard (2021), the no carryover assumption may be restrictive in many settings”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Bojinov, Rambachan and Shephard (2021)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 2×
“Combining this assumption with a further requirement that every unit receives the same version of the treatment is often referred to as the “Stable Unit Treatment Value Assumption,” in cross-sectional applications (Rubin, 1980) or the “Temporal Stable Unit Treatment Value Assumption” in time series experiments (Bojino…”
MethodDifference-in-differences with variation in treatment timing
Goodman-Bacon, A. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.03.014 · Cited by 8,046 · cited 1×
“Boryusak and Jaravel (2017), Athey and Imbens (2018), and Goodman-Bacon (2018) also provided a decomposition of the two-way fixed effects estimand in staggered designs.”
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · cited 1×
“In this section, we develop a nonparametric Horvitz and Thompson (1952) type estimator of the i, tth lag-p dynamic causal effects and derive its properties.”
MethodImpulse response analysis in nonlinear multivariate models
Koop, G., Pesaran, M. H., & Potter, S. M. (1996) · Journal of Econometrics · doi:10.1016/0304-4076(95)01753-4 · Cited by 5,049
Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects
de Chaisemartin, C., & D’Haultfœuille, X. (2020) · American Economic Review · doi:10.1257/aer.20181169 · Cited by 5,031
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293 · cited 1×
“3 Nonparametric estimation and inference We now develop a nonparametric Horvitz and Thompson (1952) type estimator of the i, t-th lag-p dynamic causal effects.”
MethodCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Martingale Limit Theory and its Application.
Loynes, R. M., Hall, P., & Heyde, C. C. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981599 · Cited by 2,685
The Production of Human Capital and the Life Cycle of Earnings
Ben-Porath, Y. (1967) · Journal of Political Economy · doi:10.1086/259291 · Cited by 2,657
Research works printed but not linked: 7
- Bojinov, 2020, Harvard Business Review, 98, 48
- Cunha, 2006, Handbook of the Economics of Education, 1, 697
- Han, 2019, Journal of Econometrics
- Imai, 2020, Political Analysis
- Kempthorne, 1955, Journal of the American Statistical Association, 50, 946
- Li, 2016, Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, 539
- Neyman, 1923, Statistical Science, 5, 465
Cited by
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027
“We note that, as described in Bojinov et al. (2021), the no carryover assumption may be restrictive in many settings — for example, if the treatment is a raise in the minimum wage and the outcome is employment, then the no carryover assumption requires that employment in period t depends only on…”
Difference-in-Differences Estimators of Intertemporal Treatment Effects
de Chaisemartin, C., & D’Haultfœuille, X. (2026) · Review of Economics and Statistics · doi:10.1162/rest_a_01414 · Cited by 740
Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: a survey
de Chaisemartin, C., & D’Haultfœuille, X. (2023) · The Econometrics Journal · doi:10.1093/ectj/utac017 · Cited by 522
Two-way fixed effects and differences-in-differences estimators with several treatments
de Chaisemartin, C., & D’Haultfœuille, X. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.105480 · Cited by 188
Singh, V. K., Nanavati, B., Kar, A. K., & Gupta, A. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10314-0 · Cited by 116
“One cannot be sure; just because creative-X performed better over creative-Y one year ago does not mean it will still perform better now (Bojinov et al., 2021).”
BackgroundInfluentialA Local Projections Approach to Difference‐in‐Differences
Dubé, A., Girardi, D., Jordà, Ò., & Taylor, A. M. (2025) · Journal of Applied Econometrics · doi:10.1002/jae.70000 · Cited by 207
Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey
de Chaisemartin, C., & D'Haultfoeuille, X. (2022) · National Bureau of Economic Research · doi:10.3386/w29734 · Cited by 70
Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A SurveyPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4034184 · Cited by 81
Design and Analysis of Switchback Experiments
Bojinov, I., Simchi‐Levi, D., & Zhao, J. (2022) · Management Science · doi:10.1287/mnsc.2022.4583 · Cited by 172
Self-citationCausal models for longitudinal and panel data: a survey
Arkhangelsky, D., & Imbens, G. (2024) · The Econometrics Journal · doi:10.1093/ectj/utae014 · Cited by 50
Difference-in-Differences for Continuous Treatments and Instruments with StayersPreprint
de Chaisemartin, C., D’Haultfœuille, X., Pasquier, F., & Vázquez-Baré, G. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4011782 · Cited by 64
“Finally, note that because it imposes parallel trends conditional on prior treatment, Assumption 1 is connected to the sequential ignorability assumption, another commonly-used identifying assumption in panel data models (see, e.g., Robins, 1986; Bojinov et al., 2021).”
MethodTwo-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A SurveyPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4021785 · Cited by 185
Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A SurveyPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3980758 · Cited by 62
“This excludes papers that have assumed randomized treatment timing (see, e.g., Athey and Imbens, 2022; Roth and Sant’Anna, 2021) or sequential treatment randomization (see, e.g., Bojinov et al., 2021), rather than parallel trends.”
BackgroundOnline Experimentation: Benefits, Operational and Methodological Challenges, and Scaling Guide
Bojinov, I., & Gupta, S. (2022) · Harvard Data Science Review · doi:10.1162/99608f92.a579756e · Cited by 30
Self-citationTwo-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey
de Chaisemartin, C., & D'Haultfoeuille, X. (2022) · doi:10.3386/w29691 · Cited by 31
Design‐robust two‐way‐fixed‐effects regression for panel data
Arkhangelsky, D., Imbens, G. W., Lei, L., & Luo, X. (2024) · Quantitative Economics · doi:10.3982/qe1962 · Cited by 66
Combining counterfactual outcomes and ARIMA models for policy evaluation
Menchetti, F., Cipollini, F., & Mealli, F. (2022) · Econometrics Journal · doi:10.1093/ectj/utac024 · Cited by 55
Doubly robust identification for causal panel data models
Arkhangelsky, D., & Imbens, G. W. (2022) · Econometrics Journal · doi:10.1093/ectj/utac019 · Cited by 51
Two-way Fixed Effects and Differences-in-Differences Estimators with Several Treatments
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · National Bureau of Economic Research · doi:10.3386/w30564 · Cited by 32
Multi-armed Bandit Experimental Design: Online Decision-Making and Adaptive Inference
Simchi‐Levi, D., & Wang, C. (2024) · Management Science · doi:10.1287/mnsc.2023.00492 · Cited by 23
Causal Models for Longitudinal and Panel Data: A Survey
Arkhangelsky, D., & Imbens, G. W. (2023) · National Bureau of Economic Research · doi:10.3386/w31942 · Cited by 28
Design and Analysis of Switchback ExperimentsPreprint
Bojinov, I., Simchi‐Levi, D., & Zhao, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3684168 · Cited by 15
“(2017), Chin (2018), Bojinov et al. (2021), Han et al. (2021). Note that, Theorem 3 does not require Var(τ̂m) to converge as n→+∞.”
InfluentialSelf-citationTwo-Way Fixed Effects and Differences-in-Differences Estimators with Several TreatmentsPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4037008 · Cited by 14
Design and Analysis of Panel Experiments with Spatial and Temporal InterferencePreprint
Ni, T., Bojinov, I., & Zhao, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4466598 · Cited by 27
Self-citationMenchetti, F., & Bojinov, I. (2022) · The Annals of Applied Statistics · doi:10.1214/21-aoas1498 · Cited by 19
“In practice, authors often sidestep the issue by aggregating units that are likely to interfere with each other, generating a single-treated time series that now satisfies the no-interference assumption (Bojinov, Chen and Liu (2020)).”
BackgroundInfluentialSelf-citationTwo-way Fixed Effects Regressions with Several Treatments
de Chaisemartin, C., & d'Haultfoeuille, X. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3751060 · Cited by 10
Design-Based Uncertainty for Quasi-Experiments
Rambachan, A., & Roth, J. (2025) · Journal of the American Statistical Association · doi:10.1080/01621459.2025.2526700 · Cited by 26
“…and Middleton (2013), Li and Ding (2017), Kang, Peck, and Keele (2018), Bojinov and Shephard (2019), Wu and Ding (2021) in statistics, and Abadie et al. (2020), Xu (2021), Bojinov, Rambachan, and Shep-hard (2021), Roth and Sant’Anna (2023), Abadie et al. (2023) in econometrics, among many others.”
Self-citationChoudhury, P., Lane, J. N., & Bojinov, I. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3855788 · Cited by 21
Self-citationDynamic regression discontinuity under treatment effect heterogeneity
Hsu, Y.-C., & Shen, S. (2024) · Quantitative Economics · doi:10.3982/qe2150 · Cited by 16
Causal Inference with Differences-in-Differences: Credible Answers to Hard QuestionsPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4487202 · Cited by 32
Wu, X., Weinberger, K. R., Wellenius, G. A., Dominici, F., & Braun, D. (2023) · Biostatistics · doi:10.1093/biostatistics/kxad002 · Cited by 15
Population interference in panel experiments
Han, K., Basse, G., & Bojinov, I. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.105565 · Cited by 17
Self-citationTwo-Way Fixed Effects and Differences-in-Differences Estimators with Several TreatmentsPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4249593 · Cited by 7
(2021) · RePEc: Research Papers in Economics · Cited by 6
Pricing Experimental Design: Causal Effect, Expected Revenue and Tail Risk
Simchi‐Levi, D., & Wang, C. (2025) · Management Science · doi:10.1287/mnsc.2023.03209 · Cited by 11
Randomization inference when N equals one
Liang, T., & Recht, B. (2025) · Biometrika · doi:10.1093/biomet/asaf013 · Cited by 5
Counterfactual inference in sequential experiments
Dwivedi, R., Tian, K., Tomkins, S., Klasnja, P., Murphy, S. A., & Shah, D. (2025) · The Annals of Statistics · doi:10.1214/25-aos2519 · Cited by 22
“[47, 14] provide strategies to estimate average treatment effects for non-stationary settings that include multiple units within an experiment and allow for non-i.”
Diao, G., Shang, D., & Wang, D. (2024) · Forests · doi:10.3390/f16010030 · Cited by 4
“], , a and predicted prices, where a greater disparity signifies a more substantial impact of the policy [41].”
Menchetti, F., & Bojinov, I. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3707723 · Cited by 7
“As a result, the unit prices should not be part of our models; nevertheless, they are important drivers of sales, especially during promotions (Neslin et al., 1985; Blattberg et al., 1995; Pauwels et al., 2002).”
BackgroundSelf-citationNonstationary Experimental Design Under Structured Trends
Simchi‐Levi, D., Wang, C., & Zheng, Z. (2026) · Management Science · doi:10.1287/mnsc.2023.03329 · Cited by 3
Detecting Interference in Online Controlled Experiments with Increasing Allocation
Han, K. W., Li, S., Mao, J., & Wu, H. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599308 · Cited by 7
“Bojinov et al. [13] and Han et al. [21] further investigate the problem in panel experiments.”
BackgroundEstimating the causal effect of an intervention in a time series setting: the C-ARIMA approachPreprint
Menchetti, F., Cipollini, F., & Mealli, F. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2103.06740 · Cited by 3
“…define and estimate causal effects under the RCM, including network data structures (VanderWeele, 2010; Forastiere et al., 2020; Noirjean et al., 2020), time series (Robins, 1986; Robins et al., 1999; Bojinov and Shephard, 2019) and panel data (Rambachan and Shephard, 2019; Bojinov et al., 2020).”
BackgroundDesign-Based Confidence Sequences: A General Approach to Risk Mitigation in Panel Experiments
Ham, D. W., Lindon, M., Tingley, M. P., & Bojinov, I. (2026) · Management Science · doi:10.1287/mnsc.2024.08235 · Cited by 23
“In most applied settings, there are often strong carryover effects such that the potential outcome is not only a function of its current treatment assignment but of the whole treatment assignment path (Bojinov and Shephard, 2019; Bojinov, Rambachan and Shephard, 2021).”
InfluentialSelf-citationCausal Estimation of User Learning in Personalized Systems
Munro, E., Jones, D., Brennan, J., Nelet, R., Mirrokni, V., & Pouget-Abadie, J. (2023) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3580507.3597702 · Cited by 2
“One strand of the literature addresses issues caused by interference, when individuals interact with one another through an equilibrium, a network, or a market platform [Eckles et al. 2017; Fradkin et al. 2021]. There is a large literature on cluster-randomized designs, see Hudgens and Halloran [2008] for a general cl…”
BackgroundInfluentialUnveiling the Unobservable: Causal Inference on Multiple Derived Outcomes
Qiu, Y., Sun, J., & Zhou, X. (2023) · Journal of the American Statistical Association · doi:10.1080/01621459.2023.2252135 · Cited by 4
“The proposed procedure can be extended to dynamic treatment effects over time as well, where the subject level potential outcomes Y i ( d , t ) and the average treatment effects τ ( t ) = E { Y i ( 1, t ) − Y i ( 0, t ) } are functions of time (Bojinov, Rambachan, and Shephard 2021).”
Two-Way Fixed Effects and Differences-in-Differences Estimators with Several TreatmentsPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4038668 · Cited by 4
Not All Differences-in-Differences Are Equally Compatible With Outcome-Based Selection ModelsPreprint
de Chaisemartin, C., & D’Haultfœuille, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4235636 · Cited by 1
Dynamic Causal Effects in Econometrics with a Focus on the Nonparametric Method: A Review PaperPreprint
Mahdavi, P., & Ehsani, M. A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4184770 · Cited by 3
Instrumental Variables with Multiple Time PeriodsPreprint
Picchetti, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4832234
Instrumental variable regression with varying-intensity repeated treatments
Choi, J., Seong, D., & Shen, S. (2026) · Journal of Econometrics · doi:10.1016/j.jeconom.2026.106305
Multiple conditional randomization tests for lagged and spillover treatment effects
Zhang, Y., & Zhao, Q. (2024) · Biometrika · doi:10.1093/biomet/asae042 · Cited by 9
Take the Load Off: Time and Technology as Determinants of Electricity Demand ResponsePreprint
Bailey, M., Brown, D. P., Shaffer, B., & Wolak, F. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5268628 · Cited by 2
Take the Load Off: Effort and Technology as Determinants of Electricity Demand Response
Bailey, M., Brown, D. P., Shaffer, B., & Wolak, F. A. (2025) · National Bureau of Economic Research · doi:10.3386/w33836 · Cited by 3
Cai, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5357637 · Cited by 1
Conditional relationships in dynamic models
Warner, Z., Kamp, G. N. V., & Jordan, S. (2026) · Political Science Research and Methods · doi:10.1017/psrm.2026.10087
“But scholars are increasingly interested in performing experiments with time series data (Bojinov and Shephard, 2019; Bojinov et al. , 2021).”
Robust evaluation of public policies to heterogeneous effects
Pasquier, F. (2024) · theses.fr (ABES) · doi:10.70675/91cfab5az188ez409dz9879zc5005fcf2fc6
Rolling Difference-in-Differences with Reversible Treatment Paths and Moderating EffectsPreprint
Lee, S. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6844479
Metadata
- Authors
- Bojinov, I., Rambachan, A., & Shephard, N.
- Affiliations
- Harvard University (US)
- Venue
- Quantitative Economics
- Publisher
- The Econometric Society
- Published
- 2021
- Volume
- 12
- Issue
- 4
- Pages
- 1171-1196
- ISSN
- 1759-7323 (ISSN-L)
- OpenAlex
- W3012766575
- Cited by
- 60
- References
- 66
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 8 · 2023: 10 · 2022: 19 · 2021: 6
- FWCI
- 8.04
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Quantitative Economics (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- Econstor (Econstor) (submittedVersion, cc-by-nc)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1745303)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- finite population inference, potential outcomes, nonparametric estimators, permutation tests
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.