CoChat

Panel experiments and dynamic causal effects: A finite population perspective

Open access
Bojinov, Iavor, Rambachan, Ashesh, Shephard, Neil · 2021 · Quantitative Economics

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

Abstract

In panel experiments, we randomly assign units to different interventions, measuring their outcomes, and repeating the procedure in several periods. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For a rich class of dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution and derive its finite population limiting distribution as either the sample size or the duration of the experiment increases. We develop two methods for inference: a conservative test for weak null hypotheses and an exact randomization test for sharp null hypotheses. We further analyze the finite population probability limit of linear fixed effects estimators. These commonly‐used estimators do not recover a causally interpretable estimand if there are dynamic causal effects and serial correlation in the assignments, highlighting the value of our proposed estimator.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3982/qe1744

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Figures

Image 1
CC-BY-NC · From the PDF
F igure 1. Simulated randomization distribution for ˆ ¯ τ( 1 commaori 0 ; 0 ) under different choices of the parameter φ (defined in equation (1)) and treatment probability p(w) . The rows index the parameter φ ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The columns index the treatment probability p(w) ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . Panel (a) plots the simulated randomization distribution with normally distributed errors ϵicommaorit ∼ N( 0 commaori 1 ) and N = 100 , T = 10 . Panel (b) plots the simulated randomization distribution with Cauchy distribution errors ϵicommaorit ∼ Cauchy and N = 500 , T = 10 . Results are computed over 5,000 simulations.
F igure 1. Simulated randomization distribution for ˆ ¯ τ( 1 commaori 0 ; 0 ) under different choices of the parameter φ (defined in equation (1)) and treatment probability p(w) . The rows index the parameter φ ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The columns index the treatment probability p(w) ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . Panel (a) plots the simulated randomization distribution with normally distributed errors ϵicommaorit ∼ N( 0 commaori 1 ) and N = 100 , T = 10 . Panel (b) plots the simulated randomization distribution with Cauchy distribution errors ϵicommaorit ∼ Cauchy and N = 500 , T = 10 . Results are computed over 5,000 simulations.CC-BY-NC · From the PDF
F igure 2. Rejection probabilities for a test of the null hypothesis H 0 : ¯ τ( 1 commaori 0 ; 0 ) = 0 and H 0 : ¯ τ † ( 1 commaori 0 ; 1 ) = 0 as the parameter β varies under different choices of the parameter φ and treatment probability p(w) . The rejection rate curve against H 0 : ¯ τ( 1 commaori 0 ; 0 ) = 0 is plotted by the circles and the rejection rate curve against H 0 : ¯ τ † ( 1 commaori 0 ; 1 ) = 0 is plotted by the triangles. The rows index the parameter φ ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The columns index the treatment probability p(w) ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The simulations are conducted with normally distributed errors ϵicommaorit ∼ N( 0 commaori 1 ) and N = 100 , T = 10 . Results are averaged over 5000 simulations.
F igure 2. Rejection probabilities for a test of the null hypothesis H 0 : ¯ τ( 1 commaori 0 ; 0 ) = 0 and H 0 : ¯ τ † ( 1 commaori 0 ; 1 ) = 0 as the parameter β varies under different choices of the parameter φ and treatment probability p(w) . The rejection rate curve against H 0 : ¯ τ( 1 commaori 0 ; 0 ) = 0 is plotted by the circles and the rejection rate curve against H 0 : ¯ τ † ( 1 commaori 0 ; 1 ) = 0 is plotted by the triangles. The rows index the parameter φ ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The columns index the treatment probability p(w) ∈ { 0 periodori 25 commaori 0 periodori 5 commaori 0 periodori 75 } . The simulations are conducted with normally distributed errors ϵicommaorit ∼ N( 0 commaori 1 ) and N = 100 , T = 10 . Results are averaged over 5000 simulations.CC-BY-NC · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 68 references

Research works printed but not linked: 7

  1. Bojinov, 2020, Harvard Business Review, 98, 48
  2. Cunha, 2006, Handbook of the Economics of Education, 1, 697
  3. Han, 2019, Journal of Econometrics
  4. Imai, 2020, Political Analysis
  5. Kempthorne, 1955, Journal of the American Statistical Association, 50, 946
  6. Li, 2016, Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, 539
  7. Neyman, 1923, Statistical Science, 5, 465

Cited by

Metadata

Authors
Bojinov, I., Rambachan, A., & Shephard, N.
Affiliations
Harvard University (US)
Venue
Quantitative Economics
Published
2021
Volume
12
Issue
4
Pages
1171-1196
ISSN
1759-7323 (ISSN-L)
OpenAlex
W3012766575
Cited by
60
References
66
Citations by year
2026: 5 · 2025: 8 · 2024: 8 · 2023: 10 · 2022: 19 · 2021: 6
FWCI
8.04
Type
journal article
Language
en
Open access
Locations
  • Quantitative Economics (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • Econstor (Econstor) (submittedVersion, cc-by-nc)
  • arXiv (Cornell University)
Funders
National Science Foundation (1745303)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
  • Social Sciences › Public Administration › Evaluation and Performance Assessment
Keywords
finite population inference, potential outcomes, nonparametric estimators, permutation tests

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.