CoChat
Sign Up
Cited by 55Advanced Causal Inference TechniquesOptimal Experimental Design MethodsComplex Network Analysis Techniques

Design and Analysis of Experiments in Networks: Reducing Bias from Interference

Open access
Eckles, Dean, Karrer, Brian, Ugander, Johan·2017·Journal of Causal Inference
DOI 10.1515/jci-2015-0021
Open this paper in CoChat — free accountOpen PDF

Abstract

Abstract Estimating the effects of interventions in networks is complicated due to interference, such that the outcomes for one experimental unit may depend on the treatment assignments of other units. Familiar statistical formalism, experimental designs, and analysis methods assume the absence of this interference, and result in biased estimates of causal effects when it exists. While some assumptions can lead to unbiased estimates, these assumptions are generally unrealistic in the context of a network and often amount to assuming away the interference. In this work, we evaluate methods for designing and analyzing randomized experiments under minimal, realistic assumptions compatible with broad interference, where the aim is to reduce bias and possibly overall error in estimates of average effects of a global treatment. In design , we consider the ability to perform random assignment to treatments that is correlated in the network, such as through graph cluster randomization. In analysis , we consider incorporating information about the treatment assignment of network neighbors. We prove sufficient conditions for bias reduction through both design and analysis in the presence of potentially global interference; these conditions also give lower bounds on treatment effects. Through simulations of the entire process of experimentation in networks, we measure the performance of these methods under varied network structure and varied social behaviors, finding substantial bias reductions and, despite a bias–variance tradeoff, error reductions. These improvements are largest for networks with more clustering and data generating processes with both stronger direct effects of the treatment and stronger interactions between units.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1515/jci-2015-0021

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Technical limitations include difficulty in performing cluster-randomized experiments testing for social influence and inability to collect the full feed inventory for a given user.

    “Other technical limitations of our approach include the difficulty in performing cluster-randomized experiments testing for social influence (54), and the inability to collect the full feed inventory for a given user.”

    From Reranking partisan animosity in algorithmic social media feeds alters affective polarization (2025) · cites this paper as [54]

    The Design section describes a limitation of graph-cluster randomization: some clustering methods can increase variance too much to justify their bias reduction. The Discussion also cautions that the clustered designs examined may not apply when the goal is to estimate peer effects themselves. These points support methodological difficulty, but the paper does not establish that such experiments are technically difficult to perform. The feed-inventory limitation is not attached to this citation. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Toulis P, Edward K. Estimation of Causal Peer Inflluence E昀ffects Proceedings of The 30th International Conference on Machine Learning. J Mach Learn Res W&CP. 2013;28(3):1489-1497.
  2. Cox DR. Planning of Experiments. Hoboken, New Jersey: Wiley, 1958.
  3. Aronow P, Samii C. Estimating average causal e昀ffects under general interference (Working Paper). 2014. Available at http://arxiv.org/abs/1305.6156. · arXiv
  4. Jackson MO. Social and Economic Networks. Princeton, NJ, USA: Princeton University Press, 2008.
  5. Ugander J, Karrer B, Backstrom L, Marlow C. 2011. Available at http://arxiv.org/abs/1111.4503. · arXiv

Cited by

Sorted by citations

Related

Authors
Eckles, D., Karrer, B., & Ugander, J.
Affiliations
  • Massachusetts Institute of Technology (US)
  • Meta (United States) (US)
  • Stanford University (US)
ORCID
Venue
Journal of Causal Inference
Publisher
Walter de Gruyter GmbH
Published
2016-02-04
Volume
5
Issue
1
ISSN
  • 2193-3677 (ISSN-L)
  • 2193-3685
DOI
10.1515/jci-2015-0021
OpenAlex
W1888549093
Cited by
49
References
42
Citations by year
2025: 3 · 2024: 4 · 2023: 6 · 2022: 2 · 2021: 3 · 2020: 6 · 2019: 5 · 2018: 3 · 2017: 6 · 2016: 7 · 2015: 2 · 2014: 2
FWCI
7.09
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Causal Inference (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Johns Hopkins University
  • Carnegie Mellon University
  • York University
  • University of California, Davis
  • Stanford Graduate School of Business
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
randomized experiments, bias mitigation, treatment allocation, causal effect estimation, bias-variance trade-off, network clustering, social behavior

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review