CoChat

Design Sensitivity and Efficiency in Observational Studies

Rosenbaum, Paul R. · 2010 · Journal of the American Statistical Association
Cited by 83

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

An observational study attempts to draw inferences about the effects caused by a treatment when subjects are not randomly assigned to treatment or control as they would be in a randomized trial. After adjustments have been made for imbalances in measured covariates, the key source of uncertainty in an observational study is the possibility that subjects were not comparable prior to treatment in terms of some unmeasured covariate, so that differing outcomes in treated and control groups are not effects caused by the treatment. A sensitivity analysis asks about the magnitude of the departure from random assignment needed to alter the qualitative conclusions of the study, and the power of a sensitivity analysis and the design sensitivity anticipate the outcome of a sensitivity analysis under an assumed model for treatment effect. Lacking theoretical guidance, we tend to select statistical methods for use in observational studies based on their efficiency in randomized experiments. This turns out to be a mistake. A highly efficient method for detecting small treatment effects in randomized experiments need not, and often does not, have the highest power in a sensitivity analysis or the largest design sensitivity. That is, the best procedure assuming the observational study is a randomized experiment need not be the best procedure under more realistic assumptions. Small effects are sensitive to small biases, and methods targeted at detecting small effects in the absence of bias may not give due emphasis to evidence that the effect is stable and not small, and hence not easily attributed to small or moderate biases. The issue is illustrated in a practical example, is explored informally using a graphical heuristic, is studied asymptotically by determining the design sensitivity for a signed rank statistic with general scores, and is evaluated in a simulation. Among robust procedures with similar Pitman efficiencies for several distributions, there are large differences in design sensitivity and hence substantial differences in the power of a sensitivity analysis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1198/jasa.2010.tm09570

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 83

Metadata

Authors
Rosenbaum, P. R.
Venue
Journal of the American Statistical Association
Published
2010-06
Volume
105
Issue
490
Pages
692-702
ISSN
  • 0162-1459 (ISSN-L)
  • 1537-274X
OpenAlex
W1988110689
Cited by
83
References
3
FWCI
5.12
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.