Sensitivity Analysis When Data Are Missing Not-at-random
The DOI matches its Crossref record.
Abstract
To the Editor: The impact of missing data on the validity of results has often been overlooked in the medical literature.1 Missing data are usually classified as ignorable (including missing completely at random [MCAR] and missing at random [MAR]), and nonignorable (missing not-at-random [MNAR]).2 It is not possible to distinguish between ignorable and nonignorable missing data using observed data. For analysis of ignorable missing data, multiple imputation—a relatively flexible and general purpose approach—is available in standard statistical software,2 and is increasingly used. In contrast, the MNAR hypothesis is rarely explored, although it has been recommended to perform sensitivity analyses through various models for the nonresponse mechanism.2 The mixture modeling principle, which assumes that the variables of interest have different distributions according to the status missing or nonmissing, is an attractive tool.3 The impact of variations in the imputation model on the overall results helps to assess their robustness. One reason for not systematically performing sensitivity analysis could be the lack of simple tools for implementation. We propose a package with an adaptation of the “mice” function from R® software to easily perform sensitivity analysis under various scenarios of nonresponse mechanisms.4,5 The algorithm MICE allows multiple imputation for data sets with mixed types of variables (continuous, binary, categorical). We propose a strategy in 3 steps: Fit an imputation model under the ignorable-missing-data hypothesis by calling the function “mice.” Modify the imputation model by specifying supplementary parameters θ as arguments for the “sens.mice” function. For binary or categorical variables, this parameter is the odds ratio comparing the odds of the modality of interest among subjects with missing value with the odds among subjects without missing value. For continuous variables, this is the difference in expected values. For standardized variables, this difference can be expressed as a coefficient of variation. Impute the missing data using the function sens.mice, resulting in a “mids” object which contains the newly imputed data sets. In the absence of historical data on the missing-data mechanism, we suggest looking for parameter values that modify the overall conclusions, ie, the test results or a point estimate outside of the original confidence interval. These values should be consistent with the targeted exposure effect and should correspond to reasonable hypotheses supported by epidemiologic evidences. As an illustrative example, we apply a sensitivity analysis supposing MNAR data on the data set “CHAIN”6 included in the package “mi.”7 We present results from 4 imputation models that relate poor mental health (as measured by a binary variable) to the self-reported viral load (taken as either a binary or a trinary variable) adjusted on 5 covariates. We assumed that nonresponders were more likely to have high viral load than responders. The magnitude of the variations was in the range of the observed odds ratio between viral load and mental health among complete cases. We compared the results with the complete cases analysis and with the multiple imputation analysis assuming MAR data (Table). The resulting odds ratio was decreased but still substantial. Estimates were robust to the explored MNAR scenarios as conclusions were not modified as compared with multiple imputation; magnitude of the effects was smaller than the assumed variations.TABLE: Example of Sensitivity Analysis on the Odds Ratio Estimating the Association Between Viral Load and Poor Mental Health Using the R Data Set “CHAIN”The “SensMice” package is provided as an eAppendix (https://links.lww.com/EDE/A455). Both the package and the reference manual are freely available at: http://lertim.fr/Members/rgiorgi/DossierPublic/fonctions-r-s/. A sensitivity analysis under different scenarios of nonresponse mechanism could be easily performed using the proposed package. It is of particular interest when the mechanism is highly suspected to be nonignorable, eg, for self-reported characteristics as psychologic disorders, quality of life, or income.8 ACKNOWLEDGMENTS We thank David Hajage and Yann De Rycke for their helpful contributions and Michel Chavance for stimulating discussions. Noémie Resseguier Biostatistics Department Institut Curie Paris, France Roch Giorgi LERTIM, EA 3283 Aix-Marseille University Marseille, France Xavier Paoletti Biostatistics Department Institut Curie Paris, France INSERM U900 Institut Curie Paris, France [email protected]
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Ang, K. K., Zhang, Q., Rosenthal, D. I., Nguyen-Tan, P. F., Sherman, E. J., Weber, R. S., Galvin, J. M., Bonner, J. A., Harris, J., El-Naggar, A. K., et al. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2013.53.5633 · Cited by 838
Goldsmith, K. A., MacKinnon, D. P., Chalder, T., White, P. D., Sharpe, M., & Pickles, A. (2018) · Psychological Methods · doi:10.1037/met0000154 · Cited by 99
van Helmondt, S. J., van der Lee, M. L., van Woezik, R. A. M., Lodder, P., & de Vries, J. (2020) · Psycho-Oncology · doi:10.1002/pon.5233 · Cited by 58
van Helmondt, S. J., Lodder, P., van Woezik, R., de Vries, J., & van der Lee, M. L. (2023) · Annals of Behavioral Medicine · doi:10.1093/abm/kaac078 · Cited by 10
Zhang, K., Xu, X., & You, H. (2023) · SSM - Population Health · doi:10.1016/j.ssmph.2023.101508 · Cited by 7
Metadata
- Authors
- Resseguier, N., Giorgi, R., & Paoletti, X.
- Venue
- Epidemiology
- Published
- 2011-03
- Volume
- 22
- Issue
- 2
- Pages
- 282
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2057270012
- Cited by
- 63
- References
- 0
- FWCI
- 2.50
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.