Packaging Data Analytical Work Reproducibly Using R (and Friends)
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Computers are a central tool in the research process, enabling complex and large-scale data analysis. As computer-based research has increased in complexity, so have the challenges of ensuring that this research is reproducible. To address this challenge, we review the concept of the research compendium as a solution for providing a standard and easily recognizable way for organizing the digital materials of a research project to enable other researchers to inspect, reproduce, and extend the research. We investigate how the structure and tooling of software packages of the R programming language are being used to produce research compendia in a variety of disciplines. We also describe how software engineering tools and services are being used by researchers to streamline working with research compendia. Using real-world examples, we show how researchers can improve the reproducibility of their work using research compendia based on R packages and related tools.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
null (2016) · Philosophy Study · doi:10.17265/2159-5313/2016.09.003 · Cited by 28,317
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 1×
“Long considered an axiom of science, the reproducibility of scientific research has recently come under scrutiny after some highly publicized failures to reproduce biomedical studies (Begley and Ellis 2012) and psychological experiments (Open Science Collaboration et al. 2015).”
BackgroundA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Long considered an axiom of science, the reproducibility of scientific research has recently come under scrutiny after some highly publicized failures to reproduce biomedical studies (Begley and Ellis 2012) and psychological experiments (Open Science Collaboration et al. 2015).”
BackgroundPromoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 1×
“Authors submitting to Political Analysis and the Quarterly Journal of Political Science are required to provide their code for review, with editors reproducing the reported analyses prior to publication (Nosek et al. 2015).”
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293 · cited 1×
“This has resulted in extensive discussion of how to define reproducibility, and how to enhance it across diverse research contexts (Stodden et al. 2016; Goodman, Fanelli, and Ioannidis 2016; Marwick 2017).”
BackgroundAn introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059
Self-citationSharing Detailed Research Data Is Associated with Increased Citation Rate
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880 · cited 2×
“…need in the proliferation of commentary and how-to publications across different areas of science (Wilson et al. 2014; Stodden and Miguez 2014; Stanisic and Legrand 2014; LeVeque, Mitchell, and Stodden 2012; Donoho 2010; Sandve et al. 2013; Rokem, Marwick, and Staneva 2017; Munafo et al. 2017).”
BackgroundBest Practices for Scientific Computing
Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723 · cited 1×
“We see evidence of this need in the proliferation of commentary and how-to publications across different areas of science (Wilson et al. 2014; Stodden and Miguez 2014; Stanisic and Legrand 2014; LeVeque, Mitchell, and Stodden 2012; Donoho 2010; Sandve et al. 2013; Rokem, Marwick, and Staneva 2017;…”
Background
Cited by
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47
Ten simple rules for building and maintaining a responsible data science workflow
Stoudt, S., Jernite, Y., Marshall, B., Marwick, B., Sharan, M., Whitaker, K., & Danchev, V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012232 · Cited by 11
Self-citation
Metadata
- Authors
- Marwick, B., Boettiger, C., & Mullen, L.
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2018-01-02
- Volume
- 72
- Issue
- 1
- Pages
- 80-88
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W4245161709
- Cited by
- 108
- References
- 12
- FWCI
- 15.76
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Packaging Data Analytical Work Reproducibly Using R (and Friends). https://app.cochat.ai/papers/10.1080/00031305.2017.1375986
- The paper
- Marwick, B., Boettiger, C., & Mullen, L. (2018). Packaging Data Analytical Work Reproducibly Using R (and Friends). The American Statistician. https://doi.org/10.1080/00031305.2017.1375986
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.