CoChat

Packaging Data Analytical Work Reproducibly Using R (and Friends)

Marwick, Ben, Boettiger, Carl, Mullen, Lincoln · 2018 · The American Statistician
Cited by 108

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

Computers are a central tool in the research process, enabling complex and large-scale data analysis. As computer-based research has increased in complexity, so have the challenges of ensuring that this research is reproducible. To address this challenge, we review the concept of the research compendium as a solution for providing a standard and easily recognizable way for organizing the digital materials of a research project to enable other researchers to inspect, reproduce, and extend the research. We investigate how the structure and tooling of software packages of the R programming language are being used to produce research compendia in a variety of disciplines. We also describe how software engineering tools and services are being used by researchers to streamline working with research compendia. Using real-world examples, we show how researchers can improve the reproducibility of their work using research compendia based on R packages and related tools.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/00031305.2017.1375986

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • null

    null (2016) · Philosophy Study · doi:10.17265/2159-5313/2016.09.003 · Cited by 28,317

  • Estimating the reproducibility of psychological science

    Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 1×

    “Long considered an axiom of science, the reproducibility of scientific research has recently come under scrutiny after some highly publicized failures to reproduce biomedical studies (Begley and Ellis 2012) and psychological experiments (Open Science Collaboration et al. 2015).”
    Background
  • A manifesto for reproducible science

    Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397

  • Raise standards for preclinical cancer research

    Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×

    “Long considered an axiom of science, the reproducibility of scientific research has recently come under scrutiny after some highly publicized failures to reproduce biomedical studies (Begley and Ellis 2012) and psychological experiments (Open Science Collaboration et al. 2015).”
    Background
  • Promoting an open research culture

    Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 1×

    “Authors submitting to Political Analysis and the Quarterly Journal of Political Science are required to provide their code for review, with editors reproducing the reported analyses prior to publication (Nosek et al. 2015).”
  • What does research reproducibility mean?

    Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293 · cited 1×

    “This has resulted in extensive discussion of how to define reproducibility, and how to enhance it across diverse research contexts (Stodden et al. 2016; Goodman, Fanelli, and Ioannidis 2016; Marwick 2017).”
    Background
  • An introduction to Docker for reproducible research

    Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059

    Self-citation
  • Sharing Detailed Research Data Is Associated with Increased Citation Rate

    Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881

  • Ten Simple Rules for Reproducible Computational Research

    Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880 · cited 2×

    “…need in the proliferation of commentary and how-to publications across different areas of science (Wilson et al. 2014; Stodden and Miguez 2014; Stanisic and Legrand 2014; LeVeque, Mitchell, and Stodden 2012; Donoho 2010; Sandve et al. 2013; Rokem, Marwick, and Staneva 2017; Munafo et al. 2017).”
    Background
  • Best Practices for Scientific Computing

    Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723 · cited 1×

    “We see evidence of this need in the proliferation of commentary and how-to publications across different areas of science (Wilson et al. 2014; Stodden and Miguez 2014; Stanisic and Legrand 2014; LeVeque, Mitchell, and Stodden 2012; Donoho 2010; Sandve et al. 2013; Rokem, Marwick, and Staneva 2017;…”
    Background

Show all 12 references

Cited by

See all 108

Metadata

Authors
Marwick, B., Boettiger, C., & Mullen, L.
Venue
The American Statistician
Published
2018-01-02
Volume
72
Issue
1
Pages
80-88
ISSN
  • 0003-1305 (ISSN-L)
  • 1537-2731
OpenAlex
W4245161709
Cited by
108
References
12
FWCI
15.76
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Packaging Data Analytical Work Reproducibly Using R (and Friends). https://app.cochat.ai/papers/10.1080/00031305.2017.1375986
The paper
Marwick, B., Boettiger, C., & Mullen, L. (2018). Packaging Data Analytical Work Reproducibly Using R (and Friends). The American Statistician. https://doi.org/10.1080/00031305.2017.1375986

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.