The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Systematic benchmarking of omics computational tools
Open accessAbstract
Computational omics methods packaged as software have become essential to modern biological research. The increasing dependence of scientists on these powerful software tools creates a need for systematic assessment of these methods, known as benchmarking. Adopting a standardized benchmarking practice could help researchers who use omics data to better leverage recent technological innovations. Our review summarizes benchmarking practices from 25 recent studies and discusses the challenges, advantages, and limitations of benchmarking across various domains of biology. We also propose principles that can make computational biology benchmarking studies more sustainable and reproducible, ultimately increasing the transparency of biomedical data and results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
A framework for human microbiome research
The Human Microbiome Project Consortium (2012) · Nature · doi:10.1038/nature11209 · Cited by 2,650
SEQC/MAQC-III Consortium (2014) · Nature Biotechnology · doi:10.1038/nbt.2957 · Cited by 983
A large‐scale experiment to assess protein structure prediction methods
Moult, J., Pedersen, J. T., Judson, R., & Fidelis, K. (1995) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.340230303 · Cited by 555
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369
Comprehensive benchmarking and ensemble approaches for metagenomic classifiers
McIntyre, A. B. R., Ounit, R., Afshinnekoo, E., Prill, R. J., Hénaff, E., Alexander, N., Minot, S. S., Danko, D., Foox, J., Ahsanuddin, S., et al. (2017) · Genome Biology · doi:10.1186/s13059-017-1299-7 · Cited by 342
Nookaew, I., Papini, M., Pornputtapong, N., Scalcinati, G., Fagerberg, L., Uhlén, M., & Nielsen, J. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks804 · Cited by 309
Thompson, J. D., Linard, B., Lecompte, O., & Poch, O. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018093 · Cited by 242
Cited by
Eleven grand challenges in single-cell data science
Lähnemann, D., Köster, J., Szczurek, E., McCarthy, D. J., Hicks, S. C., Robinson, M. D., Vallejos, C. A., Campbell, K. R., Beerenwinkel, N., Mahfouz, A., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1926-6 · Cited by 1,327
Self-citationAn R package for an integrated evaluation of statistical approaches to cancer incidence projection
Knoll, M., Furkel, J., Debus, J., Abdollahi, A., Karch, A., & Stock, C. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01133-5 · Cited by 368
State of the field in multi-omics research: from computational needs to data mining and sharing
Krassowski, M., Das, V., Sahu, S. K., & Misra, B. B. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.610798 · Cited by 351
Towards FAIR Principles for Research Software
Lamprecht, A.-L., Garcia, L., Kuzak, M., Martinez, C., Arcila, R., Martin Del Pico, E., Dominguez Del Angel, V., van de Sandt, S., Ison, J., Martinez, P. A., et al. (2020) · Data Science · doi:10.3233/ds-190026 · Cited by 322
FunRich enables enrichment analysis of OMICs datasets
Fonseka, P., Pathan, M., Chitti, S. V., Kang, T., & Mathivanan, S. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2020.166747 · Cited by 298
Metadata
- Authors
- Mangul, S., Martin, L. S., Hill, B. L., Lam, A. K.-M., Distler, M. G., Zelikovsky, A., Eskin, E., & Flint, J.
- Venue
- Nature Communications
- Published
- 2019-03-27
- Volume
- 10
- Issue
- 1
- Pages
- 1393-1393
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W2922985761
- Cited by
- 164
- References
- 12
- FWCI
- 28.60
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic benchmarking of omics computational tools. https://app.cochat.ai/papers/10.1038/s41467-019-09406-4
- The paper
- Mangul, S., Martin, L. S., Hill, B. L., Lam, A. K.-M., Distler, M. G., Zelikovsky, A., Eskin, E., & Flint, J. (2019). Systematic benchmarking of omics computational tools. Nature Communications. https://doi.org/10.1038/s41467-019-09406-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.