CoChat

Meta-research: Evaluation and Improvement of Research Methods and Practices

Open access

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Abstract

As the scientific enterprise has grown in size and diversity, we need empirical evidence on the research process to test and apply interventions that make it more efficient and its results more reliable. Meta-research is an evolving scientific discipline that aims to evaluate and improve research practices. It includes thematic areas of methods, reporting, reproducibility, evaluation, and incentives (how to do, report, verify, correct, and reward science). Much work is already done in this growing field, but efforts to-date are fragmented. We provide a map of ongoing efforts and discuss plans for connecting the multiple meta-research efforts across science worldwide.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pbio.1002264

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Number of meta-research–related publications registered by the Scopus database between January 1 and May 16 2015, by country of corresponding author and by affiliation of any coauthor. Countries are attributed based on corresponding or first author address (legend, from light yellow to red, respectively, to 1–5, 5–10, 10–20, 20–50, 50–230 publications). Blue dots indicate the 100 institutions most frequently listed amongst coauthors’ addresses. Dot size is proportional to number of papers (range: 2–37). Papers were selected for inclusion from an initial list of 1,422 papers retrieved from the Scopus database using a combination of search terms aimed at capturing the core areas described in Table 1. Of the 851 records selected for inclusion, country or affiliation data could not be retrieved for 102 Scopus records, which therefore are not included in the map. Search terms, literature lists, and further details are available at metrics.stanford.edu. The map and plots therein were generated anew, using the packages ggmap and ggplot2 implemented in the open source statistical software R. Image Credit: Daniele Fanelli
Fig 1 Number of meta-research–related publications registered by the Scopus database between January 1 and May 16 2015, by country of corresponding author and by affiliation of any coauthor. Countries are attributed based on corresponding or first author address (legend, from light yellow to red, respectively, to 1–5, 5–10, 10–20, 20–50, 50–230 publications). Blue dots indicate the 100 institutions most frequently listed amongst coauthors’ addresses. Dot size is proportional to number of papers (range: 2–37). Papers were selected for inclusion from an initial list of 1,422 papers retrieved from the Scopus database using a combination of search terms aimed at capturing the core areas described in Table 1. Of the 851 records selected for inclusion, country or affiliation data could not be retrieved for 102 Scopus records, which therefore are not included in the map. Search terms, literature lists, and further details are available at metrics.stanford.edu. The map and plots therein were generated anew, using the packages ggmap and ggplot2 implemented in the open source statistical software R. Image Credit: Daniele FanelliCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Cited by

Items 401–500 of 545

Metadata

Authors
Ioannidis, J. P. A., Fanelli, D., Dunne, D. D., & Goodman, S. N.
Affiliations
Stanford University (US)
Corresponding author
Ioannidis, John P. A.
Venue
PLOS Biology
Published
2015-10-02
Volume
13
Issue
10
Pages
e1002264
ISSN
  • 1544-9173 (ISSN-L)
  • 1545-7885
OpenAlex
W1767553509
Cited by
495
References
10
Citations by year
2026: 36 · 2025: 48 · 2024: 134 · 2023: 50 · 2022: 37 · 2021: 40 · 2020: 39 · 2019: 28 · 2018: 29 · 2017: 30 · 2016: 22 · 2015: 1
FWCI
24.65
Type
journal article
Language
en
Open access
Locations
  • PLoS Biology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Laura and John Arnold Foundation
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
Keywords
meta-research, research practices, research methodology, reproducibility, research reporting, research evaluation, research incentives, research reliability, research efficiency
MeSH terms
Animals, Humans, Meta-Analysis as Topic, Guidelines as Topic, Evidence-Based Medicine, Biomedical Research, Empirical Research, Internationality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.