The DOI matches its Crossref record; 75 of the 76 references it cites were found in a registry or an index.
Reproducible and replicable pain research: a critical review
Open accessAbstract
Recently, the degree to which scientific publications provide a reliable source of information has come under intense scrutiny. Reports suggest that a substantial amount of published literature is likely to be biased,28 distorted,27,53 and nonreproducible.7,50 This cuts across basic,10,55 preclinical,1,16 and clinical research.4,50 Although controversial, some have claimed that nonreproducible preclinical research consumes $28 billion/year (USD)16 and that 85% of biomedical research resources are wasted on biased research.35 The response from the scientific and policy community has been to identify common practices that contribute to the problem and develop methods to counteract them.47 This matter is very relevant to the pain field.1 Understanding the causes of nonreproducible and nonreplicable research and its ultimate impact on how we prevent and treat pain should assist pain researchers to improve the reproducibility and replicability of their work. The distinction between reproducibility and replicability is presented in Figure 1 and defined elsewhere.51 This article aims to: (1) outline the drivers of nonreproducible and nonreplicable research using examples from pain sciences and broader research fields; and (2) provide an overview of potential solutions and practices that could improve the reproducibility and replicability of pain research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“we submitted our protocol prior to data collection and remained transparent in all reporting.56”
Citations
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 5×
“3 Researcher degrees of freedom The term ‘researcher degrees of freedom’ refers to the decisions made by a researcher from project conception to dissemination, all of which may shift the outcome and impact of the project, usually towards false positive discoveries [10].”
BackgroundInfluentialSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Cohen, J. (1992) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10768783 · Cited by 5,870
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Reports suggest that a substantial amount of published literature is likely to be biased [28], distorted [27,53], and non-reproducible [7,50].”
BackgroundA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Research works printed but not linked: 2
- Gewandter, 2017, Checklist for the preparation and review of pain clinical trial publications, PAIN Rep, 0, 1
- Moseley, 2013, Finding the love between scientists and clinicians—a response to Dr Butler on noijam, BodyinMind
Cited by
Sorted by citations
Vægter, H. B., & Jones, M. D. (2020) · PAIN Reports · doi:10.1097/pr9.0000000000000823 · Cited by 223
Siddall, B., Ram, A., Jones, M. D., Booth, J., Perriman, D., & Summers, S. J. (2022) · Pain · doi:10.1097/j.pain.0000000000002308 · Cited by 174
Talbot, K., Madden, V. J., Jones, S., & Moseley, G. L. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2019.03.033 · Cited by 171
Self-citationWewege, M. A., & Jones, M. D. (2020) · Journal of Pain · doi:10.1016/j.jpain.2020.04.003 · Cited by 133
Karran, E. L., Cashin, A. G., Barker, T., Boyd, M. A., Chiarotto, A., Dewidar, O., Mohabir, V., Petkovic, J., Sharma, S., Tejani, S., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.09.017 · Cited by 96
Self-citation
Metadata
- Authors
- Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L.
- Affiliations
- University of Oxford (GB)
- UNSW Sydney (AU)
- Ollscoil na Gaillimhe – University of Galway (IE)
- University of South Australia (AU)
- Corresponding author
- Lee, Hopin
- Venue
- Pain
- Published
- 2018-04-23
- Volume
- 159
- Issue
- 9
- Pages
- 1683-1689
- ISSN
- 0304-3959 (ISSN-L)
- 1872-6623
- OpenAlex
- W2802823940
- Cited by
- 150
- References
- 76
- Citations by year
- 2026: 10 · 2025: 24 · 2024: 12 · 2023: 10 · 2022: 24 · 2021: 32 · 2020: 26 · 2019: 9 · 2018: 3
- FWCI
- 24.80
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Pain (publishedVersion)
- PubMed (publishedVersion)
- ARAN (University of Galway Research Repository) (Ollscoil na Gaillimhe – University of Galway) (acceptedVersion, cc-by-nc-nd)
- ARAN (University of Galway Research Repository) (Ollscoil na Gaillimhe – University of Galway) (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research (NF-SI-0611-10020)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Keywords
- reproducibility, replicability, pain research, research bias, research waste, replication crisis, research practices, preclinical research, research integrity
- MeSH terms
- Animals, Humans, Pain, Research Design, Statistics as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
