The Brazilian Reproducibility Initiative
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproducibility Initiative is a systematic, multicenter effort to repeat between 60 and 100 experiments: the project will focus on a set of common methods, repeating each experiment in three different laboratories from a countrywide network. The results, due in 2021, will allow us to estimate the level of reproducibility of biomedical science in Brazil, and to investigate what aspects of the published literature might help to predict whether a finding is reproducible.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“…reproducibility of published results in certain areas of biomedical research were initially raised by theoretical models (Ioannidis, 2005a), systematic reviews of the existing literature (Ioannidis, 2005b) and alarm calls by the pharmaceutical industry (Begley and Ellis, 2012; Prinz et al., 2011).”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Many analysts criticize the country’s research as largely composed of "salami science", growing in absolute numbers but lacking in impact, originality and influence (Righetti, 2013).”
BackgroundGenetics of Mouse Behavior: Interactions with Laboratory Environment
Crabbe, J. C., Wahlsten, D., & Dudek, B. C. (1999) · Science · doi:10.1126/science.284.5420.1670 · Cited by 1,666
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293
Research works printed but not linked: 22
- Angelo, 2016, Brazil's scientists battle to escape 20-year funding freeze, Nature, 539, 10.1038/nature.2016.21014 · doi:10.1038/nature.2016.21014
- Baker, 2017, Cancer reproducibility project releases first results, Nature, 541, 269, 10.1038/541269a · DOI · Paper page
- Barata, 2016, Dez coisas que você deveria saber sobre o Qualis, Revista Brasileira De Pós-Graduação, 13, 13, 10.21713/2358-2332.2016.v13.947 · DOI
- Ciscati R. 2018. Projeto vai replicar experimentos de cientistas brasileiros para checar sua eficiência . O Globo. https://oglobo.globo.com/sociedade/ciencia/projeto-vai-replicar-experimentos-de-cientistas-brasileiros-para-checar-sua-eficiencia-22615152 [Accessed January 25, 2019].
- Cova, 2018, Estimating the reproducibility of experimental philosophy, Review of Philosophy and Psychology, 28, 1, 10.1007/s13164-018-0400-9 · DOI · Paper page
- Coyne, 2016, Replication initiatives will not salvage the trustworthiness of psychology, BMC Psychology, 4, 28, 10.1186/s40359-016-0134-3 · DOI
- Errington, 2014, An open investigation of the reproducibility of cancer biology research, eLife, 3, 10.7554/eLife.04333 · DOI · Paper page
- Floresti F. 2017. A ciência brasileira vai quebrar?. Revista Galileu. https://revistagalileu.globo.com/Revista/noticia/2017/09/ciencia-brasileira-vai-quebrar.html [Accessed January 25, 2019].
- Harris, 2017, Rigor Mortis
- Hines, 2014, Sorting out the FACS: a devil in the details, Cell Reports, 6, 779, 10.1016/j.celrep.2014.02.021 · DOI · Paper page
- Hostins, 2006, Os planos nacionais de Pós-graduação (PNPG) e suas repercussões na pós-graduação brasileira, Perspectiva, 24, 133
- Ioannidis, 2005, Why most published research findings are false, PLOS Medicine, 2, 10.1371/journal.pmed.0020124 · DOI · Paper page
- Jamieson, 2018, Crisis or self-correction: Rethinking media narratives about the well-being of science, PNAS, 115, 2620, 10.1073/pnas.1708276114 · DOI
- Kaiser, 2018, Plan to replicate 50 high-impact cancer papers shrinks to just 18, Science, 10.1126/science.aau9619 · DOI
- Klein, 2018, Many Labs 2: Investigating variation in replicability across sample and setting, PsyArXiv, 10.31234/osf.io/9654g · DOI · Paper page
- Massonnet, 2010, Probing the reproducibility of leaf growth and molecular phenotypes: a comparison of three Arabidopsis accessions cultivated in ten laboratories, Plant Physiology, 152, 2142, 10.1104/pp.109.148338 · DOI · Paper page
- Nature Medicine, 2016, Take the long view, Nature Medicine, 22, 10.1038/nm.4033 · doi:10.1038/nm.4033
- Neves K, Amaral OB. 2018. Abrindo a caixa-preta. Ciência Hoje. http://cienciahoje.org.br/artigo/abrindo-a-caixa-preta [Accessed January 25, 2019].
- Pesquisa FAPESP. 2018. Uma rede para reproduzir experimentos. Revista Pesquisa FAPESP. http://revistapesquisa.fapesp.br/2018/05/17/uma-rede-para-reproduzir-experimentos [Accessed January 25, 2019].
- Pinto, 1999, Fator de impacto de revistas científicas: qual o significado deste parâmetro?, Química Nova, 22, 448, 10.1590/S0100-40421999000300026 · DOI
- Righetti S. 2013. Brasil cresce em produção científica, mas índice de qualidade cai. Folha De S. Paulo. https://www1.folha.uol.com.br/ciencia/2013/04/1266521-brasil-cresce-em-producao-cientifica-mas-indice-de-qualidade-cai.shtml [Accessed January 25, 2019].
- Silberzahn, 2018, Many analysts, one data set: Making transparent how variations in analytic choices affect results, Advances in Methods and Practices in Psychological Science, 1, 337, 10.1177/2515245917747646 · DOI · Paper page
Cited by
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
Loss, C. M., Melleu, F. F., Domingues, K., Lino-de-Oliveira, C., & Viola, G. G. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.763428 · Cited by 44
Addressing selective reporting of experiments through predefined exclusion criteria
Neves, K., & Amaral, O. B. (2020) · eLife · doi:10.7554/elife.56626 · Cited by 15
Self-citationEstimating the replicability of Brazilian biomedical science
Initiative, T. B. R., Amaral, O. B., Carneiro, C. F. D., Neves, K., Sampaio, A. P. W., Gomes, B. V., Abreu, M. B. D., Tan, P. B., Mota, G. P. S., Goulart, R. N., et al. (2025) · bioRxiv · doi:10.1101/2025.04.02.645026 · Cited by 13
“With this in mind, we set up the Brazilian Reproducibility Initiative, a multicenter assessment of the replicability of experiments published by Brazilian institutions using common laboratory methods (Amaral et al., 2019; Neves et al., 2020).”
Self-citation
Metadata
- Authors
- Amaral, O. B., Neves, K., Wasilewska-Sampaio, A. P., & Carneiro, C. F.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2019-02-05
- Volume
- 8
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2901258965
- Cited by
- 49
- References
- 24
- FWCI
- 7.37
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- creativecommons.org/licenses/by/4.0/ (am)
- creativecommons.org/licenses/by/4.0/ (tdm)
- Funders
- Instituto Serrapilheira
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Brazilian Reproducibility Initiative. https://app.cochat.ai/papers/10.7554/elife.41602
- The paper
- Amaral, O. B., Neves, K., Wasilewska-Sampaio, A. P., & Carneiro, C. F. (2019). The Brazilian Reproducibility Initiative. eLife. https://doi.org/10.7554/elife.41602
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.