Reporting and misreporting of sex differences in the biological sciences
Open accessPublished version of a preprint
Abstract
As part of an initiative to improve rigor and reproducibility in biomedical research, the U.S. National Institutes of Health now requires the consideration of sex as a biological variable in preclinical studies. This new policy has been interpreted by some as a call to compare males and females with each other. Researchers testing for sex differences may not be trained to do so, however, increasing risk for misinterpretation of results. Using a list of recently published articles curated by Woitowich et al. (eLife, 2020; 9:e56344), we examined reports of sex differences and non-differences across nine biological disciplines. Sex differences were claimed in the majority of the 147 articles we analyzed; however, statistical evidence supporting those differences was often missing. For example, when a sex-specific effect of a manipulation was claimed, authors usually had not tested statistically whether females and males responded differently. Thus, sex-specific effects may be over-reported. In contrast, we also encountered practices that could mask sex differences, such as pooling the sexes without first testing for a difference. Our findings support the need for continuing efforts to train researchers how to test for and report sex differences in order to promote rigor and reproducibility in biomedical research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Independently assessing males and females and noting a sex difference if an effect is observed in one sex but not the other is a common misconception of the appropriate way to assess sex differences.
“This was as opposed to independently assessing males and females and noting a sex difference if we observed an effect in one sex but not the other, a common misconception of the appropriate way to assess sex differences (Garcia-Sifuentes and Maney, 2021).”
From Social play experience in juvenile rats is indispensable for appropriate socio-sexual behavior in adulthood in males but not females (2023) · cites this paper as Garcia-Sifuentes and Maney, 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Policy: NIH to balance sex in cell and animal studies
Clayton, J. A., & Collins, F. S. (2014) · Nature · doi:10.1038/509282a · Cited by 1,816 · see citing works
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,316 · see citing works
Erroneous analyses of interactions in neuroscience: A problem of significance
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2886 · Cited by 864 · see citing works
Female rats are not more variable than male rats: a meta-analysis of neuroscience studies
Becker, J. B., Prendergast, B. J., & Liang, J. W. (2016) · Biology of Sex Differences · doi:10.1186/s13293-016-0087-5 · Cited by 473 · see citing works
Ten common statistical mistakes to watch out for when writing or reviewing a manuscript
Makin, T. R., & Orban de Xivry, J.-J. (2019) · eLife · doi:10.7554/elife.48175 · Cited by 238 · see citing works
Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Steyerberg, E. W., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i5826 · Cited by 89 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- Arnegard ME Whitten LA Hunter C Clayton JA 2020 Sex as a biological variable: A 5-year progress report and call to action Journal of Women’s Health 29 858 864 10.1089/jwh.2019.8247 31971851 PMC7476377 · DOI
- Beery AK Zucker I 2011 Sex bias in neuroscience and biomedical research Neuroscience and Biobehavioral Reviews 35 565 572 10.1016/j.neubiorev.2010.07.002 20620164 PMC3008499 · DOI · Paper page
- Beltz AM Beery AK Becker JB 2019 Analysis of sex differences in pre-clinical and clinical data sets Neuropsychopharmacology 44 2155 2158 10.1038/s41386-019-0524-3 31527863 PMC6898365 · DOI
- Clayton JA Tannenbaum C 2016 Reporting Sex, Gender, or Both in Clinical Research? JAMA 316 1863 1864 10.1001/jama.2016.16405 27802482 · DOI
- Clayton JA 2018 Applying the new SABV (sex as a biological variable) policy to research and clinical care Physiology & Behavior 187 2 5 10.1016/j.physbeh.2017.08.012 28823546 · DOI
- Cortes LR Cisternas CD Forger NG 2019 Does Gender Leave an Epigenetic Imprint on the Brain? Frontiers in Neuroscience 13 173 10.3389/fnins.2019.00173 30872999 PMC6400866 · DOI · Paper page
- Cumming G 2012 Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analyses New York, NY Routledge
- Diester CM Banks ML Neigh GN Negus SS 2019 Experimental design and analysis for consideration of sex as a biological variable Neuropsychopharmacology 44 2159 2162 10.1038/s41386-019-0458-9 31277076 PMC6897955 · DOI
- Duchesne A Pletzer B Pavlova MA Lai MC Einstein G 2020 Editorial: Bridging Gaps Between Sex and Gender in Neurosciences Frontiers in Neuroscience 14 561 10.3389/fnins.2020.00561 32595445 PMC7301887 · DOI
- Holmes MM Monks DA 2019 Bridging sex and gender in neuroscience by shedding a priori assumptions of causality Frontiers in Neuroscience 13 475 10.3389/fnins.2019.00475 31143099 PMC6521798 · DOI
- Johnson JL Greaves L Repta R 2009 Better Science with sex and gender: Facilitating the use of a sex and gender-based analysis in health research International Journal for Equity in Health 8 14 10.1186/1475-9276-8-14 19419579 PMC2689237 · DOI
- Maney DL 2014 Just like a circus: The public consumption of sex differences Current Topics in Behavioral Neuroscience 19 279 296 10.1007/7854_2014_339 25129138 · DOI
- Maney DL 2016 Perils and pitfalls of reporting sex differences Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 371 20150119 10.1098/rstb.2015.0119 26833839 PMC4785904 · DOI
- Potluri T Engle K Fink AL Vom Steeg LG Klein SL 2017 Sex reporting in preclinical microbiological and immunological research MBio 8 e01868-17 10.1128/mBio.01868-17 29138308 PMC5686541 · DOI
- Radke AK Sneddon EA Monroe SC 2021 Studying sex differences in rodent models of addictive behavior Current Protocols 1 e119 10.1002/cpz1.119 33901344 PMC8208026 · DOI
- Shansky RM Murphy AZ 2021 Considering sex as a biological variable will require a global shift in science culture Nature Neuroscience 24 457 464 10.1038/s41593-021-00806-8 33649507 PMC12900283 · DOI
- Sugimoto CR Ahn YY Smith E Macaluso B Larivière V 2019 Factors affecting sex-related reporting in medical research: A cross-disciplinary bibliometric analysis Lancet 393 550 559 10.1016/S0140-6736(18)32995-7 30739690 · DOI
- Tannenbaum C Ellis RP Eyssel F Zou J Schiebinger L 2019 Sex and gender analysis improves science and engineering Nature 575 137 146 10.1038/s41586-019-1657-6 31695204 · DOI
- Will TR Proaño SB Thomas AM Kunz LM Thompson KC Ginnari LA Jones CH Lucas S-C Reavis EM Dorris DM Meitzen J 2017 Problems and progress regarding sex bias and omission in neuroscience research ENeuro 4 ENEURO.0278-17.2017 10.1523/ENEURO.0278-17.2017 29134192 PMC5677705 · DOI
- Woitowich NC Woodruff TK 2019 Opinion: Research community needs to better appreciate the value of sex-based research PNAS 116 7154 7156 10.1073/pnas.1903586116 30971497 PMC6462069 · DOI
- Woitowich NC Beery A Woodruff T 2020 A 10-year follow-up study of sex inclusion in the biological sciences eLife 9 e56344 10.7554/eLife.56344 32513386 PMC7282816 · DOI
Cited by
Sorted by citations
Rechlin, R. K., Splinter, T. F. L., Hodges, T. E., Albert, A. Y., & Galea, L. A. M. (2022) · Nature Communications · doi:10.1038/s41467-022-29903-3 · Cited by 218 · see citing works
HUNTER, S. K., S. ANGADI, S., BHARGAVA, A., HARPER, J., HIRSCHBERG, A. L., D. LEVINE, B., L. MOREAU, K., J. NOKOFF, N., STACHENFELD, N. S., & BERMON, S. (2023) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000003300 · Cited by 241 · see citing works
Local senolysis in aged mice only partially replicates the benefits of systemic senolysis
Farr, J. N., Saul, D., Doolittle, M. L., Kaur, J., Rowsey, J. L., Vos, S. J., Froemming, M. N., Lagnado, A. B., Zhu, Y., Weivoda, M., et al. (2023) · Journal of Clinical Investigation · doi:10.1172/jci162519 · Cited by 81 · see citing works
Doolittle, M. L., Saul, D., Kaur, J., Rowsey, J. L., Vos, S. J., Pavelko, K. D., Farr, J. N., Monroe, D. G., & Khosla, S. (2023) · Nature Communications · doi:10.1038/s41467-023-40393-9 · Cited by 57 · see citing works
Doolittle, M. L., Saul, D., Kaur, J., Rowsey, J. L., Vos, S. J., Pavelko, K. D., Farr, J. N., Monroe, D. G., & Khosla, S. (2023) · doi:10.1101/2023.01.12.523760 · Cited by 57 · see citing works
Hunter, S. K., Angadi, S. S., Bhargava, A., Harper, J., Hirschberg, A. L., Levine, B. D., Moreau, K. L., Nokoff, N. J., Stachenfeld, N. S., & Bermon, S. (2023) · Translational Journal of the American College of Sports Medicine · doi:10.1249/tjx.0000000000000236 · Cited by 43 · see citing works
Marquardt, A. E., VanRyzin, J. W., Fuquen, R. W., & McCarthy, M. M. (2023) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.1076765 · Cited by 31 · see citing works
Adolescence alcohol exposure impairs fear extinction and alters medial prefrontal cortex plasticity
Lawson, K., Scarlata, M. J., Cho, W. C., Mangan, C., Petersen, D., Thompson, H. M., Ehnstrom, S., Mousley, A. L., Bezek, J. L., & Bergstrom, H. C. (2022) · Neuropharmacology · doi:10.1016/j.neuropharm.2022.109048 · Cited by 16 · see citing works
- Authors
- Garcia-Sifuentes, Y., & Maney, D. L.
- Affiliations
- Emory University (US)
- Corresponding author
- Maney, Donna L
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2021-11-02
- Volume
- 10
- ISSN
- 2050-084X (ISSN-L)
- PMC
- PMC8562995
- OpenAlex
- W3209546556
- Cited by
- 241
- References
- 6
- Citations by year
- 2026: 32 · 2025: 42 · 2024: 51 · 2023: 63 · 2022: 47 · 2021: 4 · 2020: 1 · 2013: 1
- FWCI
- 35.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- eLife (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Emory University Research Committee (00106050 - URC 2021-22)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Sex and Gender in Healthcare
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic and Clinical Aspects of Sex Determination and Chromosomal Abnormalities
- Keywords
- sex differences, sex as a biological variable, misreporting, statistical power, sex-specific effects, research rigor, reproducibility, NIH policy, research training
- MeSH terms
- Animals, Biological Science Disciplines, Female, Humans, Male, National Institutes of Health (U.S.), Sex Characteristics, Sex Factors, United States, Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
