Scientific integrity and the IAAF testosterone regulations
Open accessAbstract
In April 2018, the International Association of Athletics Federations (IAAF) announced new regulations governing the eligibility of certain female athletes with differences of sexual development accompanied by elevated levels of natural testosterone. Such women with testosterone levels above a specific threshold would be banned from competing as females unless they were to undergo medical intervention. In this paper, we examine key elements of the scientific basis offered by IAAF in support of the regulations, based on a subset of original performance data provided to us by IAAF. We identify significant flaws in the data used by IAAF leading to unreliable results. Further, these failures have not been corrected by IAAF or the academic journal which has published them, leading to a comprehensive failure of scientific integrity. We argue that the IAAF testosterone regulations are based on a flawed scientific foundation and that this case offers more general lessons for the sport governance community on the importance of upholding the standards of scientific integrity expected in other areas of policy and regulation.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper argues that the IAAF’s 2018 testosterone regulations rest on flawed scientific evidence, especially Bermon and Garnier (2017), and that IAAF and BJSM failed to correct or retract it. Using a partial dataset, the authors document pervasive data errors and show results change materially when corrected, concluding the regulations lack a reliable scientific foundation.
Key findings
- The 2018 IAAF regulations require women with differences of sexual development and elevated natural testosterone to medically reduce testosterone below 5 nmol/L to compete in five events: 400 m, 400 m hurdles, 800 m, 1500 m, and 1 mile; BG17 is cited as the empirical basis.
- In the four regulated events they recreated, problematic data made up 17–33% of values: 400 m 22/67 (32.8%), 400 m hurdles 19/67 (28.4%), 800 m 11/64 (17.2%), and 1500 m 15/66 (22.7%); errors included duplicated athletes, duplicated times, phantom times, and doping-disqualified athletes.
- Correcting the data changed all three outcomes—sample numbers, means, and standard deviations—for all four events; for example, 400 m mean changed from 52.19 to 52.85 and 1500 m from 250.15 to 245.96.
- Comparing BG17 with BHKE18, performance differences between highest and lowest testosterone tertiles decreased in 8 of 11 running events, changed from positive to negative in 3 events, and low-T tertile was faster in 6 of 11 events versus 3 of 11 in BG17; in the four regulated events the average difference fell by 0.4% absolute (from 2.0% to 1.6%), with only 1 of 4 statistically significant in BHKE18 versus 3 of 4 in BG17.
Methods
- Independent critique based on a subset of original BG17 performance data provided by Dr. Bermon after the authors’ request; they received data for the 11 women’s running events, described as 25% of the original data.
- They replicated BG17’s Table 3 summary statistics and recreated the underlying dataset for four events (400 m, 400 m hurdles, 800 m, 1500 m) by cross-checking times against publicly available 2011 Daegu and 2013 Moscow World Championships results.
- They compared original BG17 data, their corrected data, and BHKE18 results, focusing on performance differences between top and bottom testosterone tertiles.
Limitations
- The authors state they had only 25% of the original BG17 data and no linked medical data, so they cannot know what impact the problematic data had on BG17’s conclusions about testosterone’s influence on performance.
- They state their recreation was limited to four regulated events, though they consider it likely similar problems exist in other events and the anonymous medical data.
- Evident from the text: the balance of BG17 and BHKE18 data was not made available, BHKE18 was published as a letter and not peer-reviewed prior to publication, and the recreation relied on publicly available results, limiting independent verification of the full analyses.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
- Findingown data
The data provided to the authors do not match the data used in the updated BHKE18 paper.
“Further, it appears that some amount of unreliable data persists in BHKE18, since the data provided to us do not match that used in the updated BHKE18 paper.”
Quote in the paperSubstantial claim - Findingreasoning
The performance data associated with BG17 have not been reviewed by peers.
“With the exception of our analysis reported below, based on a subset of the data shared with us, the data have not been reviewed by peers and were not at the time of this claim by the IAAF.”
Quote in the paperSubstantial claim - Findingown data
In three events, the performance difference changed from positive (high T faster than low T) to negative (high T slower than low T).
“In three events, the performance difference changed from positive (high T faster than low T) to negative (high T slower than low T)”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingnone given
In the 100 m sprint, the advantage of low T athletes over high T athletes (5.4% and 3.4%, respectively) is larger than any of the advantages observed within each study of high T over low T across all events.
“In the 100 m sprint, the advantage of low T athletes over high T athletes (5.4% and 3.4%, respectively) is larger than any of the advantages observed within each study of high T over low T across all events.”
Quote in the paperNot a substantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingown data
The authors successfully reproduced the summary statistics from Table 3 of BG17 using the data provided by Dr. Bermon, with only small differences.
“Table 1 shows that we were able to successfully reproduce the summary statistics with only small differences (emphasized).This replication confirms that the data that we were provided by Dr. Bermon were in fact identical (or nearly so, we cannot explain the small differences between reported results and our replication) to that which was used in BG17.”
Quote in the paperSubstantial claim
- Findingown data
As of early January 2019, BG17 and BHKE18 have been together cited more than 40 times.
“As of early January 2019, BG17 and BHKE18 have been together cited more than 40 times (Google Scholar), providing a case study in the propagation of flawed scientific results.”
Quote in the paperNot a substantial claim - Findingown data
The results reported in BG17 change quantitatively in BHKE18 upon removal of 220 data points and introduction of new methods.
“Clearly and unambiguously, the results reported in BG17 change quantitatively in BHKE18 upon removal of 220 data points and introduction of new methods.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingown data
In comparing BG17 to BHKE18, the reported differences between the top and bottom testosterone tertiles for all women's running events changed dramatically.
“In comparing BG17 to BHKE18, the reported differences between these tertiles for all women's running events changed dramatically, as shown in Table 5.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingown data
There are remaining uncorrected data errors in BHKE18.
“The table indicates that there are remaining uncorrected data errors in BHKE18.”
Quote in the paperSubstantial claim - Findingown data
All three outcomes (sample numbers, means, standard deviations) change for all four events upon elimination of the problematic data points.
“Table 4 reveals that all three outcomes change for all four events upon the elimination of the problematic data points.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingown data
Problematic data make up between 17% and 33% of the values used in the BG17 analysis for the four events (400 m, 400 m hurdles, 800 m, 1500 m).
“Problematic data make up between 17 and 33% of the values used in the BG17 analysis for these four events.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingnone given
In BHKE18, the low T tertile is faster than the high T tertile in 6 of 11 events, compared to 3 of 11 events in BG17.
“In BHKE18, the low T tertile is faster than the high T tertile in 6 of 11 events, compared to 3 of 11 events in BG17.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Findingown data
The changes in aggregate times when using the corrected data are of a similar magnitude to the effects of testosterone that BG17 sought to identify.
“The changes in aggregate times when using the corrected data are of a similar magnitude to that of the effects of testosterone that the authors seek to identify.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Interpretationreasoning
The IAAF's use of in-house researchers to conduct research on performance effects associated with testosterone creates at a minimum a perception of a conflict of interest.
“The IAAF set itself up for problems by conducting research on performance effects associated with testosterone using in-house researchers.This creates at a minimum a perception of a conflict of interest that could have been mitigated to some degree by allowing independent researchers access to data and evidence, in order to replicate findings.”
Quote in the paperSubstantial claim - Interpretationreasoning
The large magnitude of performance changes on all athletes when correcting the problematic data points significantly undermines any conclusions that can be drawn from BG17 even before the medical data are linked to the new performance set.
“However, the large magnitude of performance changes on all athletes when correcting these problematic data points significantly undermines any conclusions that can be drawn even before the medical data are linked to the new performance set.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Interpretationreasoning
The lack of action by BJSM to uphold its stated policies has the effect that the IAAF is protected from the normal expectations of scientific research.
“Furthermore, the lack of action to uphold its stated policies from BJSM has the effect that the IAAF is protected from the normal expectations of scientific research.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The pervasive errors in the four regulated events call into question the fidelity of the entire BG17 analysis.
“Such pervasive errors in the four regulated events for which we carefully recreated data call into question the fidelity of the entire analysis.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
- Interpretationreasoning
The unwillingness of the IAAF to correct or acknowledge errors highlights its conflict of interest.
“The unwillingness of the IAAF to correct or acknowledge errors highlights its conflict of interest.”
Quote in the paperSubstantial claim - Interpretationown data
The IAAF's 2018 testosterone regulations are based on a flawed scientific foundation.
“We argue that the IAAF testosterone regulations are based on a flawed scientific foundation”
Quote in the paperSubstantial claim - Interpretationreasoning
Self-correction mechanisms in science have broken down in this instance.
“In this instance, self-correction mechanisms in science have broken down.”
Quote in the paperNot a substantial claim - Interpretationreasoning
In rejecting retraction of BG17, BJSM did not follow its own policies or international standards of science publication.
“We conclude that in rejecting retraction of BG17, BJSM did not follow its own policies or international standards of science publication.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 1
- May combine two claims. 13
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe editorial process used by BJSM to arrive at its judgements is unknown.
“The editorial process used by BJSM to arrive at these judgements is also unknown.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the four regulated events, the average difference in times was reduced by 0.4% in absolute terms (from 2.0% to 1.6%), and only 1 of 4 meets the BHKE18 standard for statistical significance (BG17 reported 3 of 4).
“In the four regulated events, the average difference in times was reduced by 0.4% in absolute terms (i.e. from 2.0 to 1.6%), and only 1 of 4 meets the BHKE18 standard for statistical significance (BG17 reported 3 of 4).”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
May combine two claims.
Findingnone givenFor 8 of 11 running events, the performance difference between the highest and lowest testosterone tertiles decreased from BG17 to BHKE18, including in 3 of 4 of the regulated events.
“For 8 of 11 running events, the performance difference between the highest and lowest tertiles decreased from BG17 to BHKE18, including in 3 of 4 of the regulated events”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
May combine two claims.
Findingnone givenThe IAAF and BJSM have not corrected the identified failures.
“Further, these failures have not been corrected by IAAF or the academic journal which has published them”
Quote in the paperSubstantial claimMay combine two claims.
FindingreasoningBHKE18 stated it excluded 230 observations, but a comparison of reported observations in BG17 and BHKE18 indicates that only 220 observations were dropped, so BHKE18 erred in its reporting of excluded observations.
“A comparison of reported observations in BG17 and BHKE18 indicates that only 220 observations were dropped from one study to the next, and thus, BHKE18 erred in its reporting of excluded observations.”
Quote in the paperSubstantial claimMay combine two claims.
FindingreasoningBJSM rejected the authors' call for retraction of BG17 and refused to require the authors of BG17 or BHKE18 to release their data to allow for independent replication.
“Similarly, once we identified errors in the data that were provided to us by Dr. Bermon, the editors of BJSM rejected our call for the paper to be retracted.”
- Also stated in: § 4 The importance of upholding scientific integrity in sports governance · plain
“BJSM has also refused to require the authors of BG17 or BHKE18 to release their data to allow for independent replication.”
Quote in the paperSubstantial claim- Also stated in: § 4 The importance of upholding scientific integrity in sports governance · plain
May combine two claims.
InterpretationreasoningNeither BG17 nor BHKE18 can be considered peer-reviewed studies because neither BJSM nor the IAAF has made available data from either analysis.
“Neither BJSM nor the IAAF has made available data from either analysis and cannot be considered peer-reviewed studies.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningBJSM refused to publish the authors' analysis after more than 3 months of deliberation, not due to any scientific shortfalls (the BJSM peer reviewers judged the analysis scientifically sound) but because the analysis was critical of how the issue was handled by BJSM.
“Further, after reviewing a submission from us on the flaws in BG17 and the irregularities in the process, BJSM refused to publish our analysis after more than 3 months of deliberation-not due to any scientific shortfalls (indeed, the BJSM peer reviewers judged our analysis to be scientifically sound) but because our analysis was critical of how the issue was handled by BJSM.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe failures in the data and their non-correction by IAAF and BJSM constitute a comprehensive failure of scientific integrity.
“Further, these failures have not been corrected by IAAF or the academic journal which has published them, leading to a comprehensive failure of scientific integrity.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningBG17 used unreliable data, and thus its results are also unreliable.
“What is unequivocal is that BG17 used unreliable data, and thus, its results are also unreliable.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
May combine two claims.
PremisereasoningThe IAAF research was produced in-house by IAAF researchers and was both funded and conducted by IAAF in support of its own regulations.
“As BG17 is both funded and conducted by IAAF in support of its own regulations, it is appropriate that independent scholars replicate or examine their work.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningBG17 forms the empirical basis for the 2018 IAAF testosterone regulations and is the primary response by IAAF to the evidence required by the 2015 CAS Chand ruling.
“There can be no doubt that BG17 forms the empirical basis for the 2018 IAAF testosterone regulations and is the primary response by IAAF to that CAS ruled must be provided by the IAAF in the 2015 Dutee Chand ruling.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe authors consider it likely that similar data problems might be found in the data for the other 17 women's events and 22 men's events, and perhaps also in the anonymous medical data.
“Given the pervasiveness of these errors, we consider it likely that similar problems might be found in the data for the other 17 women's events and 22 men's events, and perhaps also in the anonymous medical data, which are the basis for the study's main conclusions regarding the performance effects of elevated testosterone levels.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
May combine two claims.
InterpretationreasoningThe results of BG17 are clearly unreliable, and those of BHKE18 are of unknown validity.
“The results of BG17 are clearly unreliable, and those of BHKE18 are of unknown validity.”
Quote in the paperSubstantial claim- Rests on an assumption: The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
The paper relies on this without support.
The publicly available results from Wikipedia for the 2011 and 2013 World Championships are accurate and complete enough to recreate the underlying dataset used in BG17.
“recreation of the underlying dataset based on reported times from the 2011 (Daegu) and 2013 (Moscow) World Championships (via Wikipedia).”
Cites
The Honest Broker: Making Sense of Science in Policy and Politics
Pielke, Jr. (2007) · doi:10.1017/cbo9780511818110 · Cited by 2,020 · see citing works
Self-citationParkhurst, J. (2016) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.4324/9781315675008 · Cited by 214 · see citing works
Bermon, S., & Garnier, P.-Y. (2017) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-097792 · Cited by 173 · see citing works
Impossible “Choices”: The Inherent Harms of Regulating Women’s Testosterone in Sport
Karkazis, K. A., & Carpenter, M. (2018) · Journal of Bioethical Inquiry · doi:10.1007/s11673-018-9876-3 · Cited by 88 · see citing works
Hyperandrogenism controversy in elite women’s sport: an examination and critique of recent evidence
Sönksen, P. H., Bavington, L. D., Boehning, T., Cowan, D., Guha, N., Holt, R. I. G., Karkazis, K. A., Ferguson-Smith, M. A., Mircetic, J., & Böhning, D. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098446 · Cited by 60 · see citing works
Making sense of science in policy and politics
Pielke, R. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511818110.009 · Cited by 47
Self-citationBermon, S., Hirschberg, A. L., Kowalski, J. H., & Eklund, E. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099700 · Cited by 45 · see citing works
Scientific integrity and anti-doping regulation
Pielke, R. A., & Boye, E. (2019) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2019.1596968 · Cited by 32 · see citing works
Self-citationMenier, A. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098464 · Cited by 15 · see citing works
Camporesi, S. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099387 · Cited by 15 · see citing works
Sugar, spice and everything nice: how to end ‘sex testing’ in international athletics
Pielke, R. A. (2017) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2017.1369448 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Douglas HE, Bour E (2014) Scientific integrity in a politicized world. In: Logic, methodology, and philosophy of science: proceedings of the 14th international congress, pp 253–268
Cited by
Sorted by citations
Wiik, A., Lundberg, T. R., Rullman, E., Andersson, D. P., Holmberg, M., Mandić, M., Brismar, T. B., Dahlqvist Leinhard, O., Chanpen, S., Flanagan, J. N., et al. (2020) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgz247 · Cited by 116 · see citing works
Hamilton, B. R., Lima, G., Barrett, J., Seal, L., Kolliari-Turner, A., Wang, G., Karanikolou, A., Bigard, X., Löllgen, H., Zupet, P., et al. (2021) · Sports Medicine · doi:10.1007/s40279-021-01451-8 · Cited by 27 · see citing works
- Authors
- Pielke, R., Tucker, R., & Boye, E.
- Affiliations
- University of Colorado Boulder (US)
- University of Cape Town (ZA)
- Oslo University Hospital (NO)
- Corresponding author
- Pielke, Roger
- Venue
- The International Sports Law Journal
- Published
- 2019-02-07
- Volume
- 19
- Issue
- 1-2
- Pages
- 18-26
- ISSN
- 1567-7559 (ISSN-L)
- 2213-5154
- OpenAlex
- W2914586711
- Cited by
- 104
- References
- 11
- Citations by year
- 2026: 8 · 2025: 5 · 2024: 27 · 2023: 18 · 2022: 20 · 2021: 7 · 2020: 6 · 2019: 13
- FWCI
- 23.14
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-04-18)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Sports Law Journal (publishedVersion, cc-by)
- Topics
- Social Sciences › Sociology and Political Science › Doping in Sports
- Social Sciences › Gender Studies › Sports, Gender, and Society
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Sexual Differentiation and Disorders
- Keywords
- scientific integrity, sports eligibility, gender verification, data reliability, performance data, sports governance, anti-doping policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
