CoChat
Sign Up
Cited by 17Animal testing and alternativesContact Dermatitis and AllergiesAgricultural safety and regulations

Accounting for precision uncertainty of toxicity testing: methods to define borderline ranges and implications for hazard assessment of chemicals

Open access
Gabbert, Silke, Mathea, Miriam, Kolle, Susanne N., Landsiedel, Robert·2022·Risk Analysis
DOI 10.1111/risa.13648PMC PMC9292900
Open this paper in CoChat — free accountOpen PDF

Abstract

For hazard classifications of chemicals, continuous data from animal‐ or nonanimal testing methods are often dichotomized into binary positive/negative outcomes by defining classification thresholds (CT). Experimental data are, however, subject to biological and technical variability. Each test method's precision is limited resulting in uncertainty of the positive/negative outcome if the experimental result is close to the CT. Borderline ranges (BR) around the CT were suggested, which represent ranges in which the study result is ambiguous, that is, positive or negative results are equally likely. The BR reflects a method's precision uncertainty. This article explores and compares different approaches to quantify the BR. Besides using the pooled standard deviation, we determine the BR by means of the median absolute deviation (MAD), with a sequential combination of both methods, and by using nonparametric bootstrapping. Furthermore, we quantify the BR for different hazardous effects, including nonanimal tests for skin corrosion, eye irritation, skin irritation, and skin sensitization as well as for an animal test on skin sensitization (the local lymph node assay, LLNA). Additionally, for one method (direct peptide reactivity assay) the BR was determined experimentally and compared to calculated BRs. Our results demonstrate that (i) the precision of the methods is determining the size of their BRs, (ii) there is no “perfect” method to derive a BR, alas, (iii) a consensus on BR is needed to account for the limited precision of testing methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/risa.13648

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The regulatory meaning of cytotoxicity depends on its role within the test method, which may be as a classification endpoint, predictive surrogate, validity criterion, or concentration constraint.

    “Their regulatory meaning depends on assay role, where they may function as classification endpoints, predictive surrogates, validity criteria, or constraints for concentration selection.”

    From Cytotoxicity across OECD methods is not a single endpoint: context-dependent roles, variability and implications for decision-making (2026) · cites this paper as Gabbert et al., 2022

    Table I (Empirical Application, Selection of Testing Methods; page unavailable) uses relative cell viability as a classification threshold for skin irritation and corrosion, while the h-CLAT entry requires at least 50% viability for a positive result based on cell-surface markers. These examples support the point that viability has different roles across assays, but the copy does not establish the full four-role account of cytotoxicity: it does not identify cytotoxicity as a predictive surrogate or a constraint on concentration selection. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Assay variability near decision points may be large enough to affect classification, particularly where the threshold is close to the steep part of a concentration-response curve.

    “Ring-trial and validation data indicate that assay variability near decision points may be large enough to affect classification, particularly where the threshold is close to the steep part of a concentration-response curve (Kolle et al., 2021; Gabbert et al., 2022; Leontaridou, 2017).”

    From Cytotoxicity across OECD methods is not a single endpoint: context-dependent roles, variability and implications for decision-making (2026) · cites this paper as Gabbert et al., 2022

    The Conclusions state that technical and biological variability can cause results near a classification threshold to switch between positive and negative on retesting. The paper does not establish the added condition that this is particularly so when the threshold lies near the steep part of a concentration-response curve. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • The concept of borderline ranges is well established for several defined approaches and in vitro skin sensitization methods.

    “The concept is well established for several defined approaches and in vitro skin sensitization methods (Kolle et al., 2021; Gabbert et al., 2022; Leontaridou, 2017; OECD, 2025e).”

    From Cytotoxicity across OECD methods is not a single endpoint: context-dependent roles, variability and implications for decision-making (2026) · cites this paper as Gabbert et al., 2022

    The Introduction reports borderline-range analyses for three nonanimal skin-sensitization methods and one defined approach, the “2-out-of-3” approach. But it says borderline ranges had been explored for only a few methods and that evidence remained limited; it does not support calling the concept well established for several defined approaches. Page numbers were not available in our copy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • The width of borderline ranges depends on the precision of the test method.

    “Gabbert et al. provided the formal statistical basis for this approach by defining borderline ranges as expressions of precision uncertainty around classification thresholds and showed that their width depends on the precision of the test method (Gabbert et al., 2022).”

    From Cytotoxicity across OECD methods is not a single endpoint: context-dependent roles, variability and implications for decision-making (2026) · cites this paper as Gabbert et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. Agnese, G., Risso, D., & De Flora, S. (1984). Statistical evaluation of inter‐ and intra‐laboratory variations of the Ames test, as related to the genetic stability of Salmonella tester strains. Mutation Research, 130(1), 27– 44. 6420695
  2. Alépée, N., Tornier, C., Robert, C., Amsellem, C., Roux, M.‐H., Doucet, O., … de Brugerolle de Fraisinette, A. (2005). A catch‐up validation study on reconstructed human epidermis (SkinEthic RHE) for full replacement of the Draize skin irritation test. Toxicology in Vitro, 24, 257– 266. 19733228
  3. Bauch, C., Kolle, S. N., Ramirez, T., Eltze, T., Fabian, E., Mehling, A., … Landsiedel, R. (2012). Putting the parts together: Combining in vitro methods to test for skin sensitizing potentials. Regulatory Toxicology and Pharmacology, 63, 489– 504. 22659254
  4. Basketter, D., Kolle, S. N., Schrage, A., Honarvar, N., Gamer, A. O., van Ravenzwaay, B., & Landsiedel, R. (2012). Experience with local lymph node assay performance standards using standard radioactivity and nonradioactive cell count measurements. Journal of Applied Toxicology, 32(8), 590– 596. 21618258
  5. Bruner, L. H., Carr, G. J., Chamberlain, M., & Curren, R. D. (1996). Validation of alternative methods for toxicity testing. Toxicology in Vitro, 10, 479– 501. 20650229
  6. Dimitrov, S., Detroyer, A., Piroird, C., Gomes, C., Eilstein, J., Pauloin, T., … Mekenyan, O. (2016). Accounting for data variability, a key factor in in vivo/in vitro relationships: Application to the skin sensitization potency (in vivo LLNA versus in vitro DPRA) example. Journal of Applied Toxicology, 36, 1568– 1578. Retrieved from https://onlinelibrary.wiley.com/doi/full/10.1002/jat.3318 27080242 · DOI
  7. Dumont, C., Barroso, J., Matys, I., Worth, A., & Casati, S. (2016). Analysis of the local lymph node assay (LLNA) variability for assessing the prediction of skin sensitisation potential and potency of chemicals with non‐animal approaches. Toxicology in Vitro, 34, 220– 228. 27085510
  8. Hoffmann, S. (2015). LLNA variability: An essential ingredient for a comprehensive assessment of non‐animal skin sensitization test methods and strategies. ALTEX, 32(4), 379– 383. 26168096
  9. Hoffman, S., & Hartung, T. (2005). Diagnosis: Toxic! Trying to apply approaches of clinical diagnostics and prevalence in toxicology considerations. Toxicological Sciences, 85, 422– 428. 15689419
  10. Hothorn, L. A. (2002). Selected biostatistical aspects of the validation of in vitro toxicological assays. Alternatives to Laboratory Animals, 30(Suppl. 2), 93– 98. 12513657
  11. Hoffmann, S., Kinsner‐Ovaskainen, A., Prieto, P., Mangelsdorf, I., Bieler, C., & Cole, T. (2010). Acute oral toxicity: Variability, reliability, relevance and interspecies comparison of rodent LD50 data from literature surveyed for the ACuteTox project. Regulatory Toxicology and Pharmacology, 58(3), 395– 407. 20709128
  12. Hothorn, L. A. (2003). Statistics of interlaboratory in vitro toxicological studies. Alternatives to Laboratory Animals, 31(Suppl. 1), 43– 63. 15595900
  13. Kolle, S. N., Basketter, D. A., Casati, S., Stokes, W. S., Strickland, J., van Ravenzwaay, B., … Landsiedel, R. (2013). Performance standards and alternative assays: Practical insights from skin sensitization. Regulatory Toxicology and Pharmacology, 65, 278– 285. 23279805
  14. Kolle, S. N., Landsiedel, R., & Natsch, A. (2020). Replacing the refinement for skin sensitization testing: Considerations to the implementation of adverse outcome pathway (AOP)‐based defined approaches (DA) in OECD test guidelines. Regulatory Toxicology and Pharmacology, 115, 104713. 32562760
  15. Leontaridou, M., Urbisch, D., Kolle, S. N., Ott, K., Mulliner, D. S., Gabbert, S., & Landsiedel, R. (2017). Research article: The borderline range of toxicological methods: Quantification and implications for evaluating precision. ALTEX, 34(4), 525– 538. 28230889
  16. Leontaridou, M., Gabbert, S., & Landsiedel, R. (2019). The impact of precision uncertainty on predictive accuracy metrics of non‐animal testing methods. ALTEX, 36(3), 435– 446. 30964150
  17. Luechtefeld, T., Marsch, D., Rowlands, C., & Hartung, T. (2018). Machine learning of toxicological big data enables read‐across structure activity relationships (RASAR) outperforming animal test reproducibility. Toxicological Sciences, 165(1), 198– 212. 30007363 PMC6135638
  18. Ng, M., Wilcox, R. R. (2011). Bootstrap methods for comparing independent regression slopes. British Journal of Mathematical and Statistical Psychology 65, 202– 301. 21827447
  19. OECD (2019f). Supporting document for evaluation and review of draft guideline (GL) for defined approaches (DAs) for skin sensitisation. Paris, France: OECD Publishing.
  20. Paparella, M., Daneshian, M., Hornek‐Gausterer, R., Kinzl, M., Mauritz, I., & Mühleegger, S. (2013). Food for thought. Uncertainty of testing methods – What do we (want to) know? ALTEX, 30, 131– 143. 23665803
  21. Paparella, M., Colacci, A., & Jacobs, M. N. (2017). Concept article: Uncertainties of testing methods: What do we (want to) know about carcinogenicity? ALTEX, 34(2), 235– 252. 27776202
  22. Richter, S. H., Garner, J. P., & Würbel, H. (2009). Environmental standardization: Cure or cause of poor reproducibility in animal experiments? Nature Methods, 6(4), 257– 261. 19333241
  23. Rousseeuw, P. J., & Verboven, S. (2002). Robust estimation in very small samples. Computational Statistics & Data Analysis, 40(4), 741– 758.
  24. Sauer, U. G., Hill, E. H., Curren, R. D., Raabe, H. A., Kolle, S. N., Teubner, W., … Landsiedel, R. (2016). Local tolerance testing under REACH: Accepted non‐animal methods are not on equal footing with animal tests. Alternatives to Laboratory Animals, 44(3), 281– 299. 27494627
  25. Spielmann, H., Hoffmann, S., Liebsch, M., Botham, P., Fentem, J. H., Eskes, C., … Zuang, V. (2007). The ECVAM international validation study on in vitro Tests for acute skin irritation: Report on the validity of the EPISKIN and EpiDerm assays and on the skin integrity function test. Alternatives to Laboratory Animals, 35, 559– 601. 18186667
  26. Urbisch, D., Mehling, A., Guth, K., Ramirez, T., Honarvar, N., Kolle, S., … Sakaguchi, H. (2015). Assessing skin sensitization hazard in mice and men using non‐animal test methods. Regulatory Toxicology and Pharmacology, 71(2), 337– 351. 25541156
  27. Worth, A., & Balls, M. (2002). The principles of validation and the ECVAM validation process. Alternatives to Laboratory Animals, 30(2), 15– 21. 12513647

Cited by

Sorted by citations

Authors
Gabbert, S., Mathea, M., Kolle, S. N., & Landsiedel, R.
Affiliations
  • Wageningen University & Research (NL)
  • BASF (Germany) (DE)
Corresponding author
Gabbert, Silke
ORCID
Venue
Risk Analysis
Publisher
Wiley
Published
2020-12-09
Volume
42
Issue
2
Pages
224-238
ISSN
  • 0272-4332 (ISSN-L)
  • 1539-6924
DOI
10.1111/risa.13648
PMC
PMC9292900
OpenAlex
W3111970711
Cited by
17
References
2
Citations by year
2026: 3 · 2025: 5 · 2024: 3 · 2022: 3 · 2021: 3
FWCI
1.39
Type
journal article
Language
en
Open access
Licence
Locations
  • Risk Analysis (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Socio-Environmental Systems Modeling (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
Topics
  • Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
  • Health Sciences › Medicine › Dermatology › Contact Dermatitis and Allergies
  • Life Sciences › Agricultural and Biological Sciences › Food Science › Agricultural safety and regulations
Keywords
classification threshold, toxicity testing, median absolute deviation, pooled standard deviation, nonparametric bootstrap, skin sensitization, skin corrosion, eye irritation, skin irritation, local lymph node assay, direct peptide reactivity assay
MeSH terms
Animals, Animal Testing Alternatives, Skin, Local Lymph Node Assay, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review