How Adverse Outcome Pathways Can Aid the Development and Use of Computational Prediction Models for Regulatory Toxicology
Open accessAbstract
Efforts are underway to transform regulatory toxicology and chemical safety assessment from a largely empirical science based on direct observation of apical toxicity outcomes in whole organism toxicity tests to a predictive one in which outcomes and risk are inferred from accumulated mechanistic understanding. The adverse outcome pathway (AOP) framework provides a systematic approach for organizing knowledge that may support such inference. Likewise, computational models of biological systems at various scales provide another means and platform to integrate current biological understanding to facilitate inference and extrapolation. We argue that the systematic organization of knowledge into AOP frameworks can inform and help direct the design and development of computational prediction models that can further enhance the utility of mechanistic and in silico data for chemical safety assessment. This concept was explored as part of a workshop on AOP-Informed Predictive Modeling Approaches for Regulatory Toxicology held September 24-25, 2015. Examples of AOP-informed model development and its application to the assessment of chemicals for skin sensitization and multiple modes of endocrine disruption are provided. The role of problem formulation, not only as a critical phase of risk assessment, but also as guide for both AOP and complementary model development is described. Finally, a proposal for actively engaging the modeling community in AOP-informed computational model development is made. The contents serve as a vision for how AOPs can be leveraged to facilitate development of computational prediction models needed to support the next generation of chemical safety assessment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AOP facilitates mechanistic interpretation of models, provides a better understanding of toxicity, and allows the development of new in vitro tests.
“AOP facilitates mechanistic interpretation of models, provides a better understanding of toxicity, and allows the development of new in vitro tests.82”
From QSAR without borders (2020) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Ankley, G. T., Bennett, R. S., Erickson, R. J., Hoff, D. J., Hornung, M. W., Johnson, R. D., Mount, D. R., Nichols, J. W., Russom, C. L., Schmieder, P. K., et al. (2009) · Environmental Toxicology and Chemistry · doi:10.1002/etc.34 · Cited by 2,734 · see citing works
Self-citationToxicity Testing in the 21st Century: A Vision and a Strategy
Krewski, D. R., Acosta, D., Andersen, M. E., Anderson, H. A., Bailar, J. C., Boekelheide, K., Brent, R., Charnley, G., Cheung, V. G., Green, S., et al. (2010) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2010.483176 · Cited by 1,696 · see citing works
DeepTox: toxicity prediction using deep learning
Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00080 · Cited by 1,081 · see citing works
Self-citationAdverse Outcome Pathway (AOP) Development I: Strategies and Principles
Villeneuve, D. L., Crump, D., García‐Reyero, N., Hecker, M., Hutchinson, T. H., LaLone, C. A., Landesmann, B., Lettieri, T., Munn, S., Malgorzata, N., et al. (2014) · Toxicological Sciences · doi:10.1093/toxsci/kfu199 · Cited by 733 · see citing works
Self-citationScience and Decisions: Advancing Risk Assessment
Abt, E., Rodricks, J. V., Levy, J. I., Zeise, L. A., & Burke, T. A. (2010) · Risk Analysis · doi:10.1111/j.1539-6924.2010.01426.x · Cited by 643 · see citing works
Kavlock, R. J., Chandler, K. J., Houck, K. A., Hunter, S., Judson, R., Kleinstreuer, N., Knudsen, T. B., Martin, M. T., Padilla, S., Reif, D. M., et al. (2012) · Chemical Research in Toxicology · doi:10.1021/tx3000939 · Cited by 496 · see citing works
Dialogue on Reverse‐Engineering Assessment and Methods
Stolovitzky, G. A., Monroe, D., & Califano, A. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1407.021 · Cited by 377 · see citing works
Judson, R., Magpantay, F. M. G., Chickarmane, V. S., Haskell, C., Tania, N., TAYLOR, J. D., Xia, M., Huang, R., Rotroff, D. M., Filer, D. L., et al. (2015) · Toxicological Sciences · doi:10.1093/toxsci/kfv168 · Cited by 309 · see citing works
Screening Chemicals for Estrogen Receptor Bioactivity Using a Computational Model
Browne, P., Judson, R. S., Casey, W. M., Kleinstreuer, N. C., & Thomas, R. S. (2015) · Environmental Science & Technology · doi:10.1021/acs.est.5b02641 · Cited by 276 · see citing works
Becker, R. A., Ankley, G. T., Edwards, S. W., Kennedy, S. W., Linkov, I., Meek, B., Sachana, M., Segner, H. E., van der Burg, B., Villeneuve, D. L., et al. (2015) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2015.04.004 · Cited by 270 · see citing works
Self-citationThyroid disrupting chemicals: mechanisms and mixtures
Crofton, K. M. (2008) · International Journal of Andrology · doi:10.1111/j.1365-2605.2007.00857.x · Cited by 256 · see citing works
Huang, R., Xia, M., Nguyễn, Ð.-T., Zhao, T., Sakamuru, S., Zhao, J., Shahane, S. A., Rossoshek, A., & Simeonov, A. M. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00085 · Cited by 229 · see citing works
Putting the parts together: Combining in vitro methods to test for skin sensitizing potentials
Bauch, C., Kolle, S. N., Ramı́rez, T., Eltze, T., Fabian, E. J., Mehling, A., Teubner, W., van Ravenzwaay, B., & Landsiedel, R. (2012) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2012.05.013 · Cited by 226 · see citing works
Applying machine learning techniques for ADME-Tox prediction: a review
Maltarollo, V. G., Gertrudes, J. C., Oliveira, P. R., & Honorio, K. M. (2014) · Expert Opinion on Drug Metabolism & Toxicology · doi:10.1517/17425255.2015.980814 · Cited by 176 · see citing works
The Adverse Outcome Pathway for Skin Sensitisation Initiated by Covalent Binding to Proteins
OECD (2014) · OECD series on testing and assessment · doi:10.1787/9789264221444-en · Cited by 166 · see citing works
OECD (2016) · OECD series on adverse outcome pathways · doi:10.1787/5jlv1m9d1g32-en · Cited by 162 · see citing works
Teeguarden, J., Tan, Y., Edwards, S. W., Leonard, J. A., Anderson, K. A., CORLEY, R. A., Kile, M. L., Simonich, S. L. M., Stone, D., Tanguay, R. L., et al. (2016) · Environmental Science & Technology · doi:10.1021/acs.est.5b05311 · Cited by 144 · see citing works
Self-citationBayesian integrated testing strategy to assess skin sensitization potency: from theory to practice
Jaworska, J. S., Dancik, Y., Kern, P., Gerberick, F., & Natsch, A. (2013) · Journal of Applied Toxicology · doi:10.1002/jat.2869 · Cited by 138 · see citing works
Systematic evaluation of non-animal test methods for skin sensitisation safety assessment
Reisinger, K., Hoffmann, S., Alépée, N., Ashikaga, T., Barroso, J., Elcombe, C. R., Gellatly, N., Galbiati, V., Gibbs, S., Groux, H., et al. (2014) · Toxicology in Vitro · doi:10.1016/j.tiv.2014.10.018 · Cited by 135 · see citing works
Self-citationMiller, D. H., & Ankley, G. T. (2004) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2004.05.005 · Cited by 122 · see citing works
Self-citationTakenouchi, O., Fukui, S., Okamoto, K., et al. (2015) · Journal of Applied Toxicology · doi:10.1002/jat.3127 · Cited by 120 · see citing works
A Curated Database of Rodent Uterotrophic Bioactivity
Kleinstreuer, N., Ceger, P. C., Allen, D., Strickland, J. A., Chang, X., Hamm, J. T., & Casey, W. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1510183 · Cited by 118 · see citing works
Patlewicz, G. Y., Simon, T. W., Rowlands, J. C., Budinsky, R. A., & Becker, R. A. (2015) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2015.02.011 · Cited by 108 · see citing works
Mackay, C. D. A. (2013) · ALTEX · doi:10.14573/altex.2013.4.473 · Cited by 106 · see citing works
Self-citationNatsch, A., Emter, R., Gfeller, H., Haupt, T., & Ellis, G. (2014) · Toxicological Sciences · doi:10.1093/toxsci/kfu229 · Cited by 105 · see citing works
Perkins, E. J., Ankley, G. T., Crofton, K. M., García‐Reyero, N., LaLone, C. A., Johnson, M. S., Tietge, J. E., & Villeneuve, D. L. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1306638 · Cited by 103 · see citing works
Self-citationSystems biology: Leading the revolution in ecotoxicology
García‐Reyero, N., & Perkins, E. J. (2010) · Environmental Toxicology and Chemistry · doi:10.1002/etc.401 · Cited by 103 · see citing works
Self-citationPerkins, E. J., Antczak, P., Burgoon, L. D., Francesco Falciani, B., García‐Reyero, N., Gutsell, S., Hodges, G., Kienzler, A., Knapen, D., McBride, M. T., et al. (2015) · Toxicological Sciences · doi:10.1093/toxsci/kfv181 · Cited by 102 · see citing works
Self-citationPatlewicz, G. Y., Kuseva, C. D., Kesova, A., et al. (2014) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2014.06.001 · Cited by 101 · see citing works
Prediction of human population responses to toxic compounds by a collaborative competition
Eduati, F., Mangravite, L. M., Wang, T., Tang, H., Bare, J. C., Huang, R., Norman, T., Kellen, M., Menden, M. P., Yang, J., et al. (2015) · Nature Biotechnology · doi:10.1038/nbt.3299 · Cited by 95 · see citing works
Guyton, K. Z., Kyle, A. D., Aubrecht, J., Cogliano, V. J., Eastmond, D. A., Jackson, M., Keshava, N., Sandy, M. S., Sonawane, B. R., Zhang, L., et al. (2008) · Mutation Research/Reviews in Mutation Research · doi:10.1016/j.mrrev.2008.10.001 · Cited by 93 · see citing works
Verification of systems biology research in the age of collaborative competition
Meyer, P., Alexopoulos, L. G., Bonk, T., Califano, A., Cho, C. R., de la Fuente, A., de Graaf, D., Hartemink, A. J., Hoeng, J., Ivanov, N. V., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1968 · Cited by 93 · see citing works
Prediction of Human Population Responses to Toxic Compounds by a Collaborative Competition
Bynagari, N. B. (2017) · Asian Journal of Humanity Art and Literature · doi:10.18034/ajhal.v4i2.577 · Cited by 91 · see citing works
van der Veen, J. W., Rorije, E., Emter, R., Natsch, A., Loveren, H. V., & Ezendam, J. (2014) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2014.04.018 · Cited by 89 · see citing works
Rhrissorrakrai, K., Belcastro, V., Bilal, E., Norel, R., Poussin, C., Mathis, C., Dulize, R., Ivanov, N. V., Alexopoulos, L. G., Rice, J. J., et al. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu611 · Cited by 87 · see citing works
Gilbert, M. E. (2011) · Toxicological Sciences · doi:10.1093/toxsci/kfr244 · Cited by 81 · see citing works
QSAR Modeling of Tox21 Challenge Stress Response and Nuclear Receptor Signaling Toxicity Assays
Capuzzi, S. J., Politi, R., Isayev, O., Farag, S., & Tropsha, A. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2016.00003 · Cited by 79 · see citing works
Applying the skin sensitisation adverse outcome pathway (AOP) to quantitative risk assessment
Maxwell, G., Mackay, C. D. A., Cubberley, R., Davies, M., Gellatly, N. L., Glavin, S. E., Gouin, T., Jacquoilleot, S., Moore, C., Pendlington, R. U., et al. (2013) · Toxicology in Vitro · doi:10.1016/j.tiv.2013.10.013 · Cited by 72 · see citing works
Self-citationMolecular Signaling Network Motifs Provide a Mechanistic Basis for Cellular Threshold Responses
Zhang, Q., Bhattacharya, S., Conolly, R. B., M.A., H. J. C. I., Kaminski, N. E., & Andersen, M. E. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1408244 · Cited by 71 · see citing works
Hirota, M., Fukui, S., Okamoto, K., Kurotani, S., Imai, N., Fujishiro, M., Kyotani, D., Kato, Y., Kasahara, T., Fujita, M., et al. (2015) · Journal of Applied Toxicology · doi:10.1002/jat.3105 · Cited by 60 · see citing works
Large-Scale Predictive Drug Safety: From Structural Alerts to Biological Mechanisms
García-Serna, R., Vidal, D., Remez, N., & Mestres, J. (2015) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.5b00260 · Cited by 53 · see citing works
Meyer, P., Hoeng, J., Rice, J. J., Norel, R., Sprengel, J., Stolle, K., Bonk, T., Corthesy, S., Royyuru, A. K., Peitsch, M. C., et al. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts116 · Cited by 52 · see citing works
Buick, J. K., Moffat, I. D., Williams, A., et al. (2015) · Environmental and Molecular Mutagenesis · doi:10.1002/em.21940 · Cited by 50 · see citing works
Kimber, I., Maxwell, G., Gilmour, N., et al. (2011) · Toxicology · doi:10.1016/j.tox.2011.11.007 · Cited by 47 · see citing works
Self-citationMeek, M. E., Barton, H. A., Bessems, J. G. M., et al. (2013) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2013.03.005 · Cited by 42 · see citing works
Self-citationLi, Z., Kroll, K. J., Jensen, K. M., Villeneuve, D. L., Ankley, G. T., Brian, J. V., Sepúlveda, M. S., Orlando, E. F., Lazorchak, J. M., Kostich, M. S., et al. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-63 · Cited by 40 · see citing works
Self-citationBercu, J. P., Jolly, R. A., Flagella, K. M., et al. (2010) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2010.08.002 · Cited by 37 · see citing works
Stefaniak, F. (2015) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00077 · Cited by 35 · see citing works
Koutsoukas, A., Amand, J. S., Mishra, M., & Huan, J. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2016.00011 · Cited by 35 · see citing works
Davies, M. J., Pendlington, R. U., Page, L., et al. (2010) · Toxicological Sciences · doi:10.1093/toxsci/kfq326 · Cited by 32 · see citing works
Gaining acceptance for the use of in vitro toxicity assays and QIVIVE in regulatory risk assessment
Meek, M. E., & Lipscomb, J. C. (2015) · Toxicology · doi:10.1016/j.tox.2015.01.010 · Cited by 31 · see citing works
Self-citationLumen, A., McNally, K., George, N. I., Fisher, J. W., & Loizou, G. D. (2015) · Frontiers in Pharmacology · doi:10.3389/fphar.2015.00107 · Cited by 27 · see citing works
Fisher, J. W., Li, S., Crofton, K. M., et al. (2013) · Toxicological Sciences · doi:10.1093/toxsci/kfs336 · Cited by 27 · see citing works
Wise, A. R., Parham, F., Axelrad, D. A., Guyton, K. Z., Portier, C. J., Zeise, L. A., Zoeller, R. T., & Woodruff, T. J. (2012) · Environmental Research · doi:10.1016/j.envres.2012.05.013 · Cited by 26 · see citing works
Pirone, J. R. (2014) · ALTEX · doi:10.14573/altex.1310151 · Cited by 26 · see citing works
Breen, M., Villeneuve, D. L., Ankley, G. T., et al. (2013) · Toxicological Sciences · doi:10.1093/toxsci/kft067 · Cited by 25 · see citing works
Self-citationMiller, D. H., Tietge, J. E., McMaster, M. E., et al. (2015) · Environmental Toxicology and Chemistry · doi:10.1002/etc.2972 · Cited by 23 · see citing works
A computational model for asynchronous oocyte growth dynamics in a batch-spawning fish
Li, Z., Villeneuve, D. L., Jensen, K. M., et al. (2011) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f2011-066 · Cited by 23 · see citing works
Self-citationThomas, R., Thomas, R. S., Auerbach, S. S., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063308 · Cited by 18 · see citing works
A Comparative Study of Machine Learning Algorithms Applied to Predictive Toxicology Data Mining
Neagu, D. C., Guo, G., Trundle, P. R., et al. (2007) · Alternatives to Laboratory Animals · doi:10.1177/026119290703500119 · Cited by 18 · see citing works
Watanabe, K. H., Mayo, M. L., Jensen, K. M., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0146594 · Cited by 17 · see citing works
Stacking prediction for a binary outcome
Gomes, C., Noçairi, H., Thomas, M., et al. (2012) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 16 · see citing works
Cheng, W.-Y., Zhang, Q., Schroeder, A. L., et al. (2016) · Toxicological Sciences · doi:10.1093/toxsci/kfw142 · Cited by 15 · see citing works
Parham, F., Wise, A. R., Axelrad, D. A., Guyton, K. Z., Portier, C. J., Zeise, L. A., Zoeller, R. T., & Woodruff, T. J. (2012) · Environmental Research · doi:10.1016/j.envres.2012.04.003 · Cited by 11 · see citing works
Biehl, M. L., Sadowski, P., Bhanot, G. V., Bilal, E., Dayarian, A., Meyer, P., Norel, R., Rhrissorrakrai, K., Zeller, M. D., & Hormoz, S. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu407 · Cited by 11 · see citing works
Breen, M., Villeneuve, D. L., Ankley, G. T., et al. (2016) · Comparative Biochemistry and Physiology Part C Toxicology & Pharmacology · doi:10.1016/j.cbpc.2016.02.002 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- OECD. (2010). Workshop report on OECD countries activities regarding testing, assessment and management of endocrine disrupters. Environment, Health and Safety Publications, Series on Testing and Assessment, No. 118, Vol. ENV/JM/MONO(2010)2. Organisation for Economic Cooperation and Development, Paris, France.
Cited by
Sorted by citations
Muratov, E. N., Bajorath, J., Sheridan, R. P., Tetko, I. V., Filimonov, D., Poroikov, V., Oprea, T. I., Baskin, I. I., Varnek, A., Roitberg, A., et al. (2020) · Chemical Society Reviews · doi:10.1039/d0cs00098a · Cited by 913 · see citing works
Ankley, G. T., Cureton, P., Hoke, R. A., Houde, M., Kumar, A., Kurias, J., Lanno, R., McCarthy, C., Newsted, J., Salice, C. J., et al. (2020) · Environmental Toxicology and Chemistry · doi:10.1002/etc.4869 · Cited by 454 · see citing works
Self-citationCiallella, H. L., & Zhu, H. (2019) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.8b00393 · Cited by 227 · see citing works
Gellatly, N., & Sewell, F. (2019) · Computational Toxicology · doi:10.1016/j.comtox.2019.03.003 · Cited by 43 · see citing works
- Authors
- Wittwehr, C., Aladjov, H., Ankley, G., Byrne, H. J., de Knecht, J., Heinzle, E., Klambauer, G., Landesmann, B., Luijten, M., MacKay, C., Maxwell, G., Meek, M. E. (., Paini, A., Perkins, E., Sobanski, T., Villeneuve, D., Waters, K. M., & Whelan, M.
- Affiliations
- European Commission (BE)
- Bulgarian Academy of Sciences (BG)
- Environmental Protection Agency (US)
- Saarland University (DE)
- Johannes Kepler University of Linz (AT)
- National Institute for Public Health and the Environment (NL)
- Unilever (United Kingdom) (GB)
- University of Ottawa (CA)
- U.S. Army Engineer Research and Development Center (US)
- Pacific Northwest National Laboratory (US)
- Corresponding author
- Wittwehr, Clemens
- ORCID
- Venue
- Toxicological Sciences
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-12-19
- Volume
- 155
- Issue
- 2
- Pages
- 326-336
- ISSN
- 1096-0929 (ISSN-L)
- 1096-6080
- PMC
- PMC5340205
- OpenAlex
- W2563115000
- Cited by
- 161
- References
- 66
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 7 · 2023: 11 · 2022: 16 · 2021: 19 · 2020: 21 · 2019: 28 · 2018: 19 · 2017: 21 · 2016: 1
- FWCI
- 35.36
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Toxicological Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Arrow - TU Dublin (Technological University Dublin) (submittedVersion, cc-by-nc-sa)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Joint Research Centre (European Commission) (submittedVersion)
- Rivm Repository (Netherlands National Institute for Public Health and the Environment) (submittedVersion)
- Rivm (National Institute for Public Health and the Environment) (submittedVersion)
- Funders
- U.S. Department of Energy (DE-AC05–76RL01830, 76RL01830, AC05-76RL01830, DE-AC05?, Contract DE-AC05-76RL01830)
- U.S. Environmental Protection Agency
- Battelle (AC05-76RL01830, DE- AC05-76RL01830, DE-AC05)
- University of Ottawa
- University of Leeds
- European Commission
- Liverpool John Moores University
- Universität des Saarlandes
- Technological University Dublin
- Rijksinstituut voor Volksgezondheid en Milieu
- National Institute of Environmental Health Sciences (P42ES016465)
- U.S. Army Corps of Engineers
- Pacific Northwest National Laboratory (DEAC0576RL01830, 76RL01830)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Life Sciences › Agricultural and Biological Sciences › Food Science › Agricultural safety and regulations
- Keywords
- adverse outcome pathways, computational prediction model, regulatory toxicology, chemical safety assessment, in silico modeling, mechanistic risk assessment, skin sensitization, endocrine disruption, problem formulation, extrapolation, knowledge organization
- MeSH terms
- Adverse Outcome Pathways, Animals, Computer Simulation, Humans, Toxicology, Toxicity Tests
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
