Distance-based protein folding powered by deep learning
Open accessPublished version of a preprint
Abstract
Direct coupling analysis (DCA) for protein folding has made very good progress, but it is not effective for proteins that lack many sequence homologs, even coupled with time-consuming conformation sampling with fragments. We show that we can accurately predict interresidue distance distribution of a protein by deep learning, even for proteins with ∼60 sequence homologs. Using only the geometric constraints given by the resulting distance matrix we may construct 3D models without involving extensive conformation sampling. Our method successfully folded 21 of the 37 CASP12 hard targets with a median family size of 58 effective sequence homologs within 4 h on a Linux computer of 20 central processing units. In contrast, DCA-predicted contacts cannot be used to fold any of these hard targets in the absence of extensive conformation sampling, and the best CASP12 group folded only 11 of them by integrating DCA-predicted contacts into fragment-based conformation sampling. Rigorous experimental validation in CASP13 shows that our distance-based folding server successfully folded 17 of 32 hard targets (with a median family size of 36 sequence homologs) and obtained 70% precision on the top L/5 long-range predicted contacts. The latest experimental validation in CAMEO shows that our server predicted correct folds for 2 membrane proteins while all of the other servers failed. These results demonstrate that it is now feasible to predict correct fold for many more proteins lack of similar structures in the Protein Data Bank even on a personal computer.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Machine learning-based structure prediction methods explicitly incorporate constraints on protein geometries.
“significant work has been done in the opposite direction of machine learning-based structure prediction methods that explicitly incorporate constraints on protein geometries 3,5,26,87-89.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [89]
Deep learning with residue covariation significantly improved structure prediction.
“Deep learning with residue covariation significantly improved structure prediction.18,19,23–25,33”
From AlphaFold 2: Why It Works and Its Implications for Understanding the Relationships of Protein Sequence, Structure, and Function (2021) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
I-TASSER: a unified platform for automated protein structure and function prediction
Roy, A., Küçükural, A., & Zhang, Y. (2010) · Nature Protocols · doi:10.1038/nprot.2010.5 · Cited by 6,093 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
The HHpred interactive server for protein homology detection and structure prediction
Söding, J., Biegert, A., & Lupas, A. N. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki408 · Cited by 3,703 · see citing works
Scoring function for automated assessment of protein structure template quality
Zhang, Y., & Skolnick, J. (2004) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.20264 · Cited by 2,392 · see citing works
HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment
Remmert, M., Biegert, A., Hauser, A., & Söding, J. (2012) · Nature Methods · doi:10.1038/nmeth.1818 · Cited by 2,288 · see citing works
Protein structure prediction and analysis using the Robetta server
Kim, D. E., Chivian, D. C., & Baker, D. A. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh468 · Cited by 2,099 · see citing works
UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928 · see citing works
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434 · see citing works
Version 1.2 of the Crystallography and NMR system
Brünger, A. T. (2007) · Nature Protocols · doi:10.1038/nprot.2007.406 · Cited by 1,325 · see citing works
Protein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144 · see citing works
High-Resolution Comparative Modeling with RosettaCM
Song, Y., DiMaio, F., Wang, R. Y., Kim, D. E., Miles, C., Brunette, T., Thompson, J., & Baker, D. A. (2013) · Structure · doi:10.1016/j.str.2013.08.005 · Cited by 1,079 · see citing works
Identification of direct residue contacts in protein–protein interaction by message passing
Weigt, M., White, R. A., Szurmant, H., Hoch, J. A., & Hwa, T. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0805923106 · Cited by 1,025 · see citing works
Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
Wang, S., Sun, S., Li, Z., Zhang, R., & Xu, J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005324 · Cited by 931 · see citing works
Self-citationJones, D. T., Buchan, D. W. A., Cozzetto, D., & Pontil, M. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr638 · Cited by 760 · see citing works
Uniclust databases of clustered and deeply annotated protein sequences and alignments
Mirdita, M., von den Driesch, L., Galiez, C., Martin, M. J., Söding, J., & Steinegger, M. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw1081 · Cited by 710 · see citing works
Kamisetty, H., Ovchinnikov, S., & Baker, D. A. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1314045110 · Cited by 672 · see citing works
Protein structure prediction from sequence variation
Marks, D. S., Hopf, T. A., & Sander, C. (2012) · Nature Biotechnology · doi:10.1038/nbt.2419 · Cited by 622 · see citing works
Emerging methods in protein co-evolution
de Juan, D., Pazos, F., & Valencia, A. (2013) · Nature Reviews Genetics · doi:10.1038/nrg3414 · Cited by 607 · see citing works
Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields
Wang, S., Peng, J., Ma, J., & Xu, J. (2016) · Scientific Reports · doi:10.1038/srep18962 · Cited by 556 · see citing works
Self-citationRaptorX-Property: a web server for protein structure property prediction
Wang, S., Li, W., Liu, S., & Xu, J. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw306 · Cited by 553 · see citing works
Self-citationProtein structure determination using metagenome sequence data
Ovchinnikov, S., Park, H., Varghese, N., Huang, P.-S., Pavlopoulos, G. A., Kim, D. E., Kamisetty, H., Kyrpides, N. C., & Baker, D. (2017) · Science · doi:10.1126/science.aah4043 · Cited by 520 · see citing works
Tools for comparative protein structure modeling and analysis
Eswar, N. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg543 · Cited by 481 · see citing works
CCMpred—fast and precise prediction of protein residue–residue contacts from correlated mutations
Seemayer, S., Gruber, M., & Söding, J. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu500 · Cited by 416 · see citing works
End-to-End Differentiable Learning of Protein Structure
AlQuraishi, M. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.006 · Cited by 371 · see citing works
Jones, D. T., Singh, T., Kościółek, T., & Tetchner, S. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu791 · Cited by 345 · see citing works
Deep architectures for protein contact map prediction
Lena, P. D., Nagata, K., & Baldi, P. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts475 · Cited by 273 · see citing works
Jones, D. T., & Kandathil, S. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty341 · Cited by 184 · see citing works
Hanson, J., Paliwal, K. K., Litfin, T., Yang, Y., & Zhou, Y. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty481 · Cited by 179 · see citing works
Assessment of contact predictions in CASP12: Co‐evolution and deep learning coming of age
Schaarschmidt, J., Monastyrskyy, B., Kryshtafovych, A., & Bonvin, A. M. J. J. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25407 · Cited by 177 · see citing works
Haas, J., Barbato, A., Behringer, D., Studer, G., Roth, S., Bertoni, M., Mostaguir, K., Gumienny, R., & Schwede, T. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25431 · Cited by 168 · see citing works
DNCON2: improved protein contact prediction using two-level deep convolutional neural networks
Adhikari, B., Hou, J., & Cheng, J. J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx781 · Cited by 168 · see citing works
Protein structure alignment beyond spatial proximity
Wang, S., Ma, J., Peng, J., & Xu, J. (2013) · Scientific Reports · doi:10.1038/srep01448 · Cited by 164 · see citing works
Self-citationPredicting protein residue–residue contacts using deep networks and boosting
Eickholt, J., & Cheng, J. J. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts598 · Cited by 161 · see citing works
CONFOLD: Residue-residue contact-guidedab initioprotein folding
Adhikari, B., Bhattacharya, D., Cao, R., & Cheng, J. J. (2015) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24829 · Cited by 143 · see citing works
Baldassi, C., Zamparo, M., Feinauer, C., Procaccini, A., Zecchina, R., Weigt, M., & Pagnani, A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092721 · Cited by 140 · see citing works
ComplexContact: a web server for inter-protein contact prediction using deep learning
Zeng, H., Wang, S., Zhou, T., Zhao, F., Li, X., Wu, Q., & Xu, J. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky420 · Cited by 138 · see citing works
Self-citationGlobal Fold Determination from a Small Number of Distance Restraints
Aszódi, A., Gradwell, M. J., & Taylor, W. R. (1995) · Journal of Molecular Biology · doi:10.1006/jmbi.1995.0436 · Cited by 133 · see citing works
Enhancing Evolutionary Couplings with Deep Convolutional Neural Networks
Liu, Y., Palmedo, P., Ye, Q., Berger, B., & Peng, J. (2018) · Cell Systems · doi:10.1016/j.cels.2017.11.014 · Cited by 114 · see citing works
Ma, J., Wang, S., Wang, Z., & Xu, J. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv472 · Cited by 107 · see citing works
Self-citationProtein structure prediction using Rosetta in CASP12
Ovchinnikov, S., Park, H., Kim, D. E., DiMaio, F., & Baker, D. A. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25390 · Cited by 105 · see citing works
Zhang, C., Mortuza, S. M., He, B., Wang, Y., & Zhang, Y. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25414 · Cited by 93 · see citing works
A conditional neural fields model for protein threading
Ma, J., Peng, J., Wang, S., & Xu, J. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts213 · Cited by 92 · see citing works
Self-citationAnalysis of deep learning methods for blind protein contact prediction in CASP12
Wang, S., Sun, S., & Xu, J. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25377 · Cited by 90 · see citing works
Self-citationFolding Membrane Proteins by Deep Transfer Learning
Wang, S., Li, Z., Yu, Y., & Xu, J. (2017) · Cell Systems · doi:10.1016/j.cels.2017.09.001 · Cited by 83 · see citing works
Self-citationProtein threading using residue co-variation and deep learning
Zhu, J., Wang, S., Bu, D., & Xu, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty278 · Cited by 69 · see citing works
Self-citationZhao, F., & Xu, J. (2012) · Structure · doi:10.1016/j.str.2012.04.003 · Cited by 52 · see citing works
Self-citationGao, Y., Wang, S., Deng, M., & Xu, J. (2018) · BMC Bioinformatics · doi:10.1186/s12859-018-2065-x · Cited by 52 · see citing works
Self-citationDistance matrix-based approach to protein structure prediction
Kloczkowski, A., Jernigan, R. L., Wu, Z., et al. (2009) · Journal of Structural and Functional Genomics · doi:10.1007/s10969-009-9062-2 · Cited by 50 · see citing works
Kukić, P., Mirabello, C., Tradigo, G., Walsh, I., Veltri, P., & Pollastri, G. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-6 · Cited by 49 · see citing works
Modeling Errors in NOE Data with a Log-normal Distribution Improves the Quality of NMR Structures
Rieping, W., Habeck, M., & Nilges, M. (2005) · Journal of the American Chemical Society · doi:10.1021/ja055092c · Cited by 41 · see citing works
Pietal, M. J., Bujnicki, J. M., & Kozlowski, L. P. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv390 · Cited by 38 · see citing works
End-to-end differentiable learning of protein structurePreprint
AlQuraishi, M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/265231 · Cited by 33 · see citing works
Knowledge-based machine learning methods for macromolecular 3D structure predictionPreprint
Wang, Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05061 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
A structural biology community assessment of AlphaFold2 applications
Akdel, M., Pires, D. E. V., Pardo, E. P., Jänes, J., Zalevsky, A. O., Mészáros, B., Bryant, P., Good, L. L., Laskowski, R. A., Pozzati, G., et al. (2022) · Nature Structural & Molecular Biology · doi:10.1038/s41594-022-00849-w · Cited by 675 · see citing works
De novo protein design by deep network hallucination
Anishchenko, I., Chidyausiku, T. M., Ovchinnikov, S., Pellock, S. J., & Baker, D. (2020) · doi:10.1101/2020.07.22.211482 · Cited by 629 · see citing works
Critical assessment of methods of protein structure prediction (CASP)—Round XIII
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2019) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25823 · Cited by 506 · see citing works
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
“For inputs, these methods use raw frequency and covariance statistics, or parameters of maximum-entropy sequence models fitted on MSAs (Jones & Kandathil, 2018; Xu, 2018; Senior et al., 2018).”
MethodHigh-resolution <i>de novo</i> structure prediction from primary sequence
Wu, R., Ding, F., Wang, R., Shen, R., Zhang, X., Luo, S., Su, C., Wu, Z., Xie, Q., Berger, B., et al. (2022) · doi:10.1101/2022.07.21.500999 · Cited by 392 · see citing works
Transposon-associated TnpB is a programmable RNA-guided DNA endonuclease
Karvelis, T., Druteika, G., Bigelyte, G., Budre, K., Zedaveinyte, R., Silanskas, A., Kazlauskas, D., Venclovas, Č., & Siksnys, V. (2021) · Nature · doi:10.1038/s41586-021-04058-1 · Cited by 348 · see citing works
Skolnick, J., Gao, M., Zhou, H., & Singh, S. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.1c01114 · Cited by 251 · see citing works
Improved protein structure refinement guided by deep learning based accuracy estimation
Hiranuma, N., Park, H., Baek, M., Anishchenko, I., Dauparas, J., & Baker, D. (2021) · Nature Communications · doi:10.1038/s41467-021-21511-x · Cited by 243 · see citing works
Deep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227 · see citing works
Greener, J. G., Kandathil, S. M., & Jones, D. T. (2019) · Nature Communications · doi:10.1038/s41467-019-11994-0 · Cited by 178 · see citing works
Deep learning techniques have significantly impacted protein structure prediction and protein design
Pearce, R., & Zhang, Y. (2021) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2021.01.007 · Cited by 127 · see citing works
AlphaFold2 reveals commonalities and novelties in protein structure space for 21 model organisms
Bordin, N., Sillitoe, I., Nallapareddy, V., Rauer, C., Lam, S. D., Waman, V. P., Sen, N., Heinzinger, M., Littmann, M., Kim, S., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-04488-9 · Cited by 103 · see citing works
Biophysical prediction of protein–peptide interactions and signaling networks using machine learning
Cunningham, J. M., Rogers, J. R., Koytiger, G., Sorger, P. K., & AlQuraishi, M. (2020) · Nature Methods · doi:10.1038/s41592-019-0687-1 · Cited by 108 · see citing works
AlQuraishi, M., & Sorger, P. K. (2021) · Nature Methods · doi:10.1038/s41592-021-01283-4 · Cited by 96 · see citing works
AlphaFold and the amyloid landscape
Pinheiro, F., Santos, J., & Ventura, S. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167059 · Cited by 88 · see citing works
Lindorff-Larsen, K., & Kragelund, B. B. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167196 · Cited by 85 · see citing works
Protein design via deep learning
Ding, W., Nakai, K., & Gong, H. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac102 · Cited by 66 · see citing works
Protein sequence‐to‐structure learning: Is this the end(‐to‐end revolution)?
Laine, E., Eismann, S., Elofsson, A., & Grudinin, S. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26235 · Cited by 44 · see citing works
“and extended the applicability of this conceptual framework to families with fewer sequences(15,16) and to the prediction of residueresidue distances.(17,18) These advances have significantly increased the protein structure modeling coverage of genomes, and also of bacterial interactomes.”
BackgroundDeep learning methods for 3D structural proteome and interactome modeling
Lee, D., Xiong, D., Wierbowski, S., Li, L., Liang, S., & Yu, H. (2022) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2022.102329 · Cited by 26 · see citing works
Limits and potential of combined folding and docking
Pozzati, G., Zhu, W., Bassot, C., Lamb, J., Kundrotas, P., & Elofsson, A. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab760 · Cited by 25 · see citing works
The role of local versus nonlocal physicochemical restraints in determining protein native structure
Skolnick, J., & Gao, M. (2021) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2020.10.008 · Cited by 21 · see citing works
A novel sequence alignment algorithm based on deep learning of the protein folding code
Gao, M., & Skolnick, J. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btaa810 · Cited by 18 · see citing works
Protein oligomer modeling guided by predicted interchain contacts in <scp>CASP14</scp>
Baek, M., Anishchenko, I., Park, H., Humphreys, I. R., & Baker, D. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26197 · Cited by 18 · see citing works
Modeled Structure of the Cell Envelope Proteinase of Lactococcus lactis
Hansen, E. B., & Marcatili, P. (2020) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2020.613986 · Cited by 16 · see citing works
Kandathil, S. M., Greener, J. G., Lau, A. M., & Jones, D. T. (2020) · doi:10.1101/2020.11.27.401232 · Cited by 8 · see citing works
- Authors
- Xu, J.
- Affiliations
- Toyota Technological Institute at Chicago (US)
- Corresponding author
- Xu, Jinbo
- ORCID
- Xu, Jinbo
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2019-08-09
- Volume
- 116
- Issue
- 34
- Pages
- 16856-16865
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6708335
- OpenAlex
- W2949952216
- Cited by
- 401
- References
- 54
- Citations by year
- 2026: 14 · 2025: 18 · 2024: 26 · 2023: 49 · 2022: 88 · 2021: 130 · 2020: 52 · 2019: 23 · 2018: 1
- FWCI
- 24.25
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- HHS | NIH | National Institute of General Medical Sciences (R01GM089753)
- NSF | Directorate for Biological Sciences (DBI-1564955)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Keywords
- protein folding, deep learning, direct coupling analysis, sequence homology, CASP13, cameos, membrane proteins, 3D structure prediction
- MeSH terms
- Deep Learning, Algorithms, Membrane Proteins, Models, Molecular, Time Factors, Sequence Alignment, Protein Folding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
