Deep Leakage from Gradients
Abstract
Exchanging gradients is a widely used method in modern multi-node machine learning system (e.g., distributed training, collaborative learning). For a long time, people believed that gradients are safe to share: i.e., the training data will not be leaked by gradient exchange. However, we show that it is possible to obtain the private training data from the publicly shared gradients. We name this leakage as Deep Leakage from Gradient and empirically validate the effectiveness on both computer vision and natural language processing tasks. Experimental results show that our attack is much stronger than previous approaches: the recovery is pixel-wise accurate for images and token-wise matching for texts. We want to raise people's awareness to rethink the gradient's safety. Finally, we discuss several possible strategies to prevent such deep leakage. The most effective defense method is gradient pruning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The observations can indeed effectively leak personal information required for regional development, such as reversal model updates, gradients, and opponents.
“Nevertheless, the observations can indeed effectively leak personal information required for regional development, such as reversal [97] model updates, gradients [98], and opponents.”
From A comprehensive survey on federated learning techniques for healthcare informatics (2023) · cites this paper as [98]
The incentive mechanism for federated learning can be classified into three categories: incentive allocation based on end-device data, device participation, and resources available at end devices.
“In a broad sense, the incentive mechanism for FL can be classified into three categories: incentive allocation based on end-device data, device participation, and resources available at end devices [66].”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [66]
If the server can access the client's local model directly, there is a great risk of privacy leakage because the server may recover the original training data from the client's gradients.
“It is worth noting that if the server can access the client's local model directly, there is a great risk of privacy leakage because the server may recover the original training data from the client's gradients [16], [17], [18], [19].”
From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [18]
Private data can be extracted indirectly even from shared information via model inversion of model updates, gradients themselves, and adversarial attacks.
“In particular, private data can be extracted indirectly even from the shared information, by means of some emerging techniques, such as model inversion of the model updates [56], gradients themselves [57] and adversarial attacks [58].”
From Handling privacy-sensitive medical data with federated learning: Challenges and future directions (2023) · cites this paper as [57]
Based on local model updates from an IoMT device, an attacker can infer and recover local data using construction attacks and/or predict the existence of a data sample in the local training dataset via membership inference attacks.
“For example, based on the local model updates from an IoMT device, the attacker can infer and recover the local data using the construction attacks [60] and/or predict the existence of a data sample in the local training dataset such as blood type, disease name, gender information, and any other private information, which can be done via membership inference attacks [61].”
From Federated learning for smart healthcare: A survey (2023) · cites this paper as [60]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Nonlinear total variation based noise removal algorithms
Rudin, L., Osher, S., & Fatemi, E. (1992) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 17,089 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334 · see citing works
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · see citing works
Privacy-Preserving Deep Learning via Additively Homomorphic Encryption
Phong, L. T., Aono, Y., Hayashi, T., Wang, L., & Moriai, S. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2787987 · Cited by 1,662 · see citing works
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1712.01887 · Cited by 1,134 · see citing works
Self-citationInverting Gradients -- How easy is it to break privacy in federated learning?Preprint
Geiping, J., Bauermeister, H., Dröge, H., & Moeller, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14053 · Cited by 749 · see citing works
Practical Secure Aggregation for Federated Learning on User-Held DataPreprint
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04482 · Cited by 468 · see citing works
How to Securely Outsource Cryptographic Computations
Hohenberger, S., & Lysyanskaya, A. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-30576-7_15 · Cited by 366 · see citing works
A Guide to Fully Homomorphic Encryption.
Armknecht, F., Boyd, C., Carr, C., Gjøsteen, K., Jäschke, A., Reuter, C. A., & Strand, M. (2015) · MADOC (University of Mannheim) · Cited by 210 · see citing works
Jochems, A., Deist, T. M., Naqa, I. E., Kessler, M. L., Mayo, C., Reeves, J., Jolly, S., Matuszak, M. M., Haken, R. K. T., van Soest, J., et al. (2017) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2017.04.021 · Cited by 150 · see citing works
Fine-grained preference-aware location search leveraging crowdsourced digital footprints from LBSNs
Yang, D., Zhang, D., Yu, Z., & Yu, Z. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493464 · Cited by 101 · see citing works
Rossi, F., & Gégout, C. (1994) · doi:10.1109/icnn.1994.374223 · Cited by 9 · see citing works
A Transprecision Floating-Point Platform for Ultra-Low Power ComputingPreprint
Tagliavini, G., Mach, S., Rossi, D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10374 · Cited by 1 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Data Poisoning Attacks Against Federated Learning Systems
Tolpegin, V., Truex, S., Gursoy, M. E., & Liu, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58951-6_24 · Cited by 927 · see citing works
Federated learning for smart healthcare: A survey
Nguyen, D. C., Pham, Q.-V., Pathirana, P. N., Ding, M., Seneviratne, A., Lin, Z., Dobre, O., & Hwang, W.-J. (2023) · ACM Computing Surveys · doi:10.1145/3501296 · Cited by 827 · see citing works
Yin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638 · see citing works
Privacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548 · see citing works
End-to-end privacy preserving deep learning on multi-institutional medical imaging
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., Usynin, D., Trask, A., Lima, I., Mancuso, J., Jungmann, F., Steinborn, M.-M., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00337-8 · Cited by 513 · see citing works
Federated learning for medical image analysis: A survey
Guan, H., Yap, P.-T., Bozoki, A., & Liu, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110424 · Cited by 518 · see citing works
Li, X., Gu, Y., Dvornek, N., Staib, L. H., Ventola, P., & Duncan, J. S. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101765 · Cited by 480 · see citing works
Federated learning for 6G communications: Challenges, methods, and future directions
Liu, Y., Yuan, X., Xiong, Z., Kang, J., Wang, X., & Niyato, D. (2020) · China Communications · doi:10.23919/jcc.2020.09.009 · Cited by 464 · see citing works
“3) Gradient Compression : The reason why adversaries can infer the local data of the devices is that the gradient information contains rich semantic information [27].”
BackgroundInfluentialLyu, L., Yu, H., Zhao, J., & Yang, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_1 · Cited by 444 · see citing works
Privacy preservation in federated learning: An insightful survey from the GDPR perspective
Truong, N., Sun, K., Wang, S., Guitton, F., & Guo, Y. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102402 · Cited by 399 · see citing works
Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey
Ali, M., Naeem, F., Tariq, M., & Kaddoum, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3181823 · Cited by 366 · see citing works
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362 · see citing works
Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3054610 · Cited by 318 · see citing works
Communication-Efficient Federated Learning for Digital Twin Edge Networks in Industrial IoT
Lu, Y., Huang, X., Zhang, K., Maharjan, S., & Zhang, Y. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3010798 · Cited by 316 · see citing works
Byzantine-Resilient Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041404 · Cited by 313 · see citing works
Rodríguez-Barroso, N., Jiménez-López, D., Luzón, M. V., Herrera, F., & Martínez-Cámara, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.011 · Cited by 296 · see citing works
Federated learning for computational pathology on gigapixel whole slide images
Lu, M. Y., Chen, R. J., Kong, D., Lipkova, J., Singh, R., Williamson, D. F. K., Chen, T. Y., & Mahmood, F. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102298 · Cited by 267 · see citing works
Vulnerabilities in federated learning
Bouacida, N., & Mohapatra, P. (2021) · IEEE Access · doi:10.1109/access.2021.3075203 · Cited by 248 · see citing works
Poisoning attacks in federated learning: A survey
Xia, G., Chen, J., Yu, C., & Ma, J. (2023) · IEEE Access · doi:10.1109/access.2023.3238823 · Cited by 212 · see citing works
Privacy-Preserving Federated Learning for Internet of Medical Things Under Edge Computing
Wang, R., Lai, J., Zhang, Z., Li, X., Vijayakumar, P., & Karuppiah, M. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3157725 · Cited by 203 · see citing works
The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
Liu, Y., Xu, L., Yuan, X., Wang, C., & Li, B. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796721 · Cited by 202 · see citing works
Privacy Risks of General-Purpose Language Models
Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00095 · Cited by 188 · see citing works
Medical imaging deep learning with differential privacy
Ziller, A., Usynin, D., Braren, R., Makowski, M., Rueckert, D., & Kaissis, G. (2021) · Scientific Reports · doi:10.1038/s41598-021-93030-0 · Cited by 172 · see citing works
Handling privacy-sensitive medical data with federated learning: Challenges and future directions
Aouedi, O., Sacco, A., Piamrat, K., & Marchetto, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3185673 · Cited by 165 · see citing works
POSEIDON: Privacy-Preserving Federated Neural Network Learning
Sav, S., Pyrgelis, A., Troncoso-Pastoriza, J. R., Froelicher, D., Bossuat, J.-P., Sousa, J. S., & Hubaux, J.-P. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24119 · Cited by 162 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Clustered federated learning with adaptive local differential privacy on heterogeneous IoT data
He, Z., Wang, L., & Cai, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3299947 · Cited by 143 · see citing works
Li, A., Sun, J., Zeng, X., Zhang, M., Li, H., & Chen, Y. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485929 · Cited by 139 · see citing works
Provably Secure Federated Learning against Malicious Clients
Cao, X., Jia, J., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16849 · Cited by 141 · see citing works
A comprehensive survey on federated learning techniques for healthcare informatics
Dasaradharami Reddy, K., & Gadekallu, T. R. (2023) · Computational Intelligence and Neuroscience · doi:10.1155/2023/8393990 · Cited by 126 · see citing works
Lo, J., Yu, T. T., Ma, D., Zang, P., Owen, J. P., Zhang, Q., Wang, R. K., Beg, M. F., Lee, A. Y., Jia, Y., et al. (2021) · Ophthalmology Science · doi:10.1016/j.xops.2021.100069 · Cited by 105 · see citing works
Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health
Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., & Jiang, J. (2022) · Humanity Driven AI · doi:10.1007/978-3-030-72188-6_6 · Cited by 85 · see citing works
Survey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77 · see citing works
Kerkouche, R., Ács, G., Castelluccia, C., & Genevès, P. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451859 · Cited by 65 · see citing works
Zhou, J., Chen, S., Wu, Y., Li, H., Zhang, B., Zhou, L., Hu, Y., Xiang, Z., Li, Z., Chen, N., et al. (2024) · Science Advances · doi:10.1126/sciadv.adh8601 · Cited by 63 · see citing works
Privacy-Enhancing Technologies in Federated Learning for the Internet of Healthcare Things: A Survey
Mosaiyebzadeh, F., Pouriyeh, S., Parizi, R. M., Sheng, Q. Z., Han, M., Zhao, L., Sannino, G., Ranieri, C. M., Ueyama, J., & Batista, D. M. (2023) · Electronics · doi:10.3390/electronics12122703 · Cited by 51 · see citing works
RevFRF: Enabling Cross-Domain Random Forest Training With Revocable Federated Learning
Liu, Y., Ma, Z., Yang, Y., Liu, X., Ma, J., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3104842 · Cited by 49 · see citing works
Ho, T.-T., Tran, K.-D., & Huang, Y. (2022) · Sensors · doi:10.3390/s22103728 · Cited by 44 · see citing works
Secure Partial Aggregation: Making Federated Learning More Robust for Industry 4.0 Applications
Gao, J., Zhang, B., Guo, X., Baker, T., Li, M., & Liu, Z. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3145837 · Cited by 41 · see citing works
DeSMP: Differential Privacy-exploited Stealthy Model Poisoning Attacks in Federated Learning
Hossain, M. T., Islam, S., Badsha, S., & Shen, H. (2021) · 2021 17th International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn53354.2021.00038 · Cited by 44 · see citing works
Backdoor attacks and defenses in feature-partitioned collaborative learning
Liu, Y., Yi, Z., & Chen, T. (2020) · arXiv · doi:10.48550/arxiv.2007.03608 · Cited by 38 · see citing works
Federated learning vulnerabilities, threats and defenses: A systematic review and future directions
Almutairi, S., & Barnawi, A. (2023) · Internet of Things · doi:10.1016/j.iot.2023.100947 · Cited by 35 · see citing works
A federated learning architecture for secure and private neuroimaging analysis
Stripelis, D., Gupta, U., Saleem, H., Dhinagar, N., Ghai, T., Anastasiou, C., Sánchez, R., Steeg, G. V., Ravi, S., Naveed, M., et al. (2024) · Patterns · doi:10.1016/j.patter.2024.101031 · Cited by 31 · see citing works
Practical Attribute Reconstruction Attack Against Federated Learning
Chen, C., Lyu, L., Yu, H., & Chen, G. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2022.3159236 · Cited by 30 · see citing works
Esmradi, A., Yip, D. W., & Chan, C. F. (2023) · arXiv · doi:10.48550/arxiv.2312.10982 · Cited by 24 · see citing works
Esmradi, A., Yip, D. W., & Chan, C. F. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-97-1274-8_6 · Cited by 24 · see citing works
Federated transfer learning with differential privacy for multi-omics survival analysis
Wen, G., & Li, L. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf166 · Cited by 17 · see citing works
Privacy Preserving Time-Series Forecasting of User Health Data Streams
Imtiaz, S., Horchidan, S.-F., Abbas, Z., Arsalan, M., Chaudhry, H. N., & Vlassov, V. (2020) · 2020 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata50022.2020.9378186 · Cited by 15 · see citing works
Gradient Obfuscation Gives a False Sense of Security in Federated Learning
Yue, K., Jin, R., Wong, C.-W., Baron, D., & Dai, H. (2022) · arXiv · doi:10.48550/arxiv.2206.04055 · Cited by 26 · see citing works
Quantifying and Localizing Usable Information Leakage from Neural Network Gradients
Mo, F., Borovykh, A., Malekzadeh, M., Demetriou, S., Gündüz, D., & Haddadi, H. (2021) · arXiv · doi:10.48550/arxiv.2105.13929 · Cited by 1 · see citing works
“For a batch size B > 1, the optimizer can have B! different permutations, so that choosing the right gradient descent direction is complicated - as a solution a single training sample can be updated in each optimization step [7], [15].”
BackgroundMethodInfluential
- Authors
- Zhu, L., & Han, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Corresponding author
- Zhu, Ligeng
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2020
- Pages
- 17-31
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2970408908
- Semantic Scholar
- 5507d267bbf0b4cdb9f893c3c0960a45016f7010
- Cited by
- 1,906
- References
- 24
- Citations by year
- 2026: 126 · 2025: 311 · 2024: 493 · 2023: 408 · 2022: 258 · 2021: 216 · 2020: 86 · 2019: 8
- FWCI
- 92.43
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- federated learning, privacy attacks, defense strategy, deep learning privacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
