Machine Learning with Membership Privacy using Adversarial Regularization
Open accessAbstract
Machine learning models leak significant amount of information about their training sets, through their predictions. This is a serious privacy concern for the users of machine learning as a service. To address this concern, in this paper, we focus on mitigating the risks of black-box inference attacks against machine learning models. We introduce a mechanism to train models with membership privacy, which ensures indistinguishability between the predictions of a model on its training data and other data points (from the same distribution). This requires minimizing the accuracy of the best black-box membership inference attack against the model. We formalize this as a min-max game, and design an adversarial training algorithm that minimizes the prediction loss of the model as well as the maximum gain of the inference attacks. This strategy, which can guarantee membership privacy (as prediction indistinguishability), acts also as a strong regularizer and helps generalizing the model. We evaluate the practical feasibility of our privacy mechanism on training deep neural networks using benchmark datasets. We show that the min-max strategy can mitigate the risks of membership inference attacks (near random guess), and can achieve this with a negligible drop in the model's prediction accuracy (less than 4%).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two solutions have been proposed to address privacy leakage in models: adding a regularization term in a loss function to avoid overfitting and improve robustness, and adding acceptable noise into model parameters to hinder privacy inference attacks.
“To address the problem of privacy leakage in the models, two solutions have been proposed: (1) adding a regularization term in a loss function to avoid overfitting and improve robustness; for example, this method can be applied to defend against membership inference attacks, [37] and (2) adding acceptable noise into the model parameters to hinder privacy inference attacks.”
From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [37]
The Stackelberg game was adopted to model dynamic variation in end devices computation resources and stable connection between IoMT devices and central aggregator in hospital.
“For instance, the Stackelberg game was adopted to model the dynamic variation in end devices computation resources and stable connection between IoMT devices and central aggregator in hospital [67].”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [67]
The Stackelberg game algorithm requires complete knowledge of federated learning devices and networks to deploy it fully.
“The drawback with this algorithm is that one needs complete knowledge of FL devices and networks to deploy it fully.”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [67]
Based on local model updates from an IoMT device, an attacker can infer and recover local data using construction attacks and/or predict the existence of a data sample in the local training dataset via membership inference attacks.
“For example, based on the local model updates from an IoMT device, the attacker can infer and recover the local data using the construction attacks [60] and/or predict the existence of a data sample in the local training dataset such as blood type, disease name, gender information, and any other private information, which can be done via membership inference attacks [61].”
From Federated learning for smart healthcare: A survey (2023) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,989 · see citing works
(2006) · Springer eBooks · Cited by 8,123 · see citing works
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
(2017) · International Conference on Machine Learning · Cited by 6,382 · see citing works
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Self-citationPractical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956 · see citing works
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Arjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225 · see citing works
SecureML: A System for Scalable Privacy-Preserving Machine Learning
Mohassel, P., & Zhang, Y. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.12 · Cited by 1,936 · see citing works
Nonlinear Programming: Analysis and Methods
Avriel, M. (1976) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,307 · see citing works
Nonlinear Programming Analysis and Methods
Rodin, E. Y. (1977) · Computers & Mathematics with Applications · doi:10.1016/0898-1221(77)90071-2 · Cited by 1,285 · see citing works
(2016) · International Conference on Machine Learning · Cited by 1,270 · see citing works
Homer, N., Szelinger, S., Redman, M., Duggan, D., Tembe, W., Muehling, J., Pearson, J. V., Stephan, D. A., Nelson, S. F., & Craig, D. W. (2008) · PLoS Genetics · doi:10.1371/journal.pgen.1000167 · Cited by 1,221 · see citing works
Solving large scale linear prediction problems using stochastic gradient descent algorithms
Zhang, T. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015332 · Cited by 1,130 · see citing works
Privacy Preserving Data Mining
Lindell, Y., & Pinkas, B. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44598-6_3 · Cited by 1,077 · see citing works
Adversarially Learned InferencePreprint
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 1,015 · see citing works
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001 · see citing works
Revealing information while preserving privacy
Dinur, I., & Kobbi Nissim, B. (2003) · ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/773153.773173 · Cited by 993 · see citing works
(2011) · PubMed · Cited by 990 · see citing works
Game theory meets network security and privacy
Manshaei, M. H., Zhu, Q., Alpcan, T., Bacşar, T., & Hubaux, J. (2013) · ACM Computing Surveys · doi:10.1145/2480741.2480742 · Cited by 818 · see citing works
Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
Bassily, R., Smith, A., & Thakurta, A. (2014) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2014.56 · Cited by 645 · see citing works
Semi-Supervised Learning with Generative Adversarial NetworksPreprint
Odena, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01583 · Cited by 539 · see citing works
Train faster, generalize better: stability of stochastic gradient descent
Hardt, M., Recht, B., & Singer, Y. (2016) · International Conference on Machine Learning · Cited by 516 · see citing works
Good Semi-supervised Learning That Requires a Bad GANPreprint
Dai, Z.-H., Yang, Z.-L., Yang, F., Cohen, W. W., & Salakhutdinov, R. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1705.09783 · Cited by 513 · see citing works
Oblivious multi-party machine learning on trusted processors
Ohrimenko, O., Schuster, F., Cédric Fournet, A., Mehta, A., Nowozin, S., Vaswani, K., & Costa, M. (2016) · MPG.PuRe (Max Planck Society) · Cited by 402 · see citing works
Shokri, R., Theodorakopoulos, G., Troncoso, C., et al. (2012) · Proceedings of the ACM Conference on Computer and Communications Security · doi:10.1145/2382196.2382261 · Cited by 394 · see citing works
Self-citationScalable Private Learning with PATE
Papernot, N., Song, S., Mironov, I., Raghunathan, A., Talwar, K., & Erlingsson, Ú. (2018) · arXiv · doi:10.48550/arxiv.1802.08908 · Cited by 373 · see citing works
Train faster, generalize better: Stability of stochastic gradient descentPreprint
Hardt, M., Recht, B., & Singer, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01240 · Cited by 358 · see citing works
Distributional Smoothing with Virtual Adversarial TrainingPreprint
Miyato, T., Maeda, S., Koyama, M., Nakae, K., & Ishii, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00677 · Cited by 340 · see citing works
Exposed! A Survey of Attacks on Private Data
Dwork, C., Smith, A., Steinke, T., & Ullman, J. (2017) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054123 · Cited by 259 · see citing works
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised LearningPreprint
Miyato, T., Maeda, S., Koyama, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03976 · Cited by 213 · see citing works
Learning your identity and disease from research papers
Wang, R., Li, Y. F., Wang, X., Tang, H., & Zhou, X. (2009) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/1653662.1653726 · Cited by 206 · see citing works
Genomic privacy and limits of individual detection in a pool
Sankararaman, S., Obozinski, G., Jordan, M. I., & Halperin, E. (2009) · Nature Genetics · doi:10.1038/ng.436 · Cited by 204 · see citing works
Chiron: Privacy-preserving Machine Learning as a ServicePreprint
Hunt, T., Song, C., Shokri, R., Shmatikov, V., & Witchel, E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05961 · Cited by 168 · see citing works
Self-citationRobust Traceability from Trace Amounts
Dwork, C., Smith, A., Steinke, T., Ullman, J., & Vadhan, S. (2015) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2015.46 · Cited by 142 · see citing works
Membership Privacy in MicroRNA-based Studies
Michael Backes, A., Berrang, P., Humbert, M., & Manoharan, P. (2016) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/2976749.2978355 · Cited by 133 · see citing works
Li, N., Qardaji, W. H., Su, D., Wu, Y., & Yang, W. (2013) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/2508859.2516686 · Cited by 129 · see citing works
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting SecretsPreprint
Carlini, N., Liu, C., Kos, J., Erlingsson, Ú., & Song, D. X. (2018) · arXiv (Cornell University) · Cited by 124 · see citing works
AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine LearningPreprint
Jia, J., & Gong, N. Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04810 · Cited by 119 · see citing works
Privacy Games: Optimal User-Centric Data Obfuscation
Shokri, R. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0024 · Cited by 87 · see citing works
Self-citationI Know What You See: Power Side-Channel Attack on Convolutional Neural Network AcceleratorsPreprint
Wei, L., Luo, B., Li, Y., Liu, Y., & Xu, Q. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05847 · Cited by 54 · see citing works
Stealing Hyperparameters in Machine LearningPreprint
Wang, B., & Gong, N. Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05351 · Cited by 52 · see citing works
Alvim, M. S., Chatzikokolakis, K., Kawamoto, Y., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68711-7_23 · Cited by 31 · see citing works
Differential privacy for the analyst via private equilibrium computation
Hsu, J., Roth, A., & Ullman, J. (2013) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/2488608.2488651 · Cited by 29 · see citing works
An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural NetworksPreprint
Koziński, M., Simon, L., & Jurie, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.02382 · Cited by 16 · see citing works
Privacy Risk in Machine Learning: Analyzing the Connection to OverfittingPreprint
Yeom, S., Giacomelli, I., Fredrikson, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01604 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, and Haoyang Zeng. 2017. Training GANs with Optimism. arXiv preprint arXiv:1711.00141 (2017). · arXiv
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems.
- Ninghui Li, Wahbeh Qardaji, Dong Su, Yi Wu, and Weining Yang. 2013. Member- ship privacy: a unifying framework for privacy definitions. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security. ACM.
- Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro. 2017. Knock Knock, Who's There? Membership Inference on Aggregate Location Data. arXiv preprint arXiv:1708.06145 (2017). · arXiv
- Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems. 2234-2242.
- Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction apis. In USENIX Security.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760 · see citing works
Self-citationFederated learning for smart healthcare: A survey
Nguyen, D. C., Pham, Q.-V., Pathirana, P. N., Ding, M., Seneviratne, A., Lin, Z., Dobre, O., & Hwang, W.-J. (2023) · ACM Computing Surveys · doi:10.1145/3501296 · Cited by 827 · see citing works
LOGAN: Membership Inference Attacks Against Generative Models
Hayes, J., Melis, L., Danezis, G., & De Cristofaro, E. (2019) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2019-0008 · Cited by 433 · see citing works
Generative Adversarial Networks: A Survey Toward Private and Secure Applications
Cai, Z., Xiong, Z., Xu, H., Wang, P., Li, W., & Pan, Y. (2022) · ACM Computing Surveys · doi:10.1145/3459992 · Cited by 376 · see citing works
Generative AI in medical practice: in-depth exploration of privacy and security challenges
Chen, Y., & Esmaeilzadeh, P. (2024) · Journal of Medical Internet Research · doi:10.2196/53008 · Cited by 353 · see citing works
“…been proposed: (1) adding a regularization term in a loss function to avoid overfitting and improve robustness; for example, this method can be applied to defend against membership inference attacks, [37] and (2) adding acceptable noise into the model parameters to hinder privacy inference attacks.”
Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey
Ali, M., Naeem, F., Tariq, M., & Kaddoum, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3181823 · Cited by 366 · see citing works
A Taxonomy of Attacks on Federated Learning
Jere, M. S., Farnan, T., & Koushanfar, F. (2021) · IEEE Security & Privacy · doi:10.1109/msec.2020.3039941 · Cited by 217 · see citing works
Privacy Risks of General-Purpose Language Models
Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00095 · Cited by 188 · see citing works
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195 · see citing works
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
Salem, A., Bhattacharya, A., Backes, M., Fritz, M., & Zhang, Y. (2020) · Universität des Saarlandes · doi:10.22028/d291-47370 · Cited by 157 · see citing works
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
Thieves on Sesame Street! Model Extraction of BERT-based APIs
Krishna, K., Tomar, G. S., Parikh, A. P., Papernot, N., & Iyyer, M. (2019) · arXiv · doi:10.48550/arxiv.1910.12366 · Cited by 111 · see citing works
Privacy-Preserving Adversarial Networks
Tripathy, A., Wang, Y., & Ishwar, P. (2019) · 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2019.8919758 · Cited by 87 · see citing works
Survey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77 · see citing works
Property Inference from Poisoning
Mahloujifar, S., Ghosh, E., & Chase, M. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833623 · Cited by 67 · see citing works
On Utility and Privacy in Synthetic Genomic Data
Oprisanu, B., Ganev, G., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24092 · Cited by 23 · see citing works
- Authors
- Nasr, M., Shokri, R., & Houmansadr, A.
- Affiliations
- University of Massachusetts Amherst (US)
- National University of Singapore (SG)
- Venue
- Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2018-10-15
- Pages
- 634-646
- ISSN
- 1543-7221 (ISSN-L)
- OpenAlex
- W2884943453
- Cited by
- 458
- References
- 54
- Citations by year
- 2026: 24 · 2025: 57 · 2024: 71 · 2023: 69 · 2022: 65 · 2021: 77 · 2020: 53 · 2019: 35 · 2018: 6 · 2017: 1
- FWCI
- 17.45
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- National University of Singapore (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1525642)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- membership inference attacks, adversarial regularization, adversarial training, min-max game, privacy-preserving machine learning, deep neural network, privacy-utility trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
