The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks
Open accessAbstract
As cybersecurity detectors increasingly rely on machine learning mechanisms, attacks to these defenses escalate as well. Supervised classifiers are prone to adversarial evasion, and existing countermeasures suffer from many limitations. Most solutions degrade performance in the absence of adversarial perturbations; they are unable to face novel attack variants; they are applicable only to specific machine learning algorithms. We propose the first framework that can protect botnet detectors from adversarial attacks through deep reinforcement learning mechanisms. It automatically generates realistic attack samples that can evade detection, and it uses these samples to produce an augmented training set for producing hardened detectors. In such a way, we obtain more resilient detectors that can work even against unforeseen evasion attacks with the great merit of not penalizing their performance in the absence of specific attacks. We validate our proposal through an extensive experimental campaign that considers multiple machine learning algorithms and public datasets. The results highlight the improvements of the proposed solution over the state-of-the-art. Our method paves the way to novel and more robust cybersecurity detectors based on machine learning applied to network traffic analytics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This process enables the building of diffusion coefficient-based images depending on different sources and classifications.”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [73]
In the citing paper
“Moreover, GAN is used to create raw image datasets by multiple hospitals and then share those raw imprints rather than actual images for privacy preservation.”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [73]
In the citing paper
“It has been observed using experimental analysis that the algorithm provides 97% accuracy on cancer datasets and outperforms other non-FL-based schemes.”
From Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey (2023) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection
Buczak, A. L., & Guven, E. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2494502 · Cited by 3,332
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
One Pixel Attack for Fooling Deep Neural Networks
Su, J., Vargas, D. V., & Sakurai, K. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2890858 · Cited by 2,406
Chen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,864
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,658
Wild patterns: Ten years after the rise of adversarial machine learning
Biggio, B., & Roli, F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.023 · Cited by 1,563
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
Carlini, N., & Wagner, D. (2018) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00009 · Cited by 1,157
An empirical comparison of botnet detection methods
García, S., Grill, M., Stiborek, J., & Zunino, A. (2014) · Computers & Security · doi:10.1016/j.cose.2014.05.011 · Cited by 962
Evasion Attacks against Machine Learning at Test Time
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_25 · Cited by 890
Deep reinforcement learning for cyber security
Nguyen, T. T., & Reddi, V. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3121870 · Cited by 620
Adversarial Examples for Malware Detection
Grosse, K., Papernot, N., Manoharan, P., Michael Backes, A., & McDaniel, P. D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66399-9_4 · Cited by 584
SoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502
SoK: Towards the Science of security and privacy in machine learning
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 427
On the effectiveness of machine and deep learning for cyber security
Apruzzese, G., Colajanni, M., Ferretti, L., Guido, A., & Marchetti, M. (2018) · doi:10.23919/cycon.2018.8405026 · Cited by 420
Self-citationPractical Evasion of a Learning-Based Classifier: A Case Study
Šrndić, N., & Laskov, P. (2014) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2014.20 · Cited by 353
Evaluation of machine learning algorithms for intrusion detection system
Almseidin, M., Alzubi, M. M., Kovács, S., & Alkasassbeh, M. S. (2017) · IEEE International Symposium on Intelligent Systems and Informatics (SISY) · doi:10.1109/sisy.2017.8080566 · Cited by 334
Intriguing Properties of Adversarial ML Attacks in the Problem Space
Pierazzi, F., Pendlebury, F., Cortellazzi, J., & Cavallaro, L. (2020) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40000.2020.00073 · Cited by 326
Wide-scale botnet detection and characterization
Karasaridis, A., Rexroad, B., & Hoeflin, D. (2007) · Conference on Workshop on Hot Topics in Understanding Botnets · Cited by 320
Alfakih, T. M., Hassan, M. M., Gumaei, A., Savaglio, C., & Fortino, G. (2020) · IEEE Access · doi:10.1109/access.2020.2981434 · Cited by 318
Towards effective feature selection in machine learning-based botnet detection approaches
Samani, E. B. B., Jazi, H. H., Stakhanova, N., & Ghorbani, A. A. (2014) · IEEE Conference on Communications and Network Security · doi:10.1109/cns.2014.6997492 · Cited by 287
Flow-based network traffic generation using Generative Adversarial Networks
Ring, M., Schlör, D., Landes, D., & Andreas Hotho, A. (2018) · Computers & Security · doi:10.1016/j.cose.2018.12.012 · Cited by 261
Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks
Behzadan, V., & Munir, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-62416-7_19 · Cited by 247
Miller, D. J., Xiang, Z. J., & Kesidis, G. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2970615 · Cited by 246
Adversarial Machine Learning Applied to Intrusion and Malware Scenarios: A Systematic Review
Martins, N., Cruz, J. M., Cruz, T., & Abreu, P. H. (2020) · IEEE Access · doi:10.1109/access.2020.2974752 · Cited by 223
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning AttacksPreprint
Demontis, A., Melis, M., Pintor, M., Jagielski, M., Biggio, B., Oprea, A., Nita-Rotaru, C., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02861 · Cited by 219
Anderson, H. S., Woodbridge, J., & Filar, B. (2016) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/2996758.2996767 · Cited by 209
Usama, M., Asim, M., Latif, S., Qadir, J., & Al‐Fuqaha, A. (2019) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc.2019.8766353 · Cited by 208
Adversarial Deep Learning for Robust Detection of Binary Encoded Malware
Al-Dujaili, A., Huang, A., Hemberg, E., & O’Reilly, U.-M. (2018) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00020 · Cited by 206
Flow-based intrusion detection: Techniques and challenges
Umer, M. F., Sher, M., & Bi, Y. (2017) · Computers & Security · doi:10.1016/j.cose.2017.05.009 · Cited by 205
Improving DNN Robustness to Adversarial Attacks Using Jacobian Regularization
Jakubovitz, D., & Giryes, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_32 · Cited by 205
Learning to Evade Static PE Machine Learning Malware Models via Reinforcement LearningPreprint
Anderson, H. S., Kharkar, A., Filar, B., Evans, D., & Roth, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08917 · Cited by 181
Adversary Resistant Deep Neural Networks with an Application to Malware Detection
Wang, Q., Guo, W., Zhang, K., Ororbia, A. G., Xing, X., Liu, X., & Giles, C. L. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098158 · Cited by 177
An efficient flow-based botnet detection using supervised machine learning
Stevanovic, M., & Pedersen, J. M. (2014) · 2012 International Conference on Computing, Networking and Communications (ICNC) · doi:10.1109/iccnc.2014.6785439 · Cited by 129
On the Security of Machine Learning in Malware C&C Detection
Gardiner, J., & Nagaraja, S. (2016) · ACM Computing Surveys · doi:10.1145/3003816 · Cited by 125
Addressing Adversarial Attacks Against Security Systems Based on Machine Learning
Apruzzese, G., Colajanni, M., Ferretti, L., & Marchetti, M. (2019) · doi:10.23919/cycon.2019.8756865 · Cited by 123
Self-citationA P2P Botnet detection scheme based on decision tree and adaptive multilayer neural networks
Alauthman, M., Aslam, N., Zhang, L., Alasem, R., & Hossain, M. A. (2016) · Neural Computing and Applications · doi:10.1007/s00521-016-2564-5 · Cited by 117
Evading Classifiers by Morphing in the Dark
Dang, H., Huang, Y., & Chang, E.-C. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3133978 · Cited by 111
Unsupervised Anomaly Based Botnet Detection in IoT Networks
Nõmm, S., & Bahşi, H. (2018) · IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2018.00171 · Cited by 111
An enhancing framework for botnet detection using generative adversarial networks
Yin, C., Zhu, Y., Liu, S., Fei, J., & Zhang, H. (2018) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd.2018.8396200 · Cited by 102
Andbot: towards advanced mobile botnets
Cui, X., Binxing, F., Yin, L., Liu, X., & Zang, T. (2011) · Cited by 99
Evading Anti-Malware Engines With Deep Reinforcement Learning
Fang, Z., Wang, J., Li, B., Wu, S., Zhou, Y., & Huang, H. (2019) · IEEE Access · doi:10.1109/access.2019.2908033 · Cited by 97
Malwise—An Effective and Efficient Classification System for Packed and Polymorphic Malware
Cesare, S., Xiang, Y., & Zhou, W. (2012) · IEEE Transactions on Computers · doi:10.1109/tc.2012.65 · Cited by 91
Distributed response to network intrusions using multiagent reinforcement learning
Malialis, K., & Kudenko⋆, D. (2015) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2015.01.013 · Cited by 76
Coevolution of Mobile Malware and Anti-Malware
Şen, S., Aydogan, E., & Aysan, A. I. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2824250 · Cited by 75
Evading Machine Learning Botnet Detection Models via Deep Reinforcement Learning
Wu, D., Fang, B., Wang, J., Liu, Q., & Cui, X. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761337 · Cited by 72
Deep learning to detect botnet via network flow summaries
Pektaş, A., & Acarman, T. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3595-x · Cited by 57
Evading Botnet Detectors Based on Flows and Random Forest with Adversarial Samples
Apruzzese, G., & Colajanni, M. (2018) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2018.8548327 · Cited by 53
Self-citationAn analysis of network traffic classification for botnet detection
Stevanovic, M., & Pedersen, J. M. (2015) · International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA) · doi:10.1109/cybersa.2015.7361120 · Cited by 52
FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments
Chernikova, A., & Oprea, A. (2022) · ACM Transactions on Privacy and Security · doi:10.1145/3544746 · Cited by 52
Evaluating the effectiveness of Adversarial Attacks against Botnet Detectors
Apruzzese, G., Colajanni, M., & Marchetti, M. (2019) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2019.8935039 · Cited by 51
Self-citationOn the Top Threats to Cyber Systems
Kettani, H., & Wainwright, P. (2019) · doi:10.1109/infoct.2019.8711324 · Cited by 50
Biggio, B., Corona, I., He, Z.-M., Chan, P. P. K., Giacinto, G., Yeung, D. S., & Roli, F. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-20248-8_15 · Cited by 47
An Improved Sarsa($\lambda$ ) Reinforcement Learning Algorithm for Wireless Communication Systems
Jiang, H., Gui, R., Chen, Z., Wu, L., Dang, J., & Zhou, J. (2019) · IEEE Access · doi:10.1109/access.2019.2935255 · Cited by 42
BotGM: Unsupervised graph mining to detect botnets in traffic flows
Lagraa, S., François, J., Lahmadi, A., Miner, M., Hammerschmidt, C., & State, R. (2017) · Cyber Security in Networking Conference (CSNet) · doi:10.1109/csnet.2017.8241990 · Cited by 36
Enhancing network intrusion detection classifiers using supervised adversarial training
Yin, C., Zhu, Y., Liu, S., Fei, J., & Zhang, H. (2019) · The Journal of Supercomputing · doi:10.1007/s11227-019-03092-1 · Cited by 36
A Comparison of Machine Learning Approaches to Detect Botnet Traffic
Abraham, B., Mandya, A., Bapat, R., Alali, F., Brown, D. E., & Veeraraghavan, M. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489096 · Cited by 33
Soll, M., Hinz, T., Magg, S., & Wermter, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30508-6_54 · Cited by 33
Adversarial Training of Gradient-Boosted Decision Trees
Calzavara, S., Lucchese, C., & Tolomei, G. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3358149 · Cited by 30
Feature Selection Strategies for HTTP Botnet Traffic Detection
Letteri, I., Penna, G. D., & Caianiello, P. (2019) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw.2019.00029 · Cited by 22
Securing Malware Cognitive Systems against Adversarial Attacks
Ji, Y., Bowman, B., & Huang, H. H. (2019) · IEEE International Conference on Cognitive Computing (ICCC) · doi:10.1109/iccc.2019.00014 · Cited by 21
Flow based botnet detection through semi-supervised active learning
Qiu, Z., Miller, D. J., & Kesidis, G. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952584 · Cited by 20
Practical Traffic-space Adversarial Attacks on Learning-based NIDSs.Preprint
Han, D., Wang, Z., Zhong, Y. C., Chen, W., Yang, J., Lu, S., Shi, X., & Xia, Y. Y. (2020) · arXiv (Cornell University) · Cited by 20
Pavlović, D. (2011) · New Security Paradigms Workshop · doi:10.1145/2073276.2073289 · Cited by 19
TorBot Stalker: Detecting Tor Botnets Through Intelligent Circuit Data Analysis
Fajana, O., Owenson, G., & Cocea, M. (2018) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2018.8548313 · Cited by 15
Query-efficient label-only attacks against black-box machine learning models
Ren, Y., Zhou, Q., Wang, Z., Wu, T., Wu, G., & Choo, K. R. (2019) · Computers & Security · doi:10.1016/j.cose.2019.101698 · Cited by 15
Botnet detection using ensemble classifiers of network flow
Algelal, Z. M., Aldhaher, E. A. G., Abdul-wadood, D. N., & Sagheer, R. H. A. A. (2020) · International Journal of Electrical and Computer Engineering (IJECE) · doi:10.11591/ijece.v10i3.pp2543-2550 · Cited by 13
Feature partitioning for robust tree ensembles and their certification in adversarial scenarios
Calzavara, S., Lucchese, C., Marcuzzi, F., & Orlando, S. (2021) · EURASIP Journal on Information Security · doi:10.1186/s13635-021-00127-0 · Cited by 11
Why is cybersecurity not a human-scale problem anymore?
Banga, G. (2020) · Communications of the ACM · doi:10.1145/3347144 · Cited by 10
Intrusion Detection via Wide and Deep Model
Li, Z., Qin, Z., & Shen, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30490-4_57 · Cited by 8
FENCE: Feasible Evasion Attacks on Neural Networks in Constrained EnvironmentsPreprint
Chernikova, A., & Oprea, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10480 · Cited by 8
Evasion Attacks Based on Wasserstein Generative Adversarial Network
Zhang, J., Yan, Q., & Wang, M. (2019) · Computing, Communications and IoT Applications (ComComAp) · doi:10.1109/comcomap46287.2019.9018647 · Cited by 7
Simple Black-Box Adversarial Examples Generation with Very Few Queries
Senzaki, Y., Ohata, S., & Matsuura, K. (2020) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2019inp0002 · Cited by 6
Feature Partitioning for Robust Tree Ensembles and their Certification in Adversarial ScenariosPreprint
Calzavara, S., Lucchese, C., Marcuzzi, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03295 · Cited by 1
Research works printed but not linked: 2
- anderson, 2017, Evading machine learning malware detection
- 2020, Checkpoint 2020 Security Report
Cited by
Sorted by citations
Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey
Ali, M., Naeem, F., Tariq, M., & Kaddoum, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3181823 · Cited by 366
Federated Deep Learning for Zero-Day Botnet Attack Detection in IoT-Edge Devices
Popoola, S. I., Ande, R., Adebisi, B., Gui, G., Hammoudeh, M. A. A., & Jogunola, O. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3100755 · Cited by 375
Ferrag, M. A., Friha, O., Maglaras, L., Janicke, H., & Shu, L. (2021) · IEEE Access · doi:10.1109/access.2021.3118642 · Cited by 342
Zhou, X., Liang, W., Li, W., Yan, K., Shimizu, S., & Wang, K. I.-K. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3130434 · Cited by 325
The Role of Machine Learning in Cybersecurity
Apruzzese, G., Laskov, P., de, E. M., Mallouli, W., Búrdalo, L., Grammatopoulos, A. V., & Franco, F. D. (2022) · Digital Threats Research and Practice · doi:10.1145/3545574 · Cited by 359
Self-citation
Metadata
- Authors
- Apruzzese, G., Andreolini, M., Marchetti, M., Venturi, A., & Colajanni, M.
- Affiliations
- University of Liechtenstein (LI)
- University of Modena and Reggio Emilia (IT)
- University of Bologna (IT)
- Venue
- IEEE Transactions on Network and Service Management
- Published
- 2020-12
- Volume
- 17
- Issue
- 4
- Pages
- 1975-1987
- ISSN
- 1932-4537 (ISSN-L)
- 2373-7379
- OpenAlex
- W3111563271
- Cited by
- 117
- References
- 77
- Citations by year
- 2026: 12 · 2025: 19 · 2024: 28 · 2023: 21 · 2022: 23 · 2021: 12 · 2020: 1
- FWCI
- 5.60
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- IEEE Transactions on Network and Service Management (publishedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, other-oa)
- Opin vísindi (Opin vísindi) (submittedVersion, cc-by-nc-sa)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- botnet detection, evasion attacks, deep reinforcement learning, adversarial attack generation, data augmentation, network traffic analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
