Towards Proving the Adversarial Robustness of Deep Neural Networks
Open accessAbstract
Autonomous vehicles are highly complex systems, required to function reliably in a wide variety of situations. Manually crafting software controllers for these vehicles is difficult, but there has been some success in using deep neural networks generated using machine-learning. However, deep neural networks are opaque to human engineers, rendering their correctness very difficult to prove manually; and existing automated techniques, which were not designed to operate on neural networks, fail to scale to large systems. This paper focuses on proving the adversarial robustness of deep neural networks, i.e. proving that small perturbations to a correctly-classified input to the network cannot cause it to be misclassified. We describe some of our recent and ongoing work on verifying the adversarial robustness of networks, and discuss some of the open questions we have encountered and how they might be addressed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Contemporary authors continue to find value in existing data quality characterisations and conceptual structures.
“contemporary authors continue to find value in existing data quality characterisations and conceptual structures.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [41]
Long lists of requirements have been found to deter practitioners from applying traditional standards.
“Long lists of requirements have been found to deter practitioners from applying traditional standards, with Kim et al. [41] showing that there is value in simplified frameworks that are tailored to a specific use case or technology.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [41]
There is value in simplified frameworks that are tailored to a specific use case or technology.
“there is value in simplified frameworks that are tailored to a specific use case or technology.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [41]
Kim et al. developed new frameworks for assessing and improving maturity of IoT data quality processes based on ISO 8000-61 and ISO 8000-62.
“Kim et al. [41], who developed new frameworks for assessing and improving the maturity of IoT data quality processes based on the standards ISO 8000-61 and ISO 8000-62.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [41]
There is value in simplified data quality frameworks that are tailored to specific use cases or technologies.
“It was also found that there is value in simplified data quality frameworks that are tailored to specific use cases or technologies [41].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [41]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
(2011) · Cited by 6,993 · see citing works
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · see citing works
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63387-9_5 · Cited by 1,706 · see citing works
Self-citationOnline Verification of Automated Road Vehicles Using Reachability Analysis
Althoff, M., & Dolan, J. M. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2014.2312453 · Cited by 420 · see citing works
An Abstraction-Refinement Approach to Verification of Artificial Neural Networks
Pulina, L., & Tacchella, A. (2010) · International Conference on Computer Aided Verification · doi:10.1007/978-3-642-14295-6_24 · Cited by 311 · see citing works
Measuring Neural Net Robustness with ConstraintsPreprint
Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A., & Criminisi, A. (2016) · arXiv (Cornell University) · doi:10.17863/cam.26484 · Cited by 257 · see citing works
Policy compression for aircraft collision avoidance systems
Julian, K. D., Lopez, J. A., Brush, J. S., Owen, M. P., & Kochenderfer, M. J. (2016) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.2016.7778091 · Cited by 253 · see citing works
Self-citationChallenging SMT solvers to verify neural networks
Pulina, L., & Tacchella, A. (2012) · AI Communications · doi:10.3233/aic-2012-0525 · Cited by 125 · see citing works
A Formally Verified Hybrid System for the Next-Generation Airborne Collision Avoidance System
Jeannin, J.-B., Ghorbal, K., Kouskoulas, Y., Gardner, R. W., Schmidt, A., Zawadzki, E., & Platzer, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-662-46681-0_2 · Cited by 67 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021 · see citing works
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P., Zhu, Q., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886017 · Cited by 1,733 · see citing works
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,412 · see citing works
Certified Defenses against Adversarial ExamplesPreprint
Raghunathan, A., Steinhardt, J., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09344 · Cited by 572 · see citing works
The Marabou Framework for Verification and Analysis of Deep Neural Networks
Katz, G., Huang, D. A., Ibeling, D., Julian, K., Lazarus, C., Lim, R., Shah, P., Thakoor, S., Wu, H., Zeljić, A., et al. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-25540-4_26 · Cited by 517 · see citing works
“The Marabou project builds upon our previous work on the Reluplex project [2,7,12,13,15,17], which focused on applying SMT-based techniques to the verification of DNNs. Marabou follows the Reluplex spirit in that it applies an SMT-based, lazy search technique: it iteratively searches for an…”
MethodSelf-citationA review of deep learning with special emphasis on architectures, applications and recent trends
Sengupta, S., Basak, S., Saikia, P., Paul, S., Tsalavoutis, V. A., Atiah, F. D., Ravi, V., & Peters, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105596 · Cited by 468 · see citing works
Certifying Some Distributional Robustness with Principled Adversarial TrainingPreprint
Sinha, A., Namkoong, H., Volpi, R., & Duchi, J. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10571 · Cited by 515 · see citing works
Review of Artificial Intelligence Adversarial Attack and Defense Technologies
Qiu, S., Liu, Q., Zhou, S., & Wu, C. (2019) · Applied Sciences · doi:10.3390/app9050909 · Cited by 412 · see citing works
Evaluating the Robustness of Neural Networks: An Extreme Value Theory ApproachPreprint
Weng, T.-W., Zhang, H., Chen, P., Yi, J., Su, D., Gao, Y., Hsieh, C., & Daniel, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10578 · Cited by 307 · see citing works
Qayyum, A., Usama, M., Qadir, J., & Al-Fuqaha, A. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2975048 · Cited by 257 · see citing works
Adversarial Examples: Attacks and Defenses for Deep LearningPreprint
Yuan, X., He, P. L., Zhu, Q., & Li, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07107 · Cited by 212 · see citing works
Leike, J., Martic, M., Krakovna, V., Ortega, P. A., Everitt, T., Lefrancq, A., Orseau, L., & Legg, S. (2017) · arXiv · doi:10.48550/arxiv.1711.09883 · Cited by 240 · see citing works
Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
Wang, J., Dong, G., Sun, J., Wang, X., & Zhang, P. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00126 · Cited by 168 · see citing works
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A SurveyPreprint
Akhtar, N., & Mian, A. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.00553 · Cited by 156 · see citing works
Girdhar, M., Hong, J., & Moore, J. (2023) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2023.3265363 · Cited by 126 · see citing works
Benchmarking Neural Network Robustness to Common Corruptions and Surface VariationsPreprint
Hendrycks, D., & Dietterich, T. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01697 · Cited by 146 · see citing works
Towards Security Threats of Deep Learning Systems: A Survey
He, Y., Meng, G., Chen, K., Hu, X., & He, J. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3034721 · Cited by 114 · see citing works
DeepSafe: A Data-Driven Approach for Assessing Robustness of Neural Networks
Gopinath, D., Katz, G., Păsăreanu, C., & Barrett, C. W. (2018) · Automated Technology for Verification and Analysis · doi:10.1007/978-3-030-01090-4_1 · Cited by 102 · see citing works
Self-citationAn Abstraction-Based Framework for Neural Network Verification
Elboher, Y. Y., Gottschlich, J., & Katz, G. (2020) · ScholarlyCommons (University of Pennsylvania) · Cited by 102 · see citing works
Self-citationAttacks and defenses in user authentication systems: A survey
Wang, X., Yan, Z., Zhang, R., & Zhang, P. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103080 · Cited by 98 · see citing works
An Abstraction-Based Framework for Neural Network Verification
Elboher, Y. Y., Gottschlich, J. E., & Katz, G. (2019) · International Conference on Computer Aided Verification · doi:10.1007/978-3-030-53288-8_3 · Cited by 146 · see citing works
Self-citationProvably Minimally-Distorted Adversarial ExamplesPreprint
Carlini, N., Katz, G., Barrett, C., & Dill, D. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.10207 · Cited by 97 · see citing works
Self-citationAnalyzing the Robustness of Nearest Neighbors to Adversarial ExamplesPreprint
Wang, Y., Jha, S., & Chaudhuri, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.03922 · Cited by 90 · see citing works
Adversarial sample detection for deep neural network through model mutation testingPreprint
Wang, J., DONG, G., Sun, J., Wang, X., & PEIXIN, Z. (2019) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 96 · see citing works
Towards Robust CNN-based Object Detection through Augmentation with Synthetic Rain Variations
Volk, G., Müller, S., von Bernuth, A., Hospach, D., & Bringmann, O. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917269 · Cited by 85 · see citing works
Engineering problems in machine learning systems
Kuwajima, H., Yasuoka, H., & Nakae, T. (2020) · Machine Learning · doi:10.1007/s10994-020-05872-w · Cited by 84 · see citing works
A Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86 · see citing works
Sanity-Check: Boosting the Reliability of Safety-Critical Deep Neural Network Applications
Ozen, E., & Orailoğlu, A. (2019) · Proceedings - Asian Test Symposium/Proceedings · doi:10.1109/ats47505.2019.000-8 · Cited by 76 · see citing works
Model-Based and Data-Driven Strategies in Medical Image Computing
Rueckert, D., & Schnabel, J. A. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2943836 · Cited by 73 · see citing works
RobOT: Robustness-Oriented Testing for Deep Learning Systems
Wang, J., Chen, J., Sun, Y., Ma, X., Wang, D., Sun, J., & Cheng, P. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00038 · Cited by 68 · see citing works
Adversarial machine learning: a review of methods, tools, and critical industry sectors
Pelekis, S., Koutroubas, T., Blika, A., Berdelis, A., Karakolis, E., Ntanos, C., Spiliotis, E., & Askounis, D. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11147-4 · Cited by 75 · see citing works
DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural NetworksPreprint
Gopinath, D., Katz, G., Păsăreanu, C., & Barrett, C. W. (2017) · arXiv.org · doi:10.48550/arxiv.1710.00486 · Cited by 94 · see citing works
Self-citationVerifying Deep-RL-Driven Systems
Kazak, Y., Barrett, C. W., Katz, G., & Schapira, M. (2019) · NetAI@SIGCOMM · doi:10.1145/3341216.3342218 · Cited by 83 · see citing works
Self-citationZhang, J., & Li, J. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106296 · Cited by 58 · see citing works
Countering Adversarial Attacks on Autonomous Vehicles Using Denoising Techniques: A Review
Kloukiniotis, A., Papandreou, A., Lalos, A. S., Kapsalas, P., Nguyen, D.-V., & Moustakas, K. (2022) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2022.3142612 · Cited by 56 · see citing works
Verifying Recurrent Neural Networks Using Invariant Inference
Jacoby, Y., Barrett, C., & Katz, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59152-6_3 · Cited by 51 · see citing works
Self-citationCombinatorial Testing for Deep Learning SystemsPreprint
Ma, L., Zhang, F., Xue, M., Li, B., Liu, Y., Zhao, J., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07723 · Cited by 50 · see citing works
Minimal Modifications of Deep Neural Networks using Verification
Goldberger, B., Katz, G., Adi, Y., & Keshet, J. (2020) · Logic Programming and Automated Reasoning · doi:10.29007/699q · Cited by 76 · see citing works
Self-citationReluplex: a calculus for reasoning about deep neural networks
Katz, G., Barrett, C. W., Dill, D., Julian, K. D., & Kochenderfer, M. J. (2021) · Formal methods in system design · doi:10.1007/s10703-021-00363-7 · Cited by 69 · see citing works
Self-citationGround-Truth Adversarial Examples
Carlini, N., Katz, G., Barrett, C., & Dill, D. L. (2018) · Cited by 47 · see citing works
Self-citationVerifying learning-augmented systems
Eliyahu, T., Kazak, Y., Katz, G., & Schapira, M. (2021) · Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication · doi:10.1145/3452296.3472936 · Cited by 99 · see citing works
Self-citationAn SMT-Based Approach for Verifying Binarized Neural Networks
Amir, G., Wu, H.-Z., Barrett, C. W., & Katz, G. (2020) · International Conference on Tools and Algorithms for Construction and Analysis of Systems · doi:10.1007/978-3-030-72013-1_11 · Cited by 67 · see citing works
Self-citationAdversarial Attack Type I: Cheat Classifiers by Significant Changes
Tang, S., Huang, X., Chen, M., Sun, C., & Yang, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2936378 · Cited by 37 · see citing works
Detecting Adversarial Samples for Deep Neural Networks through Mutation TestingPreprint
Wang, J., Sun, J., Zhang, P., & Wang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.05010 · Cited by 43 · see citing works
A comprehensive survey of robust deep learning in computer vision
Liu, J. L., & Jin, Y. (2023) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2023.10.002 · Cited by 36 · see citing works
Fairify: Fairness Verification of Neural Networks
Biswas, S., & Rajan, H. (2022) · International Conference on Software Engineering · doi:10.1109/icse48619.2023.00134 · Cited by 50 · see citing works
A review on AI Safety in highly automated driving
Wäschle, M., Thaler, F., Berres, A., Pölzlbauer, F., & Albers, A. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.952773 · Cited by 28 · see citing works
Hains, G., Jakobsson, A., & Khmelevsky, Y. (2018) · IEEE International Systems Conference (SysCon) · doi:10.1109/syscon.2018.8369576 · Cited by 28 · see citing works
Bridging Machine Learning and Cryptography in Defence Against Adversarial Attacks
Taran, O., Rezaeifar, S., & Voloshynovskiy, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11012-3_23 · Cited by 27 · see citing works
Validation of Image-Based Neural Network Controllers through Adaptive Stress Testing
Julian, K. D., Lee, R., & Kochenderfer, M. J. (2020) · 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294549 · Cited by 43 · see citing works
Self-citationSimplifying Neural Networks Using Formal Verification
Gokulanathan, S., Feldsher, A., Malca, A., Barrett, C. W., & Katz, G. (2020) · NASA Formal Methods · doi:10.1007/978-3-030-55754-6_5 · Cited by 36 · see citing works
Self-citationParallelization Techniques for Verifying Neural NetworksPreprint
Wu, H., Ozdemir, A., Zeljić, A., Irfan, A., Julian, K. D., Gopinath, D., Fouladi, S., Katz, G., Păsăreanu, C. S., & Barrett, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08440 · Cited by 28 · see citing works
Self-citationLow-Cost Error Detection in Deep Neural Network Accelerators with Linear Algorithmic Checksums
Ozen, E., & Orailoğlu, A. (2020) · Journal of Electronic Testing · doi:10.1007/s10836-020-05920-2 · Cited by 25 · see citing works
Testing Feedforward Neural Networks Training Programs
Braiek, H. B., & Khomh, F. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3529318 · Cited by 22 · see citing works
Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path FindingPreprint
Chen, T., Niu, W., Xiang, Y., Bai, X., Liu, J., Han, Z., & Li, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06752 · Cited by 23 · see citing works
Formal Verification of a Decision-Tree Ensemble Model and Detection of Its Violation Ranges
Sato, N., Kuruma, H., Nakagawa, Y., & Ogawa, H. (2020) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2019edp7120 · Cited by 20 · see citing works
Parallelization Techniques for Verifying Neural Networks
Wu, H.-Z., Ozdemir, A., Zeljić, A., Irfan, A., Julian, K. D., Gopinath, D., Fouladi, S., Katz, G., Păsăreanu, C., & Barrett, C. W. (2020) · Formal Methods in Computer-Aided Design · doi:10.34727/2020/isbn.978-3-85448-042-6_20 · Cited by 70 · see citing works
Self-citationEnhancing Robustness Verification for Deep Neural Networks via Symbolic Propagation
Yang, P.-F., Li, J.-L., Liu, J., Huang, C.-C., Li, R.-J., Chen, L.-Q., Huang, X., & Zhang, L.-J. (2021) · Formal Aspects of Computing · doi:10.1007/s00165-021-00548-1 · Cited by 20 · see citing works
Deep Neural Network Test Coverage: How Far Are We?Preprint
Chen, J., Yan, M. X., Wang, Z., Kang, Y., & Wu, Z. J. (2020) · arXiv (Cornell University) · Cited by 19 · see citing works
QuoTe : Quality-oriented Testing for Deep Learning Systems
Chen, J., Wang, J., Ma, X., Sun, Y., Sun, J., Zhang, P., & Cheng, P. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582573 · Cited by 18 · see citing works
Improving Robustness to Adversarial Examples by Encouraging Discriminative Features
Agarwal, C., Nguyen, A., & Schonfeld, D. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803601 · Cited by 18 · see citing works
Machine Learning Verification and Safety for Unmanned Aircraft - A Literature Study
Torens, C., Juenger, F., Schirmer, S., Schopferer, S., Maienschein, T., & Dauer, J. C. (2022) · AIAA SCITECH 2022 Forum · doi:10.2514/6.2022-1133 · Cited by 17 · see citing works
Deep Learning with Convolutional Neural Networks for Histopathology Image Analysis
Bošnački, D., van Riel, N. A. W., & Veta, M. (2019) · Computational biology · doi:10.1007/978-3-030-17297-8_17 · Cited by 13 · see citing works
Black-Box Testing of Deep Neural Networks
Byun, T., Rayadurgam, S., & Heimdahl, M. P. E. (2021) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre52982.2021.00041 · Cited by 13 · see citing works
Robustness evaluation for deep neural networks via mutation decision boundaries analysis
Lin, R., Zhou, Q., Wu, B., & Nan, X. (2022) · Information Sciences · doi:10.1016/j.ins.2022.04.020 · Cited by 13 · see citing works
Enhancing Robustness of LiDAR-Based Perception in Adverse Weather using Point Cloud Augmentations
Teufel, S., Gamerdinger, J., Volk, G., Gerum, C., & Bringmann, O. (2023) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55152.2023.10186696 · Cited by 13 · see citing works
Machine learning with requirements: A manifesto
Giunchiglia, E., Imrie, F., van der Schaar, M., & Lukasiewicz, T. (2024) · Neurosymbolic Artificial Intelligence · doi:10.3233/nai-240767 · Cited by 10 · see citing works
Minimal Multi-Layer Modifications of Deep Neural Networks
Refaeli, I., & Katz, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21222-2_4 · Cited by 10 · see citing works
Self-citationQuantifying DNN Model Robustness to the Real-World Threats
Zhong, Z., Hu, Z., & Chen, X. (2020) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn48063.2020.00033 · Cited by 8 · see citing works
An SMT-Based Approach for Verifying Binarized Neural Networks
Amir, G. (2021) · doi:10.26226/morressier.604907f41a80aac83ca25cda · Cited by 9 · see citing works
Case Study: Analysis of Autonomous Center Line Tracking Neural Networks
Kadron, I. B., Gopinath, D., Păsăreanu, C., & Yu, H.-F. (2021) · Verified Software: Theories, Tools, Experiments · doi:10.1007/978-3-030-95561-8_7 · Cited by 9 · see citing works
Interpreting Neural Networks Using Flip PointsPreprint
Yousefzadeh, R., & O’Leary, D. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08789 · Cited by 8 · see citing works
Towards Scalable Verification of RL-Driven SystemsPreprint
Amir, G., Schapira, M., & Katz, G. (2021) · arXiv.org · doi:10.48550/arxiv.2105.11931 · Cited by 4 · see citing works
Self-citationAssuring Safety-Critical Machine Learning-Enabled Systems: Challenges and Promise
Goodloe, A. E. (2023) · Computer · doi:10.1109/mc.2023.3266860 · Cited by 7 · see citing works
Assuring Safety-Critical Machine Learning Enabled Systems: Challenges and Promise
Goodloe, A. E. (2022) · 2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew55968.2022.00088 · Cited by 14 · see citing works
Revisiting deep neural network test coverage from the test effectiveness perspective
Yan, M. X., Chen, J., Cao, X., Wu, Z. J., Kang, Y., & Wang, Z. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2561 · Cited by 6 · see citing works
Robustness of Neural Networks to Parameter Quantization
Murthy, A., Gupta, H. D., & Islam, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-31514-6_9 · Cited by 6 · see citing works
When are Non-Parametric Methods Robust?Preprint
Bhattacharjee, R., & Chaudhuri, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06121 · Cited by 6 · see citing works
Towards Verifiable Specifications for Neural Networks in Autonomous Driving
Remeli, V., Morapitiye, S., Rövid, A., & Szalay, Z. (2019) · IEEE International Symposium on Computational Intelligence and Informatics (CINTI) · doi:10.1109/cinti-macro49179.2019.9105190 · Cited by 5 · see citing works
Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks
Ren, H., Chandrasekar, S. K., & Murugesan, A. (2019) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc43569.2019.9081635 · Cited by 4 · see citing works
Conv-Reluplex : A Verification Framework For Convolution Neural Networks (S)
Xu, J. (2021) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2021-085 · Cited by 4 · see citing works
Sample Complexity of Adversarially Robust Linear Classification on Separated DataPreprint
Bhattacharjee, R., Jha, S., & Chaudhuri, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10794 · Cited by 4 · see citing works
Pruning and Slicing Neural Networks using Formal VerificationPreprint
Lahav, O., & Katz, G. (2021) · Formal Methods in Computer-Aided Design · doi:10.48550/arxiv.2105.13649 · Cited by 27 · see citing works
Self-citationFormal Verification and Development of Living Assistance System
Abbas, A., Arslan, M., Kumar, G., & Saghar, K. (2021) · International Bhurban Conference on Applied Sciences and Technology (IBCAST) · doi:10.1109/ibcast51254.2021.9393272 · Cited by 3 · see citing works
Safe Predictors for Enforcing Input-Output SpecificationsPreprint
Mell, S., Brown, O. M., Goodwin, J. A., & Son, S.-H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11062 · Cited by 3 · see citing works
Protecting Classifiers From Attacks. A Bayesian Approach.Preprint
Gallego, V., Naveiro, R., Redondo, A., Insua, D. R., & Ruggeri, F. (2020) · arXiv (Cornell University) · Cited by 3 · see citing works
Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing
Nirala, A., Joshi, A., Sarkar, S., & Hegde, C. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00019 · Cited by 2 · see citing works
A Parallel Optimization Method for Robustness Verification of Deep Neural Networks
Lin, R., Zhou, Q., Nan, X., & Hu, T. (2024) · Mathematics · doi:10.3390/math12121884 · Cited by 2 · see citing works
Robustness Verification Boosting for Deep Neural Networks
Feng, C. (2019) · International Conference on Information Science and Control Engineering · doi:10.1109/icisce48695.2019.00112 · Cited by 2 · see citing works
A Robustness-Oriented Data Augmentation Method for DNN
Liu, M., Hong, W., Pan, W., & Feng, C. (2021) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c55045.2021.00011 · Cited by 2 · see citing works
gRoMA: A Tool for Measuring the Global Robustness of Deep Neural Networks
Levy, N., Yerushalmi, R., & Katz, G. (2023) · AISoLA · doi:10.1007/978-3-031-46002-9_9 · Cited by 2 · see citing works
Self-citationHains, G., Jakobsson, A., & Khmelevsky, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11334 · Cited by 2 · see citing works
ROBY: Evaluating the Robustness of a Deep Model by its Decision BoundariesPreprint
Chen, J., Wang, Z., Zheng, H., Xiao, J., & Ming, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10282 · Cited by 2 · see citing works
Verifying Global Neural Network Specifications using Hyperproperties
Boetius, D., & Leue, S. (2023) · Kalpa publications in computing · doi:10.29007/pvtn · Cited by 1 · see citing works
Uncertainty-aware Metamorphic Testing for Robust Object Detection Models
Wang, J., Ishimoto, Y., Kondo, M., Kamei, Y., & Ubayashi, N. (2023) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c60940.2023.00058 · Cited by 1 · see citing works
Analysis of Airworthiness Requirements for Application of Machine Learning in Airborne Systems
Ma, Z., Zhang, T.-J., Zheng, Z., Zhong, K., & Zhang, Y.-T. (2024) · 2024 10th International Symposium on System Security, Safety, and Reliability (ISSSR) · doi:10.1109/isssr61934.2024.00022 · Cited by 1 · see citing works
Theoretical analysis of norm selection for robustness verification of neural networks
Saengsawang, S., & Li, G. (2023) · Physical Communication · doi:10.1016/j.phycom.2023.102019 · Cited by 1 · see citing works
Simplifying Neural Networks with the Marabou Verification EnginePreprint
Gokulanathan, S., Feldsher, A., Malca, A., Barrett, C. W., & Katz, G. (2019) · arXiv.org · doi:10.48550/arxiv.1910.12396 · Cited by 4 · see citing works
Self-citationFault detection method based on high-level feedback and reconstruction for image recognition
Xu, J. X. (2021) · doi:10.32657/10356/146731 · Cited by 1 · see citing works
Consistent Non-Parametric Methods for Maximizing RobustnessPreprint
Bhattacharjee, R., & Chaudhuri, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09086 · Cited by 1 · see citing works
- Authors
- Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J.
- Affiliations
- Stanford University (US)
- Venue
- Electronic Proceedings in Theoretical Computer Science
- Publisher
- Open Publishing Association
- Published
- 2017-09-07
- Volume
- 257
- Pages
- 19-26
- ISSN
- 2075-2180 (ISSN-L)
- OpenAlex
- W2754537581
- Cited by
- 115
- References
- 17
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 5 · 2023: 13 · 2022: 9 · 2021: 16 · 2020: 24 · 2019: 22 · 2018: 14 · 2017: 7
- FWCI
- 10.94
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Electronic Proceedings in Theoretical Computer Science (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Formal Methods in Verification
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- adversarial robustness, deep neural network, formal verification, autonomous vehicles, neural network verification, adversarial perturbations, correctness proofs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
