CoChat
Sign Up
Cited by 53Safety Systems Engineering in AutonomyAdversarial Robustness in Machine LearningExplainable Artificial Intelligence (XAI)

Taxonomy of Machine Learning Safety: A Survey and Primer

Open access
Mohseni, Sina, Wang, Haotao, Xiao, Chaowei, Yu, Zhiding, Wang, Zhangyang, Yadawa, Jay·2023·ACM Computing Surveys
DOI 10.1145/3551385
Open this paper in CoChat — free accountOpen PDF

Abstract

The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations. Research explores different approaches to improve ML dependability by proposing new models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. However, there is a missing connection between ongoing ML research and well-established safety principles. In this article, we present a structured and comprehensive review of ML techniques to improve the dependability of ML algorithms in uncontrolled open-world settings. From this review, we propose the Taxonomy of ML Safety that maps state-of-the-art ML techniques to key engineering safety strategies. Our taxonomy of ML safety presents a safety-oriented categorization of ML techniques to provide guidance for improving dependability of the ML design and development. The proposed taxonomy can serve as a safety checklist to aid designers in improving coverage and diversity of safety strategies employed in any given ML system.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3551385

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 264

  1. Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems.
  2. Ahmed Shihab Albahri, A. A. Zaidan, Osamah Shihab Albahri, B. B. Zaidan, and M. A. Alsalem. 2018. Real-time fault- tolerant mHealth system: Comprehensive review of healthcare services, opens issues, challenges and methodological aspects. J. Med. Syst. 42, 8 (2018), 1-56.
  3. Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In Pro- ceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice.
  4. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete prob- lems in AI safety. arXiv:1606.06565. Retrieved from https://arxiv.org/abs/1606.06565. · arXiv
  5. Rob Ashmore, Radu Calinescu, and Colin Paterson. 2021. Assuring the machine learning lifecycle: Desiderata, meth- ods, and challenges. ACM Comput. Surv. (2021).
  6. Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Cir- cumventing defenses to adversarial examples. In Proceedings of the International Conference on Machine Learning. 274-283.
  7. Muhammad Awais, Fengwei Zhou, Hang Xu, Lanqing Hong, Ping Luo, Sung-Ho Bae, and Zhenguo Li. 2021. Adversar- ial robustness for unsupervised domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision. 8568-8577.
  8. Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, 7 (2015), e0130140.
  9. Yutong Bai, Jieru Mei, Alan L. Yuille, and Cihang Xie. 2021. Are transformers more robust than CNNs? In Advances in Neural Information Processing Systems.
  10. Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. 2018. Metareg: Towards domain generalization using meta-regularization. In Advances in Neural Information Processing Systems. 1006-1016.
  11. Gagan Bansal and Daniel S. Weld. 2018. A coverage-based utility model for identifying unknown unknowns. In Proceedings of the AAAI Conference on Artificial Intelligence.
  12. Nitin Bansal, Xiaohan Chen, and Zhangyang Wang. 2018. Can we gain more from orthogonality regularizations in training deep CNNs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems.
  13. Ezio Bartocci, Jyotirmoy Deshmukh, Alexandre Donzé, Georgios Fainekos, Oded Maler, Dejan Ničković, and Sriram Sankaranarayanan. 2018. Specification-based monitoring of cyber-physical systems: A survey on theory, tools and applications. In Lectures on Runtime Verification. Springer, 135-175.
  14. Liron Bergman and Yedid Hoshen. 2020. Classification-based anomaly detection for general data. arXiv:2005.02359. Retrieved from https://arxiv.org/abs/2005.02359. · arXiv
  15. Lucas Beyer, Olivier J Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord. 2020. Are we done with ImageNet? arXiv:2006.07159. Retrieved from https://arxiv.org/abs/2006.07159. · arXiv
  16. Mariusz Bojarski, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Larry J. Ackel, Urs Muller, Phil Yeres, and Karol Zieba. [n.d.]. Visualbackprop: Efficient visualization of cnns for autonomous driving. In Proceedings of the IEEE International Conference on Robotics and Automation.
  17. Mohamed-Rafik Bouguelia, Slawomir Nowaczyk, K. C. Santosh, and Antanas Verikas. 2018. Agreeing to disagree: Active learning with noisy labels without crowdsourcing. Int. J. Mach. Learn. Cybernet. 9, 8 (2018), 1307-1319.
  18. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. Openai gym. arXiv:1606.01540. Retrieved from https://arxiv.org/abs/1606.01540. · arXiv
  19. Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. 2017. Adversarial patch. arXiv:1712.09665. Retrieved from https://arxiv.org/abs/1712.09665. · arXiv
  20. Nicholas Carlini and David Wagner. 2017. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security.
  21. Supratik Chakraborty, Daniel Fremont, Kuldeep Meel, Sanjit Seshia, and Moshe Vardi. 2014. Distribution-aware sampling and weighted model counting for SAT. In Proceedings of the AAAI Conference on Artificial Intelligence.
  22. Robin Chan, Krzysztof Lis, Svenja Uhlemeyer, Hermann Blum, Sina Honari, Roland Siegwart, Pascal Fua, Mathieu Salzmann, and Matthias Rottmann. 2021. SegmentMeIfYouCan: A benchmark for anomaly segmentation. In Ad- vances in Neural Information Processing Systems.
  23. Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Animashree Anandkumar, Sanja Fidler, and Jose M. Alvarez. 2021. Image-level or object-level? A tale of two resampling strategies for long-tailed detection. In Proceedings of the In- ternational Conference on Machine Learning.
  24. Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, and Animashree Anandku- mar. 2020. Angular visual hardness. In Proceedings of the International Conference on Machine Learning.
  25. Hanlin Chen, Baochang Zhang, Song Xue, Xuan Gong, Hong Liu, Rongrong Ji, and David Doermann. 2020. Anti- bandit neural architecture search for model defense. In Proceedings of the European Conference on Computer Vision. 70-85.
  26. Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. 2021. Atom: Robustifying out-of-distribution de- tection using outlier mining. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer.
  27. Sungik Choi and Sae-Young Chung. 2020. Novelty detection via blurring. In Proceedings of the International Conference on Learning Representations.
  28. Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. arXiv:2003.01690. Retrieved from https://arxiv.org/abs/2003.01690. · arXiv
  29. Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2018. Autoaugment: Learning augmentation policies from data. arXiv:1805.09501. Retrieved from https://arxiv.org/abs/1805.09501. · arXiv
  30. Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. 2020. Randaugment: Practical automated data augmen- tation with a reduced search space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.
  31. Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, and Duen Horng Chau. 2017. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. arXiv:1705.02900. Retrieved from https://arxiv.org/abs/1705.02900. · arXiv
  32. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248-255.
  33. Terrance DeVries and Graham W Taylor. 2018. Learning confidence for out-of-distribution detection in neural net- works. In Proceedings of the International Conference on Learning Representations.
  34. Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. 2020. MMA training: Direct input space margin maximization through adversarial training. In Proceedings of the International Conference on Learning Representations.
  35. Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, et al. 2020. On robustness and transferability of convolutional neural networks. arXiv:2007.08558. Retrieved from https://arxiv.org/abs/2007.08558. · arXiv
  36. Tommaso Dreossi, Alexandre Donzé, and Sanjit A. Seshia. 2019. Compositional falsification of cyber-physical sys- tems with machine learning components. J. Autom. Reas. 63, 4 (2019), 1031-1053.
  37. Tommaso Dreossi, Daniel J. Fremont, Shromona Ghosh, Edward Kim, Hadi Ravanbakhsh, Marcell Vazquez-Chanlatte, and Sanjit A Seshia. 2019. Verifai: A toolkit for the formal design and analysis of artificial intelligence-based systems. In International Conference on Computer Aided Verification. Springer, 432-442.
  38. Tommaso Dreossi, Shromona Ghosh, Alberto Sangiovanni-Vincentelli, and Sanjit A. Seshia. 2019. A formalization of robustness for deep neural networks. arXiv:1903.10033. Retrieved from https://arxiv.org/abs/1903.10033. · arXiv
  39. Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Kurt Keutzer, Alberto Sangiovanni-Vincentelli, and Sanjit A. Seshia. 2018. Counterexample-guided data augmentation. In Proceedings of the 27th International Joint Conference on Artificial Intelligence.
  40. Yilun Du and Igor Mordatch. 2019. Implicit generation and modeling with energy based models. In Advances in Neural Information Processing Systems. 3608-3618.
  41. Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Output range analysis for deep feedforward neural networks. In NASA Formal Methods Symposium. Springer.
  42. Krishnamurthy Dvijotham, Sven Gowal, Robert Stanforth, Relja Arandjelovic, Brendan O'Donoghue, Jonathan Ue- sato, and Pushmeet Kohli. 2018. Training verified learners with learned verifiers. arXiv:1805.10265. Retrieved from https://arxiv.org/abs/1805.10265. · arXiv
  43. Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. 2017. Detecting adversarial samples from artifacts. arXiv:1703.00410. Retrieved from https://arxiv.org/abs/1703.00410. · arXiv
  44. Lucas Fidon, Sebastien Ourselin, and Tom Vercauteren. 2020. Distributionally robust deep learning using hardness weighted sampling. arXiv:2001.02658. Retrieved from https://arxiv.org/abs/2001.02658. · arXiv
  45. Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. 2021. Exploring the limits of out-of-distribution detection. In Advances in Neural Information Processing Systems.
  46. Daniel J. Fremont, Edward Kim, Yash Vardhan Pant, Sanjit A. Seshia, Atul Acharya, Xantha Bruso, Paul Wells, Steve Lemke, Qiang Lu, and Shalin Mehta. 2020. Formal scenario-based testing of autonomous vehicles: From simulation to the real world. In Proceedings of the IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC'20). IEEE, 1-8.
  47. Yarin Gal. 2016. Uncertainty in Deep Learning. Ph.D. Dissertation. University of Cambridge.
  48. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the International Conference on Machine Learning.
  49. Yarin Gal, Jiri Hron, and Alex Kendall. 2017. Concrete dropout. In Advances in Neural Information Processing Systems.
  50. Yarin Gal, Riashat Islam, and Zoubin Ghahramani. 2017. Deep bayesian active learning with image data. In Proceed- ings of the International Conference on Machine Learning. 1183-1192.
  51. Yaroslav Ganin and Victor Lempitsky. 2014. Unsupervised domain adaptation by backpropagation. arXiv:1409.7495. Retrieved from https://arxiv.org/abs/1409.7495. · arXiv
  52. Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, and Jason D Lee. 2019. Convergence of adversarial training in overparametrized neural networks. In Advances in Neural Information Processing Systems. 13029-13040.
  53. Saurabh Garg, Sivaraman Balakrishnan, Zachary C. Lipton, Behnam Neyshabur, and Hanie Sedghi. 2022. Leveraging unlabeled data to predict out-of-distribution performance. In Proceedings of the International Conference on Learning Representations.
  54. Yonatan Geifman and Ran El-Yaniv. 2017. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems. 4878-4887.
  55. Yonatan Geifman and Ran El-Yaniv. 2019. SelectiveNet: A deep neural network with an integrated reject option. arXiv:1901.09192. Retrieved from https://arxiv.org/abs/1901.09192. · arXiv
  56. Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. 2019. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems, Vol. 32. 9277-9286.
  57. Izhak Golan and Ran El-Yaniv. 2018. Deep anomaly detection using geometric transformations. In Advances in Neural Information Processing Systems. 9758-9769.
  58. Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. 2021. MaxUp: A simple way to improve generalization of neural network training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  59. Rui Gong, Wen Li, Yuhua Chen, and Luc Van Gool. 2019. DLOW: Domain flow for adaptation and generalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2477-2486.
  60. Zhitao Gong, Wenlu Wang, and Wei-Shinn Ku. 2017. Adversarial and clean data are not twins. arXiv:1704.04960. Retrieved from https://arxiv.org/abs/1704.04960. · arXiv
  61. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of the International Conference on Learning Representations.
  62. Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. 2018. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv:1810.12715. Retrieved from https://arxiv.org/abs/1810.12715. · arXiv
  63. Sachin Goyal, Aditi Raghunathan, Moksh Jain, Harsha Vardhan Simhadri, and Prateek Jain. 2020. DROCC: Deep robust one-class classification. In Proceedings of the 37th International Conference on Machine Learning.
  64. Will Grathwohl, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swer- sky. 2019. Your classifier is secretly an energy based model and you should treat it like one. In Proceedings of the International Conference on Learning Representations.
  65. Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. 2017. On the (statis- tical) detection of adversarial examples. arXiv:1702.06280. Retrieved from https://arxiv.org/abs/1702.06280. · arXiv
  66. Shupeng Gui, Haotao Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, and Ji Liu. 2019. Model compression with adversarial robustness: A unified optimization framework. In Advances in Neural Information Processing Systems.
  67. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. 2018. Local rule-based explanations of black box decision systems. arXiv:1805.10820. Retrieved from https://arxiv.org/abs/ 1805.10820. · arXiv
  68. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning.
  69. Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. 2018. Countering adversarial images using input transformations. In Proceedings of the International Conference on Learning Representations.
  70. Minghao Guo, Yuzhe Yang, Rui Xu, Ziwei Liu, and Dahua Lin. 2020. When NAS meets robustness: In search of robust architectures against adversarial attacks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  71. Guo Haixiang, Li Yijing, Jennifer Shang, Gu Mingyun, Huang Yuanyue, and Gong Bing. 2017. Learning from class- imbalanced data: Review of methods and applications. Expert Syst. Appl. 73 (2017), 220-239.
  72. Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co- teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems.
  73. Jiangfan Han, Ping Luo, and Xiaogang Wang. 2019. Deep self-learning from noisy labels. In Proceedings of the IEEE International Conference on Computer Vision. 5138-5147.
  74. Elmar Haussmann, Michele Fenzi, Kashyap Chitta, Jan Ivanecky, Hanson Xu, Donna Roy, Akshita Mittel, Nicolas Koumchatzky, Clement Farabet, and Jose M. Alvarez. 2020. Scalable active learning for object detection. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV'20).
  75. Richard Hawkins, Colin Paterson, Chiara Picardi, Yan Jia, Radu Calinescu, and Ibrahim Habli. 2021. Guidance on the assurance of machine learning in autonomous systems (AMLAS). arXiv:2102.01564. Retrieved from https://arxiv.org/ abs/2102.01564. · arXiv
  76. Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. 2019. Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem. arXiv:1812.05720. Retrieved from https://arxiv.org/abs/1812.05720. · arXiv
  77. Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. 2019. Scaling out-of-distribution detection for real-world settings. arXiv:1911.11132. Retrieved from https://arxiv.org/abs/ 1911.11132. · arXiv
  78. Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. 2020. The many faces of robustness: A critical analysis of out-of-distribution generalization. arXiv:2006.16241. Retrieved from https://arxiv.org/abs/2006.16241. · arXiv
  79. Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. 2021. Unsolved problems in ml safety. arXiv:2109.13916. Retrieved from https://arxiv.org/abs/2109.13916. · arXiv
  80. Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. In Proceedings of the International Conference on Learning Representations.
  81. Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv:1610.02136. Retrieved from https://arxiv.org/abs/1610.02136. · arXiv
  82. Dan Hendrycks and Kevin Gimpel. 2016. Early methods for detecting adversarial images. arXiv:1608.00530. Retrieved from https://arxiv.org/abs/1608.00530. · arXiv
  83. Dan Hendrycks, Kimin Lee, and Mantas Mazeika. 2019. Using pre-training can improve model robustness and un- certainty. arXiv:1901.09960. Retrieved from https://arxiv.org/abs/1901.09960. · arXiv
  84. Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. 2018. Deep anomaly detection with outlier exposure. In Proceedings of the International Conference on Learning Representations.
  85. Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. 2019. Using self-supervised learning can im- prove model robustness and uncertainty. In Advances in Neural Information Processing Systems. 15663-15674.
  86. Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. 2019. Aug- mix: A simple data processing method to improve robustness and uncertainty. arXiv:1912.02781. Retrieved from https://arxiv.org/abs/1912.02781. · arXiv
  87. Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. 2019. Natural adversarial examples. arXiv:1907.07174. Retrieved from https://arxiv.org/abs/1907.07174. · arXiv
  88. José Hernández-Orallo, Fernando Martínez-Plumed, Shahar Avin, and Seán Ó hÉigeartaigh. 2019. Surveying safety- relevant AI characteristics. In Proceedings of the AAAI Conference on Artificial Intelligence Workshop.
  89. Fred Hohman, Haekyu Park, Caleb Robinson, and Duen Horng Polo Chau. 2019. Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE Trans. Vis. Comput. Graph. (2019).
  90. Junyuan Hong, Haotao Wang, Zhangyang Wang, and Jiayu Zhou. 2021. Federated robustness propagation: Sharing adversarial robustness in federated learning. arXiv:2106.10196. Retrieved from https://arxiv.org/abs/2106.10196. · arXiv
  91. Junyuan Hong, Haotao Wang, Zhangyang Wang, and Jiayu Zhou. 2022. Efficient split-mix federated learning for on-demand and in-situ customization. In Proceedings of the International Conference on Learning Representations.
  92. Ting-Kuei Hu, Tianlong Chen, Haotao Wang, and Zhangyang Wang. 2020. Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference. arXiv:2002.10025. Retrieved from https://arxiv.org/abs/ 2002.10025. · arXiv
  93. Xiaowei Huang, Daniel Kroening, Wenjie Ruan, James Sharp, Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. 2020. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Comput. Sci. Rev. 37 (2020), 100270.
  94. Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety verification of deep neural networks. In International Conference on Computer Aided Verification. Springer, 3-29.
  95. Zeyi Huang, Haohan Wang, Eric P. Xing, and Dong Huang. 2020. Self-challenging improves cross-domain general- ization. In Proceedings of the European Conference on Computer Vision.
  96. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntac- tically controlled paraphrase networks. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT'18). 1875-1885.
  97. Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. 2020. Robust pre-training by adversarial contrastive learning. arXiv:2010.13337. Retrieved from https://arxiv.org/abs/2010.13337. · arXiv
  98. Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, and Duen Horng Polo Chau. 2018. ActiVis: Visual exploration of industry-scale deep neural network models. IEEE Trans. Vis. Comput. Graph. 24, 1 (2018), 88-97.
  99. Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. 2019. Testing robustness against unforeseen adversaries. arXiv:1908.08016. Retrieved from https://arxiv.org/abs/1908.08016. · arXiv
  100. Archit Karandikar, Nicholas Cain, Dustin Tran, Balaji Lakshminarayanan, Jonathon Shlens, Michael C. Mozer, and Rebecca Roelofs. 2021. Soft calibration objectives for neural networks. In Advances in Neural Information Processing Systems.
  101. Angelos Katharopoulos and François Fleuret. 2018. Not all samples are created equal: Deep learning with importance sampling. In Proceedings of the International Conference on Machine Learning. 2525-2534.
  102. Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems. 5574-5584.
  103. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In Proceedings of the Interna- tional Conference on Machine Learning.
  104. Edward Kim, Divya Gopinath, Corina Pasareanu, and Sanjit Seshia. 2019. A programmatic and semantic approach to explaining and debuggingneural network based object detectors. arXiv:1912.00289. Retrieved from https://arxiv. org/abs/1912.00289. · arXiv
  105. Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019. The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer.
  106. Philip Koopman and Michael Wagner. 2017. Autonomous vehicle safety: An interdisciplinary challenge. IEEE Intell. Transport. Syst. Mag. 9, 1 (2017), 90-96.
  107. Ranganath Krishnan and Omesh Tickoo. 2020. Improving model calibration with accuracy versus uncertainty opti- mization. In Advances in Neural Information Processing Systems. 18237-18248.
  108. Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. 2018. Trainable calibration measures for neural networks from kernel mean embeddings. In Proceedings of the International Conference on Machine Learning.
  109. Isaac Lage, Andrew Slavin Ross, Samuel J. Gershman, Been Kim, and Finale Doshi-Velez. 2018. Human-in-the-loop interpretability prior. In Advances in Neural Information Processing Systems. 10180-10189.
  110. Himabindu Lakkaraju, Nino Arsov, and Osbert Bastani. 2020. Robust and stable black box explanations. In Proceedings of the International Conference on Machine Learning. 5628-5638.
  111. Himabindu Lakkaraju, Stephen H. Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
  112. Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Eric Horvitz. 2017. Identifying unknown unknowns in the open world: Representations and policies for guided exploration. In Proceedings of the AAAI Conference on Artificial Intelligence.
  113. Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2019. Faithful and customizable explanations of black box models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 131-138.
  114. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems.
  115. John Lambert, Zhuang Liu, Ozan Sener, James Hays, and Vladlen Koltun. 2020. MSeg: A composite dataset for multi- domain semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  116. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436-444.
  117. Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. 2017. Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv:1711.09325. Retrieved from https://arxiv.org/abs/1711.09325. · arXiv
  118. Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of- distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems. 7167-7177.
  119. Kimin Lee, Kibok Lee, Jinwoo Shin, and Honglak Lee. 2020. Network randomization: A simple technique for gener- alization in deep reinforcement learning. In Proceedings of the International Conference on Learning Representations.
  120. Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A. Ortega, Tom Everitt, Andrew Lefrancq, Laurent Orseau, and Shane Legg. 2017. Ai safety gridworlds. arXiv:1711.09883. Retrieved from https://arxiv.org/abs/1711.09883. · arXiv
  121. Piyawat Lertvittayakumjorn and Francesca Toni. 2019. Human-grounded evaluations of explanation methods for text classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP'19). 5198-5208.
  122. Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the AAAI Conference on Artificial Intelligence.
  123. Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017. Deeper, broader and artier domain generaliza- tion. In Proceedings of the IEEE International Conference on Computer Vision. 5542-5550.
  124. Xin Li and Fuxin Li. 2017. Adversarial examples detection in deep networks with convolutional filter statistics. In Proceedings of the IEEE International Conference on Computer Vision. 5764-5772.
  125. Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, et al. 2020. Shape-texture debiased neural network training. In Proceedings of the International Conference on Learning Representations.
  126. Zhizhong Li and Derek Hoiem. 2020. Improving confidence estimates for unfamiliar examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  127. Ji Lin, Chuang Gan, and Song Han. 2019. Defensive quantization: When efficiency meets robustness. In Proceedings of the International Conference on Learning Representations.
  128. Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. 2018. Detecting and correcting for label shift with black box predictors. In Proceedings of the International Conference on Machine Learning. 3122-3130.
  129. Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss, and Balaji Lakshminarayanan. 2020. Sim- ple and principled uncertainty estimation with deterministic deep learning via distance awareness. arXiv:2006.10108. Retrieved from https://arxiv.org/abs/2006.10108. · arXiv
  130. Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. 2020. Energy-based Out-of-distribution detection. arXiv:2010.03759. Retrieved from https://arxiv.org/abs/2010.03759. · arXiv
  131. Qinghua Lu, Liming Zhu, Xiwei Xu, Jon Whittle, David Douglas, and Conrad Sanderson. 2021. Software engineering for responsible AI: An empirical study and operationalised patterns. arXiv:2111.09478. Retrieved from https://arxiv. org/abs/2111.09478. · arXiv
  132. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765-4774.
  133. Alexandra L'heureux, Katarina Grolinger, Hany F. Elyamany, and Miriam A. M. Capretz. 2017. Machine learning with big data: Challenges and approaches. IEEE Access 5 (2017), 7776-7797.
  134. Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. 2018. Characterizing adversarial subspaces using local intrinsic dimensional- ity. arXiv:1801.02613. Retrieved from https://arxiv.org/abs/1801.02613. · arXiv
  135. Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. 2018. Exploring the limits of weakly supervised pretraining. In Proceedings of the European Conference on Computer Vision. 181-196.
  136. Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. 2021. On the robustness of vision transformers to adversarial examples. arXiv:2104.02610. Retrieved from https://arxiv.org/abs/2104.02610. · arXiv
  137. Andrey Malinin, Neil Band, Yarin Gal, Mark Gales, Alexander Ganshin, German Chesnokov, Alexey Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, et al. 2021. Shifts: A dataset of real distributional shift across multiple large-scale tasks. In Advances in Neural Information Processing Systems.
  138. Rowan McAllister, Yarin Gal, Alex Kendall, Mark Van Der Wilk, Amar Shah, Roberto Cipolla, and Adrian Vivian Weller. 2017. Concrete problems for autonomous vehicle safety: Advantages of bayesian deep learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI '17).
  139. Alexander Meinke and Matthias Hein. 2019. Towards neural networks that provably know when they don't know. In Proceedings of the International Conference on Learning Representations.
  140. Dongyu Meng and Hao Chen. 2017. Magnet: A two-pronged defense against adversarial examples. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. ACM, 135-147.
  141. Qinghao Meng, Wenguan Wang, Tianfei Zhou, Jianbing Shen, Yunde Jia, and Luc Van Gool. 2021. Towards a weakly supervised framework for 3d point cloud object detection and annotation. IEEE Trans. Pattern Anal. Mach. Intell. (2021).
  142. Takeru Miyato, Shin-Ichi Maeda, Masanori Koyama, and Shin Ishii. 2019. Virtual adversarial training: A regular- ization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41, 8 (2019), 1979-1993.
  143. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Foundations of Machine Learning. MIT Press.
  144. Sina Mohseni, Jeremy E. Block, and Eric Ragan. 2021. Quantitative evaluation of machine learning explanations: A human-grounded benchmark. In Proceedings of the 26th International Conference on Intelligent User Interfaces. https:// doi.org/10.1145/3397481.3450689 · DOI
  145. Sina Mohseni, Mandar Pitale, Vasu Singh, and Zhangyang Wang. 2019. Practical solutions for machine learning safety in autonomous vehicles. arXiv:1912.09630. Retrieved from https://arxiv.org/abs/1912.09630. · arXiv
  146. Sina Mohseni, Mandar Pitale, J. B. S. Yadawa, and Zhangyang Wang. 2020. Self-supervised learning for generalizable out-of-distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence.
  147. Sina Mohseni, Arash Vahdat, and Jay Yadawa. 2021. Shifting transformation learning for out-of-distribution detection. arXiv:2106.03899. Retrieved from https://arxiv.org/abs/2106.03899. · arXiv
  148. Jisoo Mok, Byunggook Na, Hyeokjun Choe, and Sungroh Yoon. 2021. AdvRush: Searching for adversarially robust neural architectures. In Proceedings of the IEEE International Conference on Computer Vision. 12322-12332.
  149. Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. 2020. Confidence-aware learning for deep neural networks. In Proceedings of the International Conference on Machine Learning. 7034-7044.
  150. Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip H. S. Torr, and Yarin Gal. 2021. Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty. arXiv:2102.11582. Retrieved from https://arxiv.org/abs/2102.11582. · arXiv
  151. Rafael Müller, Simon Kornblith, and Geoffrey Hinton. 2019. When does label smoothing help? arXiv:1906.02629. Retrieved from https://arxiv.org/abs/1906.02629. · arXiv
  152. Preetum Nakkiran. 2019. Adversarial robustness may be at odds with simplicity. arXiv:1901.00532. Retrieved from https://arxiv.org/abs/1901.00532. · arXiv
  153. Nina Narodytska, Shiva Kasiviswanathan, Leonid Ryzhyk, Mooly Sagiv, and Toby Walsh. 2018. Verifying properties of binarized deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence.
  154. Xuefei Ning, Junbo Zhao, Wenshuo Li, Tianchen Zhao, Huazhong Yang, and Yu Wang. 2020. Multi-shot NAS for dis- covering adversarially robust convolutional neural architectures at targeted capacities. arXiv:2012.11835. Retrieved from https://arxiv.org/abs/2012.11835. · arXiv
  155. Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordv- intsev. 2018. The building blocks of interpretability. Distill (2018). https://doi.org/10.23915/distill.00010 · DOI · Paper page
  156. Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang. 2018. Two at once: Enhancing learning and generalization capacities via ibn-net. In Proceedings of the European Conference on Computer Vision. 464-479.
  157. Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. Deepxplore: Automated whitebox testing of deep learn- ing systems. In Proceedings of the 26th Symposium on Operating Systems Principles. 1-18.
  158. Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. Towards practical verification of machine learning: The case of computer vision systems. arXiv:1712.01785. Retrieved from https://arxiv.org/abs/1712.01785. · arXiv
  159. Yi Qin, Huiyan Wang, Chang Xu, Xiaoxing Ma, and Jian Lu. 2018. SynEva: Evaluating ml programs by mirror program synthesis. In Proceedings of the IEEE International Conference on Software Quality, Reliability and Security (QRS'18). IEEE, 171-182.
  160. Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, and Bo Li. 2020. SemanticAdv: Generating ad- versarial examples via attribute-conditioned image editing. In Proceedings of the European Conference on Computer Vision. Springer, 19-37.
  161. Joaquin Quiñonero-Candela, Masashi Sugiyama, Neil D. Lawrence, and Anton Schwaighofer. 2009. Dataset Shift in Machine Learning. MIT Press.
  162. Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019. Do ImageNet classifiers generalize to imagenet? In Proceedings of the International Conference on Machine Learning. 5389-5400.
  163. Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshmi- narayanan. 2019. Likelihood ratios for out-of-distribution detection. In Advances in Neural Information Processing Systems. 14707-14718.
  164. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the Annual Meeting of the Association for Computa- tional Linguistics (ACL'19). 1085-1097.
  165. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you? Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
  166. Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Em- manuel Müller, and Marius Kloft. 2018. Deep one-class classification. In Proceedings of the International Conference on Machine Learning. 4393-4402.
  167. Lukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus-Robert Müller, and Marius Kloft. 2019. Deep semi-supervised anomaly detection. In Proceedings of the International Conference on Learn- ing Representations.
  168. Dorsa Sadigh, Shankar Sastry, Sanjit A. Seshia, and Anca D. Dragan. 2016. Planning for autonomous cars that leverage effects on human actions. In Robotics: Science and Systems, Vol. 2. Ann Arbor, MI.
  169. Dorsa Sadigh, S. Shankar Sastry, Sanjit A. Seshia, and Anca Dragan. 2016. Information gathering actions over human internal state. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'16). IEEE, 66-73.
  170. Rick Salay, Rodrigo Queiroz, and Krzysztof Czarnecki. 2017. An analysis of ISO 26262: Using machine learning safely in automotive software. arXiv:1709.02435. Retrieved from https://arxiv.org/abs/1709.012435. · arXiv
  171. Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. 2021. A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: Solutions and future challenges. arXiv:2110.14051. Retrieved from https://arxiv.org/abs/2110.14051. · arXiv
  172. Pouya Samangouei, Maya Kabkab, and Rama Chellappa. 2018. Defense-GAN: Protecting classifiers against adversar- ial attacks using generative models. arXiv:1805.06605. · arXiv
  173. Chandramouli Shama Sastry and Sageev Oore. 2020. Detecting out-of-distribution examples with in-distribution examples and gram matrices. In Proceedings of the International Conference on Machine Learning. 8491-8501.
  174. Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. 2017. Unsuper- vised anomaly detection with generative adversarial networks to guide marker discovery. arXiv:1703.05921. · arXiv
  175. Vikash Sehwag, Mung Chiang, and Prateek Mittal. 2021. SSD: A unified framework for self-supervised outlier detec- tion. In Proceedings of the International Conference on Learning Representations.
  176. Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. 2020. On pruning adversarially robust neural networks. arXiv:2002.10509. Retrieved from https://arxiv.org/abs/2002.10529. · arXiv
  177. Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision.
  178. Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F Núñez, and Jordi Luque. 2019. Input complexity and out-of-distribution detection with likelihood-based generative models. arXiv:1909.11480. Retrieved from https:// arxiv.org/abs/1909.11480. · arXiv
  179. Sanjit A. Seshia, Ankush Desai, Tommaso Dreossi, Daniel J. Fremont, Shromona Ghosh, Edward Kim, Sumukh Shiv- akumar, Marcell Vazquez-Chanlatte, and Xiangyu Yue. 2018. Formal specification for deep neural networks. In Au- tomated Technology for Verification and Analysis. Springer.
  180. Sanjit A. Seshia, Dorsa Sadigh, and S. Shankar Sastry. 2016. Towards verified artificial intelligence. arXiv:1606.08514. Retrieved from https://arxiv.org/abs/1606.08514. · arXiv
  181. Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. 2019. Adversarial training for free! arXiv:1904.12843. Retrieved from https://arxiv.org/abs/1904. 12843. · arXiv
  182. Uri Shaham, James Garritano, Yutaro Yamada, Ethan Weinberger, Alex Cloninger, Xiuyuan Cheng, Kelly Stanton, and Yuval Kluger. 2018. Defending against adversarial images using basis functions transformations. arXiv:1803.10840. Retrieved from https://arxiv.org/abs/1803.10840. · arXiv
  183. Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In Proceedings of the International Conference on Machine Learning. 3145-3153.
  184. Yawar Siddiqui, Julien Valentin, and Matthias Nießner. 2020. ViewAL: Active learning with viewpoint entropy for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9433-9443.
  185. Julien Siebert, Lisa Joeckel, Jens Heidrich, Koji Nakamichi, Kyoko Ohashi, Isao Namba, Rieko Yamamoto, and Mikio Aoyama. 2020. Towards guidelines for assessing qualities of machine learning systems. arXiv:2008.11007. Retrieved from https://arxiv.org/abs/2008.11007. · arXiv
  186. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv:1312.6034. Retrieved from https://arxiv.org/abs/1312.6034. · arXiv
  187. Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: Removing noise by adding noise. arXiv:1706.03825. Retrieved from https://arxiv.org/abs/1706.03825. · arXiv
  188. Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. 2021. Learning and evaluating represen- tations for deep one-class classification. In Proceedings of the International Conference on Learning Representations.
  189. Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014. Striving for simplicity: The all convolutional net. arXiv:1412.6806. Retrieved from https://arxiv.org/abs/1412.6806. · arXiv
  190. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929-1958.
  191. Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, and Alexander M Rush. 2018. LstmVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Vis. Comput. Graph. 24, 1 (2018), 667-676.
  192. Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, and Rob Ashmore. 2018. Testing deep neural networks. arXiv:1803.04792. Retrieved from https://arxiv.org/abs/1803.04792. · arXiv
  193. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  194. Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. 2020. CSI: Novelty detection via contrastive learning on distributionally shifted instances. In Advances in Neural Information Processing Systems.
  195. Zheng Tang, Milind Naphade, Stan Birchfield, Jonathan Tremblay, William Hodge, Ratnesh Kumar, Shuo Wang, and Xiaodong Yang. 2019. PAMTRI: Pose-aware multi-task learning for vehicle re-identification using highly randomized synthetic data. In Proceedings of the IEEE International Conference on Computer Vision.
  196. Mohammadamin Tavakoli, Forest Agostinelli, and Pierre Baldi. 2021. SPLASH: Learnable activation functions for improving accuracy and adversarial robustness. Neural Netw. 140 (2021), 1-12.
  197. Engkarat Techapanurak, Masanori Suganuma, and Takayuki Okatani. 2020. Hyperparameter-free out-of-distribution detection using cosine similarity. In Proceedings of the Asian Conference on Computer Vision.
  198. Sunil Thulasidasan, Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. 2019. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. arXiv:1905.11001. Retrieved from https://arxiv.org/abs/1905.11001. · arXiv
  199. Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. 2020. On adaptive attacks to adversarial example defenses. In Advances in Neural Information Processing Systems. 1633-1645.
  200. Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cam- eracci, Shaad Boochoon, and Stan Birchfield. 2018. Training deep networks with synthetic data: Bridging the reality gap by domain randomization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 969-977.

200 of 264 shown

Cited by

Sorted by citations

Authors
Mohseni, S., Wang, H., Xiao, C., Yu, Z., Wang, Z., & Yadawa, J.
Affiliations
  • Nvidia (United States) (US)
  • The University of Texas at Austin (US)
ORCID
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2022-12-23
Volume
55
Issue
8
Pages
1-38
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3551385
OpenAlex
W4288050713
Cited by
53
References
22
Citations by year
2026: 11 · 2025: 18 · 2024: 8 · 2023: 14 · 2022: 2
FWCI
6.66
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Computing Surveys (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
machine learning safety, safety-critical systems, autonomous vehicles, interpretability, verifiability, generalization error, domain adaptation, outlier detection, adversarial attacks, safety checklist

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review