Steps Toward Robust Artificial Intelligence
Open accessAbstract
Recent advances in artificial intelligence are encouraging governments and corporations to deploy AI in high‐stakes settings including driving cars autonomously, managing the power grid, trading on stock exchanges, and controlling autonomous weapons systems. Such applications require AI methods to be robust to both the known unknowns (those uncertain aspects of the world about which the computer can reason explicitly) and the unknown unknowns (those aspects of the world that are not captured by the system's models). This article discusses recent progress in AI and then describes eight ideas related to robustness that are being pursued within the AI research community. While these ideas are a start, we need to devote more attention to the challenges of dealing with the known and unknown unknowns. These issues are fascinating, because they touch on the fundamental question of how finite systems can survive and thrive in a complex and dangerous world.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653 · see citing works
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,726 · see citing works
(2012) · Cited by 9,062 · see citing works
Markov Chain Monte Carlo in Practice
Gilks, W. R., Richardson, S. T., & Spiegelhalter, D. J. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b14835 · Cited by 7,927 · see citing works
(2009) · Cited by 6,153 · see citing works
Combining labeled and unlabeled data with co-training
Blum, A. L., & Mitchell, T. M. (1998) · Conference on Learning Theory · doi:10.1145/279943.279962 · Cited by 5,597 · see citing works
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · see citing works
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvari, C. (2006) · European Conference on Machine Learning · doi:10.1007/11871842_29 · Cited by 4,073 · see citing works
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-009-5152-4 · Cited by 3,515 · see citing works
Probabilistic Reasoning in Intelligent Systems
(1988) · Elsevier eBooks · doi:10.1016/c2009-0-27609-4 · Cited by 3,371 · see citing works
A machine program for theorem-proving
Davis, M. D., Logemann, G. W., & Loveland, D. W. (1962) · Communications of the ACM · doi:10.1145/368273.368557 · Cited by 3,240 · see citing works
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · see citing works
A Computing Procedure for Quantification Theory
Davis, M. D., & Putnam, H. (1960) · Journal of the ACM · doi:10.1145/321033.321034 · Cited by 2,657 · see citing works
Kitano, H. (2004) · Nature Reviews Genetics · doi:10.1038/nrg1471 · Cited by 2,278 · see citing works
Isolation-Based Anomaly Detection
Liu, F. T., Ting, K. M., & Zhou, Z. (2012) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/2133360.2133363 · Cited by 2,112 · see citing works
Analysis of Representations for Domain Adaptation
Ben-David, S., Blitzer, J., Crammer, K., & Pereira, F. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0022 · Cited by 2,019 · see citing works
Steps toward Artificial Intelligence
Minsky, M. (1961) · Proceedings of the IRE · doi:10.1109/jrproc.1961.287775 · Cited by 1,706 · see citing works
Correcting Sample Selection Bias by Unlabeled Data
Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., & Schölkopf, B. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0080 · Cited by 1,617 · see citing works
Scheirer, W. J., de Rezende Rocha, A., Sapkota, A., & Boult, T. E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.256 · Cited by 1,573 · see citing works
Domain adaptation with structural correspondence learning
Blitzer, J., McDonald, R., & Pereira, F. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610094 · Cited by 1,543 · see citing works
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Neal, R. M. (2011) · Cited by 1,035 · see citing works
Learning Structural Descriptions From Examples
Winston, P. H. (1970) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 995 · see citing works
SATzilla: Portfolio-based Algorithm Selection for SAT
Xu, L., Hutter, F., Hoos, H. H., et al. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2490 · Cited by 866 · see citing works
The Flash Crash: High‐Frequency Trading in an Electronic Market
KIRILENKO, A., KYLE, A. S., SAMADI, M., & TUZUN, T. (2017) · The Journal of Finance · doi:10.1111/jofi.12498 · Cited by 795 · see citing works
Covariate Shift Adaptation by Importance Weighted Cross Validation
Sugiyama, M., Krauledat, M., & Müller, K. (2007) · Cited by 710 · see citing works
A Bayesian Analysis for Change Point Problems
Barry, D., & Hartigan, J. A. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594323 · Cited by 666 · see citing works
A tutorial on conformal prediction
Shafer, G., & Vovk, V. (2007) · arXiv · doi:10.48550/arxiv.0706.3188 · Cited by 655 · see citing works
A test for a change in a parameter occurring at an unknown point
Page, E. S. (1955) · Biometrika · doi:10.1093/biomet/42.3-4.523 · Cited by 647 · see citing works
Supervised autonomous robotic soft tissue surgery
Shademan, A., Decker, R., Opfermann, J. D., Léonard, S., Krieger, A., & Kim, P. C. W. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aad9398 · Cited by 626 · see citing works
Bayesian Online Changepoint Detection
Adams, R. P., & MacKay, D. J. C. (2007) · arXiv · doi:10.48550/arxiv.0710.3742 · Cited by 601 · see citing works
Cyc: toward programs with common sense
Lenat, D., Guha, R., Pittman, K., Pratt, D., & Shepherd, M. (1990) · CACM · doi:10.1145/79173.79176 · Cited by 585 · see citing works
A Conceptual Introduction to Hamiltonian Monte CarloPreprint
Betancourt, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.02434 · Cited by 538 · see citing works
Towards Open World Recognition
Bendale, A., & Boult, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298799 · Cited by 529 · see citing works
Introduction to “This is Watson”
Ferrucci, D. A. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2184356 · Cited by 488 · see citing works
Governing Lethal Behavior in Autonomous Robots
Arkin, R. C. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420085952 · Cited by 435 · see citing works
Density Ratio Estimation in Machine Learning
Sugiyama, M., Suzuki, T., & Kanamori, T. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9781139035613 · Cited by 432 · see citing works
Oksanen, T., Tiusanen, J., & Kostamo, J. (2009) · Wageningen Academic Publishers eBooks · doi:10.3920/9789086866649_102 · Cited by 416 · see citing works
Gordon, A. D., Henzinger, T. A., Nori, A. V., & Rajamani, S. K. (2014) · Future of Software Engineering (FOSE) · doi:10.1145/2593882.2593900 · Cited by 372 · see citing works
Robustness and Regularization of Support Vector MachinesPreprint
Xu, H., Caramanis, C., & Mannor, S. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0803.3490 · Cited by 359 · see citing works
Heads-up limit hold’em poker is solved
Bowling, M., BURCH, N. R., Johanson, M., & Tammelin, O. (2015) · Science · doi:10.1126/science.1259433 · Cited by 349 · see citing works
Cozman, F. G. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00029-1 · Cited by 345 · see citing works
Pervasive robustness in biological systems
Félix, M., & Barkoulas, M. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3949 · Cited by 319 · see citing works
The Third Culture: Beyond the Scientific Revolution
Brockman, J. R., & Scott, A. C. (1996) · American Journal of Physics · doi:10.1119/1.18425 · Cited by 287 · see citing works
Sample Selection Bias Correction Theory
Cortes, C., Mohri, M., Riley, M., & Rostamizadeh, A. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-87987-9_8 · Cited by 272 · see citing works
Learning at the Knowledge Level
Dietterich, T. G. (1986) · Machine Learning · doi:10.1023/a:1022858530318 · Cited by 227 · see citing works
Self-citationScaling and Probabilistic Smoothing: Efficient Dynamic Local Search for SAT
Hutter, F., Tompkins, D. A. D., & Hoos, H. H. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46135-3_16 · Cited by 206 · see citing works
Biological Robustness: Paradigms, Mechanisms, and Systems Principles
Whitacre, J. M. (2012) · Frontiers in Genetics · doi:10.3389/fgene.2012.00067 · Cited by 182 · see citing works
Transportability of Causal and Statistical Relations: A Formal Approach
Pearl, J., & Bareinboim, E. (2011) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2011.169 · Cited by 178 · see citing works
Robustness and Regularization of Support Vector Machines
XuHuan, CaramanisConstantine, & MannorShie (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755834 · Cited by 175 · see citing works
Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (1997)Preprint
Geiger, D., & Shenoy, P. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.3846 · Cited by 139 · see citing works
Direct Density Ratio Estimation for Large-scale Covariate Shift Adaptation
Tsuboi, Y., Kashima, H., Hido, S., Bickel, S., & Sugiyama, M. (2009) · Journal of Information Processing · doi:10.2197/ipsjjip.17.138 · Cited by 135 · see citing works
Practical Probabilistic Programming
Pfeffer, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21295-6_2 · Cited by 122 · see citing works
Learning with Augmented Class by Exploiting Unlabeled Data
Da, Q., Yang, Y., & Zhou, Z. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8997 · Cited by 89 · see citing works
Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (2013)Preprint
Nicholson, A. E., & Smyth, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.7971 · Cited by 89 · see citing works
Transportability of Causal Effects: Completeness Results
Bareinboim, E., & Pearl, J. (2012) · doi:10.21236/ada557446 · Cited by 87 · see citing works
Thinking Backward for Knowledge Acquisition
Shachter, R. D., & Heckerman, D. E. (1987) · doi:10.1609/aimag.v8i3.600 · Cited by 65 · see citing works
Learning Algorithm Portfolios for Parallel Execution
Yun, X., & Epstein, S. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34413-8_23 · Cited by 32 · see citing works
Tutorials in Operations Research: Models, Methods and Applications for Innovative Decisionmaking
Johnson, M. P., Norman, B. A., & Secomandi, N. (2006) · Cited by 23 · see citing works
Algorithms and Applications for the Same-Decision Probability
Chen, S. J., Choi, A., & Darwiche, A. Y. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4218 · Cited by 20 · see citing works
Solis, M. (2016) · IEEE Pulse · doi:10.1109/mpul.2016.2606470 · Cited by 13 · see citing works
Theory and Applications of Satisfiability Testing – SAT 2017
Walsh, T., & Gaspers, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66263-3 · Cited by 11 · see citing works
Value of Information Based on Decision Robustness
Chen, S., Choi, A., & Darwiche, A. Y. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9684 · Cited by 10 · see citing works
Fast algorithms for robust classification with Bayesian nets
Antonucci, A., & Zaffalon, M. (2006) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2006.07.011 · Cited by 9 · see citing works
Pathirikkat, G., Reddy, M. J. B., & Mohanta, D. K. (2014) · Electric Power Components and Systems · doi:10.1080/15325008.2014.913745 · Cited by 7 · see citing works
Schaeffer, J., Müller, M., & Kishimoto, A. (2014) · IEEE Spectrum · doi:10.1109/mspec.2014.6840803 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 15
- Ben-David, S.; Blitzer, J.; Crammer, K.; Kulesza, A.; Pereira, F.; and Vaughan, J. W. 2010. A Theory of Learning from Differ- ent Domains. Machine Learning 79(1): 151-175.
- Bendale, A., and Boult, T. 2015. Towards Open World Recog- nition. In 2015 IEEE Conference on Computer Vision and Pat- tern Recognition, 1893-1902.
- Chow, Y.; Tamar, A.; Mannor, S.; and Pavone, M. 2015. Risk- Sensitive and Robust Decision-Making: A CVaR Optimiza- tion Approach. In Advances in Neural Information Process- ing Systems 28: Annual Conference on Neural Information Processing Systems 2015, (NIPS 2015), 1522-1530. Decem- ber 7-12, 2015, Montréal, Québec, Canada.
- Gretton, A.; Borgwardt, K. M.; Rasch, M. J.; Schölkopf, B.; and Smola, A. 2012. A Kernel Two-Sample Test. Journal of Machine Learning Research 13: 723-773.
- Grinberg, Y.; Precup, D.; and Gendreau, M. 2014. Optimiz- ing Energy Production Using Policy Search and Predictive State Representations. In Advances in Neural Information Pro- cessing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 3657-3665. La Jolla, CA: Neural Information Processing Systems Foundation, Inc.
- Heule, M.; Dufour, M.; van Zwieten, J.; and van Maaren, H. 2004. March_eq: Implementing Additional Reasoning into an Efficient Look-Ahead SAT Solver. In Theory and Applica- tions of Satisfiability Testing. SAT 2004 Lecture Notes in Com- puter science volume 3542, 345-359. Berlin: Springer. doi.org/10.1007/11527695_26 Human Rights Watch. 2016. Killer Robots and the Concept of Meaningful Human Control: Memorandum to Conven- tion on Conventional Weapons (CCW) Delegates. 11 April. New York: Humans Rights Watch, Inc. (www.hrw.org/ news/2016/04/11/killer-robots-and-concept-meaningful- human-control). · DOI
- Knight, W. 2016. An AI with 30 Years' Worth of Knowledge Finally Goes to Work. MIT Technology Review March 14.
- McCarthy, J.; Minsky, M.; Rochester, N.; and Shannon, C. E. 1955. A Proposal for the Dartmouth Summer Research Proj- ect on Artificial Intelligence. AI Magazine 27(4): 12-14. doi. org/10.1609/aimag.v27i4.1904 · DOI
- Meek, C., and Heckerman, D. 1997. Structure and Parame- ter Learning for Causal Independence and Causal Interac- tion Models. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, 366-375. San Francisco: Morgan Kaufmann Publishers Inc.
- Mitchell, T. M.; Cohen, W.; Hruschka, E.; Talukdar, P.; Bet- teridge, J.; Carlson, A.; Mishra, B. D.; Gardner, M.; Kisiel, B.; Krishnamurthy, J.; Lao, N.; Mazaitis, K.; Mohamed, T.; Nakashole, N.; Platanios, E. A.; Ritter, A.; Samadi, M.; Settles, B.; Wang, R.; Wijaya, D.; Gupta, A.; Chen, X.; Saparov, A.; Greaves, M.; and Welling, J. 2015. Never-Ending Learning. In Proceedings of the Twenty-Ninth AAAI Conference on Artifi- cial Intelligence (AAAI-15), 2302-2310. Palo Alto, CA: AAAI Press.
- Rudd, E. M.; Jain, L. P.; Scheirer, W. J.; and Boult, T. E. 2016. The Extreme Value Machine. Preprint of a manuscript accepted to the IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI) journal. arXiv:1506.06112 [cs.LG]. Ithaca, NY: Cornell University Library. · arXiv
- Steinhardt, J., and Liang, P. S. 2016. Unsupervised Risk Esti- mation Using Only Conditional Independence Structure. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 3657-3665.
- Sugiyama, M.; Krauledat, M.; and Müller, K.-R. 2007. Covari- ate Shift Adaptation by Importance Weighted Cross Valida- tion. Journal of Machine Learning Research 8(May): 985-1005.
- Surdeanu, M., and Ji, H. 2014. Overview of the English Slot Filling Track at the TAC2014 Knowledge Base Population Evaluation. In Proceedings of the Text Analysis Conference (TAC2014). Gaithersburg, MD: National Institute of Stan- dards and Technology, US Department of Commerce.
- Thrun, S.; Burgard, W.; and Fox, D. 2005. Probabilistic Robot- ics. Cambridge, MA: The MIT Press.
Cited by
Sorted by citations
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352 · see citing works
“A human operator can be notified whenever the predictive model expresses skepticism about the veracity of its prognosis [15], [19], [18].”
BackgroundInfluentialThe challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253 · see citing works
“In order to trust deployed AI systems, we must not only improve their robustness [5], but also develop ways to make their reasoning intelligible.”
BackgroundToward Trustworthy Artificial Intelligence (TAI) in the Context of Explainability and Robustness
chander, B., John, C., Warrier, L., & Gopalakrishnan, K. (2025) · ACM Computing Surveys · doi:10.1145/3675392 · Cited by 108 · see citing works
Adoption of artificial intelligence: a TOP framework-based checklist for digital leaders
Tursunbayeva, A., & Chalutz-Ben Gal, H. (2024) · Business Horizons · doi:10.1016/j.bushor.2024.04.006 · Cited by 102 · see citing works
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92 · see citing works
- Authors
- Dietterich, T. G.
- Affiliations
- Association for the Advancement of Artificial Intelligence (US)
- Corresponding author
- Dietterich, Thomas G.
- Venue
- AI Magazine
- Publisher
- Wiley
- Published
- 2017-09
- Volume
- 38
- Issue
- 3
- Pages
- 3-24
- ISSN
- 0738-4602 (ISSN-L)
- 2371-9621
- OpenAlex
- W2763152536
- Cited by
- 139
- References
- 67
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 19 · 2023: 25 · 2022: 21 · 2021: 16 · 2020: 22 · 2019: 11 · 2018: 8
- FWCI
- 6.69
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- AI Magazine (publishedVersion, cc-by)
- Funders
- National Science Foundation (0832804, 0705765)
- Future of Life Institute
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- uncertainty, unknown unknowns, known unknowns, autonomous systems, autonomous weapons, autonomous driving, power grid management, stock trading
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
