The DOI matches its DataCite record; 27 of the 27 references it cites were found in a registry or an index.
Cooperative Inverse Reinforcement Learning
Open accessAbstract
For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to the maximization of value for the humans. We propose a formal definition of the value alignment problem as cooperative inverse reinforcement learning (CIRL). A CIRL problem is a cooperative, partial-information game with two agents, human and robot; both are rewarded according to the human's reward function, but the robot does not initially know what this is. In contrast to classical IRL, where the human is assumed to act optimally in isolation, optimal CIRL solutions produce behaviors such as active teaching, active learning, and communicative actions that are more effective in achieving value alignment. We show that computing optimal joint policies in CIRL games can be reduced to solving a POMDP, prove that optimality in isolation is suboptimal in CIRL, and derive an approximate CIRL algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Assistance games, such as cooperative inverse RL (CIRL) (Hadfield-Menell et al., 2016), provide a framework to analyze algorithms like RLHF.”
From Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (2023) · cites this paper as Hadfield-Menell et al., 2016
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Theory of the firm: Managerial behavior, agency costs and ownership structure
Jensen, M. C., & Meckling, W. H. (1976) · Journal of Financial Economics · doi:10.1016/0304-405x(76)90026-x · Cited by 71,491
Multitask principal – agent analyses: Incentive contracts, asset ownership, and job design
Holmstrom, B., & Milgrom, P. (1991) · The Journal of Law, Economics, and Organization · doi:10.1093/jleo/7.special_issue.24 · Cited by 6,046
Aggregation and Linearity in the Provision of Intertemporal Incentives
Holmstrom, B. R., & Milgrom, P. (1987) · Econometrica · doi:10.2307/1913238 · Cited by 2,870
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015430 · Cited by 2,855
Self-citationSuperintelligence: paths, dangers, strategies
Bostrom, N. (2015) · Choice Reviews Online · doi:10.5860/choice.187536 · Cited by 1,998
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868
The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
Smallwood, R., & Sondik, E. (1973) · Operational Research · doi:10.1287/opre.21.5.1071 · Cited by 1,787
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
On the folly of rewarding A, while hoping for B
Kerr, S. (1975) · Academy of Management Journal · doi:10.2307/255378 · Cited by 1,585
An investigation of the Therac-25 accidents
Leveson, N. G., & Turner, C. S. (1993) · Computer · doi:10.1109/mc.1993.274940 · Cited by 1,245
Research works printed but not linked: 2
- Russell, S. and Norvig, P. Artificial Intelligence . Pearson, 2010.
- Waugh, K, Ziebart, B, and Bagnell, J. Computational rationalization: The inverse equilibrium problem. In ICML , 2011.
Cited by
Sorted by citations
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Artificial Intelligence, Values, and Alignment
Gabriel, I. (2020) · Minds and Machines · doi:10.1007/s11023-020-09539-2 · Cited by 896
Recent Advances in Robot Learning from Demonstration
Ravichandar, H., Polydoros, A. S., Chernova, S., & Billard, A. (2020) · Annu. Rev. Control. Robotics Auton. Syst. · doi:10.1146/annurev-control-100819-063206 · Cited by 947
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405
Self-citationAn Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G. M., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 394
Self-citationA Survey of Robot Learning Strategies for Human-Robot Collaboration in Industrial Settings
Mukherjee, D., Gupta, K., Chang, L., & Najjaran, H. (2021) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2021.102231 · Cited by 389
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications
Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, Y.-X., & Knoll, A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3457538 · Cited by 723
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. G. (2024) · IEEE Access · doi:10.1109/access.2024.3431437 · Cited by 459
Russell, S. (2020) · doi:10.1093/oso/9780190905033.003.0012 · Cited by 289
Self-citationDu, W., & Ding, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09938-y · Cited by 295
Rabinowitz, N. C., Perbet, F., Song, H. F., Zhang, C., Eslami, S. M. A., & Botvinick, M. (2018) · arXiv · doi:10.48550/arxiv.1802.07740
Vallor, S. (2024) · doi:10.1093/oso/9780197759066.001.0001 · Cited by 264
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 256
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
Zare, M., Kebria, P. M., Khosravi, A., & Nahavandi, S. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3395626 · Cited by 307
Improved Trust in Human-Robot Collaboration With ChatGPT
Ye, Y., You, H., & Du, J. (2023) · IEEE Access · doi:10.1109/access.2023.3282111 · Cited by 213
Theory of mind as inverse reinforcement learning
Jara‐Ettinger, J. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.010 · Cited by 220
An Analysis of the Interaction Between Intelligent Software Agents and Human Users
Burr, C., Cristianini, N., & Ladyman, J. (2018) · Minds and Machines · doi:10.1007/s11023-018-9479-0 · Cited by 205
A review on reinforcement learning for contact-rich robotic manipulation tasks
Elguea-Aguinaco, Í., Serrano-Muñoz, A., Chrysostomou, D., Inziarte-Hidalgo, I., Bøgh, S., & Arana-Arexolaleiba, N. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102517 · Cited by 177
Leike, J., Martic, M., Krakovna, V., Ortega, P. A., Everitt, T., Lefrancq, A., Orseau, L., & Legg, S. (2017) · arXiv · doi:10.48550/arxiv.1711.09883 · Cited by 240
Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge
Wang, P., & Chan, C. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317735 · Cited by 157
Kim, S. K., Kirchner, E. A., Stefes, A., & Kirchner, F. (2017) · Scientific Reports · doi:10.1038/s41598-017-17682-7 · Cited by 146
From inverse optimal control to inverse reinforcement learning: A historical review
Azar, N. A., Shahmansoorian, A., & Davoudi, M. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.06.001 · Cited by 127
Modeling Others using Oneself in Multi-Agent Reinforcement LearningPreprint
Răileanu, R., Denton, E., Szlam, A. D., & Fergus, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09640 · Cited by 142
Hadfield-Menell, D., Milli, S., Abbeel, P., Russell, S., & Dragan, A. (2017) · arXiv · doi:10.48550/arxiv.1711.02827 · Cited by 125
Self-citationHadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/32 · Cited by 119
Self-citationHadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S. (2016) · arXiv · doi:10.48550/arxiv.1611.08219 · Cited by 119
Self-citationA Deep Reinforcement Learning Framework for Fast Charging of Li-Ion Batteries
Park, S., Pozzi, A., Whitmeyer, M., Perez, H. E., Kandel, A., Kim, G., Choi, Y., Joe, W. T., Raimondo, D. M., & Moura, S. J. (2022) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2022.3140316 · Cited by 117
Perspectives on Digital Humanism
Werthner, H., Prem, E., Lee, E. A., & Ghezzi, C. (2021) · Springer eBooks · doi:10.1007/978-3-030-86144-5 · Cited by 133
Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration
Nikolaidis, S., Nath, S., Procaccia, A. D., & Srinivasa, S. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2909824.3020253 · Cited by 102
Learning to Teach in Cooperative Multiagent Reinforcement Learning
Omidshafiei, S., Kim, D.-K., Liu, M., Tesauro, G. J., Riemer, M., Amato, C., Campbell, M. S., & How, J. P. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v33i01.33016128 · Cited by 105
Artificial Intelligence and the Problem of Control
Russell, S. (2021) · Springer eBooks · doi:10.1007/978-3-030-86144-5_3 · Cited by 100
Self-citationA Survey of Mental Modeling Techniques in Human–Robot Teaming
Tabrez, A., Luebbers, M. B., & Hayes, B. (2020) · Current Robotics Reports · doi:10.1007/s43154-020-00019-0 · Cited by 98
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Inverse Reinforcement Q-Learning Through Expert Imitation for Discrete-Time Systems
Xue, W., Lian, B., Fan, J., Kolaric, P., Chai, T., & Lewis, F. L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3106635 · Cited by 95
(2021) · Journal of Machine Learning Research · Cited by 224
Open Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93
Understanding Human Intelligence through Human Limitations
Griffiths, T. L. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.001 · Cited by 88
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101
“Assistance games, such as cooperative inverse RL (CIRL) [Hadfield-Menell et al., 2016], provide a framework to analyze algorithms like RLHF.”
MethodSelf-citationValiente, R., Toghi, B., Pedarsani, R., & Fallah, Y. P. (2022) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2022.3172981 · Cited by 82
Reinforcement Learning with a Corrupted Reward Channel
Everitt, T., Krakovna, V., Orseau, L., & Legg, S. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/656 · Cited by 74
Trustworthy human-AI partnerships
Ramchurn, S. D., Stein, S., & Jennings, N. R. (2021) · iScience · doi:10.1016/j.isci.2021.102891 · Cited by 73
Showing versus Doing: Teaching by DemonstrationPreprint
Ho, M. K., Littman, M. L., MacGlashan, J., Cushman, F. A., & Austerweil, J. L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14004 · Cited by 71
Learning a commonsense moral theory
Max Kleiman-weiner, A., Saxe, R. R., & Tenenbaum, J. B. (2017) · Cognition · doi:10.1016/j.cognition.2017.03.005 · Cited by 73
Rakhsha, A., Radanović, G., Devidze, R., Zhu, X., & Singla, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12909 · Cited by 73
Inferring Human Intent and Predicting Human Action in Human–Robot Collaboration
Hoffman, G., Bhattacharjee, T., & Nikolaidis, S. (2023) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-071223-105834 · Cited by 66
Find the Gap: AI, Responsible Agency and Vulnerability
Vallor, S., & Vierkant, T. (2024) · Minds and Machines · doi:10.1007/s11023-024-09674-0 · Cited by 66
Hard choices in artificial intelligence
Dobbe, R., Krendl Gilbert, T., & Mintz, Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103555 · Cited by 65
Langley, C., Cirstea, B. I., Fabio Cuzzolin, A., & Sahakian, B. J. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.778852 · Cited by 72
Flexible human–robot cooperation models for assisted shop-floor tasks
Darvish, K., Wanderlingh, F., Bruno, B., Simetti, E., Mastrogiovanni, F., & Casalino, G. (2018) · Mechatronics · doi:10.1016/j.mechatronics.2018.03.006 · Cited by 62
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., & Halgamuge, M. N. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3377445 · Cited by 62
Zhang, H., Zhang, Y., Wang, Z., Zhang, S., Li, H., & Chen, M. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.04.005 · Cited by 58
A goal-centric outlook on learning
Molinaro, G., & Collins, A. G. E. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.08.011 · Cited by 63
Alignment for Advanced Machine Learning Systems
Taylor, J., Yudkowsky, E., LaVictoire, P., & Critch, A. (2020) · Oxford University Press eBooks · doi:10.1093/oso/9780190905033.003.0013 · Cited by 63
Robot learning from demonstration for path planning: A review
Xie, Z., Zhang, Q., Jiang, Z., & Liu, H. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1648-4 · Cited by 58
LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planning
Cleac’h, S. L., Schwager, M., & Manchester, Z. R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3074880 · Cited by 58
Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography
Gschwindt, M., Camci, E., Bonatti, R., Wang, W., Kayacan, E., & Scherer, S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967592 · Cited by 59
Maximum-Entropy Multi-Agent Dynamic Games: Forward and Inverse Solutions
Mehr, N., Wang, M., Bhatt, M., & Schwager, M. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3232300 · Cited by 57
When code isn’t law: rethinking regulation for artificial intelligence
Judge, B., Nitzberg, M., & Russell, S. (2024) · Policy and Society · doi:10.1093/polsoc/puae020 · Cited by 91
Self-citationAwad, E., Levine, S., Anderson, M., Anderson, S., Conitzer, V., Crockett, M. J., Everett, J. A. C., Evgeniou, T., Alison Gopnik, C., Jamison, J., et al. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.02.009 · Cited by 70
Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications
Brown, D. S., & Niekum, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017749 · Cited by 16
The Challenge of Value Alignment: From Fairer Algorithms to AI Safety
Gabriel, I., & Ghazavi, V. (2022) · Oxford Handbook of Digital Ethics · doi:10.1093/oxfordhb/9780198857815.013.18 · Cited by 52
Personalizing Mobile Fitness Apps using Reinforcement Learning.
Zhou, M., Mintz, Y. D., Fukuoka, Y., Goldberg, K. Y., Flowers, E., Kaminsky, P., Castillejo, A., & Aswani, A. J. (2018) · PubMed · Cited by 51
Foundations of explanations as model reconciliation
Sreedharan, S., Chakraborti, T., & Subbarao Kambhampati, B. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103558 · Cited by 55
Agentic AI: The age of reasoning—A review
Nisa, U., Shirazi, M., Saip, M. A., & Pozi, M. S. M. (2025) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2025.08.003 · Cited by 83
Continuous-time inverse quadratic optimal control problem
Li, Y., Yao, Y., & Xiaoming Hu, A. (2020) · Automatica · doi:10.1016/j.automatica.2020.108977 · Cited by 50
Franceschelli, G., & Musolesi, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15278 · Cited by 54
Everitt, T., Hutter, M., Kumar, R., & Krakovna, V. (2021) · Synthese · doi:10.1007/s11229-021-03141-4 · Cited by 80
Safe Imitation Learning via Fast Bayesian Reward Inference from PreferencesPreprint
Brown, D. S., Coleman, R., Srinivasan, R., & Niekum, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09089 · Cited by 51
Clustering Trust Dynamics in a Human-Robot Sequential Decision-Making Task
Bhat, S., Lyons, J. B., Shi, C., & Yang, X. J. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3188902 · Cited by 46
AI Safety Needs Social Scientists
Irving, G., & Askell, A. (2019) · Distill · doi:10.23915/distill.00014 · Cited by 61
The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail
Bowman, S. R. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.516 · Cited by 55
Learning from Interventions: Human-robot interaction as both explicit and implicit feedback
Spencer, J. C., Choudhury, S., Barnes, M., Schmittle, M., Chiang, M., Ramadge, P. J., & Srinivasa, S. S. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.055 · Cited by 44
Who's Learning? Using Demographics in EDM Research
Paquette, L., Ocumpaugh, J. L., Li, Z., Andres, A., & Baker, R. S. (2020) · Journal of Educational Data Mining · doi:10.5281/zenodo.4143611 · Cited by 48
Butz, M. V. (2021) · KI - Künstliche Intelligenz · doi:10.1007/s13218-021-00705-x · Cited by 42
Ho, M. K., & Griffiths, T. L. (2021) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-042920-015547 · Cited by 53
Quantum Computing and AI: The Synergy of Two Revolutionary Technologies
Ahmadi, A. A. (2023) · Asian Journal of Electrical Sciences · doi:10.51983/ajes-2023.12.2.4118 · Cited by 46
Handling Model Uncertainty and Multiplicity in Explanations via Model Reconciliation
Sreedharan, S., Chakraborti, T., & Subbarao Kambhampati, B. (2018) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v28i1.13930 · Cited by 42
Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks
Gao, X.-F., Gong, R., Zhao, Y.-Z., Wang, S., Shu, T., & Zhu, S.-C. (2020) · IEEE International Symposium on Robot and Human Interactive Communication · doi:10.1109/ro-man47096.2020.9223595 · Cited by 48
Multiagent Graphical Games With Inverse Reinforcement Learning
Donge, V. S., Lian, B., Lewis, F. L., & Davoudi, A. A. (2023) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3210856 · Cited by 39
Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games
Lin, X., Adams, S. C., & Beling, P. A. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11541 · Cited by 39
A Survey of Opponent Modeling in Adversarial Domains
Nashed, S. B., & Zilberstein, S. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12889 · Cited by 43
Physical interaction as communication: Learning robot objectives online from human corrections
Losey, D. P., Bajcsy, A. V., O’Malley, M. K., & Dragan, A. D. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211050958 · Cited by 37
Self-citationInteractive Teaching Algorithms for Inverse Reinforcement Learning
Kamalaruban, P., Devidze, R., Cevher, V., & Singla, A. K. (2019) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 37
Conservative Agency via Attainable Utility Preservation
Turner, A. M., Hadfield-Menell, D., & Tadepalli, P. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375851 · Cited by 36
Self-citationLai, K.-H., Zha, D., Li, Y., & Hu, X. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/435 · Cited by 40
Human-in-the-Loop Imitation Learning using Remote Teleoperation
Mandlekar, A., Xu, D., Martín-Martín, R., Zhu, Y., Fei-Fei, L., & Savarese, S. (2020) · arXiv · doi:10.48550/arxiv.2012.06733 · Cited by 35
Decision-Oriented Dialogue for Human-AI Collaboration
Lin, J., Tomlin, N., Andreas, J., & Eisner, J. M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00679 · Cited by 68
Beyond Preferences in AI Alignment
Zhi‐Xuan, T., Carroll, M., Franklin, M., & Ashton, H. (2024) · Philosophical Studies · doi:10.1007/s11098-024-02249-w · Cited by 80
Task Learning, Intent Prediction, and Adaptive Blended Shared Control With Application to Excavators
Jin, Z., Pagilla, P. R., Maske, H., & Chowdhary, G. V. (2020) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2019.2959536 · Cited by 35
AI ethical biases: normative and information systems development conceptual framework
Chowdhury, T., & Oredo, J. O. (2022) · Journal of Decision System · doi:10.1080/12460125.2022.2062849 · Cited by 38
AGENT: A Benchmark for Core Psychological Reasoning
Shu, T., Bhandwaldar, A., Gan, C., Smith, K. A., Liu, S., Gutfreund, D., Spelke, E., Tenenbaum, J. B., & Ullman, T. D. (2021) · arXiv · doi:10.48550/arxiv.2102.12321 · Cited by 34
Algorithmic Equity: A Framework for Social Applications
Osoba, O., Boudreaux, B., Saunders, J. M., Irwin, J., Mueller, P. A., & Cherney, S. (2019) · RAND Corporation eBooks · doi:10.7249/rr2708 · Cited by 35
Game-Theoretic Planning for Risk-Aware Interactive Agents
Wang, M., Mehr, N., Gaidon, A., & Schwager, M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341137 · Cited by 33
SRL-ORCA: A Socially Aware Multi-Agent Mapless Navigation Algorithm in Complex Dynamic Scenes
Qin, J., Qin, J., Qiu, J., Liu, Q., Li, M., & Ma, Q. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3331621 · Cited by 33
Instilling moral value alignment by means of multi-objective reinforcement learning
Rodríguez-Soto, M., Serramia, M., López-Sánchez, M., & Rodríguez-Aguilar, J. A. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09635-0 · Cited by 32
Bridging adaptive management and reinforcement learning for more robust decisions
Chapman, M., Xu, L., Lapeyrolerie, M., & Boettiger, C. (2023) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2022.0195 · Cited by 37
Bobu, A., Bajcsy, A., Fisac, J. F., Deglurkar, S., & Dragan, A. D. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2971415 · Cited by 39
Self-citationChallenges of Aligning Artificial Intelligence with Human Values
Sutrop, M. (2020) · Acta Baltica Historiae et Philosophiae Scientiarum · doi:10.11590/abhps.2020.2.04 · Cited by 44
Metadata
- Authors
- Hadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- Volume
- 29
- Pages
- 3909-3917
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.03137
- OpenAlex
- W2410842990
- Cited by
- 413
- References
- 25
- Citations by year
- 2026: 1 · 2025: 18 · 2024: 59 · 2023: 56 · 2022: 37 · 2021: 66 · 2020: 80 · 2019: 41 · 2018: 27 · 2017: 25 · 2016: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- value alignment, human-robot interaction, partially observable Markov decision processes, active teaching methods, active learning (machine learning), communicative action, robot learning from humans, reward function learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
