Explainable, Normative, and Justified Agency
Open accessAbstract
In this paper, we pose a new challenge for AI researchers – to develop intelligent systems that support justified agency. We illustrate this ability with examples and relate it to two more basic topics that are receiving increased attention – agents that explain their decisions and ones that follow societal norms. In each case, we describe the target abilities, consider design alternatives, note some open questions, and review prior research. After this, we return to justified agency, offering a hypothesis about its relation to explanatory and normative behavior. We conclude by proposing testbeds and experiments to evaluate this empirical claim and encouraging other researchers to contribute to this crucial area.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The framework introduced in [51] relies on norms (called 'principles') inferred from ethical judgments using inductive logic programming.
“For example, the framework introduced in [51] relies on norms (called 'principles') inferred from ethical judgments using inductive logic programming.”
From Beyond explainability: justifiability and contestability of algorithmic decision systems (2022) · cites this paper as [51]
A lot of work has been done on argumentation and dialog games, but the focus in these areas is generally the logical structure of the framework to express and to relate arguments or the protocol to exchange arguments.
“A lot of work has also been done on argumentation and dialog games [6, 11, 51, 53] but the focus in these areas is generally the logical structure of the framework to express and to relate arguments or the protocol to exchange arguments.”
From Beyond explainability: justifiability and contestability of algorithmic decision systems (2022) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Universal moral grammar: theory, evidence and the future
Mikhail, J. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.12.007 · Cited by 885 · see citing works
Argumentation in artificial intelligence
Bench-Capon, T. J. M., & Dunne, P. E. S. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.05.001 · Cited by 806 · see citing works
The epistemology of a rule-based expert system —a framework for explanation
Clancey, W. J. (1983) · Artificial Intelligence · doi:10.1016/0004-3702(83)90008-5 · Cited by 712 · see citing works
Integrating planning and learning: the PRODIGY architecture
Veloso, M., Carbonell, J., PÉREZ, A., Borrajo, D., Fink, E., & Blythe, J. (1995) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/09528139508953801 · Cited by 365 · see citing works
An explainable artificial intelligence system for small-unit tactical behavior
van Lent, M., Fisher, W. M., & Mancuso, M. R. (2004) · Cited by 215 · see citing works
Toward a General Logicist Methodology for Engineering Ethically Correct Robots
Bringsjord, S., Arkoudas, K., & Bello, P. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.82 · Cited by 185 · see citing works
Explanation in Second Generation Expert Systems
Swartout, W. R., & Moore, J. D. (1993) · Second Generation Expert Systems · doi:10.1007/978-3-642-77927-5_24 · Cited by 160 · see citing works
Explainable Agency for Intelligent Autonomous Systems
Langley, P., Meadows, B., Sridharan, M., & Choi, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i2.19108 · Cited by 156 · see citing works
An Approach to Computing Ethics
Anderson, M., Anderson, S., & Armen, C. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.64 · Cited by 113 · see citing works
Agents that Learn to Explain Themselves
Johnson, W. L. (1994) · Cited by 93 · see citing works
Evaluating Explanations: A Content Theory
Leake, D. B. (1992) · Cited by 80 · see citing works
An Integrated Reasoning Approach to Moral Decision Making
Dehghani, M., Forbus, K., Tomai, E., & Klenk, M. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511978036.028 · Cited by 78 · see citing works
Reasoning with Rules and Precedents - A Computational Model of Legal Analysis
Branting, L. K. (1999) · Medical Entomology and Zoology · Cited by 51 · see citing works
Networks of Social and Moral Norms in Human and Robot Agents
Malle, B. F., Scheutz, M. J., & Austerweil, J. L. (2017) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-46667-5_1 · Cited by 46 · see citing works
Briggs, G., & Scheutz, M. J. (2015) · National Conference on Artificial Intelligence · Cited by 41 · see citing works
A CONSTRAINED ARCHITECTURE FOR LEARNING AND PROBLEM SOLVING
Jones, R. M., & Pat Langley, B. (2005) · Computational Intelligence · doi:10.1111/j.1467-8640.2005.00283.x · Cited by 28 · see citing works
Lessons in Machine Ethics from the Perspective of Two Computational Models of Ethical Reasoning
McLaren, B. M. (2005) · doi:10.1184/r1/6470222 · Cited by 13 · see citing works
There Is No Agency Without Attention
Bello, P., & Bridewell, W. B. (2017) · AI Magazine · doi:10.1609/aimag.v38i4.2742 · Cited by 13 · see citing works
Exploring Moral Reasoning in a Cognitive Architecture.
Iba, W., & Langley, P. (2011) · eScholarship (California Digital Library) · Cited by 11 · see citing works
Self-citationExperimental Studies of Integrated Cognitive Systems
Pat Langley, B., & Messina, E. R. (2004) · Cited by 10 · see citing works
A Testbed for Evaluation of Architectures for Physical Agents
Choi, D., Morgan, M. L., Park, C., & Pat Langley, B. (2007) · Cited by 7 · see citing works
Particularism and Generalism: How AI can Help us to Better Understand Moral Cognition
Guarini, M. (2020) · Routledge eBooks · doi:10.4324/9781003074991-26 · Cited by 3 · see citing works
A Computational Account of Complex Moral Judgement
Liu, L., Langley, P., & Meadows, B. (2013) · ResearchSpace (University of Auckland) · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Colaco, Z., and Sridharan, M. 2015. What Happened and Why? A Mixed Architecture for Planning and Explanation Generation in Robotics. Australasian Conference on Robotics and Automation . Canberra, Australia.
- Erwig, M.; Fern, A.; Murali, M.; and Koul, A. 2018. Explaining Deep Adaptive Programs via Reward Decomposition. Proceedings of the 2018 IJCAI Workshop on Explainable Artificial Intelligence , 40-44. Stockholm.
- Langley, P.; Barley, M.; Meadows, B.; Choi, D.; and Katz, E. P. 2016. Goals, Utilities, and Mental Simulation in Continuous Planning. Proceedings of the Fourth Annual Conference on Cognitive Systems . Evanston, IL.
Cited by
Sorted by citations
Argumentation and explainable artificial intelligence: a survey
Vassiliades, A., Bassiliades, N., & Patkos, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000011 · Cited by 160 · see citing works
Beyond explainability: justifiability and contestability of algorithmic decision systems
Henin, C., & Le Métayer, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01251-8 · Cited by 71 · see citing works
Conversational Interfaces for Explainable AI: A Human-Centred Approach
Jentzsch, S. F., Höhn, S., & Hochgeschwender, N. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30391-4_5 · Cited by 37 · see citing works
NormLime: A New Feature Importance Metric for Explaining Deep Neural NetworksPreprint
Ahern, I., Noack, A., Guzman-Nateras, L., Dou, D., Li, B., & Huan, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04200 · Cited by 37 · see citing works
Artificial Intelligence and Agency: Tie-breaking in AI Decision-Making
Swanepoel, D., & Corks, D. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00476-2 · Cited by 23 · see citing works
Generating justifications for norm-related agent decisions
Kasenberg, D., Roque, A., Thielstrom, R., Chita-Tegmark, M., & Scheutz, M. J. (2019) · International Conference on Natural Language Generation · doi:10.18653/v1/w19-8660 · Cited by 18 · see citing works
What Did You Think Would Happen? Explaining Agent Behaviour Through Intended OutcomesPreprint
Yau, H., Russell, C., & Hadfield, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05064 · Cited by 17 · see citing works
An explainable assistant for multiuser privacy
Mosca, F., & Such, J. M. (2022) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-021-09543-5 · Cited by 16 · see citing works
A framework to contest and justify algorithmic decisions
Henin, C., & Le Métayer, D. (2021) · AI and Ethics · doi:10.1007/s43681-021-00054-3 · Cited by 14 · see citing works
Agent EXPRI: Licence to Explain
Mosca, F., Sarkadi, Ş., Such, J. M., & McBurney, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-51924-7_2 · Cited by 11 · see citing works
Automated Justification of Collective Decisions via Constraint Solving
Boixel, A., & Endriss, U. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/nukv4871 · Cited by 9 · see citing works
Explaining the Behavior of Remote Robots to Humans : An Agent-based Approach
Mualla, Y. (2020) · doi:10.70675/387da533zcca9z4ec0zaeb2z4257ce794921 · Cited by 6 · see citing works
Makokha, J. M. (2021) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--36700 · Cited by 5 · see citing works
Kear, A., & Sasha, L. (2020) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2020.0110307 · Cited by 4 · see citing works
Explainability in Mechanism Design: Recent Advances and the Road Ahead
Suryanarayana, S. A., Sarne, D., & Kraus, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20614-6_21 · Cited by 4 · see citing works
Start Using Justifications When Explaining AI Systems to Decision Subjects
Kolářová, K., & Schmude, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11108-1_14 · Cited by 3 · see citing works
Achamu, G., Berhan, E., & Gebeyehu, S. G. (2021) · Journal of Data Information and Management · doi:10.1007/s42488-021-00060-4 · Cited by 2 · see citing works
Applying artificial intelligence in the science & technology cycle
Weber, R. O. (2019) · Information Services & Use · doi:10.3233/isu-190062 · Cited by 2 · see citing works
Engineering Inter-agent Explainability in BDI Agents
Beaumont, K., Yan, E., Burattini, S., & Collier, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-01399-6_9 · Cited by 1 · see citing works
Towards a Framework for Challenging ML-Based Decisions
Henin, C., & Métayer, D. L. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-91779-1_10
Towards a framework for challenging ML-based decisions
Henin, C., & Métayer, D. L. (2020) · HAL
Achamu, G., Berhan, E., & Geremaw, S. (2022) · Research Square · doi:10.21203/rs.3.rs-901071/v1
Managing the sXAI Interaction: Beliefs, Goals, and Decisional Processes
Alami, R., & Wrede, B. (2026) · Springer eBooks · doi:10.1007/978-981-96-5290-7_11
Collaborate and explain on-the-fly: knowledge-based reasoning and learning in ad hoc teamwork
Dodampegama, H., & Sridharan, M. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1765191
Evaluating Actionability in Explainable AI
Mansi, G., Kim, J., & Riedl, M. O. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_5
- Authors
- Langley, P.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 9775-9779
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2966649526
- Cited by
- 25
- References
- 23
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 1 · 2022: 3 · 2021: 6 · 2020: 6 · 2019: 4
- FWCI
- 1.82
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- Office of Naval Research (N00014-17-1-2434, N00014-17-1, N00014-15-1-2517, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- explainable AI, normative behavior, social norms, decision explanation, AI agents, testbed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
