CoChat
Sign Up
PreprintCited by 28Law, Economics, and Judicial SystemsEthics and Social Impacts of AILiterary and Philosophical Studies

The DOI matches its Crossref record.

Incomplete Contracting and AI Alignment

Open access
Hadfield-Menell, Dylan, Hadfield, Gillian K.·2018·Proceedings of the AAAI/ACM Conference on AI Ethics and Society
DOI 10.2139/ssrn.3165793
Open this paper in CoChat — free accountOpen PDF

Abstract

We suggest that the analysis of incomplete contracting developed by law and economics researchers can provide a useful framework for understanding the AI alignment problem and help to generate a systematic approach to finding solutions. We first provide an overview of the incomplete contracting literature and explore parallels between this work and the problem of AI alignment. As we emphasize, misalignment between principal and agent is a core focus of economic analysis. We highlight some technical results from the economics literature on incomplete contracts that may provide insights for AI alignment researchers. Our core contribution, however, is to bring to bear an insight that economists have been urged to absorb from legal scholars and other behavioral scientists: the fact that human contracting is supported by substantial amounts of external structure, such as generally available institutions (culture, law) that can supply implied terms to fill the gaps in incomplete contracts. We propose a research agenda for AI alignment work that focuses on the problem of how to build AI that can replicate the human cognitive processes that connect individual incomplete contracts with this supporting external structure.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2139/ssrn.3165793

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • To really grasp the intention behind instructions, AI may require a complete model of human language and interaction, including an understanding of the culture, institutions, and practices that allow people to understand the implied meaning of terms.

    “To really grasp the intention behind instructions, AI may require a complete model of human language and interaction, including an understanding of the culture, institutions, and practices that allow people to understand the implied meaning of terms (HadfieldMenell and Hadfield 2018).”

    From Artificial Intelligence, Values, and Alignment (2020) · cites this paper as Hadfield-Menell and Hadfield 2018

    Section 5 (page not available in our copy) says that interpreting implied terms depends on cultural norms and institutions, and that aligned AI needs tools to read and predict human normative structures. It does not say AI requires a complete model of human language and interaction; that requirement is stronger and broader than the paper's claim. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 80

  1. Philippe Aghion, 2011, Incomplete contracts and the theory of the firm: What have we learned over the past 25 years?, The Journal of Economic Perspectives, 25, 181, 10.1257/jep.25.2.181 · DOI
  2. Philippe Aghion, 1997, Formal and real authority in organizations, Journal of Political Economy, 105, 1, 10.1086/262063 · DOI
  3. Hana Ajakan, 2015, Domain-adversarial neural networks
  4. Armen A Alchian, 1972, Production, information costs, and economic organization, The American Economic Review, 62, 777, 10.1017/cbo9780511817410.015 · DOI
  5. Dario Amodei, 2016, Faulty reward functions in the wild
  6. Dario Amodei, 2016, Concrete problems in AI safety, 10.1063/pt.5.028530 · DOI
  7. Mike Ananny, 2016, Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability, 10.1177/1461444816676645 · DOI · Paper page
  8. Stuart Armstrong, 2015, Motivated value selection for artificial agents, AAAI Workshop: AI and Ethics
  9. Kenneth J Arrow, 1951, An extension of the basic theorems of classical welfare economics, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 10.1525/9780520411586-038 · DOI
  10. Ian Ayres, 1989, Filling gaps in incomplete contracts: An economic theory of default rules, The Yale Law Journal, 99, 87, 10.2307/796722 · DOI
  11. George Baker, 1994, Subjective performance measures in optimal incentive contracts, The Quarterly Journal of Economics, 109, 1125, 10.2307/2118358 · DOI
  12. George Baker, 1999, Informal authority in organizations, Economics & Organization, 15, 56, 10.1093/jleo/15.1.56 · DOI
  13. Andrew Barto, 2009, Where do rewards come from, Proceedings of the Cognitive Science Society
  14. B Bernheim, 1998, Incomplete contracts and strategic ambiguity, The American Economic Review, 88, 902
  15. Lisa Bernstein, 1992, Opting out of the legal system: Extralegal contractual relations in the diamond industry, The Journal of Legal Studies, 21, 115, 10.1086/467902 · DOI
  16. Lisa Bernstein, 2015, Beyond relational contracts: Social capital and network governance in procurement contracts, Journal of Legal Analysis, 7, 561, 10.1093/jla/law001 · DOI
  17. Cristina Bicchieri, 2006, The Grammar of Society: The Nature and Dynamics of Social Norms
  18. Patrick Bolton, 2010, Satisficing contracts. The Review of Economic Studies, 77, 937, 10.1111/j.1467-937X.2009.00597.x · DOI
  19. Nick Bostrom, 2014, Superintelligence: Paths, Dangers, Strategies
  20. Paul Christiano, 2017, Deep reinforcement learning from human preferences
  21. Christian Daniel, 2014, Active reward learning, Robotics: Science and Systems, 10.15607/rss.2014.x.031 · DOI
  22. Gerard Debreu, 1959, A Theory of Value: An Axiomatic Analysis of Economic Equilibrium
  23. Dorsa Sadigh, 2017, Active preference-based learning of reward functions, Robotics: Science and Systems, 10.15607/rss.2017.xiii.053 · DOI
  24. Anca D Dragan, 2017, Robot planning with mathematical models of human state and action
  25. Eugene F Fama, 1983, Separation of ownership and control, Journal of Law and Economics, 26, 1301, 10.1086/467037 · DOI
  26. Allan E Farnsworth, 1968, Disputes over omission in contracts, Columbia Law Review, 68, 860, 10.2307/1121035 · DOI
  27. Ricard Gil, 2017, Formal and informal contracting: Theory and evidence, Annual Review of Law and Social Science, 13, 141, 10.1146/annurev-lawsocsci-110316-113413 · DOI · Paper page
  28. David Ginsberg, 2016, Hard questions: Is spending time on social media bad for us?
  29. Charles J Goetz, 1981, Principles of relational contracts, Virginia Law Review, 67, 1089, 10.2307/1072743 · DOI
  30. Mark Granovetter, 1985, Economic action and social structure: The problem of embeddedness, American Journal of Sociology, 91, 481, 10.1086/228311 · DOI · Paper page
  31. Sanford Grossman, 1986, The costs and benefits of ownership: A theory of vertical and lateral integration, Journal of Political Economy, 94, 691, 10.1086/261404 · DOI
  32. Gillian K Hadfield, 1990, Problematic relations: Franchising and the law of incomplete contracts, Stanford Law Review, 42, 927, 10.2307/1228908 · DOI
  33. Gillian K Hadfield, 1994, Judicial competence and the interpretation of incomplete contracts, Journal of Legal Studies, 23, 159, 10.1086/467919 · DOI
  34. Gillian K Hadfield, 2005, fairness and welfare: An invitation to feminist law and economics, Annual Review of Law and Social Science, 1, 285, 10.1146/annurev.lawsocsci.1.041604.115915 · DOI
  35. Gillian K Hadfield, 2017, Rules for a Flat World: Why Humans Invented Law and How to Reinvent It for a Complex Global Economy
  36. Gillian K Hadfield, 2016, Scaffolding: Using formal contracts to support informal relations in support of innovation, Wisconsin Law Review, 981
  37. Gillian K Hadfield, 2014, Microfoundations of the rule of law, Annual Review of Political Science, 17, 21, 10.1146/annurev-polisci-100711-135226 · DOI
  38. Hadfield-Menell, 2016, Cooperative inverse reinforcement learning, Advances in Neural Information Processing Systems, 3909
  39. Hadfield-Menell, 2017, The off-switch game, 10.24963/ijcai.2017/32 · DOI · Paper page
  40. Hadfield-Menell, 2017, Inverse reward design, Advances in Neural Information Processing Systems
  41. Halonen-Akatwijuka, 2013, More is less: Why partiews may deliberately write incomplete contracts, 10.3386/w19001 · DOI
  42. Hardt, 2016, Equality of opportunity in supervised learning
  43. Oliver D Hart, 1988, Incomplete contracts and the theory of the firm, Economics & Organization, 4, 119, 10.1093/oxfordjournals.jleo.a036940 · DOI
  44. Oliver D Hart, 1988, Incomplete contracts and renegotiation, Econometrica, 56, 755, 10.2307/1912698 · DOI
  45. Bengt Holmstrom, 1991, Multitask principal-agent analyses: Incentive contracts, asset ownership, and job design, Economics, & Organization, 7, 24, 10.1093/jleo/7.special_issue.24 · DOI · Paper page
  46. Benjamin Klein, 1980, Transaction cost determinants of 'unfair' contractual arrangements, The American Economic Review, 70, 356, 10.1002/9780470752135.ch14 · DOI
  47. Benjamin Klein, 1978, Vertical integration, appropriable rents, and the competitive contracting process, Journal of Law and Economics, 21, 297, 10.1086/466922 · DOI
  48. Jon Kleinberg, 2016, Inherent trade-offs in the fair determination of risk
  49. Jean-Jacques Laffont, 1989, The Economics of Uncertainty and Information
  50. Jonathan Levin, 2003, Relational incentive contracts, The American Economic Review, 93, 835, 10.1257/000282803322157115 · DOI
  51. Kristian Lum, 2016, A statistical framework for fair predictive algorithms
  52. Stewart Macaulay, 1963, Non-contractual relations in business: A preliminary study, American Sociological Review, 28, 55, 10.2307/2090458 · DOI
  53. Ian R Macneil, 1973, The many futures of contracts, Southern California Law Review, 691
  54. Ian R Macneil, 1978, Contracts: Adjustment of long-term economic relations under classical, neoclassical, and relational contract law, Northwestern University Law Review, 72, 854
  55. Ian R Macneil, 1983, Values in contract: internal and external, Northwestern University Law Review, 340
  56. Eric Maskin, 1999, Unforeseen contingencies and incomplete contracts. The Review of Economic Studies, 66, 83
  57. Andrew Y Ng, 2000, Algorithms for inverse reinforcement learning, ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning, 663
  58. Stephen M Omohundro, 2008, The basic ai drives
  59. Laurent Orseau, 2016, Safely interruptible agents
  60. Hannes Rakoczy, 2013, The early ontogeny of social norms, Child Development Perspectives, 7, 17, 10.1111/cdep.12010 · DOI
  61. Rakoczy, 2009, Young children's understanding of the context-relativity of normative rules in conventional games, British Journal of Developmental Psychology, 27, 445, 10.1348/026151008x337752 · DOI
  62. Francesca Rossi, 2011, A short introduction to preferences between artificial intelligence and social choice, 10.2200/s00372ed1v01y201107aim014 · DOI
  63. Alan Schwartz, 2003, Contract theory and the limits of contract law, The Yale Law Review, 113, 541, 10.2307/3657531 · DOI
  64. Robert E Scott, 2006, Anticipating litigation in contract design, Yale Law Journal, 115, 10.2307/20455639 · DOI
  65. Amartya Sen, 1985, Social choice and justice: A review article, Journal of Economic Literature, 23, 1764
  66. Steven Shavell, 1980, Damage measures for breach of contract, The Bell Journal of Economics, 11, 466, 10.2307/3003374 · DOI
  67. Steven Shavell, 2006, On the writing and interpretation of contracts, Economics and Organization, 22, 289, 10.1093/jleo/ewj017 · DOI
  68. Steven Shavell, 2006, On the writing and interpretation of contracts, Economics and Organization, 22, 289, 10.1093/jleo/ewj017 · DOI
  69. Herbert A Simon, 1955, A behavioral model of rational choice, The Quarterly Journal of Economics, 69, 99, 10.2307/1884852 · DOI · Paper page
  70. Singh, 2010, Intrinsically motivated reinforcement learning: An evolutionary perspective, IEEE Transactions on Autonomous Mental Development, 2, 70, 10.1109/tamd.2010.2051031 · DOI
  71. Adam Smith, The Theory of Moral Sentiments. A Millar, 10.1017/cbo9780511800153.005 · DOI
  72. Nate Soares, 2015, Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence
  73. Michael Spence, 1973, Job market signaling, The Quarterly Journal of Economics, 87, 355, 10.2307/1882010 · DOI
  74. Kathryn E Spier, 1992, Incomplete contracts and signalling, The RAND Journal of Economics, 23, 432, 10.2307/2555872 · DOI
  75. Jean Tirole, 1999, Incomplete contracts: Where do we stand?, Econometrica, 67, 741, 10.1111/1468-0262.00052 · DOI
  76. Jean Tirole, 2009, Cognition and incomplete contracts, The American Economic Review, 99, 265, 10.1257/aer.99.1.265 · DOI
  77. Polly Wiessner, 2005, Norm enforcement among the ju/'hoansi bushmen: A case of strong reciprocity?, Human Nature, 16, 115, 10.1007/s12110-005-1000-9 · DOI
  78. Oliver E Williamson, 1975, Markets and Hierarchies
  79. Muhammad Zafar, 2016, Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment
  80. Richard Zemel, Learning fair representations, Proceedings of Machine Learning Research, 28, 325

Cited by

Sorted by citations

Metadata

Authors
Hadfield-Menell, D., & Hadfield, G. K.
Affiliations
  • University of California, Berkeley (US)
  • University of Toronto (CA)
ORCID
Venue
Proceedings of the AAAI/ACM Conference on AI Ethics and Society
Publisher
Elsevier BV
Published
2018
Pages
417-422
ISSN
3065-8365 (ISSN-L)
DOI
10.2139/ssrn.3165793
OpenAlex
W2796639635
Cited by
28
References
0
Citations by year
2026: 2 · 2025: 4 · 2024: 6 · 2023: 3 · 2022: 1 · 2021: 4 · 2020: 1 · 2019: 1 · 2018: 6
FWCI
6.49
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
  • SSRN Electronic Journal (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • bepress Legal Repository (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Law, Economics, and Judicial Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Law › Literary and Philosophical Studies
Keywords
incomplete contracting, AI alignment, principal-agent problem, implied terms, external structure, institutions, law and economics, human cognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review