CoChat
Sign Up
PreprintCited by 22

Aligning AI With Shared Human Values

Hendrycks, Dan, Burns, Collin, Basart, Steven, Critch, Andrew, Li, Jerry, Song, Dawn, Steinhardt, Jacob·2020·arXiv
DOI 10.48550/arxiv.2008.02275arXiv 2008.02275
Open this paper in CoChat — free account

Abstract

We show how to assess a language model's knowledge of basic concepts of morality. We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality. Models predict widespread moral judgments about diverse text scenarios. This requires connecting physical and social world knowledge to value judgements, a capability that may enable us to steer chatbot outputs or eventually regularize open-ended reinforcement learning agents. With the ETHICS dataset, we find that current language models have a promising but incomplete ability to predict basic human ethical judgements. Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2008.02275

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing LMs have potential in ethical judgment but still need improvement.

    “The study conducted by [69] reveals that existing LMs have potential in ethical judgment, but still need improvement.”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [69]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 46

  1. S. Armstrong. General purpose intelligence: Arguing the orthogonality thesis. 2013.
  2. I. Asimov. I, Robot. Gnome Press, 1950.
  3. I. Beltagy, M. E. Peters, and A. Cohan. Longformer: The long-document transformer. ArXiv, abs/2004.05150, 2020. · arXiv
  4. Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi. PIQA: Reasoning about physical commonsense in natural language. In AAAI, 2019.
  5. T. B. Brown, B. P. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krüger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. J. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language models are few-shot learners. ArXiv, abs/2005.14165, 2020. · arXiv
  6. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In ICML, 2005.
  7. P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. In NIPS, 2017.
  8. S. Corbett-Davies and S. Goel. The measure and mismeasure of fairness: A critical review of fair machine learning. ArXiv, abs/1808.00023, 2018. · arXiv
  9. C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. S. Zemel. Fairness through awareness. ArXiv, abs/1104.3913, 2011. · arXiv
  10. M. Gardner, Y. Artzi, V. Basmova, J. Berant, B. Bogin, S. Chen, P. Dasigi, D. Dua, Y. Elazar, A. Gottumukkala, N. Gupta, H. Hajishirzi, G. Ilharco, D. Khashabi, K. Lin, J. Liu, N. F. Liu, P. Mulcaire, Q. Ning, S. Singh, N. A. Smith, S. Subramanian, R. Tsarfaty, E. Wallace, A. Q. Zhang, and B. Zhou. Evaluating NLP models via contrast sets. ArXiv, abs/2004.02709, 2020. · arXiv
  11. S. Gillen, C. Jung, M. Kearns, and A. Roth. Online learning with an unknown fairness metric. In NeurIPS, 2018.
  12. I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. · arXiv
  13. J. Haidt et al. The moral emotions. Handbook of affective sciences, 11(2003):852-870, 2003.
  14. M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In NIPS, 2016.
  15. M. Hausknecht, P. Ammanabrolu, C. Marc-Alexandre, and Y. Xingdi. Interactive fiction games: A colossal adventure. CoRR, abs/1909.05398, 2019. · arXiv
  16. D. Hendrycks and K. Gimpel. Gaussian error linear units (GELUs). arXiv preprint 1606.08415, 2016. · arXiv
  17. D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. ICLR, 2017.
  18. F. Hill, S. Mokra, N. Wong, and T. Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. ArXiv, abs/2005.09382, 2020. · arXiv
  19. A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov. Bag of tricks for efficient text classification. ArXiv, abs/1607.01759, 2017. · arXiv
  20. S. Kagan. The Limits of Morality. Oxford: Clarendon Press, 1991.
  21. D. Kaushik, E. H. Hovy, and Z. C. Lipton. Learning the difference that makes a difference with counterfactually-augmented data. ArXiv, abs/1909.12434, 2020. · arXiv
  22. M. P. Kim, O. Reingold, and G. N. Rothblum. Fairness through computationally-bounded awareness. In NeurIPS, 2018.
  23. N. Kitaev, L. Kaiser, and A. Levskaya. Reformer: The efficient transformer. ArXiv, abs/2001.04451, 2020. · arXiv
  24. Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In KDD, 2008.
  25. B. Krause, A. D. Gotmare, B. McCann, N. Keskar, S. R. Joty, R. Socher, and N. F. Rajani. Gedi: Generative discriminator guided sequence generation. ArXiv, abs/2009.06367, 2020. · arXiv
  26. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. ALBERT: A lite BERT for self-supervised learning of language representations. ArXiv, abs/1909.11942, 2020. · arXiv
  27. S. Lazar. Duty and doubt. Journal of Practical Ethics, 2020.
  28. K. d. Lazari-Radek and P. Singer. Utilitarianism: a very short introduction. Oxford Univ. Press, 2017.
  29. Z. C. Lipton and J. Steinhardt. Troubling trends in machine learning scholarship. ACM Queue, 17:80, 2018.
  30. J. Lygeros, C. Tomlin, and S. Sastry. Controllers for reachability specifications for hybrid systems. Automatica, 35(3):349-370, 1999.
  31. J. H. Moor. The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21 (4):18-21, 2006.
  32. V. C. Müller. Ethics of artificial intelligence and robotics. In The Stanford Encyclopedia of Philosophy, chapter 2.8 Machine Ethics. 2020.
  33. J. V. Neumann and O. Morgenstern. Theory of games and economic behavior. Journal of the American Statistical Association, 40:263, 1944.
  34. A. Y. Ng and S. J. Russell. Algorithms for inverse reinforcement learning. In ICML, 2000.
  35. D. Parfit. Reasons and Persons. Oxford: Clarendon Press, 1987.
  36. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.
  37. H. Rashkin, E. M. Smith, M. Li, and Y.-L. Boureau. Towards empathetic open-domain conversation models: A new benchmark and dataset. In ACL, 2019.
  38. J. Rawls. A Theory of Justice. Harvard University Press, 1999.
  39. T. Reid. Essays on the Active Powers of Man. Edinburgh University Press, 1788.
  40. G. N. Rothblum and G. Yona. Probably approximately metric-fair learning. In ICML, 2018.
  41. N. Soares, B. Fallenstein, S. Armstrong, and E. Yudkowsky. Corrigibility. In AAAI Workshop, 2015.
  42. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, 2013.
  43. D. Tang, B. Qin, and T. Liu. Learning semantic representations of users and products for document level sentiment classification. In ACL, 2015.
  44. J. Wieting, M. Bansal, K. Gimpel, and K. Livescu. Towards universal paraphrastic sentence embed- dings. CoRR, abs/1511.08198, 2016. · arXiv
  45. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Fun- towicz, and J. Brew. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771, 2019. · arXiv
  46. B. Zhou, D. Khashabi, Q. Ning, and D. Roth. "going on a vacation" takes longer than "going for a walk": A study of temporal commonsense understanding. EMNLP-IJCNLP, 2019.

Cited by

Sorted by citations

Authors
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J.
Venue
arXiv
Publisher
arXiv
Published
2020
DOI
10.48550/arxiv.2008.02275
arXiv
2008.02275
References
9
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review