The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Why we should have seen that coming: comments on Microsoft's tay "experiment," and wider implications
Abstract
In this paper we examine the case of Tay, the Microsoft AI chatbot that was launched in March, 2016. After less than 24 hours, Microsoft shut down the experiment because the chatbot was generating tweets that were judged to be inappropriate since they included racist, sexist, and anti-Semitic language. We contend that the case of Tay illustrates a problem with the very nature of learning software (LS is a term that describes any software that changes its program in response to its interactions) that interacts directly with the public, and the developer's role and responsibility associated with it. We make the case that when LS interacts directly with people or indirectly via social media, the developer has additional ethical responsibilities beyond those of standard software. There is an additional burden of care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777
Distributed Morality in an Information Society
Floridi, L. (2013) · Science and Engineering Ethics · doi:10.1007/s11948-012-9413-4 · Cited by 201
Effects of Online Comments on Smokers' Perception of Antismoking Public Service Announcements
Shi, R., Messaris, P., & Cappella, J. N. (2014) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12057 · Cited by 117
Cited by
Vision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147
Social media? It's serious! Understanding the dark side of social media
Baccarella, C. V., Wagner, T. F., Kietzmann, J. H., & McCarthy, I. P. (2018) · European Management Journal · doi:10.1016/j.emj.2018.07.002 · Cited by 502
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
The Ethics of ChatGPT-Exploring the Ethical Issues of an Emerging Technology
Stahl, B. C., & Eke, D. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102700 · Cited by 476
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack
Dinan, E., Humeau, S., Chintagunta, B., & Weston, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1461 · Cited by 161
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152
ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia
Halfaker, A., & Geiger, R. S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415219 · Cited by 82
“approach that seeks to expand access to a potentially harmful technology should also be mindful about unintended uses and abuses, as adversaries and harms in the content moderation space are numerous [70, 81, 108]. Finally, we also see potential in allowing Wikipedians to freely train, test, and use their own predicti…”
BackgroundScruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes
Lourie, N., Le Bras, R., & Choi, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17589 · Cited by 73
Believe in artificial intelligence? A user study on the chatGPT’s fake information impact.
Amaro, I., Barra, P., Greca, A. D., Francese, R., & Tucci, C. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3291539 · Cited by 60
Hagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47
Personalized chatbot trustworthiness ratings
Srivastava, B., Rossi, F., Usmani, S., & Bernagozzi, M. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3023919 · Cited by 40
“extreme example is the Tay [1], [24] Twitter chatbot, released by Microsoft in 2016, that was designed to engage with people on open topics and learn from feedback, but ended up”
BackgroundSarkar, S., Gaur, M., Chen, L. K., Garg, M., & Srivastava, B. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1229805 · Cited by 38
What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S., & Viviano, J. D. (2021) · arXiv · doi:10.48550/arxiv.2105.02732 · Cited by 17
Risks and Harms of Conversational Artificial Intelligence (CAI) Chatbot Use Among US Youth
Hinduja, S., & Patchin, J. W. (2026) · Journal of Adolescence · doi:10.1002/jad.70164 · Cited by 9
Metadata
- Authors
- Wolf, M. J., Miller, K., & Grodzinsky, F. S.
- Venue
- ACM SIGCAS Computers and Society
- Published
- 2017-09-25
- Volume
- 47
- Issue
- 3
- Pages
- 54-64
- ISSN
- 0095-2737 (ISSN-L)
- OpenAlex
- W2758912220
- Cited by
- 172
- References
- 3
- FWCI
- 21.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.