Harms from Increasingly Agentic Algorithmic Systems
Open accessThe DOI matches its Crossref record; 165 of the 165 references it cites were found in a registry or an index.
Abstract
Research in Fairness, Accountability, Transparency, and Ethics (FATE)1 has established many sources and forms of algorithmic harm, in domains as diverse as health care, finance, policing, and recommendations. Much work remains to be done to mitigate the serious harms of these systems, particularly those disproportionately affecting marginalized communities. Despite these ongoing harms, new systems are being developed and deployed, typically without strong regulatory barriers, threatening the perpetuation of the same harms and the creation of novel ones. In response, the FATE community has emphasized the importance of anticipating harms, rather than just responding to them. Anticipation of harms is especially important given the rapid pace of developments in machine learning (ML). Our work focuses on the anticipation of harms from increasingly agentic systems. Rather than providing a definition of agency as a binary property, we identify 4 key characteristics which, particularly in combination, tend to increase the agency of a given algorithmic system: underspecification, directness of impact, goal-directedness, and long-term planning. We also discuss important harms which arise from increasing agency – notably, these include systemic and/or long-range impacts, often on marginalized or unconsidered stakeholders. We emphasize that recognizing agency of algorithmic systems does not absolve or shift the human responsibility for algorithmic harms. Rather, we use the term agency to highlight the increasingly evident fact that ML systems are not fully under human control. Our work explores increasingly agentic algorithmic systems in three parts. First, we explain the notion of an increase in agency for algorithmic systems in the context of diverse perspectives on agency across disciplines. Second, we argue for the need to anticipate harms from increasingly agentic systems. Third, we discuss important harms from increasingly agentic systems and ways forward for addressing them. We conclude by reflecting on implications of our work for anticipating algorithmic harms from emerging systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Without the ability to represent and reason about these causal links, Agentic AI systems are inherently constrained in their ability to safely generalize beyond narrow, predefined training scenarios [186,215].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [186]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Self-defined and executed goal-oriented behavior”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“More advanced models, like Vectara-agentic [27] and AutoGPT [34], can take text input, read the user's prompt to create outputs or undertake actions, and coordinate with other agents without human assistance.”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [34]
Experimental — accuracy not yet measuredIn the citing paper
“This possibility is important for safety - and for future AI ecosystems made of specialised 'AI services' (Drexler, 2019) - as generally intelligent autonomous systems could pose much greater risks than narrow AI tools (Chan et al., 2023).”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Chan et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Theory of the firm: Managerial behavior, agency costs and ownership structure
Jensen, M. C., & Meckling, W. H. (1976) · Journal of Financial Economics · doi:10.1016/0304-405x(76)90026-x · Cited by 71,491
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · cited 2×
“Although the deployment of algorithmic systems may result in increased safety or material improvements to human well-being [Abebe and Goldner 2018; Jumper et al. 2021; Li et al. 2017], diverse lines of work in Fairness, Accountability, Transparency, and Ethics (FATE) have established the roles that…”
BackgroundMastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 2×
“…performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [Brown and Sandholm 2019; Perolat et al. 2022; Silver et al. 2016, 2017] and video games [Mnih et al. 2013; Schrittwieser et al. 2020; Ye et al. 2021].”
BackgroundAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Agency Theory: An Assessment and Review
Eisenhardt, K. M. (1989) · The Academy of Management Review · doi:10.5465/amr.1989.4279003 · Cited by 13,338
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 3×
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1×
“…performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [Brown and Sandholm 2019; Perolat et al. 2022; Silver et al. 2016, 2017] and video games [Mnih et al. 2013; Schrittwieser et al. 2020; Ye et al. 2021].”
BackgroundReinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 4×
“the past, such as environmental risks [23], concentration of power”
BackgroundInfluentialSocial Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 2×
“models, recent studies [Ji et al. 2022; Valmeekam et al. 2022] have shown that large language models can perform poorly on planning and reasoning tasks, and such systems are prone to hallucinate unintended text, which fails to meet users’ intents on many real-world scenarios.”
BackgroundResultSurvey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.03629 · Cited by 4,981
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 2×
“” For example, LLMs are trained to model a distribution of internet text; this training leads to emergent behavior such as learning from very few examples [Olsson et al. 2022], or arithmetic [Brown et al. 2020], or even the ability itself to perform sequential reasoning [Wei et al. 2023].”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
A Deep Learning Approach to Antibiotic Discovery
Stokes, J. M., Yang, K., Swanson, K., Jin, W., Cubillos-Ruiz, A., Donghia, N. M., MacNair, C. R., French, S., Carfrae, L. A., Bloom-Ackermann, Z., et al. (2020) · Cell · doi:10.1016/j.cell.2020.01.021 · Cited by 2,463
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Keles, B., McCrae, N., & Grealish, A. (2020) · International Journal of Adolescence and Youth · doi:10.1080/02673843.2019.1590851 · Cited by 1,909
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 2×
“…particular, many have identified the impact of computational modeling and development in social change [Abebe et al. 2020; Jacobs and Wallach 2021; Selbst et al. 2019] and scoped numerous taxonomies of risks, harms, and failures of algorithmic systems [Raji et al. 2022; Shelby et al. 2022;…”
BackgroundFlamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · cited 2×
“Recent models [Alayrac et al. 2022; Reed et al. 2022; Zeng et al. 2022] can even handle multiple data modalities simultaneously.”
BackgroundMethodMastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · cited 1×
“…performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [Brown and Sandholm 2019; Perolat et al. 2022; Silver et al. 2016, 2017] and video games [Mnih et al. 2013; Schrittwieser et al. 2020; Ye et al. 2021].”
BackgroundMastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.01815 · Cited by 1,102 · cited 2×
“deep learning to RL systems produced superhuman performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [35, 146, 168, 169] and video games [129, 161, 199].”
BackgroundSpelke, E. S., & Kinzler, K. D. (2007) · Developmental Science · doi:10.1111/j.1467-7687.2007.00569.x · Cited by 1,089
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 3×
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 2×
“…or over-hype a model’s performance by ignoring model failures [Bender et al. 2021; Bender and Koller 2020; Blodgett et al. 2021; Collins et al. 2022; Lin et al. 2022; Raji et al. 2022; Vinsel 2021], it is also important not to understate (and thus fail to anticipate negative consequences of) what…”
BackgroundTruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2021) · arXiv · doi:10.48550/arxiv.2109.07958 · Cited by 899
Braveman, P. A., Arkin, E., Proctor, D., Kauh, T., & Holm, N. (2022) · Health Affairs · doi:10.1377/hlthaff.2021.01394 · Cited by 858 · cited 1×
“More broadly, many have identified the systemic nature of general classes of harms, such as financial risk [Armour and Gordon 2014], racism [Braveman et al. 2022], and misogyny [Manne 2017].”
BackgroundApplications of artificial intelligence in intelligent manufacturing: a review
Li, B.-H., Hou, B.-C., Yu, W.-T., Lu, X.-B., & Yang, C.-W. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1601885 · Cited by 826
Social media addiction: Its impact, mediation, and intervention
Hou, Y., Xiong, D., Jiang, T., Song, L., & Wang, Q. (2019) · Cyberpsychology: Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2019-1-4 · Cited by 815 · cited 2×
“Social media is speculated to be a contributing factor to many systemic and delayed harms, including mental health issues [Hou et al. 2019; Yoon et al. 2019], the amplification of political polarisation [Whittaker et al.”
BackgroundDown Girl: The Logic of Misogyny
Manne, K. (2017) · doi:10.1093/oso/9780190604981.001.0001 · Cited by 791
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 2×
“197], the generation of toxic language [7, 76], and informational harms [42, 99, 119, 193].”
BackgroundRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · cited 2×
“…and failure [Barabas et al. 2020; Buolamwini and Gebru 2018; Obermeyer and Mullainathan 2019; Piantadosi and Hill 2022; Ribeiro et al. 2020; Wallace et al. 2019; Wolfe and Caliskan 2022] suggest that increasingly agentic systems will likely be deployed according to industry interests, and…”
BackgroundSuperhuman AI for multiplayer poker
Brown, N., & Sandholm, T. (2019) · Science · doi:10.1126/science.aay2400 · Cited by 592 · cited 1×
“…performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [Brown and Sandholm 2019; Perolat et al. 2022; Silver et al. 2016, 2017] and video games [Mnih et al. 2013; Schrittwieser et al. 2020; Ye et al. 2021].”
BackgroundAuditing radicalization pathways on YouTube
Ribeiro, M. H., Ottoni, R., West, R., Almeida, V. A. F., & Meira, W. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372879 · Cited by 580 · cited 2×
“Several landmark works in the FATE community have involved audits [Raji et al. 2020] of algorithmic systems [Black et al. 2022; Buolamwini and Gebru 2018; Obermeyer and Mullainathan 2019; Ribeiro et al. 2020].”
BackgroundBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Recommender systems and their ethical challenges
Milano, S., Taddeo, M., & Floridi, L. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00950-y · Cited by 556
Autonomous Robots: From Biological Inspiration to Implementation and Control
Bekey, G. A. (2005) · Cited by 507
Reinforcement Learning based Recommender Systems: A Survey
Afsar, M. M., Crump, T., & Far, B. (2023) · ACM Computing Surveys · doi:10.1145/3543846 · Cited by 502 · cited 1×
“Additionally, there has been an increasing amount of research in recommender systems to optimize long-term metrics such as engagement via reinforcement learning [Afsar et al. 2022].”
BackgroundPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 1×
“…and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020], and informational harms [Carroll et al. 2022; Jiang et al. 2019; Li et al. 2022; Weidinger et al. 2022].”
BackgroundOvertrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468 · cited 1×
“Similarly, Robinette et al. [2016] examines (over)trust of autonomous systems and find in emergency situations, people will follow robots into further danger, because they attribute the agency of the robot to the (assumed capable and responsible) designers.”
BackgroundBoxell, L., Gentzkow, M., & Shapiro, J. M. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706588114 · Cited by 467
Burrell, J., & Fourcade, M. (2021) · Annual Review of Sociology · doi:10.1146/annurev-soc-090820-020800 · Cited by 430
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398 · cited 2×
“(1) Underspecification: the degree to which the algorithmic system can accomplish a goal provided by operators or designers, without a concrete specification of how the goal is to be accomplished [D’Amour et al. 2020a].”
BackgroundJacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362 · cited 1×
“In particular, many have identified the impact of computational modeling and development in social change [Abebe et al. 2020; Jacobs and Wallach 2021; Selbst et al. 2019] and scoped numerous taxonomies of risks, harms, and failures of algorithmic systems [Raji et al. 2022; Shelby et al. 2022;…”
BackgroundArtificial Intelligence and Journalism
Broussard, M., Diakopoulos, N., Guzman, A. L., Abebe, R., Dupagne, M., & Chuan, C.-H. (2019) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699019859901 · Cited by 356
Is social network site usage related to depression? A meta-analysis of Facebook–depression relations
Yoon, S., Kleinman, M., Mertz, J., & Brannick, M. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.01.026 · Cited by 355
Accountability in a computerized society
Nissenbaum, H. (1996) · Science and Engineering Ethics · doi:10.1007/bf02639315 · Cited by 346
Predictability and Surprise in Large Generative Models
Ganguli, D., Hernandez, D., Lovitt, L., DasSarma, N., Henighan, T., Jones, A., Joseph, N., Kernion, J., Mann, B., Askell, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.07785 · Cited by 324
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 4×
“” And in a systematic review on algorithmic accountability, Wieringa [196] defines algorithms as “basically instructions fed to a computer”.”
BackgroundInfluentialToward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306 · cited 4×
“The likely failures of a more agentic system (relative to what has been deployed already) are certainly a barrier to deployment – given the disproportionate impact of these failures on already marginalized groups, we would hope and advocate for restrictions on deployment [Brundage et al. 2020].”
BackgroundMethodInfluentialSelf-citationInner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · cited 2×
“For instance, recent work has shown that foundation models can perform planning tasks [93].”
BackgroundSociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Self-citationDisparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275
Defining and Characterizing Reward HackingPreprint
Skalse, J., Howe, N. H. R., Krasheninnikov, D., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.48550/arxiv.2209.13085 · Cited by 274 · cited 2×
“This kind of failure is called reward hacking [Krakovna et al. 2020; Skalse et al. 2022], which is when a system exploits a reward signal to achieve a goal in an unforeseen, perhaps undesirable way.”
BackgroundSelf-citationWebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · cited 4×
“…that was more agentic than the search engine, still for the same task, could simply be queried with the topic of the desired literature review, and would automatically look through related works on the internet without user intervention, like WebGPT can do to some extent [Nakano et al. 2022].”
BackgroundInfluentialImagining the thinking machine: Technological myths and the rise of artificial intelligence
Natale, S., & Ballatore, A. (2020) · Convergence: The International Journal of Research into New Media Technologies · doi:10.1177/1354856517715164 · Cited by 264
Enchanted Determinism: Power without Responsibility in Artificial Intelligence
Campolo, A., & Crawford, K. (2020) · Engaging Science, Technology, and Society · doi:10.17351/ests2020.277 · Cited by 254 · cited 1×
“Related to this is the adoption of discourse around ML systems that their capabilities are both scientifically impossible to explain, and yet deterministic in their societal impact [Campolo and Crawford 2020].”
BackgroundThe Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251 · cited 6×
“[153] highlight could disincentivize adoption of increasingly agentic systems, even if they were”
BackgroundInfluentialFairness in Recommendation: Foundations, Methods, and Applications
Li, Y., Chen, H., Xu, S., Ge, Y., Tan, J., Liu, S., & Zhang, Y. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3610302 · Cited by 250
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00234 · Cited by 248
Johnson, D. G., & Verdicchio, M. (2017) · Minds and Machines · doi:10.1007/s11023-017-9417-6 · Cited by 241
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 2×
“Before the current era of large language models (LLMs) [30, 36, 150, 172], few systems competently performed out-of-the-box on a range of natural language tasks [36].”
BackgroundDegenerate Feedback Loops in Recommender Systems
Jiang, R., Chiappa, S., Lattimore, T., György, A., & Kohli, P. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314288 · Cited by 232 · cited 4×
“…and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020], and informational harms [Carroll et al. 2022; Jiang et al. 2019; Li et al. 2022; Weidinger et al. 2022].”
BackgroundInfluentialRoles for computing in social change
Abebe, R., Barocas, S., Kleinberg, J., Levy, K., Raghavan, M., & Robinson, D. G. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372871 · Cited by 219 · cited 1×
“In particular, many have identified the impact of computational modeling and development in social change [Abebe et al. 2020; Jacobs and Wallach 2021; Selbst et al. 2019] and scoped numerous taxonomies of risks, harms, and failures of algorithmic systems [Raji et al. 2022; Shelby et al. 2022;…”
BackgroundA Deep Learning Approach to Antibiotic Discovery
Stokes, J. M., Yang, K., Swanson, K., Jin, W., Cubillos-Ruiz, A., Donghia, N. M., MacNair, C. R., French, S., Carfrae, L. A., Bloom-Ackermann, Z., et al. (2020) · Cell · doi:10.1016/j.cell.2020.04.001 · Cited by 212
Social Simulacra: Creating Populated Prototypes for Social Computing Systems
Park, J. S., Popowski, L., Cai, C., Morris, M. R., Liang, P., & Bernstein, M. S. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545616 · Cited by 212 · cited 2×
“Assuming either that we have the system in question or that we can simulate it faithfully [11, 63, 143], we can formulate and test hypotheses in simple experiments or simulation.”
BackgroundPredictability and Surprise in Large Generative Models
Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533229 · Cited by 207 · cited 1×
“For current AI models, there are strong incentives for continued investment and development despite uncertainty around how their future capabilities will emerge [Ganguli et al. 2022].”
BackgroundSocratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Zeng, A., Attarian, M., Ichter, B., Choromanski, K., Wong, A., Welker, S., Tombari, F., Purohit, A., Ryoo, M., Sindhwani, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.00598 · Cited by 205 · cited 4×
“For example, the raw task performance of generalist systems [Reed et al. 2022; Zeng et al. 2022] is still limited to tasks where expert data is available and has not achieved human level on all tasks.”
BackgroundMethodInfluential"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196 · cited 2×
“2020], health-care systems [Fogliato et al. 2022; Obermeyer and Mullainathan 2019; Sendak et al. 2020], the judicial sector [Barabas et al.”
BackgroundThe Nooscope manifested: AI as instrument of knowledge extractivism
Pasquinelli, M., & Joler, V. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01097-6 · Cited by 188
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Moral Judgments in the Age of Artificial Intelligence
Sullivan, Y. W., & Wamba, S. F. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05053-w · Cited by 175
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Aher, G., Arriaga, R. I., & Kalai, A. T. (2022) · arXiv · doi:10.48550/arxiv.2208.10264 · Cited by 161
Mastering the game of Stratego with model-free multiagent reinforcement learning
Perolat, J., De Vylder, B., Hennes, D., Tarassov, E., Strub, F., de Boer, V., Muller, P., Connor, J. T., Burch, N., Anthony, T., et al. (2022) · Science · doi:10.1126/science.add4679 · Cited by 159
Fairness is not static: deeper understanding of long term fairness via simulation studies
D'Amour, A., Srinivasan, H., Atwood, J., Baljekar, P., Sculley, D., & Halpern, Y. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372878 · Cited by 156
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.81 · Cited by 147 · cited 2×
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundFairness, Equality, and Power in Algorithmic Decision-Making
Kasy, M., & Abebe, R. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445919 · Cited by 144 · cited 3×
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundRecommender systems and the amplification of extremist content
Whittaker, J., Looney, S., Reed, A., & Votta, F. (2021) · Internet Policy Review · doi:10.14763/2021.2.1565 · Cited by 135
Broken Neural Scaling LawsPreprint
Caballero, E., Gupta, K., Rish, I., & Krueger, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.14891 · Cited by 134 · cited 1×
“There is also initial work into developing scaling laws for robotics [Caballero et al. 2023].”
BackgroundSelf-citationCooper, A. F., Moss, E., Laufer, B., & Nissenbaum, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533150 · Cited by 129 · cited 5×
“ing the many meanings of agency, as well as the need not to absolve humans of responsibility pertaining to algorithmic harms [48, 134, 196], we use the term agency consciously to counter the somewhat prevalent view that the developers of an algorithmic sys-”
BackgroundInfluentialMastering Diverse Domains through World Models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2301.04104 · Cited by 124 · cited 2×
“For instance, DreamerV3 [87] collected diamonds from scratch without human data or curricula in MineCraft, which has been a longstanding challenge because the task is extremely complex and open-ended.”
BackgroundDissecting Racial Bias in an Algorithm that Guides Health Decisions for 70 Million People
Obermeyer, Z., & Mullainathan, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287593 · Cited by 124 · cited 3×
“Continued cycles of deployment and failure [Barabas et al. 2020; Buolamwini and Gebru 2018; Obermeyer and Mullainathan 2019; Piantadosi and Hill 2022; Ribeiro et al. 2020; Wallace et al. 2019; Wolfe and Caliskan 2022] suggest that increasingly agentic systems will likely be deployed according to…”
BackgroundThe Forgotten Margins of AI Ethics
Birhane, A., Ruane, E., Laurent, T., S. Brown, M., Flowers, J., Ventresque, A., & L. Dancy, C. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533157 · Cited by 122 · cited 1×
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundStudying up: reorienting the study of algorithmic fairness around issues of power
Barabas, C., Doyle, C., Rubinovitz, J., & Dinakar, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372859 · Cited by 108
Systemic Harms and Shareholder Value
Armour, J. H., & Gordon, J. N. (2014) · The Journal of Legal Analysis · doi:10.1093/jla/lau004 · Cited by 103
Superforecasting: the art and science of prediction
Rötheli, T. F. (2017) · foresight · doi:10.1108/fs-12-2016-0061 · Cited by 103
“Who goes first? Influences of human-AI workflow on decision making in clinical imaging,”
Fogliato, R., Chappidi, S., Lungren, M., Fisher, P., Wilson, D., Fitzke, M., Parkinson, M., Horvitz, E., Inkpen, K., & Nushi, B. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533193 · Cited by 103
The Grey Hoodie Project: Big Tobacco, Big Tech, and the Threat on Academic Integrity
Abdalla, M., & Abdalla, M. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462563 · Cited by 99 · cited 2×
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundIncomplete Contracting and AI Alignment
Hadfield, G. K., & Hadfield-Menell, D. (2019) · TSpace (University of Toronto) · doi:10.1145/3306618.3314250 · Cited by 94 · cited 2×
“In our context, we view the principal as humans and the agent as algorithmic systems, as done in prior work [Hadfield-Menell and Hadfield 2019].”
MethodA Tutorial on Meta-Reinforcement Learning
Beck, J., Vuorio, R., Zheran Liu, E., Xiong, Z., Zintgraf, L., Finn, C., & Whiteson, S. (2025) · Foundations and Trends in Machine Learning · doi:10.1561/2200000080 · Cited by 93 · cited 1×
“In the current scientific paradigm of large-scale deep-learning, one instead provides high-level learning algorithms that tend to be task-agnostic or adapt to new tasks efficiently [Beck et al. 2023; Dong et al. 2022].”
BackgroundOpen Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93 · cited 1×
“Additional problems remain in understanding how to manage the interests of multiple stakeholders [Dafoe et al. 2020].”
BackgroundLanguage Models as Agent Models
Andreas, J. (2022) · arXiv · doi:10.48550/arxiv.2212.01681 · Cited by 92
Language Models as Agent Models
Andreas, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.423 · Cited by 92
Recommender Systems and their Ethical ChallengesPreprint
Milano, S., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3378581 · Cited by 88
Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
Gauci, J., Conti, E., Liang, Y., Virochsiri, K., He, Y., Kaden, Z., Narayanan, V., Ye, X., Chen, Z., & Fujimoto, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00260 · Cited by 83 · cited 2×
“Major recommendation companies such as Meta [Gauci et al. 2019], YouTube [Association for Computing Machinery (ACM) 2019], and Spotify [Engineering 2021] have already deployed RL-based recommender systems on their live products.”
BackgroundTowards Accountability in the Use of Artificial Intelligence for Public Administrations
Loi, M., & Spielkamp, M. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462631 · Cited by 81 · cited 1×
“…2019; Sendak et al. 2020], the judicial sector [Barabas et al. 2020; Green 2020; Zilka et al. 2022], and public services [Black et al. 2022; Loi and Spielkamp 2021; Stapleton et al. 2022], ADM can often exhibit similar kinds of diffuse and long-term harms to those we discuss coming from…”
BackgroundThe false promise of risk assessments: epistemic reform and the limits of fairness
Green, B. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372869 · Cited by 80 · cited 2×
“…the FATE community has extensively studied the ways in which automated decision-making can disempower individuals, by impairing human decision-making [Green and Chen 2019; Green 2021] or subjecting individuals to oppressive institutions [Barabas et al. 2020, 2018; Green 2020; Zilka et al. 2022].”
BackgroundStapleton, L., Lee, M. H., Qing, D., Wright, M., Chouldechova, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533177 · Cited by 76 · cited 2×
“2022], and public services [Black et al. 2022; Loi and Spielkamp 2021; Stapleton et al. 2022], ADM can often exhibit similar kinds of diffuse and long-term harms to those we discuss coming from increased agency.”
BackgroundData Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 75 · cited 1×
“[97] highlight the imbalance of power between AI developers and the rest of society, and propose frameworks and”
BackgroundValmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2022) · arXiv · doi:10.48550/arxiv.2206.10498 · Cited by 70
Shavit, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.11341 · Cited by 70 · cited 2×
“pute usage tracking [37, 166], while a source of serious privacy risks, could help to control the pace at which systems become in-”
BackgroundMethodValmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1693 · Cited by 70
Meaning without reference in large language models
Piantadosi, S. T., & Hill, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.02957 · Cited by 65
Confronting Power and Corporate Capture at the FAccT Conference
Young, M., Katell, M., & Krafft, P. M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533194 · Cited by 63 · cited 1×
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundReward Reports for Reinforcement Learning
Gilbert, T. K., Lambert, N., Dean, S., Zick, T., Snoswell, A., & Mehta, S. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604698 · Cited by 63
Scaling Laws for Reward Model Overoptimization
Gao, L., Schulman, J., & Hilton, J. (2022) · arXiv · doi:10.48550/arxiv.2210.10760 · Cited by 62 · cited 2×
“Of particular interest for increasing agency is that scaling laws have been derived for reinforcement learning [Gao et al. 2022; Hilton et al. 2023; Team et al. 2023] and generative modeling [Hoffmann et al.”
BackgroundAgent Incentives: A Causal Perspective
Everitt, T., Carey, R., Langlois, E. D., Ortega, P. A., & Legg, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17368 · Cited by 60
Scaling laws for single-agent reinforcement learningPreprint
Hilton, J. K., Tang, J., & Schulman, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.13442 · Cited by 60 · cited 2×
“Of particular interest for increasing agency is that scaling laws have been derived for reinforcement learning [Gao et al. 2022; Hilton et al. 2023; Team et al. 2023] and generative modeling [Hoffmann et al.”
BackgroundLegitimacy, Authority, and Democratic Duties of Explanation
Lazar, S. (2024) · Oxford University Press eBooks · doi:10.1093/oso/9780198909460.003.0002 · Cited by 60
Sühr, T., Hilgard, S., & Lakkaraju, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462602 · Cited by 54 · cited 2×
“2021], concentration of power [Abdalla and Abdalla 2021; Pratyusha 2020], unfair algorithmic hiring decisions [Sühr et al. 2021], and privacy infringements [Ekstrand et al.”
BackgroundUser Tampering in Reinforcement Learning Recommender Systems
Kasirzadeh, A., & Evans, C. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604669 · Cited by 54 · cited 2×
“In particular, Carroll et al. [2022]; Evans and Kasirzadeh [2022]; Krueger et al. [2020] show that long time-horizon systems, such as RLRS, will have incentives to change or manipulate users’ internal states (e.g. preferences, beliefs, and psychology) for the purposes of increasing the metrics the RLRS systems are opt…”
BackgroundTeaching language models to support answers with verified quotes
Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.11147 · Cited by 53 · cited 2×
“on next-token prediction, they can be leveraged to interact with APIs and accomplish a wide variety of multi-step digital tasks with increasingly less explicit human intervention [43, 125, 131].”
BackgroundDiscovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · cited 2×
“[145] show that increased training of a LLM with”
BackgroundPPG Reports Fourth Quarter and Full-Year 2022 Financial Results
(2023) · Focus on Powder Coatings · doi:10.1016/j.fopow.2023.02.013 · Cited by 52
Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48 · cited 1×
“Dehghani et al. [2021] provide evidence that factors other than “fundamental algorithmic superiority” may lead to the perception that a particular method is superior.”
BackgroundThe Alberta Plan for AI ResearchPreprint
Sutton, R. S., Bowling, M., & Pilarski, P. M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.11173 · Cited by 46 · cited 2×
“Indeed, many in the ML community are explicitly building such systems as a research goal [Chen et al. 2021; Reed et al. 2022; Sutton et al. 2022].”
BackgroundMechanism design for social good
Abebe, R., & Goldner, K. (2018) · AI Matters · doi:10.1145/3284751.3284761 · Cited by 45
Kenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., & Irving, G. (2021) · arXiv · doi:10.48550/arxiv.2103.14659 · Cited by 45
The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail
Bowman, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.516 · Cited by 44 · cited 1×
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundMind the Gap: Autonomous Systems, the Responsibility Gap, and Moral Entanglement
Goetze, T. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533106 · Cited by 41 · cited 1×
“Goetze [80] identifies a responsibility gap between engineers”
BackgroundEhsan, U., Singh, R., Metcalf, J., & Riedl, M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533186 · Cited by 40 · cited 1×
“Examples include the perpetuation of existing, unjust power relations [Barabas et al. 2018; Buolamwini and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020],…”
BackgroundAgents and Devices: A Relative Definition of AgencyPreprint
Orseau, L., McGill, S. M., & Legg, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12387 · Cited by 39
Path-Specific Objectives for Safer Agent Incentives
Farquhar, S., Carey, R., & Everitt, T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21186 · Cited by 38
Human-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning
Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., Jacob, A. P., et al. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7236699 · Cited by 35
Tech Worker Organizing for Power and Accountability
Boag, W., Suresh, H., Lepe, B., & D'Ignazio, C. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533111 · Cited by 34 · cited 1×
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundPath-Specific Objectives for Safer Agent IncentivesPreprint
Farquhar, S., Carey, R., & Everitt, T. (2022) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2204.10018 · Cited by 33 · cited 1×
“While some work has also investigated potential solutions [42, 69], how to practically measure and address these issues in real-world RLRS remains an open problem.”
BackgroundPan, A., Chan, J. S., Zou, A., Li, N., Basart, S., Woodside, T., Ng, J., Zhang, H., Emmons, S., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2304.03279 · Cited by 32 · cited 2×
“Some existing work already measures aspects of agency, such as long-term planning [Valmeekam et al. 2022] and goal-directedness [Pan et al. 2023].”
BackgroundAmerican == White in Multimodal Language-and-Image AI
Wolfe, R., & Caliskan, A. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534136 · Cited by 31 · cited 2×
“…power relations [Barabas et al. 2018; Buolamwini and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020], and informational harms [Carroll et al. 2022;…”
BackgroundZilka, M., Sargeant, H., & Weller, A. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534200 · Cited by 28 · cited 4×
“extensively studied the ways in which automated decision-making can disempower individuals, by impairing human decision-making [Green and Chen 2019; Green 2021] or subjecting individuals to oppressive institutions [Barabas et al. 2020, 2018; Green 2020; Zilka et al. 2022].”
BackgroundInfluentialSelf-citationWhy We Need to Bust Some Myths about AI
Leufer, D. (2020) · Patterns · doi:10.1016/j.patter.2020.100124 · Cited by 25
Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models
Black, E., Elzayn, H., Chouldechova, A., Goldin, J., & Ho, D. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533204 · Cited by 24 · cited 2×
“…Obermeyer and Mullainathan 2019; Sendak et al. 2020], the judicial sector [Barabas et al. 2020; Green 2020; Zilka et al. 2022], and public services [Black et al. 2022; Loi and Spielkamp 2021; Stapleton et al. 2022], ADM can often exhibit similar kinds of diffuse and long-term harms to those we…”
BackgroundSociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm ReductionPreprint
Shelby, R. M., Rismani, S., Henne, K., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.05791 · Cited by 22 · cited 2×
“…have identified the impact of computational modeling and development in social change [Abebe et al. 2020; Jacobs and Wallach 2021; Selbst et al. 2019] and scoped numerous taxonomies of risks, harms, and failures of algorithmic systems [Raji et al. 2022; Shelby et al. 2022; Weidinger et al. 2022].”
BackgroundSelf-citationAn AI Ethics Course Highlighting Explicit Ethical Agents
Green, N. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462552 · Cited by 21 · cited 1×
“…the FATE community has extensively studied the ways in which automated decision-making can disempower individuals, by impairing human decision-making [Green and Chen 2019; Green 2021] or subjecting individuals to oppressive institutions [Barabas et al. 2020, 2018; Green 2020; Zilka et al. 2022].”
BackgroundHuman-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Bauer, J., Baumli, K., Baveja, S., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.07608 · Cited by 19 · cited 5×
“One particular example to highlight is AdA [181], which adapts to”
BackgroundInfluentialThe AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement LearningPreprint
Zheng, S., Trott, A., Srinivasa, S., Parkes, D. C., & Socher, R. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3900018 · Cited by 18
Fairness in Recommendation: Foundations, Methods and ApplicationsPreprint
Li, Y., Chen, H., Xu, S., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13619 · Cited by 17 · cited 3×
“…and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020], and informational harms [Carroll et al. 2022; Jiang et al. 2019; Li et al. 2022; Weidinger et al. 2022].”
BackgroundKenton, Z., Kumar, R., Farquhar, S., Richens, J., MacDermott, M., & Everitt, T. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103963 · Cited by 17
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
Shah, R., Varma, V., Kumar, R., Phuong, M., Krakovna, V., Uesato, J., & Kenton, Z. (2022) · arXiv · doi:10.48550/arxiv.2210.01790 · Cited by 16 · cited 1×
“Even a system successfully trained under a correctly specified objective function may do something completely different in a different environment [Langosco et al. 2022; Shah et al. 2022].”
BackgroundAI Ethics for Systemic Issues: A Structural ApproachPreprint
van der Loeff, A. S., Bassi, I. G., Kapila, S., & Gamper, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03216 · Cited by 14 · cited 1×
“A developer is likely blameworthy at least to some extent when a system causes harm, but structural factors like economic incentives or company culture to push forward likely also play significant roles [van der Loeff et al. 2019; Zwetsloot and Dafoe 2019].”
BackgroundHidden Incentives for Auto-Induced Distributional Shift
Krueger, D., Maharaj, T., & Leike, J. (2020) · arXiv · doi:10.48550/arxiv.2009.09153 · Cited by 14 · cited 3×
“[111] show that long time-horizon systems, such as RLRS, will have incentives to change or manipulate users’ internal states (e.”
BackgroundSelf-citationCollins, K. M., Wong, C., Feng, J., Wei, M., & Tenenbaum, J. B. (2022) · arXiv · doi:10.48550/arxiv.2205.05718 · Cited by 13 · cited 2×
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundSelf-citationEstimating and Penalizing Induced Preference Shifts in Recommender Systems
Carroll, M., Dragan, A., Russell, S., & Hadfield-Menell, D. (2022) · arXiv · doi:10.48550/arxiv.2204.11966 · Cited by 11 · cited 5×
“While some work has also investigated potential solutions [Carroll et al. 2022; Farquhar et al. 2022], how to practically measure and address these issues in real-world RLRS remains an open problem [Carroll et al. 2023].”
BackgroundInfluentialSelf-citationThe effects of competition and regulation on error inequality in data-driven markets
Elzayn, H., & Fish, B. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372842 · Cited by 10 · cited 1×
“Assuming either that we have the system in question or that we can simulate it faithfully [Aher et al. 2022; Elzayn and Fish 2020; Park et al. 2022], we can formulate and test hypotheses in simple experiments or simulation.”
BackgroundReinforcement learning based recommender systems: A surveyPreprint
Afsar, M. M., Crump, T., & Far, B. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06286 · Cited by 8 · cited 1×
“Additionally, there has been an increasing amount of research in recommender systems to optimize long-term metrics such as engagement via reinforcement learning [Afsar et al. 2022].”
BackgroundLegitimacy, Authority, and Democratic Duties of ExplanationPreprint
Lazar, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.08628 · Cited by 8
Kenton, Z., Kumar, R., Farquhar, S., Richens, J., MacDermott, M., & Everitt, T. (2022) · arXiv · doi:10.48550/arxiv.2208.08345 · Cited by 7
Reward Reports for Reinforcement LearningPreprint
Gilbert, T. K., Lambert, N., Dean, S., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.10817 · Cited by 5
Fairness in Learning-Based Sequential Decision Algorithms: A SurveyPreprint
Zhang, X., & Liu, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04861 · Cited by 4 · cited 2×
“fairness implications of decisions [25, 54, 94, 101, 121, 203].”
BackgroundRegulating Lethal and Harmful Autonomy
Welsh, S. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314295 · Cited by 2
(2023) · Inorganic Chemistry Frontiers · doi:10.1039/d3qi90031b · Cited by 1
A Tutorial on Meta-Reinforcement Learning
Beck, J., Vuorio, R., Liu, E. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.08028 · cited 1×
“In the current scientific paradigm of large-scale deep-learning, one instead provides high-level learning algorithms that tend to be task-agnostic or adapt to new tasks efficiently [Beck et al. 2023; Dong et al. 2022].”
BackgroundThe AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning
Zheng, S., Trott, A., Srinivasa, S., Parkes, D. C., & Socher, R. (2021) · arXiv · doi:10.48550/arxiv.2108.02755 · cited 1×
“power to an particular system, such as one that decides taxation policy [Zheng et al. 2021].”
BackgroundUser Tampering in Reinforcement Learning Recommender Systems
Evans, C., & Kasirzadeh, A. (2021) · arXiv · doi:10.48550/arxiv.2109.04083 · cited 2×
“In particular, Carroll et al. [2022]; Evans and Kasirzadeh [2022]; Krueger et al. [2020] show that long time-horizon systems, such as RLRS, will have incentives to change or manipulate users’ internal states (e.g. preferences, beliefs, and psychology) for the purposes of increasing the metrics the RLRS systems are opt…”
Background
Research works printed but not linked: 46
- Julius Adebayo, Michael Muelly, Harold Abelson, and Been Kim. 2022. Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation. In International Conference on Learning Representations. https://openreview. net/forum?id=xNOVfCCvDpM
- 10.1093/jla · doi:10.1093/jla
- Association for Computing Machinery (ACM). 2019. "Reinforcement Learning for Recommender Systems: A Case Study on Youtube, " by Minmin Chen. https: //www.youtube.com/watch?v=HEqQ2_1XRTs
- Chelsea Barabas, Madars Virza, Karthik Dinakar, Joichi Ito, and Jonathan Zittrain. 2018. Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 62-76. https://proceedings.mlr.press/ v81/barabas18a.html
- Joslyn Barnhart. 2022. Emerging Technologies, Prestige Motiva- tions and the Dynamics of International Competition. (Jan. 2022). https://www.governance.ai/research-paper/emerging-technologies-prestige- motivations-and-the-dynamics-of-international-competition
- 10.18653/v1 · doi:10.18653/v1
- Sarah Bird, Solon Barocas, Kate Crawford, Fernando Diaz, and Hanna Wallach. 2016. Exploring or Exploiting? Social and Ethical Implications of Autonomous Experimentation in AI. https://papers.ssrn.com/abstract=2846909
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learn- ers. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 1877-1901. https://proceedings.neurips.cc/paper/2020/hash/ 1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accu- racy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 77-91. https://proceedings.mlr.press/v81/buolamwini18a.html
- Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. 2021. De- cision Transformer: Reinforcement Learning via Sequence Modeling. In Advances in Neural Information Processing Systems, Vol. 34. Curran Asso- ciates, Inc., 15084-15097. https://proceedings.neurips.cc/paper/2021/hash/ 7f489f642a0ddb10272b5c31057f0663-Abstract.html
- Rémi Coulom. 2002. Reinforcement Learning Using Neural Networks, with Applications to Motor Control. (June 2002).
- Carla Zoe Cremer and Luke Kemp. 2021. Democratising Risk: In Search of a Methodology to Study Existential Risk. https://papers.ssrn.com/abstract= 3995225
- Daniel Clement Dennett. 1981. The Intentional Stance. MIT Press.
- Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Rec- ommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 172-186. https://proceedings.mlr.press/v81/ekstrand18b.html
- Mustafa Emirbayer and Ann Mische. 1998. What is Agency? American journal of sociology 103, 4 (1998), 962-1023. Publisher: The University of Chicago Press.
- 10.18653/v1 · doi:10.18653/v1
- Charlie Giattino, Edouard Mathieu, Julia Broden, and Max Roser. 2022. Artificial Intelligence. Our World in Data (2022).
- Charles Goodhart. 1975. Problems of monetary management: the UK experience in papers in monetary economics. Monetary Economics 1 (1975).
- Mary L. Gray and Siddharth Suri. 2019. Ghost work: how to stop Silicon Valley from building a new global underclass. Houghton Mifflin Harcourt, Boston.
- Stacy A. Hickox. 2010. Employer Liability of Negligent Hiring of Ex-Offenders. Saint Louis University Law Journal 55, 3 (2010), 1001-1046. https://heinonline. org/HOL/P?h=hein.journals/stlulj55&i=1029
- Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. 2017. Fairness in Reinforcement Learning. In Proceedings of the 34th International Conference on Machine Learning. PMLR, 1617-1626. https://proceedings.mlr.press/v70/jabbari17a.html ISSN: 2640-3498.
- Victoria Krakovna, Jonathan Uesato, Vladimir Mikulik, Matthew Rahtz, Tom Everitt, Ramana Kumar, Zac Kenton, Jan Leike, and Shane Legg. 2020. Specifica- tion gaming: the flip side of AI ingenuity. DeepMind Blog (2020).
- T.S. Kuhn and I. Hacking. 2012. The Structure of Scientific Revolutions. University of Chicago Press. https://books.google.co.uk/books?id=3eP5Y_OOuzwC
- Lauro Langosco Di Langosco, Jack Koch, Lee D Sharkey, Jacob Pfau, and David Krueger. 2022. Goal Misgeneralization in Deep Reinforcement Learning. In Proceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 162), Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (Eds.). PMLR, 12004- 12019. https://proceedings.mlr.press/v162/langosco22a.html
- Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed impact of fair machine learning. In International Conference on Machine Learning. PMLR, 3150-3158.
- Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Con- erly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. In-context Learning and Induction Heads. Transformer Circuits Thread (2022).
- Stephen M Omohundro. 2008. The Basic AI Drives. In AGI, Vol. 171. 483-492.
- Alexander Pan, Kush Bhatia, and Jacob Steinhardt. 2022. The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. In International Conference on Learning Representations. https://openreview.net/forum?id= JYtwGwIL7ye
- By Pratyusha. 2020. World view. Nature 583 (2020), 169.
- Deborah Raji, Emily Denton, Emily M. Bender, Alex Hanna, and Amandalynne Paullada. 2021. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (Dec. 2021). https://datasets-benchmarks-proceedings.neurips.cc/ paper/2021/hash/084b6fbb10729ed4da8c3d3f5a3ae7c9-Abstract-round2.html
- Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexan- der Novikov, Gabriel Barth-maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. 2022. A Generalist Agent. Transactions on Machine Learning Research (2022). https://openreview.net/forum?id=1ikK0kHjvj
- Stuart J. Russell and Peter Norvig. 2021. Artificial Intelligence: A Modern Approach (4 ed.).
- Anne Schulz, P Howard, and R Nielsen. 2019. Industry, Experts, or Industry Experts? Academic Sourcing in News Coverage of AI. (2019). Publisher: Reuters Institute for the Study of Journalism.
- steven t. piantadosi [@spiantado]. 2022. Yes, ChatGPT is amazing and impressive. No, @OpenAI has not come close to addressing the problem of bias. Filters appear to be bypassed with simple tricks, and superficially masked. And what is lurking inside is egregious. @Abebab @sama tw racism, sexism. https://t.co/V4fw1fY9dY. https://twitter.com/spiantado/status/ 1599462375887114240
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An intro- duction. MIT press.
- Robert Trager. 2022. Deliberating Autonomous Weapons. Issues in Science and Technology XXXVIII, 4 (2022). https://issues.org/autonomous-weapons-russell- forum/
- Andrew Tutt. 2017. An FDA for Algorithms. Administrative Law Review 69, 1 (2017), 83-124. https://heinonline.org/HOL/P?h=hein.journals/admin69&i=95
- Axel Volkery and Teresa Ribeiro. 2009. Scenario planning in public policy: Under- standing use, impacts and the role of institutional context factors. Technological forecasting and social change 76, 9 (2009), 1198-1207. Publisher: Elsevier.
- 10.18653/v1 · doi:10.18653/v1
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent Abilities of Large Language Models. Transactions on Ma- chine Learning Research (2022). https://openreview.net/forum?id=yzkSU5zdwD
- S. Wyatt. 2008. Technological determinism is dead; Long live technological determinism. In Handbook of Science and Technology Studies, E. Hackett, O. Am- sterdamska, M. Lynch, and J. Wajcman (Eds.). MIT Press, Cambridge, 165-180.
- Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. 2021. Mastering Atari Games with Limited Data. In Advances in Neural Information Processing Systems, Vol.
- Curran Associates, Inc., 25476-25488. https://proceedings.neurips.cc/paper/2021/hash/ d5eca8dc3820cad9fe56a3bafda65ca1-Abstract.html
- Remco Zwetsloot and Allan Dafoe. 2019. Thinking about risks from AI: Acci- dents, misuse and structure. Lawfare. February 11 (2019), 2019.
Cited by
Agentic AI: Autonomous intelligence for complex goals–a comprehensive survey
Acharya, D. B., Kuppan, K., & Divya, B. (2025) · IEEE Access · doi:10.1109/access.2025.3532853 · Cited by 717
“…the situation on the exchange, independently making decisions regarding the purchase or sale of securities, and adjusting their strategies to changing conditions in real time, which significantly improves the quality of managerial performance and reduces the participation of human beings [1].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
“Without the ability to represent and reason about these causal links, Agentic AI systems are inherently constrained in their ability to safely generalize beyond narrow, predefined training scenarios [186,215].”
Agentic AI: a comprehensive survey of architectures, applications, and future directions
Abou Ali, M., Dornaika, F., & Charafeddine, J. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11422-4 · Cited by 165
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
Metadata
- Authors
- Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., Lin, M., Mayhew, A., Collins, K., Molamohammadi, M., Burden, J., Zhao, W., Rismani, S., Voudouris, K., Bhatt, U., Weller, A., Krueger, D., & Maharaj, T.
- Affiliations
- Université de Montréal (CA)
- University of Cambridge (GB)
- University of California, Berkeley (US)
- McGill University (CA)
- Western University (CA)
- Mila - Quebec Artificial Intelligence Institute (CA)
- University of Toronto (CA)
- ORCID
- Chan, Alan
- Salganik, Rebecca
- Markelius, Alva
- Pang, Chris
- Rajkumar, Nitarshan
- Krasheninnikov, Dmitrii
- Langosco, Lauro
- He, Zhonghao
- Duan, Yawen
- Carroll, Micah
- Lin, Michelle
- Mayhew, Alex
- Collins, Katherine
- Molamohammadi, Maryam
- Burden, John
- Zhao, Wanru
- Rismani, Shalaleh
- Voudouris, Konstantinos
- Bhatt, Umang
- Weller, Adrian
- Krueger, David
- Maharaj, Tegan
- Venue
- 2023 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2023-06-12
- Pages
- 651-666
- OpenAlex
- W4376632242
- Cited by
- 149
- References
- 135
- Citations by year
- 2026: 60 · 2025: 63 · 2024: 21 · 2023: 5
- FWCI
- 37.30
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Leverhulme Trust
- EPSRC
- Open Philanthropy Project
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- algorithmic agency, algorithmic harms, underspecification, goal-directed behavior, long-term planning, marginalized communities, systemic impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.