The DOI matches its Crossref record; 115 of the 115 references it cites were found in a registry or an index.
Machine behaviour
Open accessAbstract
Machines powered by artificial intelligence increasingly mediate our social, cultural, economic and political interactions. Understanding the behaviour of artificial intelligence systems is essential to our ability to control their actions, reap their benefits and minimize their harms. Here we argue that this necessitates a broad scientific research agenda to study machine behaviour that incorporates and expands upon the discipline of computer science and includes insights from across the sciences. We first outline a set of questions that are fundamental to this emerging field and then explore the technical, legal and institutional constraints on the study of machine behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such endogenous coordination failures blur the line between attack and accident (Leibo et al., 2017;Rahwan et al., 2019), reinforcing that defenses must account for both malicious swarms and emergent collective behaviors.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Augmentation is therefore a coevolutionary process during which humans learn from machines and machines learn from humans (Amershi, Cakmak, Knox, & Kulesza, 2014;Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Only such studies can examine the feedback loops between human influence on machine behavior and machine influence on human behavior, which cause the emergent behaviors that are otherwise impossible to predict (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Computer scientists may be expert technologists, but they are generally not trained social or behavioral scientists (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“These differences lead to entirely new machine behaviors (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Openly available, robust natural language generation (NLG) algorithms have an ability to generate human-like texts across multiple domains”
From Value attributed to text-based archives generated by artificial intelligence (2023) · cites this paper as [1]
Experimental — accuracy not yet measuredIn the citing paper
“This means that research into the beliefs and cognitive processes underlying LLMs need to resort to more indirect and interdisciplinary methods (Rahwan et al., 2019).”
From Exploring the psychology of LLMs’ moral and legal reasoning (2024) · cites this paper as Rahwan et al., 2019
In the citing paper
“First, we demonstrate the effectiveness of experimental economics methods in studying choice behavior of artificial intelligence (43), which adds earlier studies of children (23, 30), monkeys (31), rats, and pigeons (32).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
Self-citation
Research works printed but not linked: 25
- Simon, H. A. The Sciences of the Artificial (MIT Press, Cambridge, 1969). Simon asks whether there can be a science of the ‘artificial’ that produces knowledge about artificial objects and phenomena.
- Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J. & Mullainathan, S. Human decisions and machine predictions. Q. J. Econ. 133, 237–293 (2018).
- Amodei, D. et al. Concrete problems in AI safety. Preprint at https://arxiv.org/abs/1606.06565 (2016). · arXiv
- Bessi, A. & Ferrara, E. Social bots distort the 2016 U.S. Presidential election online discussion. First Monday 21, 11 (2016).
- Tufekci, Z. Engineering the public: big data, surveillance and computational politics. First Monday 19, 7 (2014).
- Cartlidge, J., Szostek, C., De Luca, M. & Cliff, D. Too fast too furious—faster financial-market trading agents can give less efficient markets. In Proc. 4th International Conference on Agents and Artificial Intelligence 126–135 (2012).
- Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https://arxiv.org/abs/1702.08608 (2017). · arXiv
- Kleinberg, J., Mullainathan, S. & Raghavan, M. Inherent trade-offs in the fair determination of risk scores. Preprint at https://arxiv.org/abs/1609.05807 (2016). · arXiv
- Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. In Proc. 1st Conference on Fairness, Accountability and Transparency (eds Friedler, S. A. & Wilson, C.) 81, 77–91 (PMLR, 2018).
- Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. & Kalai, A. T. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proc. Advances in Neural Information Processing Systems 4349–4357 (2016).
- Sweeney, L. Discrimination in online ad delivery. Queueing Syst. 11, 10 (2013).
- Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C. & Venkatasubramanian, S. Runaway feedback loops in predictive policing. Preprint at https://arxiv.org/abs/1706.09847 (2017). · arXiv
- Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proc. 1st Conference on Fairness, Accountability and Transparency (eds Friedler, S. A. & Wilson, C.) 81, 134–148 (PMLR, 2018).
- Zhou, Z., Zhang, W. & Wang, J. Inception score, label smoothing, gradient vanishing and -log(D(x)) alternative. Preprint at https://arxiv.org/abs/1708.01729 (2017). · arXiv
- Tinbergen, N. On aims and methods of ethology. Ethology 20, 410–433 (1963).
- Das, R., Hanson, J. E., Kephart, J. O. & Tesauro, G. Agent–human interactions in the continuous double auction. In Proc. 17th International Joint Conference on Artificial Intelligence 1169–1178 (Lawrence Erlbaum, 2001).
- Smilkov, D., Thorat, N., Kim, B., Viégas, F. & Wattenberg, M. SmoothGrad: removing noise by adding noise. Preprint at https://arxiv.org/abs/1706.03825 (2017). · arXiv
- Edwards, H. & Storkey, A. Censoring representations with an adversary. Preprint at https://arxiv.org/abs/1511.05897 (2015). · arXiv
- Zemel, R., Wu, Y., Swersky, K., Pitassi, T. & Dwork, C. learning fair representations. In Proc. International Conference on Machine Learning 325–333 (2013).
- Leibo, J. Z. et al. Psychlab: a psychology laboratory for deep reinforcement learning agents. Preprint at https://arxiv.org/abs/1801.08116 (2018). In this study, the authors use behavioural tools from the life sciences in the study of machine behaviours. · arXiv
- Lazaridou, A., Peysakhovich, A. & Baroni, M. Multi-agent cooperation and the emergence of (natural) language. Preprint at https://arxiv.org/abs/1612.07182 (2016). · arXiv
- Kamar, E., Hacker, S. & Horvitz, E. Combining human and machine intelligence in large-scale crowdsourcing. in Proc. 11th International Conference on Autonomous Agents and Multiagent Systems 467–474 (International Foundation for Autonomous Agents and Multiagent Systems, 2012).
- Wang, D., Khosla, A., Gargeya, R., Irshad, H. & Beck, A. H. Deep learning for identifying metastatic breast cancer. Preprint at https://arxiv.org/abs/1606.05718 (2016). · arXiv
- Pentland, A. Social Physics: How Social Networks Can Make Us Smarter (Penguin, 2015).
- Meyer, M. N. Two cheers for corporate experimentation: The A/B illusion and the virtues of data-driven innovation. Colorado Technol. Law J. 13, 273 (2015).
Cited by
Sorted by citations
Items 401–500 of 1,008
Training on the Test Set: Mapping the System-Problem Space in AI
Hernández‐Orallo, J., Schellaert, W., & Martínez‐Plumed, F. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21487 · Cited by 16
Angelucci, A., Li, Z., Stoimenova, N., & Canali, S. (2022) · AI & Society · doi:10.1007/s00146-022-01562-4 · Cited by 22
“Decisions made by AI systems often lead to perpetuating harmful biases and result in discrimination based on gender, race, or sexual preferences (Rahwan et al. 2019).”
BackgroundNg, Y.-L. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2111273 · Cited by 16
“Thus, they focus on maximising the algorithmic performance that can accomplish their intended functions instead of the observational causal inference of the human – AI interaction from a human psychology perspective (Rahwan et al. 2019).”
BackgroundImpact and Assessment of Suspension Stiffness on Vibration Propagation into Vehicle
Burdzik, R. (2023) · Sensors · doi:10.3390/s23041930 · Cited by 16
Cai, J., Fu, X., Gu, Z., & Wu, R. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2212224 · Cited by 18
“Artificial intelligence (AI), a novel generation of technologies that can interact with the environment, not only can sensitively sense its surroundings, but also timely adjust its own behavior based on cues from the environment (Frantz, 2003; Rahwan et al., 2019).”
Discussions of machine versus living intelligence need more clarity
Rouleau, N., & Levin, M. G. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00955-y · Cited by 19
Sparse oblique decision trees: a tool to understand and manipulate neural net features
Hada, S. S., Carreira-Perpiñán, M. Á., & Zharmagambetov, A. S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00892-7 · Cited by 21
“More generally, there is a need to understand AI systems experimentally, and different perspectives (including but not limited to computer science) will likely be necessary (Rahwan et al. 2019).”
Beyond efficiency: Trust, AI, and surprise in knowledge work environments
Brown, A. S., Dishop, C. R., Kuznetsov, A. V., Chao, P.-Y., & Woolley, A. W. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108605 · Cited by 19
Qian, P., & Xie, X.-Z. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2526580 · Cited by 17
“Such perception introduces psychological and emotional dimensions to the user-technology relationship that were largely absent in interactions with previous technologies (Rahwan et al., 2019).”
Cristofaro, M., & Bañón-Gomis, A. J. (2025) · Journal of Knowledge Management · doi:10.1108/jkm-06-2025-0870 · Cited by 20
Engineering psychology in the era of artificial intelligence
Xu, W. M., & Ge, L. (2020) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2020.01409 · Cited by 15
Building Global Societies on Collective Intelligence: Challenges and Opportunities
Suran, S., Pattanaik, V., Kurvers, R. H. J. M., Hallin, C. A., Liddo, A. D., Krimmer, R., & Draheim, D. (2021) · Digital Government Research and Practice · doi:10.1145/3568169 · Cited by 15
“…these superminds (specifically their MI counterparts) can have, it is important to look beyond the ‘explainability’ of such systems and examine them through the emerging field of “machine behavior” (described only as “the scientific study of behavior exhibited by intelligent machines”) [23].”
BackgroundTegtmeier, P., Weber, C., Sommer, S., Tisch, A., & Wischniewski, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315506 · Cited by 18
Human control redressed: Comparing AI and human predictability in a real-effort task
Kandul, S., Micheli, V., Beck, J., Burri, T., Fleuret, F., Kneer, M., & Christen, M. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100290 · Cited by 19
Public aversion against ChatGPT in creative fields?
Chu, Y., & Liu, P. (2023) · The Innovation · doi:10.1016/j.xinn.2023.100449 · Cited by 19
Siderska, J., Aini, S. N. B. M., & Kedziora, D. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-53960-2_4 · Cited by 17
Jia, Z., Yu, D., Wang, Z., Chen, C. L. P., & Li, X. (2024) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2024.3384509 · Cited by 15
“With artificial intelligence technology advancing rapidly, game theory holds an increasingly critical position in social [4], machine [5], and cooperative intelligence [6].”
BackgroundHussain, M. H., Iacovides, I., Lawton, T., Sharma, V., Porter, Z., Cunningham, A., Habli, I., Hickey, S., Jia, Y., Morgan, P., et al. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660697 · Cited by 18
“Firstly, the potential for unexpected machine behaviour and biased outcomes, which may evolve as the machine learns [61].”
BackgroundFrom captain to button-presser: operators’ perspectives on navigating highly automated ferries
Veitch, E., Christensen, K. A., Log, M. M., Valestrand, E. T., Lundheim, S. H., Nesse, M., Alsos, O. A., & Steinert, M. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2311/1/012028 · Cited by 15
“In Rahwan et al. [32] the authors argue that insofar that AI agents exhibit “machine behaviour,” human interaction with these agents will inevitably influence human behaviour.”
BackgroundBiomimetic Leadership for 21st Century Companies
Olaizola, E., Morales‐Sánchez, R., & Huerta, M. E. (2021) · Biomimetics · doi:10.3390/biomimetics6030047 · Cited by 15
Du, T., Li, X., Jiang, N., Xu, Y., & Zhou, Y. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2462120 · Cited by 14
“…advanced technology equipped AI with cognitive capabilities (Razmerita et al., 2022; Xie et al., 2021), thus, compared with automation, AI will have more autonomy, tends to conduct tasks independently without direct human supervision (Rahwan et al., 2019; Razmerita et al., 2022; Xu & Ge, 2020).”
Wang, Y., Zhou, H.-B., Luo, X.-Q., Deng, S.-W., Xu, H., Song, Y., Chen, J.-J., Yin, W.-X., Cheng, H.-Y., Wang, A.-J., et al. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c06126 · Cited by 14
Shetty, S. S., Suvarna, S., & Aithal, P. S. (2025) · Poornaprajna International Journal of Teaching & Research Case Studies (PIJTRCS) · doi:10.64818/pijtrcs.3107.8494.0035 · Cited by 14
Houwer, J. D., & Hughes, S. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221114886 · Cited by 19
“The idea that also inorganic machines can learn has given rise to an extensive literature on machine learning (e.g., Burgos, 2018; Rahwan et al., 2019).”
BackgroundDeepCreativity: measuring creativity with deep learning techniques
Franceschelli, G., & Musolesi, M. (2022) · Intelligenza Artificiale · doi:10.3233/ia-220136 · Cited by 19
“At the same time, there is also a need to derive new metrics for examining the behavior of algorithms in different environments and in relation with society [34].”
Analysis of Expert System Implementation in Computer Damage Diagnosis with Forward Chaining Method
Anwar, M. R. (2023) · International Transactions on Artificial Intelligence (ITALIC) · doi:10.33050/italic.v1i2.213 · Cited by 14
Stairway to heaven or highway to hell: A model for assessing cognitive automation use cases
Engel, C., Elshan, E., Ebel, P., & Leimeister, J. M. (2023) · Journal of Information Technology · doi:10.1177/02683962231185599 · Cited by 17
“If a use case ’ s relationship requirements are high, a machine must establish social presence (Short et al., 1976) and human-like behavior (Rahwan et al., 2019; Seeger et al., 2021).”
BackgroundEmergence and collapse of reciprocity in semiautomatic driving coordination experiments with humans
Shirado, H., Kasahara, S., & Christakis, N. A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2307804120 · Cited by 16
Self-citationThe Effects of Sentiment Evolution in Financial Texts: A Word Embedding Approach
Zheng, J., Ng, K. C., Zheng, R., & Tam, K. Y. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2023.2301176 · Cited by 12
López, C., Davidoff, A., Luco, F., Humeres, M., & Correa, T. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2356910 · Cited by 17
“…ubiquity of Artificial Intelligence (AI) and algorithm-mediated interactions in our daily lives, ethical considerations regarding its development and societal implications have assumed a prominent position within academic, regulatory, and policy-making discourses (Cath, 2018; Rahwan et al., 2019).”
Sohail, M., Wang, F., Archer, N., Wang, W., & Yuan, Y. (2024) · Information & Management · doi:10.1016/j.im.2024.104033 · Cited by 14
Coiera, E. W. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32987-3 · Cited by 14
Intelligent machines as social catalysts
Rahwan, I., Crandall, J. W., & Bonnefon, J. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2002744117 · Cited by 16
Self-citationMachinic Encounters: A Relational Approach to the Sociology of AI
Yolgörmez, C. (2020) · Springer eBooks · doi:10.1007/978-3-030-56286-1_5 · Cited by 17
Schneider, T., Hois, J., Rosenstein, A., Metzl, S., Gerlicher, A. R. S., Ghellal, S., & Love, S. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584085 · Cited by 14
Zhang, S., Zhao, B., Tian, Y., & Chen, S. (2020) · Annals of the American Association of Geographers · doi:10.1080/24694452.2020.1782166 · Cited by 14
“Social media content recommendation algorithms are nonhuman actors (Rahwan et al. 2019) that played an essential role in this particular movement and in the supposed post-truth phenomena.”
BackgroundComputational Psychiatry for Computers
Schulz, E., & Dayan, P. (2020) · iScience · doi:10.1016/j.isci.2020.101772 · Cited by 14
Defining deliberation for dual-process models of reasoning
Neys, W. D. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00466-6 · Cited by 14
“The conceptualization of deliberation will be equally critical for the development of machine thinking accounts in artificial intelligence (AI) research 95,96 .”
Na, R., Hu, A., Zeng, Q., Ge, R., & Hao, R. (2025) · Electronic Markets · doi:10.1007/s12525-025-00813-8 · Cited by 16
“The interactive characteristics of virtual streamers to learn and adapt to consumer cues, further strengthening the bond between the streamer and the consumer (Rahwan et al., 2019; Wu et al., 2023a, 2023b).”
An HCAI Methodological Framework: Putting it Into Action to Enable Human-Centered AI
Xu, W., Gao, Z., & Dainoff, M. J. (2026) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3631590 · Cited by 92
Raman, R., Gandhi, A. V., Banerjee, U., & Banerjee, S. (2026) · Global Journal of Flexible Systems Management · doi:10.1007/s40171-026-00483-1 · Cited by 13
“…systems apart is how they operate as a responsive entity, which is capable of: (a) dismantling complex organizational aims into manageable steps (Rahwan et al., 2019), (b) impro-vising with evolving managerial needs, keeping business priorities aligned (Calvaresi et al., 2019), and (c)…”
Remote Medical Applications of Artificial Intelligence
Hudiono, R. K., & Watini, S. (2023) · International Transactions on Artificial Intelligence (ITALIC) · doi:10.33050/italic.v1i2.292 · Cited by 13
Winner-Take-All? Visibility, Availability, and Heterogeneity on Webcam Sex Platforms
Jokubauskaitė, E., Rieder, B., & Burkhardt, S. (2023) · Social Media + Society · doi:10.1177/20563051231214807 · Cited by 15
“Rahwan et al. (2019) thus argue that “[m]achine behavior . . . cannot be fully understood without the integrated study of algorithms and the social environments in which algorithms operate” (p. 477), referring to the idea that users co-produce the decision procedures encapsulated in machine…”
BackgroundHilbert, M., Thakur, A., Flores, P. M., Zhang, X., Bhan, J. Y., Bernhard, P., & Ji, F. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102743 · Cited by 20
Kuckling, J., Luckey, R., Аврутин, В., Vardy, A., Reina, A., & Hamann, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610771 · Cited by 16
“[5].”
Machine learning in electrochemical oxidation process: A mini-review
Li, Z., Zou, S., Wang, Z., Postole, G., Hu, L., & Zhao, H. (2024) · Chinese Chemical Letters · doi:10.1016/j.cclet.2024.110526 · Cited by 13
Sharma, S., & Shrestha, S. (2024) · Journal of Future Artificial Intelligence and Technologies · doi:10.62411/faith.3048-3719-47 · Cited by 17
“AI has become increasingly pervasive, resulting in debates in academic and research areas about its regulatory influence [44].”
Predictability limit of partially observed systems
Abeliuk, A., Huang, Z., Ferrara, E., & Lerman, K. (2020) · Scientific Reports · doi:10.1038/s41598-020-77091-1 · Cited by 14
Modeling and prediction of human negotiation behavior in human-computer negotiation
Cao, M., Wang, G. A., & Kiang, M. Y. (2021) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2021.101099 · Cited by 13
Symbolic AI for XAI: Evaluating LFIT Inductive Programming for Explaining Biases in Machine Learning
de la Puente, A. O., Fiérrez, J., Morales, A., Wang, Z., de la Cruz, M., Alonso, C. L., & Ribeiro, T. (2021) · Computers · doi:10.3390/computers10110154 · Cited by 14
“…deep learning methods usually have good generalisation ability on similarly distributed new data, they have some weaknesses, including their trend to hide the reasons for their behaviours [4,5] , and having a clear explanation of the machine behaviour can be crucial in many practical applications.”
BackgroundWhy robots should be technical
Hindemith, L., Göpfert, J. P., Wiebel-Herboth, C. B., Wrede, B., & Vollmer, A.-L. (2021) · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.20023.hin · Cited by 13
Measuring Bias in AI Models: An Statistical Approach Introducing N-Sigma
DeAlcala, D., Serna, I., Morales, A., Fiérrez, J., & Ortega-García, J. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00176 · Cited by 13
“The absence of international standards for the development of Responsible Artificial Intelligence has motivated a wide variety of approaches [2], [3].”
BackgroundAnokwuru, E. A., Omachi, A., & Enyejo, L. A. (2022) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit2541333 · Cited by 23
People Copy the Actions of Artificial Intelligence
Klichowski, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01130 · Cited by 13
“These interactions become deeper and deeper, and start to have an increasing influence on human functioning (Iqbal and Riek, 2019; Rahwan et al., 2019; Strengers, 2019).”
BackgroundConfronting barriers to human-robot cooperation: balancing efficiency and risk in machine behavior
Whiting, T., Gautam, A., Tye, J., Simmons, M., Henstrom, J., Oudah, M., & Crandall, J. W. (2020) · iScience · doi:10.1016/j.isci.2020.101963 · Cited by 16
Self-citationEliciting Fairness in N‐Player Network Games through Degree‐Based Role Assignment
Teixeira, A. S., Santos, F. C., Francisco, A. P., & Santos, F. P. (2021) · Complexity · doi:10.1155/2021/6851477 · Cited by 13
“…assignment elicits fairness in rather complicated scenarios — as multiplayer bargaining games — suggests that such an approach could also be used within the broader context of active interventions aiming at fostering fairness in hybrid populations comprising humans and machines [39, 43, 31, 44].”
BackgroundAbstract argumentation and the rational man
Kampik, T., & Nieves, J. C. (2021) · Journal of Logic and Computation · doi:10.1093/logcom/exab003 · Cited by 13
Chen, Q., & Heydari, B. (2022) · Journal of Mechanical Design · doi:10.1115/1.4055057 · Cited by 12
Artificial intelligence in hospitality & tourism: ethics, values and cognitive work ability
Khan, A., & Soomro, M. A. (2025) · Current Issues in Tourism · doi:10.1080/13683500.2025.2530590 · Cited by 15
Williams, B. A. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2575897 · Cited by 15
“Artificial Intelligence (AI) is widely accepted as one of the driving forces behind the fourth industrial revolution and it is slowly dominating all sectors including the banking, clothing, transport, and health sectors (Paul & Ahmed, 2024; Rahwan et al., 2019; Schwab, 2017).”
Evaluating the ethics of machines assessing humans
Fischer, I. (2023) · Journal of Information Technology Teaching Cases · doi:10.1177/20438869231178844 · Cited by 12
“Complexities included how imperfections in data might signi fi cantly impact AI-generated results and how the algorithms underpinning particular AI-based tools could be quite simple, however, the results too complex for the users (Rahwan et al., 2019).”
Blinding to Circumvent Human Biases: Deliberate Ignorance in Humans, Institutions, and Machines
Hertwig, R., Herzog, S. M., & Kozyreva, A. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231188052 · Cited by 20
“A blind algorithm cannot be presumed to be bias-free; it must be empirically demonstrated as such in its natural habitat (Rahwan et al., 2019)—that is, in the situations within which it is designed to operate.”
BackgroundOpacity, Machine Learning and Explainable AI
Fernández, A. (2023) · The International library of ethics, law and technology · doi:10.1007/978-3-031-48135-2_3 · Cited by 13
AI Transforming Business and Everyday Life
Bialkova, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-56471-0_9 · Cited by 12
Smale, M. C., Fox, J. D., & Fox, A. K. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108405 · Cited by 12
Zhang, T., Goldstein, A., & Levin, M. G. (2024) · Adaptive Behavior · doi:10.1177/10597123241269740 · Cited by 14
“We strongly support (Bongard & Levin, 2023; Rahwan et al., 2019; Rouleau & Levin, 2023) a de fi nition of basal intelligence as problem-solving, utilizing an objectively observable, third person perspective to distinguish these highly tractable and empirically testable questions from the thorny…”
The AI transparency dilemma: when more is less for trust and adoption
Ngo, V. M. (2025) · Information Discovery and Delivery · doi:10.1108/idd-03-2025-0056 · Cited by 17
Melzer, S. M., & Diewald, M. (2020) · Social Sciences · doi:10.3390/socsci9090156 · Cited by 13
“More recent publications on the human–machine interface emphasize the need to integrate human emotions and perceptions in the understanding of how digitalization impacts work processes (Frank et al. 2019; Rahwan et al. 2019; Riedl 2019).”
BackgroundBorch, C. (2023) · Sociology · doi:10.1177/00380385221146877 · Cited by 12
“…is algorithmic rather than human-generated and where a growing part of the interactions taking place among these algorithmic trading systems involves ML-based systems whose interactions might give rise to complex and unanticipated forms of ‘collective machine behaviour’ (Rahwan et al., 2019: 482).”
BackgroundAutonomous Systems in Ethical Dilemmas: Attitudes towards Randomization
Bodenschatz, A., Uhl, M., & Walkowitz, G. (2020) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100145 · Cited by 12
Noorian, S. S., Qiu, S., Gadiraju, U., Yang, J., & Bozzon, A. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512040 · Cited by 13
Expl(Ai)Ned: The Impact of Explainable Artificial Intelligence on Cognitive ProcessesPreprint
Bauer, K., von Zahn, M., & Hinz, O. F. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3872711 · Cited by 13
Algorithmic transparency and assessing effects of algorithmic ranking
Eckles, D. (2022) · doi:10.31235/osf.io/c8za6 · Cited by 12
Xu, Z., Wang, G., Zhai, S., & Liu, P. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2176986 · Cited by 12
“…only in human work-places but also in social lives, performing from physical and cognitive to social activities, and cooperating with or replacing humans in various sectors such as entertainment, education, health and social care, and transportation (Rahwan et al., 2019; Stephanidis et al., 2019).”
Buckley, P. J., Elia, S., Gaur, A. S., Krakowski, S. M., Kuivalainen, O., & Pedota, M. (2026) · Journal of World Business · doi:10.1016/j.jwb.2025.101710 · Cited by 14
AI is turning research into a scientific monoculture
Traberg, C. S., Roozenbeek, J., & van der Linden, S. (2026) · Communications Psychology · doi:10.1038/s44271-026-00428-5 · Cited by 11
Bias and fairness in machine learning and artificial intelligence
Cirillo, D., & Rementeria, M. J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-821392-6.00006-6 · Cited by 14
Affective Computing for Human-Robot Interaction Research: Four Critical Lessons for the Hitchhiker
Güneş, H., & Churamani, N. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309450 · Cited by 11
“Firstly, in the human-machine interaction context, humans shape machine behaviour and vice-versa [21], [22].”
BackgroundOn the Emergence of Symmetrical Reality
Zhang, Z., Zhang, Z., Jiao, Z., Su, Y., Liu, H., Wang, W., & Zhu, S. C. (2024) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr58804.2024.00084 · Cited by 11
“In this section, we provide a brief overview of five of these seven components, excluding the two components ( i.e. , physical environments and physical humans) that are already well-known.”
BackgroundMateriality and risk in the age of pervasive AI sensors
Sloane, M., Moss, E., Kennedy, S., Stewart, M. P., Warden, P., Plancher, B., & Reddi, V. J. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01017-7 · Cited by 16
“This is a time-consuming process which requires a domain expert to be done effectively and cannot be easily scaled to meet demand [90].”
Guo, X., Liu, Y., Ma, X., & Fu, H. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103647 · Cited by 12
Yi, C., Zhu, R., & Wang, Q. (2021) · Internet Research · doi:10.1108/intr-08-2020-0459 · Cited by 11
The presence of automation enhances deontological considerations in moral judgments
Schurr, A., & Moran, S. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107590 · Cited by 12
Höper, L., & Schulte, C. (2024) · Informatics in Education · doi:10.15388/infedu.2024.19 · Cited by 11
“Through such data practices, they have the potential to influence individuals and societies (Tufekci, 2014; Rahwan et al. , 2019).”
BackgroundAlgorithmic CooperationPreprint
Kasberger, B., Martin, S., Normann, H., & Werner, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4389647 · Cited by 14
Multi-Agent Risks from Advanced AI
Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14143 · Cited by 305
Self-citationPrzygoda, W., Rynio, A., & Kalisz, M. (2025) · Religions · doi:10.3390/rel16080948 · Cited by 23
Reversible gender privacy enhancement via adversarial perturbations
Xie, Y., Zhou, Y., Wang, T., Wen, W., Yi, S., & Zhang, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106130 · Cited by 10
Why People Judge Humans Differently from Machines: The Role of Perceived Agency and Experience
Zhang, J., Conway, J., & Hidalgo, C. A. (2023) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom59411.2023.10397474 · Cited by 10
“The four machine machine A + , E + to a low-agency and low-experience machine A − , E − (passing through a high-agency low-experience machine”
BackgroundZeng, Y. Z., Liu, X., Zhang, X., & Li, Z. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102636 · Cited by 11
Social bots spoil activist sentiment without eroding engagement
Li, L., Vásárhelyi, O., & Vedres, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-74032-0 · Cited by 16
Morality on the road: Should machine drivers be more utilitarian than human drivers?
Liu, P., Chu, Y., Zhai, S., Zhang, T., & Awad, E. (2024) · Cognition · doi:10.1016/j.cognition.2024.106011 · Cited by 10
Phelps, S., & Russell, Y. I. (2025) · Journal of Physics Complexity · doi:10.1088/2632-072x/ada711 · Cited by 76
“Nonetheless, there is a strongly emerging consensus that scientists should study the behaviour of an LLM-chatbot in the same manner that one should study the behaviour of a human or animal [51; 57; 55; 58; 59; 60; 61; 62; 63; 64; 65].”
BackgroundInfluentialDigital Twins and Artificial Collective Intelligence: Synergies for the Future
Pretel, E., Navarro, E., Pujol, V. C., & Dustdar, S. (2025) · IEEE Internet Computing · doi:10.1109/mic.2024.3521607 · Cited by 10
Miazek, K., & Bocian, K. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100719 · Cited by 16
Li, S., Yang, Y., Han, C., Sun, Y., Attar, R. W., & Gupta, B. B. (2025) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2025.102951 · Cited by 12
Rieger, T., Onnasch, L., Roesler, E., & Manzey, D. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3584662 · Cited by 12
“In the age of the science of machine behavior [32], the reliability of systems keeps increasing.”
The AI intelligence playbook: Decoding GenAI capabilities for strategic advantage
Campbell, C., Sands, S., Whittaker, L., & Mavrommatis, A. (2026) · Business Horizons · doi:10.1016/j.bushor.2025.08.004 · Cited by 10
Measuring transparency in intelligent robots
Angelopoulos, G., Lacroix, D., Wullenkord, R., Rossi, A., Rossi, S., & Eyssel, F. (2024) · Scientific Reports · doi:10.1038/s41598-025-29685-w · Cited by 15
“Transparency in robots can also serve as a tool for developers and researchers to better understand and improve robot’s functioning 11 .”
Voice-cloning artificial-intelligence speakers can also mimic human-specific vocal expression
Chen, W., & Jiang, X. (2026) · Current Psychology · doi:10.1007/s12144-026-09991-w · Cited by 10
Metadata
- Authors
- Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., Jennings, N. R., Kamar, E., Kloumann, I. M., Larochelle, H., Lazer, D., McElreath, R., Mislove, A., Parkes, D. C., Pentland, A. ‘., Roberts, M. E., Shariff, A., Tenenbaum, J. B., & Wellman, M.
- Affiliations
- Max Planck Institute for Human Development (DE)
- MIT Media Lab (US)
- University of Vermont (US)
- Centre National de la Recherche Scientifique (FR)
- Brigham Young University (US)
- Yale University (US)
- Max Planck Society (DE)
- Canadian Institute for Advanced Research (CA)
- Imperial College London (GB)
- Microsoft (United States) (US)
- Meta (United States) (US)
- Northeastern University (US)
- Max Planck Institute for Evolutionary Anthropology (DE)
- Harvard University (US)
- University of California San Diego (US)
- University of British Columbia (CA)
- Massachusetts Institute of Technology (US)
- University of Michigan (US)
- Corresponding author
- Rahwan, Iyad
- ORCID
- Rahwan, Iyad
- Obradovich, Nick
- Bongard, Josh
- Bonnefon, Jean-François
- Breazeal, Cynthia
- Crandall, Jacob W.
- Christakis, Nicholas A.
- Couzin, Iain D.
- Jackson, Matthew O.
- Jennings, Nicholas R.
- Lazer, David
- McElreath, Richard
- Mislove, Alan
- Parkes, David C.
- Pentland, Alex ‘Sandy’
- Roberts, Margaret E.
- Shariff, Azim
- Tenenbaum, Joshua B.
- Wellman, Michael
- Venue
- Nature
- Published
- 2019-04
- Volume
- 568
- Issue
- 7753
- Pages
- 477-486
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4211017739
- Cited by
- 1,364
- References
- 115
- Citations by year
- 2026: 330 · 2025: 258 · 2024: 187 · 2023: 157 · 2022: 142 · 2021: 143 · 2020: 113 · 2019: 31
- FWCI
- 29.97
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- KOPS (University of Konstanz) (submittedVersion)
- HAL (submittedVersion, cc-by)
- Toulouse 1 Capitole Publications (Université Toulouse I Capitole) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- Funders
- National Science Foundation (N00014-14-1-0635, 1355061, 1447634, N00014-09-1-1074, 1231216, 1344227)
- Robert Wood Johnson Foundation
- Office of the Director of National Intelligence (2017-17061500006)
- Future of Life Institute
- Deutsche Forschungsgemeinschaft (422037984)
- Agence Nationale de la Recherche (ANR-17-EURE-0010, 2017-)
- Max-Planck-Gesellschaft (W911NG-11-1-0385, N00014-09-1-1074, IOS-1355061, N00014-14-1-0635)
- Companhia Brasileira de Metalurgia e Mineração (CCF-1231216)
- Defense Advanced Research Projects Agency
- Intelligence Advanced Research Projects Activity (2017-17061500006)
- Office of Naval Research (N00014-09-1-1074, N00014-14-1 0635, IOS-1355061, W911NG-11-1-0385, N00014)
- NIH Office of the Director
- Army Research Office (W911NF-14-1-0431, W911NF-16-1-0304, N00014-14-1-0635, W911NG-11-1- 0385, IOS-1355061, W911NF-16-1-, W911NF, N00014-09-1-1074)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- artificial intelligence, artificial intelligence systems, AI regulation, interdisciplinary research agenda, institutional constraints, societal impact of AI
- MeSH terms
- Artificial Intelligence, Humans, Motivation, Robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
