The DOI matches its Crossref record; 115 of the 115 references it cites were found in a registry or an index.
Machine behaviour
Open accessAbstract
Machines powered by artificial intelligence increasingly mediate our social, cultural, economic and political interactions. Understanding the behaviour of artificial intelligence systems is essential to our ability to control their actions, reap their benefits and minimize their harms. Here we argue that this necessitates a broad scientific research agenda to study machine behaviour that incorporates and expands upon the discipline of computer science and includes insights from across the sciences. We first outline a set of questions that are fundamental to this emerging field and then explore the technical, legal and institutional constraints on the study of machine behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such endogenous coordination failures blur the line between attack and accident (Leibo et al., 2017;Rahwan et al., 2019), reinforcing that defenses must account for both malicious swarms and emergent collective behaviors.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Augmentation is therefore a coevolutionary process during which humans learn from machines and machines learn from humans (Amershi, Cakmak, Knox, & Kulesza, 2014;Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Only such studies can examine the feedback loops between human influence on machine behavior and machine influence on human behavior, which cause the emergent behaviors that are otherwise impossible to predict (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Computer scientists may be expert technologists, but they are generally not trained social or behavioral scientists (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“These differences lead to entirely new machine behaviors (Rahwan et al., 2019).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Rahwan et al., 2019
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Openly available, robust natural language generation (NLG) algorithms have an ability to generate human-like texts across multiple domains”
From Value attributed to text-based archives generated by artificial intelligence (2023) · cites this paper as [1]
Experimental — accuracy not yet measuredIn the citing paper
“This means that research into the beliefs and cognitive processes underlying LLMs need to resort to more indirect and interdisciplinary methods (Rahwan et al., 2019).”
From Exploring the psychology of LLMs’ moral and legal reasoning (2024) · cites this paper as Rahwan et al., 2019
In the citing paper
“First, we demonstrate the effectiveness of experimental economics methods in studying choice behavior of artificial intelligence (43), which adds earlier studies of children (23, 30), monkeys (31), rats, and pigeons (32).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
Self-citationExperimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationExposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Diabetes and Cardiovascular Disease: The Framingham Study
Kannel, W. B. (1979) · JAMA · doi:10.1001/jama.1979.03290450033020 · Cited by 2,936
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Further, when systems learn from data, their failures are linked to imperfections in data or how data was collected, leading some to argue for adapted reporting mechanisms for datasets 47 and models 48 .”
BackgroundGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Bainbridge, L. (1983) · Automatica · doi:10.1016/0005-1098(83)90046-8 · Cited by 2,205
Ensemble Adversarial Training: Attacks and Defenses
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.07204 · Cited by 2,176
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859
Self-citationEvaluating Color Descriptors for Object and Scene Recognition
van de Sande, K. E. A., Gevers, T., & Snoek, C. G. M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.154 · Cited by 1,837
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,658
The social dilemma of autonomous vehicles
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016) · Science · doi:10.1126/science.aaf2654 · Cited by 1,527
Self-citationProgrammable self-assembly in a thousand-robot swarm
Rubenstein, M., Cornejo, A., & Nagpal, R. (2014) · Science · doi:10.1126/science.1254295 · Cited by 1,308
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
What can machine learning do? Workforce implications
Brynjolfsson, E., & Mitchell, T. (2017) · Science · doi:10.1126/science.aap8062 · Cited by 1,131
Dalvi, N. N., Domingos, P. M., Mausam, Sanghai, S. K., & Verma, D. (2004) · Knowledge Discovery and Data Mining · doi:10.1145/1014052.1014066 · Cited by 1,085
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Visual interpretability for deep learning: a survey
Zhang, Q., & Zhu, S. C. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700808 · Cited by 1,018
Robots that can adapt like animals
Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015) · Nature · doi:10.1038/nature14422 · Cited by 979
Feeling robots and human zombies: Mind perception and the uncanny valley
Gray, K., & Wegner, D. M. (2012) · Cognition · doi:10.1016/j.cognition.2012.06.007 · Cited by 941
The High-Frequency Trading Arms Race: Frequent Batch Auctions as a Market Design Response
Budish, E. B., Cramton, P., & Shim, J. J. (2013) · The Quarterly Journal of Economics · doi:10.1093/qje/qjv027 · Cited by 901
Evasion Attacks against Machine Learning at Test Time
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_25 · Cited by 890
Every planar map is four colorable. Part I: Discharging
Appel, K. I., & Haken, W. (1977) · Illinois Journal of Mathematics · doi:10.1215/ijm/1256049011 · Cited by 888
Robustness and Evolvability in Living Systems
Wagner, A. (2013) · Princeton University Press eBooks · doi:10.1515/9781400849383 · Cited by 886
Credit scoring with a data mining approach based on support vector machines
Huang, C.-L., Chen, M., & Wang, C.-J. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.07.007 · Cited by 830
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822 · cited 1×
“More sophisticated agents may compute strategies based on adaptive heuristics or explicit maximization of expected utility 86 .”
MethodOnline Dating: A Critical Analysis From the Perspective of Psychological Science
Finkel, E. J., Eastwick, P. W., Karney, B. R., Reis, H. T., & Sprecher, S. (2012) · Psychological Science in the Public Interest · doi:10.1177/1529100612436522 · Cited by 812
Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., & Osawa, E. (1997) · doi:10.1145/267658.267738 · Cited by 776
Every planar map is four colorable. Part II: Reducibility
Appel, K. I., Haken, W., & Koch, J. (1977) · Illinois Journal of Mathematics · doi:10.1215/ijm/1256049012 · Cited by 713
Resilient Machines Through Continuous Self-Modeling
Bongard, J., Zykov, V., & Lipson, H. (2006) · Science · doi:10.1126/science.1133687 · Cited by 709
Self-citationPredictive Policing: The Role of Crime Forecasting in Law Enforcement Operations
Perry, W., McInnis, B., Price, C., Smith, S., & Hollywood, J. (2013) · doi:10.7249/rr233 · Cited by 693
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
Exploring the filter bubble: the effect of using recommender systems on content diversity
Nguyen, T. T., Hui, P.-M., Harper, F., Terveen, L., & Konstan, J. (2014) · The Web Conference · doi:10.1145/2566486.2568012 · Cited by 690
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676
Matching and Sorting in Online Dating
Hitsch, G. J., Hortaçsu, A., & Ariely, D. (2010) · American Economic Review · doi:10.1257/aer.100.1.130 · Cited by 668
A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots
Giusti, A., Guzzi, J., Ciresan, D. C., He, F.-L., Rodriguez, J. P., Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Caro, G. D., et al. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2015.2509024 · Cited by 665
Interdisciplinary research has consistently lower funding success
Bromham, L. D., Dinnage, R., & Hua, X. (2016) · Nature · doi:10.1038/nature18315 · Cited by 602
Socially Assistive Robots in Elderly Care: A Systematic Review into Effects and Effectiveness
Bemelmans, R., Gelderblom, G. J., Jonker, P., & de Witte, L. (2012) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2010.10.002 · Cited by 585
Emergent Sensing of Complex Environments by Mobile Animal Groups
Berdahl, A. M., Torney, C. J., Ioannou, C. C., Faria, J. J., & Couzin, I. D. (2013) · Science · doi:10.1126/science.1225883 · Cited by 545
Self-citationUsing neural network ensembles for bankruptcy prediction and credit scoring
TSAI, C., & Wu, J. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.05.019 · Cited by 517
The DARPA Twitter Bot Challenge
Subrahmanian, V. S., Azaria, A., Durst, S., Kagan, V., Galstyan, A., Lerman, K., Zhu, L., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Computer · doi:10.1109/mc.2016.183 · Cited by 480
Social fMRI: Investigating and shaping social mechanisms in the real world
Aharony, N., Pan, W., Ip, C., Khayal, I. S., & Pentland, A. (2011) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2011.09.004 · Cited by 458
Self-citationSchaeffer, J., BURCH, N. R., Björnsson, Y., Kishimoto, A., Müller, M., Lake, R., Lu, P., & Sutphen, S. (2007) · Science · doi:10.1126/science.1144079 · Cited by 452
Uninformed individuals promote democratic consensus in animal groups
Couzin, I. D., Ioannou, C. C., Demirel, G., Gross, T., Torney, C. J., Hartnett, A., Conradt, L., Levin, S. A., & Leonard, N. E. (2011) · Science · doi:10.1126/science.1210280 · Cited by 451
Self-citationTeachable robots: Understanding human teaching behavior to build more effective robot learners
Thomaz, A. L., & Breazeal, C. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.09.009 · Cited by 414
Self-citationLEE, T., & Chen, I.-F. (2005) · Expert Systems with Applications · doi:10.1016/j.eswa.2004.12.031 · Cited by 398
How to solve the world's biggest problems
Ledford, H. K. (2015) · Nature · doi:10.1038/525308a · Cited by 396
Self-organized criticality in the 'Game of Life"
Bak, P., Chen, K., & Creutz, M. J. (1989) · Nature · doi:10.1038/342780a0 · Cited by 362
Heads-up limit hold’em poker is solved
Bowling, M., BURCH, N. R., Johanson, M., & Tammelin, O. (2015) · Science · doi:10.1126/science.1259433 · Cited by 349
Evidence of complex contagion of information in social media: An experiment using Twitter bots
Mønsted, B., Sapieżyński, P., Ferrara, E., & Lehmann, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184148 · Cited by 340
The Economics of High-Frequency Trading: Taking Stock
Menkveld, A. J. (2016) · Annual Review of Financial Economics · doi:10.1146/annurev-financial-121415-033010 · Cited by 314
Nightmare at test time: robust learning by feature deletion
Globerson, A., & Roweis, S. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143889 · Cited by 309
An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace
Chen, L., Mislove, A. E., & Wilson, C. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883089 · Cited by 309
Self-citationLocally noisy autonomous agents improve global human coordination in network experiments
Shirado, H., & Christakis, N. A. (2017) · Nature · doi:10.1038/nature22332 · Cited by 288
Self-citationBias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr
Hannák, A., Wagner, C., Garcia, D., Mislove, A., Strohmaier, M., & Wilson, C. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998327 · Cited by 286 · cited 1×
“Online pricing algorithms shape the cost of products differentially across consumers 23–25 .”
BackgroundSelf-citationReinforcement learning for optimized trade execution
Nevmyvaka, Y., Feng, Y., & Kearns, M. J. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143929 · Cited by 272
Crandall, J. W., Oudah, M., Tennom, Ishowo-Oloko, F., Abdallah, S., Bonnefon, J., Cebrián, M., Shariff, A., Goodrich, M. A., & Rahwan, I. (2018) · Nature Communications · doi:10.1038/s41467-017-02597-8 · Cited by 255
Self-citationEconomic reasoning and artificial intelligence
Parkes, D. C., & Wellman, M. P. (2015) · Science · doi:10.1126/science.aaa8403 · Cited by 245
Self-citationMoore's Law versus Murphy's Law: Algorithmic Trading and Its Discontents
Kirilenko, A., & Lo, A. W. (2013) · The Journal of Economic Perspectives · doi:10.1257/jep.27.2.51 · Cited by 212
Artificial Intelligence and Life in 2030
Stone, P. (2024) · IEEE RESOURCE CENTERS · doi:10.17023/ta87-9545 · Cited by 202
Even good bots fight: The case of Wikipedia
Tsvetkova, M., García-Gavilanes, R., Floridi, L., & Yasseri, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0171774 · Cited by 200
Re-embodiment of Honeybee Aggregation Behavior in an Artificial Micro-Robotic System
Kernbach, S., Thenius, R., Kernbach, O., & Schmickl, T. (2009) · Adaptive Behavior · doi:10.1177/1059712309104966 · Cited by 185
Jennings, N. R., Moreau, L., Nicholson, D., Ramchurn, S. D., Roberts, S., Rodden, T. A., & Rogers, A. (2014) · Communications of the ACM · doi:10.1145/2629559 · Cited by 171
Self-citationGalceran, E., Cunningham, A. G., Eustice, R. M., & Olson, E. (2017) · Autonomous Robots · doi:10.1007/s10514-017-9619-z · Cited by 168
Abrupt rise of new machine ecology beyond human response time
Johnson, N. F., Zhao, G., Hunsader, E., Qi, H., Johnson, N., Meng, J., & Tivnan, B. F. (2013) · Scientific Reports · doi:10.1038/srep02627 · Cited by 163
Designing the market game for a trading agent competition
Wellman, M. P., Wurman, P. R., O’Malley, K., Bangera, R., Lin, S.-D., Reeves, D., & Walsh, W. E. (2001) · IEEE Internet Computing · doi:10.1109/4236.914647 · Cited by 160
Self-citationRoberts, M. E. (2018) · Princeton University Press · doi:10.23943/9781400890057 · Cited by 159
Tinbergen's four questions, organized: a response to Bateson and Laland
Nesse, R. M. (2013) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2013.10.008 · Cited by 155
The Strategic Robot Problem: Lethal Autonomous Weapons in War
Roff, H. M. (2014) · Journal of Military Ethics · doi:10.1080/15027570.2014.975010 · Cited by 154
A modal characterisation of observable machine-behaviour
Milner, R. (1981) · Lecture notes in computer science · doi:10.1007/3-540-10828-9_52 · Cited by 154
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Lakkaraju, H., Kamar, E. S., Caruana, R., & Horvitz, E. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10821 · Cited by 146
Self-citationGrowing Growth Mindset with a Social Robot Peer
Park, H. W., Rosenberg-Kima, R., Rosenberg, M., Gordon, G., & Breazeal, C. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020213 · Cited by 140
Self-citationKory-Westlund, J. M., Jeong, S., Park, H. W., Ronfard, S., Adhikari, A., Harris, P. L., DeSteno, D. A., & Breazeal, C. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00295 · Cited by 137
Self-citationThe rise of the social algorithm
Lazer, D. (2015) · Science · doi:10.1126/science.aab1422 · Cited by 133
Self-citationAnalyzing Uber's Ride-sharing Economy
Kooti, F., Grbovic, M., Aiello, L. M., Djuric, N., Radosavljević, V., & Lerman, K. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054194 · Cited by 124
Voosen, P. (2017) · Science · doi:10.1126/science.357.6346.22 · Cited by 101
Synchrony and Reciprocity: Key Mechanisms for Social Companion Robots in Therapy and Care
Lorenz, T., Weiss, A., & Hirche, S. (2015) · International Journal of Social Robotics · doi:10.1007/s12369-015-0325-8 · Cited by 96
Measuring young children's long-term relationships with social robots
Kory-Westlund, J. M., Park, H. W., Williams, R., & Breazeal, C. (2018) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3202185.3202732 · Cited by 91
Self-citationRobot Vacuum Cleaner Personality and Behavior
Hendriks, B., Meerbeek, B., Boess, S. U., Pauws, S. C., & Sonneveld, M. H. (2010) · International Journal of Social Robotics · doi:10.1007/s12369-010-0084-5 · Cited by 88
Salisbury, E., Kamar, E. S., & Morris, M. R. (2017) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v5i1.13301 · Cited by 85
Self-citationAn ecological perspective on the future of computer trading
Farmer, J. D., & Skouras, S. (2013) · Quantitative Finance · doi:10.1080/14697688.2012.757636 · Cited by 84
I always feel like somebody's watching me
Carrascosa, J. M., Mikians, J., Cuevas, R., Erramilli, V., & Nikolaos Laoutaris, B. (2015) · ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) · doi:10.1145/2716281.2836098 · Cited by 79
Bank Lending Policy, Credit Scoring, and the Survival of Loans
Roszbach, K. F. (2004) · The Review of Economics and Statistics · doi:10.1162/0034653043125248 · Cited by 71
Understanding Human-Machine Networks: A Cross-Disciplinary Survey
Tsvetkova, M., Yasseri, T., Meyer, E. T., Pickering, J. B., Engen, V., Walland, P., Lüders, M., Følstad, A., & Bravos, G. (2018) · ACM Computing Surveys · doi:10.1145/3039868 · Cited by 67
Ethical Issues for Autonomous Trading Agents
Wellman, M. P., & Rajan, U. (2017) · Minds and Machines · doi:10.1007/s11023-017-9419-4 · Cited by 67
Self-citationMechanisms for (mis)allocating scientific credit
Kleinberg, J. M., & Oren, S. (2011) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/1993636.1993707 · Cited by 67
Jeong, S., Breazeal, C., Logan, D. E., & Weinstock, P. H. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174069 · Cited by 58
Self-citationDrone Warfare: Blowback from the New American Way of War
Hudson, L., Owens, C. S., & Flannes, M. (2011) · Middle East Policy · doi:10.1111/j.1475-4967.2011.00502.x · Cited by 51
Hilbert, M., Ahmed, S., Cho, J., Liu, B., & Luu, J. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1479843 · Cited by 49
Take this personally: pollution attacks on personalized services
Xing, X., Meng, W., Doozan, D., Snoeren, A. C., Feamster, N., & Lee, W. (2013) · Cited by 41
Empirical Limitations on High-Frequency Trading Profitability
Kearns, M. J., Kulesza, A., & Nevmyvaka, Y. (2010) · The Journal of Trading · doi:10.3905/jot.2010.5.4.050 · Cited by 39
Closing the AI Knowledge GapPreprint
Epstein, Z., Payne, B. H., Shen, J. H., Dubey, A., Felbo, B., Groh, M., Obradovich, N., Cebrián, M., & Rahwan, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07233 · Cited by 39
Self-citationKrishnan, A. (2016) · doi:10.4324/9781315591070 · Cited by 32
Krafft, P. M., Macy, M. W., & Pentland, A. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998354 · Cited by 24
Self-citationObserving algorithmic marketplaces in-the-wild
Chen, L., & Wilson, C. (2017) · ACM SIGecom Exchanges · doi:10.1145/3055589.3055594 · Cited by 18
Design and performance evaluation of voice activated wireless home devices
Zeng, X., Fapojuwo, A. O., & Davies, R. J. (2006) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2006.1706497 · Cited by 16
Patel, K. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3046847 · Cited by 6
Research works printed but not linked: 25
- Simon, H. A. The Sciences of the Artificial (MIT Press, Cambridge, 1969). Simon asks whether there can be a science of the ‘artificial’ that produces knowledge about artificial objects and phenomena.
- Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J. & Mullainathan, S. Human decisions and machine predictions. Q. J. Econ. 133, 237–293 (2018).
- Amodei, D. et al. Concrete problems in AI safety. Preprint at https://arxiv.org/abs/1606.06565 (2016). · arXiv
- Bessi, A. & Ferrara, E. Social bots distort the 2016 U.S. Presidential election online discussion. First Monday 21, 11 (2016).
- Tufekci, Z. Engineering the public: big data, surveillance and computational politics. First Monday 19, 7 (2014).
- Cartlidge, J., Szostek, C., De Luca, M. & Cliff, D. Too fast too furious—faster financial-market trading agents can give less efficient markets. In Proc. 4th International Conference on Agents and Artificial Intelligence 126–135 (2012).
- Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https://arxiv.org/abs/1702.08608 (2017). · arXiv
- Kleinberg, J., Mullainathan, S. & Raghavan, M. Inherent trade-offs in the fair determination of risk scores. Preprint at https://arxiv.org/abs/1609.05807 (2016). · arXiv
- Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. In Proc. 1st Conference on Fairness, Accountability and Transparency (eds Friedler, S. A. & Wilson, C.) 81, 77–91 (PMLR, 2018).
- Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. & Kalai, A. T. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proc. Advances in Neural Information Processing Systems 4349–4357 (2016).
- Sweeney, L. Discrimination in online ad delivery. Queueing Syst. 11, 10 (2013).
- Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C. & Venkatasubramanian, S. Runaway feedback loops in predictive policing. Preprint at https://arxiv.org/abs/1706.09847 (2017). · arXiv
- Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proc. 1st Conference on Fairness, Accountability and Transparency (eds Friedler, S. A. & Wilson, C.) 81, 134–148 (PMLR, 2018).
- Zhou, Z., Zhang, W. & Wang, J. Inception score, label smoothing, gradient vanishing and -log(D(x)) alternative. Preprint at https://arxiv.org/abs/1708.01729 (2017). · arXiv
- Tinbergen, N. On aims and methods of ethology. Ethology 20, 410–433 (1963).
- Das, R., Hanson, J. E., Kephart, J. O. & Tesauro, G. Agent–human interactions in the continuous double auction. In Proc. 17th International Joint Conference on Artificial Intelligence 1169–1178 (Lawrence Erlbaum, 2001).
- Smilkov, D., Thorat, N., Kim, B., Viégas, F. & Wattenberg, M. SmoothGrad: removing noise by adding noise. Preprint at https://arxiv.org/abs/1706.03825 (2017). · arXiv
- Edwards, H. & Storkey, A. Censoring representations with an adversary. Preprint at https://arxiv.org/abs/1511.05897 (2015). · arXiv
- Zemel, R., Wu, Y., Swersky, K., Pitassi, T. & Dwork, C. learning fair representations. In Proc. International Conference on Machine Learning 325–333 (2013).
- Leibo, J. Z. et al. Psychlab: a psychology laboratory for deep reinforcement learning agents. Preprint at https://arxiv.org/abs/1801.08116 (2018). In this study, the authors use behavioural tools from the life sciences in the study of machine behaviours. · arXiv
- Lazaridou, A., Peysakhovich, A. & Baroni, M. Multi-agent cooperation and the emergence of (natural) language. Preprint at https://arxiv.org/abs/1612.07182 (2016). · arXiv
- Kamar, E., Hacker, S. & Horvitz, E. Combining human and machine intelligence in large-scale crowdsourcing. in Proc. 11th International Conference on Autonomous Agents and Multiagent Systems 467–474 (International Foundation for Autonomous Agents and Multiagent Systems, 2012).
- Wang, D., Khosla, A., Gargeya, R., Irshad, H. & Beck, A. H. Deep learning for identifying metastatic breast cancer. Preprint at https://arxiv.org/abs/1606.05718 (2016). · arXiv
- Pentland, A. Social Physics: How Social Networks Can Make Us Smarter (Penguin, 2015).
- Meyer, M. N. Two cheers for corporate experimentation: The A/B illusion and the virtues of data-driven innovation. Colorado Technol. Law J. 13, 273 (2015).
Cited by
Sorted by citations
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
“The interactional properties of AI make it capable of learning and changing its behavior based on the cues from the environment (Frantz, 2003; Rahwan et al., 2019).”
BackgroundHow artificial intelligence will change the future of marketing
Davenport, T., Guha, A., Grewal, D., & Bressgott, T. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00696-0 · Cited by 2,875
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Vision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435
“Topics related to human-arti fi cial congition such as machine behavior (Rahwan et al., 2019), cognitive of fl oading (Risko & Gilbert, 2016), and new de fi nitions of cognition (Bayne et al., 2019) are particularly relevat to study as the role of AI in education is growing in signi fi cance.”
BackgroundZheng, Y. H., Yao, Y., Ou, J., Li, M., Luo, D., Dou, H., Li, Z., Amine, K., Yu, A., & Chen, Z. (2020) · Chemical Society Reviews · doi:10.1039/d0cs00305k · Cited by 1,047
Metadata
- Authors
- Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., Jennings, N. R., Kamar, E., Kloumann, I. M., Larochelle, H., Lazer, D., McElreath, R., Mislove, A., Parkes, D. C., Pentland, A. ‘., Roberts, M. E., Shariff, A., Tenenbaum, J. B., & Wellman, M.
- Affiliations
- Max Planck Institute for Human Development (DE)
- MIT Media Lab (US)
- University of Vermont (US)
- Centre National de la Recherche Scientifique (FR)
- Brigham Young University (US)
- Yale University (US)
- Max Planck Society (DE)
- Canadian Institute for Advanced Research (CA)
- Imperial College London (GB)
- Microsoft (United States) (US)
- Meta (United States) (US)
- Northeastern University (US)
- Max Planck Institute for Evolutionary Anthropology (DE)
- Harvard University (US)
- University of California San Diego (US)
- University of British Columbia (CA)
- Massachusetts Institute of Technology (US)
- University of Michigan (US)
- Corresponding author
- Rahwan, Iyad
- ORCID
- Rahwan, Iyad
- Obradovich, Nick
- Bongard, Josh
- Bonnefon, Jean-François
- Breazeal, Cynthia
- Crandall, Jacob W.
- Christakis, Nicholas A.
- Couzin, Iain D.
- Jackson, Matthew O.
- Jennings, Nicholas R.
- Lazer, David
- McElreath, Richard
- Mislove, Alan
- Parkes, David C.
- Pentland, Alex ‘Sandy’
- Roberts, Margaret E.
- Shariff, Azim
- Tenenbaum, Joshua B.
- Wellman, Michael
- Venue
- Nature
- Published
- 2019-04
- Volume
- 568
- Issue
- 7753
- Pages
- 477-486
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4211017739
- Cited by
- 1,364
- References
- 115
- Citations by year
- 2026: 330 · 2025: 258 · 2024: 187 · 2023: 157 · 2022: 142 · 2021: 143 · 2020: 113 · 2019: 31
- FWCI
- 29.97
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- KOPS (University of Konstanz) (submittedVersion)
- HAL (submittedVersion, cc-by)
- Toulouse 1 Capitole Publications (Université Toulouse I Capitole) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- Funders
- National Science Foundation (N00014-14-1-0635, 1355061, 1447634, N00014-09-1-1074, 1231216, 1344227)
- Robert Wood Johnson Foundation
- Office of the Director of National Intelligence (2017-17061500006)
- Future of Life Institute
- Deutsche Forschungsgemeinschaft (422037984)
- Agence Nationale de la Recherche (ANR-17-EURE-0010, 2017-)
- Max-Planck-Gesellschaft (W911NG-11-1-0385, N00014-09-1-1074, IOS-1355061, N00014-14-1-0635)
- Companhia Brasileira de Metalurgia e Mineração (CCF-1231216)
- Defense Advanced Research Projects Agency
- Intelligence Advanced Research Projects Activity (2017-17061500006)
- Office of Naval Research (N00014-09-1-1074, N00014-14-1 0635, IOS-1355061, W911NG-11-1-0385, N00014)
- NIH Office of the Director
- Army Research Office (W911NF-14-1-0431, W911NF-16-1-0304, N00014-14-1-0635, W911NG-11-1- 0385, IOS-1355061, W911NF-16-1-, W911NF, N00014-09-1-1074)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- artificial intelligence, artificial intelligence systems, AI regulation, interdisciplinary research agenda, institutional constraints, societal impact of AI
- MeSH terms
- Artificial Intelligence, Humans, Motivation, Robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
