The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Machine Learning, Functions and Goals
Open accessAbstract
Machine learning researchers distinguish between reinforcement learning and supervised learning and refer to reinforcement learning systems as “agents”. This paper vindicates the claim that systems trained by reinforcement learning are agents while those trained by supervised learning are not. Systems of both kinds satisfy Dretske’s criteria for agency, because they both learn to produce outputs selectively in response to inputs. However, reinforcement learning is sensitive to the instrumental value of outputs, giving rise to systems which exploit the effects of outputs on subsequent inputs to achieve good performance over episodes of interaction with their environments. Supervised learning systems, in contrast, merely learn to produce better outputs in response to individual inputs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Reinforcement learning (RL) research explicitly aims to build artificial agents which pursue goals (Sutton & Barto 2018), and typical RL systems meet all three of these conditions.”
From Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) · cites this paper as Butlin 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Language, Thought, and Other Biological Categories: New Foundations for Realism
Millikan, R. G. (1984) · doi:10.7551/mitpress/4124.001.0001 · Cited by 4,758
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Research works printed but not linked: 19
- Brafman, R. and Tennenholtz, M. 2002. "R-Max: A general polynomial time algorithm for near-optimal reinforcement learning." Journal of Machine Learning Research 3: 213-231.
- Bommasani, R. et al. 2022. "On the opportunities and risks of foundation models." arXiv preprint.
- Boorse, C. 1976. "Wright on functions." Philosophical Review 85: 70-86.
- Dennett, D. 1991. "Ways of establishing harmony". In McLaughlin (ed.). Dretske and His Critics. Oxford: Blackwell.
- Dolan, R. and P. Dayan. 2013. "Goals and habits in the brain." Neuron 80 (2): 312-325.
- Dretske, F. 1985. "Machines and the mental." Proceedings and Addresses of the American Philosophical Association 59: 23-33.
- Dretske, F. 1993. "Can intelligence be artifi cial?" Philosophical Studies 71 (2): 201-216.
- Garson, J. 2016. What Biological Functions Are and Why They Matter. Cambridge: Cambridge University Press.
- Lycan, W. 1987. Consciousness. Cambridge: The MIT Press.
- McKenna, M. 2016. "A modest historical theory of moral responsibility." The Journal of Ethics 20: 83-105.
- Millikan, R. G. 1984. Language, Thought and Other Biological Categories. Cambridge: The MIT Press.
- Millikan, R. G. 1996. "On swampkinds." Mind and Language 11 (1): 103- 117.
- Mnih, V. et al. 2015. "Human-level control through deep reinforcement learning." Nature 518 (7540): 529-533.
- Niv, Y. 2009. "Reinforcement learning in the brain." Journal of Mathemati- cal Psychology 53: 139-154.
- Papineau, D. 1993. Philosophical Naturalism. Oxford: Blackwell.
- Russell, S. and Norvig, P. 2010. Artifi cial Intelligence: A Modern Approach (3 rd edition). London: Pearson.
- Sutton, R and Barto, A. 2018. Reinforcement Learning: An Introduction (2 nd edition). Cambridge: The MIT Press.
- Wright, L. 1973. "Functions." Philosophical Review 82: 139-168.
- Zimmerman, D. 2003. "That was then, this is now: Personal history v. psy- chological structure in compatibilist theories of autonomy." Noûs 37 (4): 638-671.
Cited by
Sorted by citations
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., Deane, G., Fleming, S. M., Frith, C., Ji, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08708 · Cited by 196
Self-citationUnderstanding artificial agency
Dung, L. (2025) · The Philosophical Quarterly · doi:10.1093/pq/pqae010 · Cited by 69
Reinforcement learning and artificial agency
Butlin, P. (2024) · Mind & Language · doi:10.1111/mila.12458 · Cited by 40
Self-citation
Metadata
- Authors
- Butlin, P.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Butlin, Patrick
- ORCID
- Butlin, Patrick
- Venue
- Croatian journal of philosophy
- Published
- 2022-12-27
- Volume
- 22
- Issue
- 66
- Pages
- 351-370
- ISSN
- 1333-1108 (ISSN-L)
- 1847-6139
- OpenAlex
- W4313412039
- Cited by
- 22
- References
- 5
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 15 · 2023: 1
- FWCI
- 4.37
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Croatian Journal of Philosophy (publishedVersion)
- University of Zagreb University Computing Centre (SRCE) (submittedVersion, other-oa)
- Topics
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Keywords
- machine learning, agency, reinforcement learning, supervised learning, instrumental values, functions, goals, artificial agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
