Understanding from Machine Learning Models
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Simple idealized models seem to provide more understanding than opaque, complex, and hyper-realistic models. However, an increasing number of scientists are going in the opposite direction by utilizing opaque machine learning models to make predictions and draw inferences, suggesting that scientists are opting for models that have less potential for understanding. Are scientists trading understanding for some other epistemic or pragmatic good when they choose a machine learning model? Or are the assumptions behind why minimal models provide understanding misguided? In this article, using the case of deep neural networks, I argue that it is not the complexity or black box nature of a model that limits how much understanding the model provides. Instead, it is a lack of scientific and empirical evidence supporting the link that connects a model to the target phenomenon that primarily prohibits understanding.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“2 The type of opacity I am concerned with is close to Burrell’s ([2016]) use of opacity in regards to the way algorithms “operate at the scale of application”.”
BackgroundHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · cited 1×
“14 That said, even in the simple handwritten number classifier case, computational neuroscientists seek answers to contrastive explanatory questions between the way machines learn versus humans (Lake et al. [2015]).”
BackgroundThe information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626
Deep learning for visual understanding: A review
Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710 · cited 2×
“For example, they did not seek predictions of HIV because of the large behavioural aspect to the disease (Miotto et al. [2016], p.”
BackgroundShickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2767063 · Cited by 1,688
Inductive Risk and Values in Science
Douglas, H. (2000) · Philosophy of Science · doi:10.1086/392855 · Cited by 885
Wang, Y., & Kosinski, M. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000098 · Cited by 766 · cited 1×
“Researchers developed a facial recognition model that seeks to identify the sexual orientation of individuals (Wang and Kosinski [2018]).”
BackgroundThe philosophical novelty of computer simulation methods
Humphreys, P. (2009) · Synthese · doi:10.1007/s11229-008-9435-2 · Cited by 362
Empiricism without magic: transformational abstraction in deep convolutional neural networks
Buckner, C. (2018) · Synthese · doi:10.1007/s11229-018-01949-1 · Cited by 149 · cited 1×
“See Buckner ([2018]) for a philosophical argument that DNNs (CNNs in particular) capture processes of the brain.”
Background
Research works printed but not linked: 45
- Abelson H., [1996]: Structure and Interpretation of Computer Programs
- Bailer-Jones D. M., Simulated Annealing, and Genetic Algorithms, 65
- Baldi P., Advances in Neural Information Processing Systems, 2834
- 10.1086/676677 · DOI · Paper page
- Bobo L., 1996, Social Forces, 74, 883, 10.2307/2580385 · DOI
- Bokulich A., Reexamining the Quantum–Classical Relation, 10.1017/CBO9780511751813 · DOI
- Clark W. A. V., 1991, Demography, 28, 1, 10.2307/2061333 · DOI · Paper page
- Clark W. A. V., 1992, Demography, 29, 451, 10.2307/2061828 · DOI · Paper page
- Cristianini N., 2010, Neural Networks, 23, 466, 10.1016/j.neunet.2010.01.006 · doi:10.1016/j.neunet.2010.01.006
- De Regt H. W., Understanding Scientific Understanding, 10.1093/oso/9780190652913.001.0001 · DOI · Paper page
- Elliott K. C., 2013, Studies in History and Philosophy of Science Part A, 44, 375, 10.1016/j.shpsa.2013.06.003 · DOI
- Glorot X., 2011, Proceedings of Machine Learning Research, 15, 315
- Goodfellow I., [2016]: Deep Learning
- Grimm S., 2010, Studies in History and Philosophy of Science A, 41, 337, 10.1016/j.shpsa.2010.10.006 · DOI
- Grimm S., Virtue Epistemology Naturalized: Bridges between Virtue Epistemology and Philosophy of Science, 45
- Grüne-Yanoff T., 2009, Erkenntnis, 70, 81, 10.1007/s10670-008-9138-6 · DOI
- Hills A., 2016, Noûs, 50, 661, 10.1111/nous.12092 · DOI · Paper page
- Humphreys P., Extending Ourselves: Computational Science, Empiricism, and Scientific Method
- Khalifa K., Understanding, Explanation, and Scientific Knowledge, 10.1017/9781108164276 · DOI · Paper page
- Kuorikoski J., 2015, Synthese, 192, 3817, 10.1007/s11229-014-0591-2 · DOI · Paper page
- Lawler I., Synthese, 21
- LeVay S., Queer Science: The Use and Abuse of Research into Homosexuality, 10.7551/mitpress/5726.001.0001 · doi:10.7551/mitpress/5726.001.0001
- LeVay S., Gay, Straight, and the Reason Why: The Science of Sexual Orientation
- Lipton P., H. W. de Regt, 63
- Mäki U., 2009, Erkenntnis, 70, 29, 10.1007/s10670-008-9135-9 · DOI · Paper page
- Miller A. E., 2018, Psychology and Sexuality, 9, 188, 10.1080/19419899.2018.1468353 · DOI
- Mitchell T. M., Machine Learning, 10.1007/978-1-4613-2279-5 · DOI
- 10.1086/663236 · DOI
- Mustanski B. S., 2002, Annual Review of Sex Research, 13, 89
- Potochnik A., 2011, European Journal for Philosophy of Science, 1, 29, 10.1007/s13194-010-0002-6 · DOI
- Potochnik A., 2015, Studies in History and Philosophy of Science Part A, 53, 71, 10.1016/j.shpsa.2015.05.008 · DOI
- Reutlinger A., 2017, British Journal for the Philosophy of Science, 69, 1069, 10.1093/bjps/axx005 · DOI
- Ribeiro M. T., Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York: ACM, 44
- 10.1086/671399 · DOI
- Schelling T. C., 1971, The Journal of Mathematical Sociology, 1, 143, 10.1080/0022250X.1971.9989794 · DOI · Paper page
- Strevens M., Depth: An Account of Scientific Explanation, 10.2307/j.ctv1dv0tnw · doi:10.2307/j.ctv1dv0tnw
- Strevens M., Explanation and Integration in Mind and Brain Science
- Sullivan E., 2018, Philosophical Studies, 175, 221, 10.1007/s11098-017-0863-z · DOI
- Sullivan E., 2019, Australasian Journal of Philosophy, 97, 673, 10.1080/00048402.2018.1564337 · DOI
- Tishby N., Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, Urbana-Champaign, IL: University of Illinois, 77
- Tishby N., 2015, IEEE Information Theory Workshop 2015
- Valentova J. V., 2014, Archives of Sexual Behavior, 43, 353, 10.1007/s10508-013-0194-x · DOI
- van Riel R., 2015, Synthese, 192, 3839, 10.1007/s11229-015-0728-y · DOI
- Wan L., 2013, Proceedings of Machine Learning Research, 28, 1058
- Winsberg E., 2012, Kennedy Institute of Ethics Journal, 22, 111, 10.1353/ken.2012.0008 · DOI · Paper page
Cited by
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136
“Clinicians make diagnostic decisions by considering and weighing a set of features, data points and clinical markers [74]; see also the example in §5.”
BackgroundBeyond generalization: a theory of robustness in machine learning
Freiesleben, T., & Grote, T. (2023) · Synthese · doi:10.1007/s11229-023-04334-9 · Cited by 106
“5 Secondly, the opacity of ML models (Boge, 2022; Creel, 2020; Sullivan, 2022) makes it challenging to achieve a degree of transparency, necessary for detecting common model properties.”
BackgroundConceptual challenges for interpretable machine learning
Watson, D. S. (2022) · Synthese · doi:10.1007/s11229-022-03485-5 · Cited by 87
Metadata
- Authors
- Sullivan, E.
- Venue
- The British Journal for the Philosophy of Science
- Publisher
- University of Chicago Press
- Published
- 2022-03-01
- Volume
- 73
- Issue
- 1
- Pages
- 109-133
- ISSN
- 0007-0882 (ISSN-L)
- 1464-3537
- OpenAlex
- W2965860133
- Cited by
- 221
- References
- 13
- FWCI
- 49.50
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.