CoChat

Two Dimensions of Opacity and the Deep Learning Predicament

Open access
Boge, Florian J. · 2022 · Minds and Machines
Cited by 110

The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.

Abstract

Deep neural networks (DNNs) have become increasingly successful in applications from biology to cosmology to social science. Trained DNNs, moreover, correspond to models that ideally allow the prediction of new phenomena. Building in part on the literature on ‘eXplainable AI’ (XAI), I here argue that these models are instrumental in a sense that makes them non-explanatory, and that their automated generation is opaque in a unique way. This combination implies the possibility of an unprecedented gap between discovery and explanation: When unsupervised models are successfully used in exploratory contexts, scientists face a whole new challenge in forming the concepts required for understanding underlying mechanisms.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11023-021-09569-4

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 14 references

Research works printed but not linked: 90

  1. Albertsson, K., Altoe, P., Anderson, D., Andrews, M., Espinosa, J. P. A., Aurisano, A., Basara, L., Bevan, A., Bhimji, W., Bonacorsi, D., Calafiura, P., Campanelli, M., Capps, L., Carminati, F., Carrazza, S., Childers, T., Coniavitis, E., Cranmer, K., David, C., … Zapata, O. (2018). Machine learning in high energy physics community white paper. Journal of Physics: Conference Series, 1085(2), 022008.
  2. Alvarez Melis, D., & Jaakkola, T. (2018). Towards robust interpretability with self-explaining neural networks. Advances in Neural Information Processing Systems, 31, 7775–7784.
  3. Baldi, P., Sadowski, P., & Whiteson, D. (2014). Searching for exotic particles in high-energy physics with deep learning. Nature Communications, 5, 4308. · DOI · Paper page
  4. Basso, A., Lisciandra, C., & Marchionni, C. (2017). Hypothetical models in social science. In L. Magnani & T. Bertolotti (Eds.), Springer handbook of model-based science (pp. 413–433). Springer. · DOI
  5. Beisbart, C. (2012). How can computer simulations produce new knowledge? European Journal for Philosophy of Science, 2(3), 395–434. · DOI
  6. Boge, F. J. (2019a). How to infer explanations from computer simulations. Studies in History and Philosophy of Science Part A. https://doi.org/10.1016/j.shpsa.2019.12.003 · DOI
  7. Boge, F. J. (2019b). Why computer simulations are not inferences, and in what sense they are experiments. European Journal for Philosophy of Science, 9(1), 13. · DOI
  8. Boge, F. J., & Zeitnitz, C. (2020). Polycratic hierarchies and networks: What simulation-modeling at the LHC can teach us about the epistemology of simulation. Synthese. https://doi.org/10.1007/s11229-020-02667-3 · DOI
  9. Bokulich, A. (2011). How scientific models can explain. Synthese, 180(1), 33–45. · DOI · Paper page
  10. Bokulich, A. (2014). How the tiger bush got its stripes: ‘How possibly’ vs. ‘how actually’ model explanations’. The Monist, 97(3), 321–338. · doi:10.5840/monist201497321
  11. Buckner, C. (2019). Deep learning: A philosophical introduction. Philosophy Compass, 14(10), e12625. · DOI · Paper page
  12. Buckner, C. (2020). Understanding adversarial examples requires a theory of artefacts for deep learning. Nature Machine Intelligence, 2(12), 731–736. · DOI
  13. Buckner, C. J. (2021). Black boxes, or unflattering mirrors? Comparative bias in the science of machine behavior. The British Journal for the Philosophy of Science. https://doi.org/10.1086/714960 · DOI
  14. Card, D., Mas, A., & Rothstein, J. (2008). Tipping and the dynamics of segregation*. The Quarterly Journal of Economics, 123(1), 177–218. · DOI · Paper page
  15. Chang, S., Cohen, T., & Ostdiek, B. (2018). What is the machine learning? Physical Review D, 97(5), 6.
  16. Chattopadhyay, A., Hassanzadeh, P., & Pasha, S. (2020). Predicting clustered weather patterns: A test case for applications of convolutional neural networks to spatio-temporal climate data. Scientific Reports, 10(1), 1–13. · DOI · Paper page
  17. Chirimuuta, M. (2020). Prediction versus understanding in computationally enhanced neuroscience. Synthese. https://doi.org/10.1007/s11229-020-02713-0 · DOI
  18. Cho, A. (2017). Ai’s early proving ground: The hunt for new particles. Science, 357(6346), 20. · doi:10.1126/science.357.6346.20
  19. Clark, W. A. (1991). Residential preferences and neighborhood racial segregation: A test of the schelling segregation model. Demography, 28(1), 1–19. · DOI · Paper page
  20. Craver, C. F. (2006). When mechanistic models explain. Synthese, 153(3), 355–376. · DOI · Paper page
  21. Creel, K. A. (2020). Transparency in complex computational systems. Philosophy of Science, 87(4), 568–589. · DOI · Paper page
  22. Davies, P. L. (2014). Data analysis and approximate models. CRC Press. · doi:10.1201/b17146
  23. de Regt, H. (2017). Understanding scientific understanding. Oxford University Press. · DOI · Paper page
  24. Douglas, H. E. (2009). Reintroducing prediction to explanation. Philosophy of Science, 76(4), 444–463. · DOI · Paper page
  25. Duncan, A., & Janssen, M. (2019). Constructing quantum mechanics (Vol. 1). Oxford University Press. · DOI
  26. Durán, J. M. (2017). Varying the explanatory span: Scientific explanation for computer simulations. International Studies in the Philosophy of Science, 31(1), 27–45. · DOI
  27. Durán, J. M. (2018). Computer simulations in science and engineering. Springer Nature. · DOI · Paper page
  28. Durán, J. M. (2020). What is a simulation model? Minds and Machines, 30(3), 301–323. · DOI
  29. Farina, M., Nakai, Y., & Shih, D. (2020). Searching for new physics with deep autoencoders. Physical Review D, 101(7), 075021. · DOI · Paper page
  30. Franklin, L. R. (2005). Exploratory experiments. Philosophy of Science, 72(5), 888–899. · DOI · Paper page
  31. Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169(3), 593–613. · DOI
  32. Giere, R. (2006). Scientific perspectivism. University of Chicago Press. · DOI · Paper page
  33. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. The MIT Press.
  34. Grimm, S. R. (2010). The goal of explanation. Studies in History and Philosophy of Science Part A, 41(4), 337–344. · DOI
  35. Grüne-Yanoff, T. (2013). Appraising models nonrepresentationally. Philosophy of Science, 80(5), 850–861. · DOI
  36. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), 1–42. · DOI · Paper page
  37. Hacking, I. (1992). The self-vindication of the laboratory sciences. In A. Pickering (Ed.), Science as practice and culture (pp. 29–64). The University of Chicago Press.
  38. Harris, T. (2003). Data models and the acquisition and manipulation of data. Philosophy of Science, 70(5), 1508–1517. · doi:10.1086/377426
  39. Hasse, H., & Lenhard, J. (2017). Boon and bane: On the role of adjustable parameters in simulation models. In J. Lenhard & M. Carrier (Eds.), Mathematics as a tool (pp. 93–116). Springer. · DOI
  40. Hastie, T., Tibshirani, R., & Friedman, J. (2013). The elements of statistical learning: Data mining, inference, and prediction. Springer.
  41. Hornik, K., Stinchcombe, M., White, H., et al. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366. · DOI · Paper page
  42. Humphreys, P. (2004). Extending ourselves: Computational science, empiricism, and scientific method. Oxford University Press. · DOI · Paper page
  43. Humphreys, P. (2013). Data analysis: Models or techniques? Foundations of Science, 18(3), 579–581. · doi:10.1007/s10699-012-9317-4
  44. Karaca, K. (2013). The strong and weak senses of theory-ladenness of experimentation. Science in Context, 26(01), 93–136. · DOI
  45. Karaca, K. (2018). Lessons from the large hadron collider for model-based experimentation. Synthese, 195(12), 5431–5452. · DOI
  46. Kasabov, N. (2019). Time-space, spiking neural networks and brain-inspired artificial intelligence. Springer. · doi:10.1007/978-3-662-57715-8
  47. Khalifa, K. (2017). Understanding, explanation, and scientific knowledge. Cambridge University Press. · DOI · Paper page
  48. Lenhard, J. (2006). Surprised by a nanowire: Simulation, control, and understanding. Philosophy of Science, 73(5), 605–616. · doi:10.1086/518330
  49. Lenhard, J. (2007). Computer simulation: The cooperation between experimenting and modeling. Philosophy of Science, 74(2), 176–194. · DOI
  50. Lenhard, J. (2019). Calculated surprises. Oxford University Press. · DOI
  51. Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Modern Physics, 41(3), 253–262. · DOI · Paper page
  52. Leonelli, S. (2016). Data-centric biology. University of Chicago Press. · DOI · Paper page
  53. Leonelli, S. (2019). What distinguishes data from models? European Journal for Philosophy of Science, 9(2), 22. · DOI
  54. López-Rubio, E. (2020). Throwing light on black boxes: Emergence of visual categories from deep learning. Synthese. https://doi.org/10.1007/s11229-020-02700-5 · DOI
  55. Lyre, H. (2008). Does the Higgs mechanism exist? International Studies in the Philosophy of Science, 22(2), 119–133. · doi:10.1080/02698590802496664
  56. Lyre, H. (2020). The state space of artificial intelligence. Minds and Machines. https://doi.org/10.1007/s11023-020-09538-3 · DOI
  57. Maher, P. (1988). Prediction, accommodation, and the logic of discovery. Philosophy of Science, 1988(1), 273–285.
  58. Massimi, M. (2005). Pauli’s exclusion principle: The origin and validation of a scientific principle. Cambridge University Press. · DOI
  59. McCrea, R. (2009). Explaining sociospatial patterns in South East Queensland, Australia. Economy and Space, 41(9), 2201–2214.
  60. Minsky, M., & Papert, S. A. (1969). Perceptrons: An introduction to computational geometry. MIT Press.
  61. Morello, V., Barr, E., Bailes, M., Flynn, C., Keane, E., & van Straten, W. (2014). Spinn: A straightforward machine learning solution to the pulsar candidate selection problem. Monthly Notices of the Royal Astronomical Society, 443(2), 1651–1662. · DOI
  62. Morrison, M. (1999). Models as autonomous agents. In M. Morrison & M. S. Morgan (Eds.), Models as mediators (pp. 38–65). Cambridge University Press. · DOI · Paper page
  63. Morrison, M. (2015). Reconstructing reality: Models, mathematics, and simulations. Oxford University Press. · DOI · Paper page
  64. Morrison, M., & Morgan, M. S. (1999). Models as mediating instruments. In M. Morrison & M. S. Morgan (Eds.), Models as mediators (pp. 10–37). Cambridge University Press. · DOI · Paper page
  65. Napoletani, D., Panza, M., & Struppa, D. C. (2011). Agnostic science. Towards a philosophy of data analysis. Foundations of Science, 16(1), 1–20. · DOI
  66. Patterson, J., & Gibson, A. (2017). Deep learning: A practitioner’s approach. O’Reilly Media.
  67. Poggio, T., Banburski, A., & Liao, Q. (2020). Theoretical issues in deep networks. Proceedings of the National Academy of Sciences, 117(48), 30039–30045. · DOI · Paper page
  68. Potochnik, A. (2017). Idealization and the aims of science. University of Chicago Press. · DOI · Paper page
  69. Raghu, M., & Schmidt, E. (2020). A survey of deep learning for scientific discovery. arXiv:2003.11755 (arXiv preprint). · arXiv
  70. Reutlinger, A., Hangleiter, D., & Hartmann, S. (2018). Understanding (with) toy models. The British Journal for the Philosophy of Science, 69(4), 1069–1099. · DOI
  71. Saxe, A. M., Bansal, Y., Dapello, J., Advani, M., Kolchinsky, A., Tracey, B. D., & Cox, D. D. (2019). On the information bottleneck theory of deep learning. Journal of Statistical Mechanics, 2019(12), 124020. · DOI · Paper page
  72. Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1(2), 143–186. · DOI · Paper page
  73. Schwartz-Ziv, R., & Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv:1703.00810 (arXiv preprint). · arXiv
  74. Skansi, S. (2018). Introduction to deep learning: From logical calculus to artificial intelligence. Springer International Publishing. · DOI · Paper page
  75. Smeenk, C. (2006). The elusive Higgs mechanism. Philosophy of Science, 73(5), 487–499. · DOI
  76. Sober, E. (2002). Instrumentalism, parsimony, and the Akaike framework. Philosophy of Science, 69(S3), S112–S123. · DOI · Paper page
  77. Steinle, F. (1997). Entering new fields: Exploratory uses of experimentation. Philosophy of Science, 64, S65–S74. · DOI · Paper page
  78. Steinle, F. (2016). Exploratory experiments: Ampère, Faraday, and the origins of electrodynamics. University of Pittsburgh Press. · doi:10.2307/j.ctt1f89r1p
  79. Strevens, M. (2008). Depth: An account of scientific explanation. Harvard University Press.
  80. Strevens, M. (2013). No understanding without explanation. Studies in History and Philosophy of Science Part A, 44(3), 510–515. · DOI · Paper page
  81. Suárez, M., & Cartwright, N. (2008). Theories: Tools versus models. Studies in History and Philosophy of Modern Physics, 39(1), 62–81. · DOI
  82. Sullivan, E. (2019). Understanding from machine learning models. The British Journal for the Philosophy of Science. https://doi.org/10.1093/bjps/axz035 · DOI · Paper page
  83. Suthaharan, S. (2016). Machine learning models and algorithms for big data classification. Springer. · DOI · Paper page
  84. Voss, H. (2013). Classification. In O. Behnke, K. Kröninger, G. Schott, & T. Schörner-Sadenius (Eds.), Data analysis in high energy physics: A practical guide to statistical methods (pp. 153–186). Wiley. · DOI
  85. Wallace, D. (2020). On the plurality of quantum theories. In S. French & J. Saatsi (Eds.), Scientific realism and the quantum (pp. 78–102). Oxford University Press. · DOI
  86. Wilholt, T. (2005). Explaining models: Theoretical and phenomenological models and their role for the first explanation of the hydrogen spectrum. Foundations of Chemistry, 7(2), 149–169. · DOI
  87. Winsberg, E. (1999). The hierarchy of models in simulation. In L. Magnani, N. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 255–269). Kluwer Academic/Plenum Publishers. · doi:10.1007/978-1-4615-4813-3_16
  88. Winsberg, E. (2010). Science in the age of computer simulation. University of Chicago Press. · DOI · Paper page
  89. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82. · DOI · Paper page
  90. Worrall, J. (1985). Scientific discovery and theory-confirmation. In J. C. Pitt (Ed.), Change and progress in modern science (pp. 301–331). Springer. · DOI

Cited by

See all 110

Metadata

Authors
Boge, F. J.
Venue
Minds and Machines
Published
2021-09-03
Volume
32
Issue
1
Pages
43-75
ISSN
  • 0924-6495 (ISSN-L)
  • 1572-8641
OpenAlex
W3197302864
Cited by
110
References
14
FWCI
4.32
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
  • Deutsche Forschungsgemeinschaft (FOR 2063)
  • Bergische Universität Wuppertal

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Two Dimensions of Opacity and the Deep Learning Predicament. https://app.cochat.ai/papers/10.1007/s11023-021-09569-4
The paper
Boge, F. J. (2022). Two Dimensions of Opacity and the Deep Learning Predicament. Minds and Machines. https://doi.org/10.1007/s11023-021-09569-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.