Deep neural network models of sound localization reveal how perception is adapted to real-world environments
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Mammals localize sounds using information from their two ears. Localization in real-world conditions is challenging, as echoes provide erroneous information and noises mask parts of target sounds. To better understand real-world localization, we equipped a deep neural network with human ears and trained it to localize sounds in a virtual environment. The resulting model localized accurately in realistic conditions with noise and reverberation. In simulated experiments, the model exhibited many features of human spatial hearing: sensitivity to monaural spectral cues and interaural time and level differences, integration across frequency, biases for sound onsets and limits on localization of concurrent sources. But when trained in unnatural environments without reverberation, noise or natural sounds, these performance characteristics deviated from those of humans. The results show how biological hearing is adapted to the challenges of real-world environments and illustrate how artificial neural networks can reveal the real-world constraints that shape perception. Francl and McDermott use deep neural networks to reveal the behavioural phenotype of systems optimized for tasks in simulated environments, showing that many characteristics of human sound localization are adapted to real-world environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple emitter location and signal parameter estimation
Schmidt, R. (1986) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.1986.1143830 · Cited by 14,718 · cited 1×
“These comprised: steered-response power phase transform (SRP) 65 , multiple signal classification (MUSIC) 64 , the coherent signal-subspace method (CSSM) 63 , weighted average of signal subspaces (WAVES) 66 , test of orthogonality of projected subspaces (TOPS) 67 and the WavLoc neural network 68 .”
MethodLarge-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · cited 2×
“Next, we trained each architecture (that is, optimized the weights of the convolutional and fully connected layers) using mini-batch SGD for 15,000 steps, each with a batch size of 16, for a total of 240,000 unique training examples, randomly drawn from the training set described above.”
MethodBuilding machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Using goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Motion illusions as optimal percepts
Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002) · Nature Neuroscience · doi:10.1038/nn0602-858 · Cited by 1,076
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716
Cited by
Computational bioacoustics with deep learning: a review and roadmap
Stowell, D. (2022) · PeerJ · doi:10.7717/peerj.13152 · Cited by 471
Goldstein, A., Grinstein-Dabush, A., Schain, M., Wang, H., Hong, Z., Aubrey, B., Nastase, S. A., Zada, Z., Ham, E., Feder, A., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-46631-y · Cited by 64
Model metamers reveal divergent invariances between biological and artificial neural networks
Feather, J., Leclerc, G., Mądry, A., & McDermott, J. H. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01442-0 · Cited by 59
“Models optimized for object recognition 78 , speech recognition 4 , sound localization 79 and pitch recognition 80 all exhibit qualitative and often quantitative similarities to human judgments when run in traditional psychophysical experiments with natural or relatively naturalistic stimuli (that…”
BackgroundSelf-citationThe Umwelt Representation Hypothesis: rethinking Universality
Bosch, V., Sommers, R. P., Doerig, A., & Kietzmann, T. C. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2026.07.004 · Cited by 2
Metadata
- Authors
- Francl, A., & McDermott, J. H.
- Venue
- Nature Human Behaviour
- Published
- 2022-01-27
- Volume
- 6
- Issue
- 1
- Pages
- 111-133
- ISSN
- 2397-3374 (ISSN-L)
- OpenAlex
- W4226064907
- Cited by
- 92
- References
- 15
- FWCI
- 8.63
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep neural network models of sound localization reveal how perception is adapted to real-world environments. https://app.cochat.ai/papers/10.1038/s41562-021-01244-z
- The paper
- Francl, A., & McDermott, J. H. (2022). Deep neural network models of sound localization reveal how perception is adapted to real-world environments. Nature Human Behaviour. https://doi.org/10.1038/s41562-021-01244-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.