The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Of bits and wows: A Bayesian theory of surprise with applications to attention
Open accessAbstract
The amount of information contained in a piece of data can be measured by the effect this data has on its observer. Fundamentally, this effect is to transform the observer's prior beliefs into posterior beliefs, according to Bayes theorem. Thus the amount of information can be measured in a natural way by the distance (relative entropy) between the prior and posterior distributions of the observer over the available space of hypotheses. This facet of information, termed “surprise”, is important in dynamic situations where beliefs change, in particular during learning and adaptation. Surprise can often be computed analytically, for instance in the case of distributions from the exponential family, or it can be numerically approximated. During sequential Bayesian learning, surprise decreases like the inverse of the number of training examples. Theoretical properties of surprise are discussed, in particular how it differs and complements Shannon's definition of information. A computer vision neural network architecture is then presented capable of computing surprise over images and video stimuli. Hypothesizing that surprising data ought to attract natural or artificial attention systems, the output of this architecture is used in a psychophysical experiment to analyze human eye movements in the presence of natural video stimuli. Surprise is found to yield robust performance at predicting human gaze (ROC-like ordinal dominance score ∼ 0.7 compared to ∼ 0.8 for human inter-observer repeatability, ∼ 0.6 for simpler intensity contrast-based predictor, and 0.5 for chance). The resulting theory of surprise is applicable across different spatio-temporal scales, modalities, and levels of abstraction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
John Lu, Z. Q. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,390
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Self-citationTable of Integrals, Series, and Products.
Kolbig, K. S., Gradshteyn, I. S., Ryzhik, I. M., Jeffrey, A., & Scripta Technica, I. (1995) · Mathematics of Computation · doi:10.2307/2153347 · Cited by 9,550
Table of Integrals, Series and Products.
L., Y., Gradshteyn, I. S., & Ryzhik, I. M. (1981) · Mathematics of Computation · doi:10.2307/2007757 · Cited by 8,436
Cited by
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365
When the world becomes ‘too real’: a Bayesian explanation of autistic perception
Pellicano, E., & Burr, D. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.009 · Cited by 1,156
Arousal-Biased Competition in Perception and Memory
Mather, M., & Sutherland, M. R. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611400234 · Cited by 926
SanMiguel, I., Saupe, K., & Schröger, E. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00407 · Cited by 89
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68
Metadata
- Authors
- Baldi, P., & Itti, L.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2010-06
- Volume
- 23
- Issue
- 5
- Pages
- 649-666
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2019515298
- Cited by
- 302
- References
- 61
- FWCI
- 7.33
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Of bits and wows: A Bayesian theory of surprise with applications to attention. https://app.cochat.ai/papers/10.1016/j.neunet.2009.12.007
- The paper
- Baldi, P., & Itti, L. (2010). Of bits and wows: A Bayesian theory of surprise with applications to attention. Neural Networks. https://doi.org/10.1016/j.neunet.2009.12.007
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.