The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A Principled Approach to Detecting Surprising Events in Video
Abstract
Primates demonstrate unparalleled ability at rapidly orienting towards important events in complex dynamic environments. During rapid guidance of attention and gaze towards potential objects of interest or threats, often there is no time for detailed visual analysis. Thus, heuristic computations are necessary to locate the most interesting events in quasi real-time. We present a new theory of sensory surprise, which provides a principled and computable shortcut to important information. We develop a model that computes instantaneous low-level surprise at every location in video streams. The algorithm significantly correlates with eye movements of two humans watching complex video clips, including television programs (17,936 frames, 2,152 saccadic gaze shifts). The system allows more sophisticated and time-consuming image analysis to be efficiently focused onto the most surprising subsets of the incoming data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Self-citationInformation theory and statistics
(1959) · Journal of the Franklin Institute · doi:10.1016/0016-0032(59)90368-0 · Cited by 7,094
Computational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733
Self-citationThe principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
What attributes guide the deployment of visual attention and how do they do it?
Wolfe, J. M., & Horowitz, T. S. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1411 · Cited by 1,766 · cited 1×
“Saliency models explain a wealth of psychophysical results on human visual search behavior [34, 37, 38], and have been shown to correlate with human eye movements over static images [25].”
BackgroundSoftky, W., & Koch, C. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-01-00334.1993 · Cited by 1,689
Novelty detection: a review—part 1: statistical approaches
Markou, M., & Singh, S. (2003) · Signal Processing · doi:10.1016/j.sigpro.2003.07.018 · Cited by 1,432
Modeling the role of salience in the allocation of overt visual attention
Parkhurst, D., Law, K., & Niebur, E. (2002) · Vision Research · doi:10.1016/s0042-6989(01)00250-4 · Cited by 1,388
Probability, Frequency and Reasonable Expectation
Cox, R. T. (1946) · American Journal of Physics · doi:10.1119/1.1990764 · Cited by 1,308
Modeling visual attention via selective tuning
Tsotsos, J. K., Culhane, S. M., Kei Wai, W. Y., Lai, Y., Davis, N., & Nuflo, F. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00025-9 · Cited by 1,143
Embree, L., & James, W. (1983) · Philosophy and Phenomenological Research · doi:10.2307/2107586 · Cited by 1,117
The Dynamic Representation of Scenes
Rensink, R. A. (2000) · Visual Cognition · doi:10.1080/135062800394667 · Cited by 1,078
The representation of visual salience in monkey parietal cortex
Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998) · Nature · doi:10.1038/35135 · Cited by 1,078
Henderson, J. M., & Hollingworth, A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.243 · Cited by 967
Information Theory and Statistics.
Lindley, D. V., & Kullback, S. (1959) · Journal of the American Statistical Association · doi:10.2307/2282528 · Cited by 960
Image Segmentation in Video Sequences: A Probabilistic Approach
Friedman, N., & Russell, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1539 · Cited by 833
Visual cortical mechanisms detecting focal orientation discontinuities
Slllito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davls, J. (1995) · Nature · doi:10.1038/378492a0 · Cited by 649
Scanpaths in Eye Movements during Pattern Perception
Noton, D., & Stark, L. (1971) · Science · doi:10.1126/science.171.3968.308 · Cited by 645
Algorithms for defining visual regions-of-interest: comparison with eye fixations
Privitera, C. M., & Stark, L. W. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.877520 · Cited by 535
Using adaptive tracking to classify and monitor activities in a site
Grimson, W. E. L., Stauffer, C., Romano, R., & Lee, L. (2002) · Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231) · doi:10.1109/cvpr.1998.698583 · Cited by 504
Rapid Adaptation in Visual Cortex to the Structure of Images
Müller, J. R., Metha, A. B., Krauskopf, J., & Lennie, P. (1999) · Science · doi:10.1126/science.285.5432.1405 · Cited by 457
Novelty Detection and Neural Network Validation
Bishop, C. M. (1993) · ICANN ’93 · doi:10.1007/978-1-4471-2063-6_225 · Cited by 436
Natural scene statistics at the centre of gaze
Reinagel, P., & Zador, A. M. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x/10/4/304 · Cited by 434
Novelty detection and neural network validation
Bishop, C. M. (1994) · IEE Proceedings - Vision, Image, and Signal Processing · doi:10.1049/ip-vis:19941330 · Cited by 394
Fundamentals of Statistical Exponential Families
Brown, L. D. (1987) · Cited by 386
Saccade Target Selection in the Superior Colliculus During a Visual Search Task
McPeek, R. M., & Keller, E. L. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.4.2019 · Cited by 355
Novelty detection for the identification of masses in mammograms
Tarassenko, L. (1995) · 4th International Conference on Artificial Neural Networks · doi:10.1049/cp:19950597 · Cited by 350
Neural Correlate of Perceptual Adaptation to Gratings
Maffei, L., Fiorentini, A., & Bisti, S. (1973) · Science · doi:10.1126/science.182.4116.1036 · Cited by 338
Effects of similarity and history on neural mechanisms of visual selection
Bichot, N. P., & Schall, J. D. (1999) · Nature Neuroscience · doi:10.1038/9205 · Cited by 302
Goal-Related Activity in V4 during Free Viewing Visual Search
Mazer, J. A., & Gallant, J. L. (2003) · Neuron · doi:10.1016/s0896-6273(03)00764-5 · Cited by 262
Contextual influences in V1 as a basis for pop out and asymmetry in visual search
Li, Z. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.18.10530 · Cited by 198
Neural Correlates of the Automatic and Goal-Driven Biases in Orienting Spatial Attention
Fecteau, J. H., Bell, A. H., & Munoz, D. P. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00184.2004 · Cited by 166
Detecting Novel Features of an Environment Using Habituation
Marsland, S., Nehmzow, U., & Shapiro, J. (2000) · The MIT Press eBooks · doi:10.7551/mitpress/3120.003.0021 · Cited by 36
Computational resources do constrain behavior
Tsotsos, J. K. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00071053 · Cited by 13
Cited by
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
Self-citationSUN: A Bayesian framework for saliency using natural statistics
Zhang, L., Tong, M. H., Marks, T. K., Shan, H., & Cottrell, G. W. (2008) · Journal of Vision · doi:10.1167/8.7.32 · Cited by 1,227
Visual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
Learning to Detect A Salient Object
Liu, T., Sun, J., Zheng, N.-N., Tang, X., & Shum, H.-Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383047 · Cited by 946
Guo, C., & Zhang, L. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2030969 · Cited by 934
Metadata
- Authors
- Itti, L., & Baldi, P.
- Venue
- 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
- Publisher
- IEEE
- Published
- 2005-07-27
- Volume
- 1
- Pages
- 631-637
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2170610624
- Cited by
- 376
- References
- 39
- FWCI
- 16.41
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.