The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Predicting the Driver's Focus of Attention: The DR(eye)VE Project
Open accessAbstract
In this work we aim to predict the driver's focus of attention. The goal is to estimate what a person would pay attention to while driving, and which part of the scene around the vehicle is more critical for the task. To this end we propose a new computer vision model based on a multi-branch deep architecture that integrates three sources of information: raw video, motion and scene semantics. We also introduce DR(eye)VE, the largest dataset of driving scenes for which eye-tracking annotations are available. This dataset features more than 500,000 registered frames, matching ego-centric views (from glasses worn by drivers) and car-centric views (from roof-mounted camera), further enriched by other sensors measurements. Results highlight that several attention patterns are shared across drivers and can be reproduced to some extent. The indication of which elements in the scene are likely to capture the driver's attention may benefit several applications in the context of human-vehicle interaction and driver attention analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Frequency-tuned salient region detection
Achanta, R., Hemami, S., Estrada, F., & Susstrunk, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206596 · Cited by 4,167
Global contrast based salient region detection
Cheng, M.-M., Zhang, G.-X., Mitra, N. J., Huang, X., & Hu, S.-M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995344 · Cited by 2,976
Global Contrast Based Salient Region Detection
Cheng, M.-M., Mitra, N. J., Huang, X., Torr, P. H. S., & Hu, S.-M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345401 · Cited by 2,463
Guided search: An alternative to the feature integration model for visual search.
Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.15.3.419 · Cited by 2,147
Understanding Convolution for Semantic Segmentation
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., & Cottrell, G. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00163 · Cited by 1,957
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.766 · Cited by 1,639
Context-Aware Saliency Detection
Goferman, S., Zelnik-Manor, L., & Tal, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.272 · Cited by 1,567
Inhibition of return: Neural basis and function
Posner, M. I., Rafal, R. D., Choate, L. S., & Vaughan, J. (1985) · Cognitive Neuropsychology · doi:10.1080/02643298508252866 · Cited by 1,441
Human gaze control during real-world scene perception
Henderson, J. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.09.006 · Cited by 1,404
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,386
Top–down and bottom–up control of visual selection
Theeuwes, J. (2010) · Acta Psychologica · doi:10.1016/j.actpsy.2010.02.006 · Cited by 1,385
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129
Tatler, B. W. (2007) · Journal of Vision · doi:10.1167/7.14.4 · Cited by 1,028
Contrast-based image attention analysis by using fuzzy growing
Ma, Y.-F., & Zhang, H.-J. (2003) · Proceedings of the eleventh ACM international conference on Multimedia · doi:10.1145/957013.957094 · Cited by 965
Visual attention detection in video sequences using spatiotemporal cues
Zhai, Y., & Shah, M. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180824 · Cited by 959
What Do Different Evaluation Metrics Tell Us About Saliency Models?
Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., & Durand, F. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2815601 · Cited by 802
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
(2015) · Cited by 656
SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks
Huang, X., Shen, C., Boix, X., & Zhao, Q. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.38 · Cited by 597
Saliency-aware geodesic video object segmentation
Wang, W., Shen, J., & Porikli, F. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298961 · Cited by 524
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Pan, P., Xu, Z., Yang, Y., Wu, F., & Zhuang, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.117 · Cited by 412
Learning-based approach for online lane change intention prediction
Kumar, P., Perrollaz, M., Lefevre, S., & Laugier, C. (2013) · 2013 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2013.6629564 · Cited by 410
Computational visual attention systems and their cognitive foundations: A survey
Frintrop, S., Rome, E., & Christensen, H. I. (2010) · ACM Transactions on Applied Perception · doi:10.1145/1658349.1658355 · Cited by 389
Driver Gaze Tracking and Eyes Off the Road Detection System
Vicente, F., Huang, Z., Xiong, X., De la Torre, F., Zhang, W., & Levi, D. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2396031 · Cited by 362
A deep multi-level network for saliency prediction
Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900174 · Cited by 360
Self-citationOn the plausibility of the discriminant center-surround hypothesis for visual saliency
Gao, D., Mahadevan, V., & Vasconcelos, N. (2008) · Journal of Vision · doi:10.1167/8.7.13 · Cited by 278
Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models
Jain, A., Koppula, H. S., Raghavan, B., Soh, S., & Saxena, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.364 · Cited by 274
The contributions of central versus peripheral vision to scene gist recognition
Larson, A. M., & Loschky, L. C. (2009) · Journal of Vision · doi:10.1167/9.10.6 · Cited by 270
Peters, R. J., & Itti, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383337 · Cited by 244
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
Kümmerer, M., Theis, L., & Bethge, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1045 · Cited by 231
Lane change intent prediction for driver assistance: On-road design and evaluation
Morris, B., Doshi, A., & Trivedi, M. (2011) · 2011 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2011.5940538 · Cited by 228
Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition
Mathe, S., & Sminchisescu, C. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366154 · Cited by 217
Semantic Video Segmentation by Gated Recurrent Flow Propagation
Nilsson, D., & Sminchisescu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00713 · Cited by 204
Hierarchical Boundary-Aware Neural Encoder for Video Captioning
Baraldi, L., Grana, C., & Cucchiara, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.339 · Cited by 199
Self-citationVideo Saliency Incorporating Spatiotemporal Cues and Uncertainty Weighting
Fang, Y., Wang, Z., Lin, W., & Fang, Z. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2336549 · Cited by 193
POSEidon: Face-from-Depth for Driver Pose Estimation
Borghi, G., Venturelli, M., Vezzani, R., & Cucchiara, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.583 · Cited by 174
Self-citationLearning Video Saliency from Human Gaze Using Candidate Selection
Rudoy, D., Goldman, D. B., Shechtman, E., & Zelnik-Manor, L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.152 · Cited by 159
Information-theoretic model comparison unifies saliency metrics
Kümmerer, M., Wallis, T. S. A., & Bethge, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510393112 · Cited by 158
Recurrent Mixture Density Network for Spatiotemporal Visual AttentionPreprint
Bazzani, L., Larochelle, H., & Torresani, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08199 · Cited by 139
DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving
Alletto, S., Palazzi, A., Solera, F., Calderara, S., & Cucchiara, R. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.14 · Cited by 137
Self-citationWhat/Where to Look Next? Modeling Top-Down Visual Attention in Complex Interactive Environments
Borji, A., Sihite, D. N., & Itti, L. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2013.2279715 · Cited by 122
Looking at Faces: Local and Global Aspects of Scanpaths
Groner, R., Walder, F., & Groner, M. T. (1984) · Advances in psychology · doi:10.1016/s0166-4115(08)61874-9 · Cited by 121
Rogé, J., Pébayle, T., Lambilliotte, E., Spitzenstetter, F., Giselbrecht, D., & Muzet, A. G. (2004) · Vision Research · doi:10.1016/j.visres.2004.05.026 · Cited by 120
A Bayesian model for efficient visual search and recognition
Elazary, L., & Itti, L. (2010) · Vision Research · doi:10.1016/j.visres.2010.01.002 · Cited by 114
Applying computational tools to predict gaze direction in interactive visual environments
Peters, R. J., & Itti, L. (2008) · ACM Transactions on Applied Perception · doi:10.1145/1279920.1279923 · Cited by 106
Fixation sequences made during visual examination of briefly presented 2D images
Mannan, S., Ruddock, K. H., & Wooding, D. S. (1997) · Spatial Vision · doi:10.1163/156856897x00177 · Cited by 98
Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos
Tawari, A., & Trivedi, M. M. (2014) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2014.6856607 · Cited by 97
How Much of Driving Is Preattentive?
Pugeault, N., & Bowden, R. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2487826 · Cited by 50
Central and peripheral vision for scene recognition: A neurocomputational modeling exploration
Wang, P., & Cottrell, G. W. (2017) · Journal of Vision · doi:10.48550/arxiv.1705.00816 · Cited by 47
3D Bounding Box Estimation Using Deep Learning and GeometryPreprint
Mousavian, A., Anguelov, D., Flynn, J., & Košecká, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00496 · Cited by 45
Underwood, G., Humphrey, K., & van Loon, E. M. (2011) · Vision Research · doi:10.1016/j.visres.2011.07.020 · Cited by 39
Will this car change the lane? - Turn signal recognition in the frequency domain
Fröhlich, B., Enzweiler, M., & Franke, U. (2014) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2014.6856477 · Cited by 38
Learning where to attend like a human driver
Palazzi, A., Solera, F., Calderara, S., Alletto, S., & Cucchiara, R. (2016) · 2017 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2017.7995833 · Cited by 35
Self-citationFormal Modelling of Salience and Cognitive Load
Rukšėnas, R., Back, J., Curzon, P., & BLANDFORD, A. E. (2008) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2008.03.107 · Cited by 27
Semantic Video Segmentation by Gated Recurrent Flow PropagationPreprint
Nilsson, D., & Sminchisescu, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08871 · Cited by 21
Vanishing point attracts gaze in free-viewing and visual search tasks
Borji, A., Feng, M., & Lu, H. (2016) · Journal of Vision · doi:10.1167/16.14.18 · Cited by 16
Where we look when we drive: A multidisciplinary approach
Brémond, R., Auberlet, J., Cavallo, V., Désiré, L., Faure, V., Lemonnier, S., Lobjois, R., & Tarel, J. (2014) · HAL · Cited by 10
Eye Movements Converge on Vanishing Points during Visual Search
Ueda, Y., Kamakura, Y., & Saiki, J. (2017) · Japanese Psychological Research · doi:10.1111/jpr.12144 · Cited by 8
Guiding optical flow estimation using superpixels
Gkamas, T., & Nikou, C. (2011) · IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1109/icdsp.2011.6004871 · Cited by 7
Where Should You Attend While DrivingPreprint
Palazzi, A., Solera, F., Calderara, S., Alletto, S., & Cucchiara, R. (2016) · arXiv (Cornell University) · Cited by 7
Self-citation
Cited by
Latent Space Autoregression for Novelty Detection
Abati, D., Porrello, A., Calderara, S., & Cucchiara, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00057 · Cited by 508
Self-citationA Review of Sensor Technologies for Perception in Automated Driving
Marti, E., de Miguel, M. A., Garcia, F., & Perez, J. (2019) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2019.2907630 · Cited by 315
Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach
Xu, M., Song, Y., Wang, J., Qiao, M., Huo, L., & Wang, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858783 · Cited by 259
Explainable Object-Induced Action Decision for Autonomous Vehicles
Xu, Y., Yang, X., Gong, L., Lin, H.-C., Wu, T.-Y., Li, Y., & Vasconcelos, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00954 · Cited by 129
Human Gaze Assisted Artificial Intelligence: A Review
Zhang, R., Saran, A., Liu, B., Zhu, Y., Guo, S., Niekum, S., Ballard, D., & Hayhoe, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/689 · Cited by 48
Metadata
- Authors
- Palazzi, A., Abati, D., Calderara, S., Solera, F., & Cucchiara, R.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-07-01
- Volume
- 41
- Issue
- 7
- Pages
- 1720-1733
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2612448678
- Cited by
- 313
- References
- 70
- FWCI
- 13.58
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.