The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
3D Traffic Scene Understanding From Movable Platforms
Abstract
In this paper, we present a novel probabilistic generative model for multi-object traffic scene understanding from movable platforms which reasons jointly about the 3D scene layout as well as the location and orientation of objects in the scene. In particular, the scene topology, geometry, and traffic activities are inferred from short video sequences. Inspired by the impressive driving capabilities of humans, our model does not rely on GPS, lidar, or map knowledge. Instead, it takes advantage of a diverse set of visual cues in the form of vehicle tracklets, vanishing points, semantic scene labels, scene flow, and occupancy grids. For each of these cues, we propose likelihood functions that are integrated into a probabilistic generative model. We learn all model parameters from training data using contrastive divergence. Experiments conducted on videos of 113 representative intersections show that our approach successfully infers the correct layout in a variety of very challenging scenarios. To evaluate the importance of each feature cue, experiments using different feature combinations are conducted. Furthermore, we show how by employing context derived from the proposed method we are able to improve over the state-of-the-art in terms of object detection and object orientation estimation in challenging and cluttered urban environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Self-citationThe Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
de Boor, C. (1978) · Applied Mathematical Sciences · doi:10.1007/978-1-4612-6333-3 · Cited by 11,944
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Thrun, S. (2002) · Communications of the ACM · doi:10.1145/504729.504754 · Cited by 7,994
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 4×
“For more details and derivations we refer the interested reader to [28], [37].”
BackgroundMethodResultInfluentialAn introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451
Make3D: Learning 3D Scene Structure from a Single Still Image
Saxena, A., Sun, M., & Ng, A. Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.132 · Cited by 1,736
StereoScan: Dense 3d reconstruction in real-time
Geiger, A., Ziegler, J., & Stiller, C. (2011) · 2011 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2011.5940405 · Cited by 1,089
Self-citationVideo-Based Lane Estimation and Tracking for Driver Assistance: Survey, System, and Evaluation
McCall, J. C., & Trivedi, M. M. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.869595 · Cited by 1,028
Putting Objects in Perspective
Hoiem, D., Efros, A. A., & Hebert, M. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0137-5 · Cited by 805
MCMC-based particle filtering for tracking a variable number of interacting targets
Khan, Z., Balch, T., & Dellaert, F. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.223 · Cited by 771
Efficient Large-Scale Stereo Matching
Geiger, A., Roser, M., & Urtasun, R. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-19315-6_3 · Cited by 761
Self-citationReal time detection of lane markers in urban streets
Aly, M. (2008) · 2008 IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2008.4621152 · Cited by 678
Recovering Surface Layout from an Image
Hoiem, D., Efros, A. A., & Hebert, M. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0031-y · Cited by 674
3-D Depth Reconstruction from a Single Still Image
Saxena, A., Chung, S. H., & Ng, A. Y. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0071-y · Cited by 600
Real time Detection of Lane Markers in Urban Streets
Aly, M. (2008) · Cited by 592
Sharing features: efficient boosting procedures for multiclass object detection
Torralba, A., Murphy, K. P., & Freeman, W. T. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315241 · Cited by 577
Recursive 3-D road and relative ego-state recognition
Dickmanns, E. D., & Mysliwetz, B. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121789 · Cited by 553
Robust tracking-by-detection using a detector confidence particle filter
Breitenstein, M. D., Reichlin, F., Leibe, B., Koller-Meier, E., & Van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459278 · Cited by 545
Wang, X., Ma, X., & Grimson, W. E. L. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.87 · Cited by 534
A survey of video processing techniques for traffic applications
Kastrinaki, V, Zervakis, M., & Kalaitzakis, K. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00004-0 · Cited by 527
Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle
Gavrila, D. M., & Munder, S. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9038-7 · Cited by 524
Monocular 3D pose estimation and tracking by detection
Andriluka, M., Roth, S., & Schiele, B. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540156 · Cited by 508
Who are you with and where are you going?
Yamaguchi, K., Berg, A. C., Ortiz, L. E., & Berg, T. L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995468 · Cited by 484
Unsupervised Activity Perception by Hierarchical Bayesian Models
Wang, X., Ma, X., & Grimson, E. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383072 · Cited by 359
Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles
Leibe, B., Schindler, K., Cornelis, N., & Van Gool, L. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.170 · Cited by 354
A Unified Framework for Multi-target Tracking and Collective Activity Recognition
Choi, W., & Savarese, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_16 · Cited by 351
The DARPA Urban Challenge: Autonomous Vehicles in City Traffic
(2009) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-03991-1 · Cited by 341
Self-supervised Monocular Road Detection in Desert Terrain
Dahlkamp, H., Kaehler, A., Stavens, D., Thrun, S., & Bradski, G. (2006) · Robotics: Science and Systems II · doi:10.15607/rss.2006.ii.005 · Cited by 329
RALPH: rapidly adapting lateral position handler
Pomerleau, D. (2002) · Proceedings of the Intelligent Vehicles '95. Symposium · doi:10.1109/ivs.1995.528333 · Cited by 309
Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics
Gupta, A., Efros, A. A., & Hebert, M. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_35 · Cited by 308
Košecká, J., & Zhang, W. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_32 · Cited by 293
Discriminative models for multi-class object layout
Desai, C., Ramanan, D., & Fowlkes, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459256 · Cited by 274
A Bayesian, Exemplar-Based Approach to Hierarchical Shape Matching
Gavrila, D. M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1062 · Cited by 268
Finding and tracking people from the bottom up
Ramanan, D., & Forsyth, D. A. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211504 · Cited by 268
Robust Multiperson Tracking from a Mobile Platform
Ess, A., Leibe, B., Schindler, K., & van Gool, L. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.109 · Cited by 251
SCARF: a color vision system that tracks roads and intersections
Crisman, J. D., & Thorpe, C. E. (1993) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.210794 · Cited by 245
What's going on? Discovering spatio-temporal dependencies in dynamic scenes
Kuettel, D., Breitenstein, M. D., Van Gool, L., & Ferrari, V. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539869 · Cited by 241
A hierarchical field framework for unified context-based classification
Kumar, S., & Hebert, M. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.9 · Cited by 223
Team AnnieWAY's Entry to the 2011 Grand Cooperative Driving Challenge
Geiger, A., Lauer, M., Moosmann, F., Ranft, B., Rapp, H., Stiller, C., & Ziegler, J. (2012) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2012.2189882 · Cited by 179
Self-citationLost! Leveraging the Crowd for Probabilistic Visual Self-Localization
Brubaker, M. A., Geiger, A., & Urtasun, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.393 · Cited by 152
Self-citationSegmentation-Based Urban Traffic Scene Understanding
Ess, A., Mueller, T., Gräbner, H., et al. (2009) · British Machine Vision Conference (BMVC) · doi:10.5244/c.23.84 · Cited by 148
Autonomous vehicles control in the VisLab Intercontinental Autonomous Challenge
Broggi, A., Medici, P., Zani, P., Coati, A., & Panciroli, M. (2012) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2012.03.012 · Cited by 135
Multiple Target Tracking in World Coordinate with Single, Minimally Calibrated Camera
Choi, W., & Savarese, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15561-1_40 · Cited by 133
Probabilistic Lane Tracking in Difficult Road Scenarios Using Stereovision
Danescu, R., & Nedevschi, S. (2009) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2009.2018328 · Cited by 127
Off-road Path Following using Region Classification and Geometric Projection Constraints
Alon, Y., Ferencz, A., & Shashua, A. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.213 · Cited by 118
A Dynamic Conditional Random Field Model for Joint Labeling of Object and Scene Classes
Wojek, C., & Schiele, B. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88693-8_54 · Cited by 110
Self-citation3D Scene priors for road detection
Álvarez, J. M., Gevers, T., & López, A. M. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540228 · Cited by 98
Joint 3D Estimation of Objects and Scene Layout
Geiger, A., Wojek, C., & Urtasun, R. (2011) · Cited by 93
Self-citationMultiple-Target Tracking by Spatiotemporal Monte Carlo Markov Chain Data Association
Yu, Q., & Medioni, G. G. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.253 · Cited by 90
Monocular Visual Scene Understanding: Understanding Multi-Object Traffic Scenes
Wojek, C., Walk, S., Roth, S., Konrad Schindler, B., & Schiele, B. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.174 · Cited by 87
Self-citationGeometric Image Parsing in Man-Made Environments
Barinova, O., Lempitsky, V., Tretiak, E., & Kohli, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15552-9_5 · Cited by 77
3D LIDAR point cloud based intersection recognition for autonomous driving
Zhu, Q., Chen, L., Li, Q., Li, M., Nüchter, A., & Wang, J. (2012) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2012.6232219 · Cited by 75
A generative model for 3D urban scene understanding from movable platforms
Geiger, A., Lauer, M., & Urtasun, R. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995641 · Cited by 58
Self-citationStages as Models of Scene Geometry
Nedović, V., Smeulders, A. W. M., Redert, A., & Geusebroek, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.174 · Cited by 56
Toward coherent object detection and scene layout understanding
Bao, S. Y., Sun, M., & Savarese, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540229 · Cited by 53
Acquiring semantics induced topology in urban environments
Singh, G., & Košecká, J. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225282 · Cited by 18
Probabilistic Models for 3D Urban Scene Understanding from Movable Platforms
Geiger, A. (2013) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ksp/1000036064 · Cited by 11
Self-citation
Cited by
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433
Object scene flow for autonomous vehicles
Menze, M., & Geiger, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298925 · Cited by 2,270
Self-citationDeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876
Discriminative Scale Space Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2609928 · Cited by 1,315
Metadata
- Authors
- Geiger, A., Lauer, M., Wojek, C., Stiller, C., & Urtasun, R.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-05
- Volume
- 36
- Issue
- 5
- Pages
- 1012-1025
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2137097255
- Cited by
- 397
- References
- 61
- FWCI
- 57.56
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.