Stable multi-target tracking in real-time surveillance video
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
The majority of existing pedestrian trackers concentrate on maintaining the identities of targets, however systems for remote biometric analysis or activity recognition in surveillance video often require stable bounding-boxes around pedestrians rather than approximate locations. We present a multi-target tracking system that is designed specifically for the provision of stable and accurate head location estimates. By performing data association over a sliding window of frames, we are able to correct many data association errors and fill in gaps where observations are missed. The approach is multi-threaded and combines asynchronous HOG detections with simultaneous KLT tracking and Markov-Chain Monte-Carlo Data Association (MCM-CDA) to provide guaranteed real-time tracking in high definition video. Where previous approaches have used ad-hoc models for data association, we use a more principled approach based on a Minimal Description Length (MDL) objective which accurately models the affinity between observations. We demonstrate by qualitative and quantitative evaluation that the system is capable of providing precise location estimates for large crowds of pedestrians in real-time. To facilitate future performance comparisons, we make a new dataset with hand annotated ground truth head locations publicly available.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
(1981) · Cited by 11,963
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918
Detection and Tracking of Point Features
Tomasi, C. (1991) · Cited by 2,231
Learning to associate: HybridBoosted multi-target tracker for crowded scene
Li, Y., Huang, C., & Nevatia, R. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206735 · Cited by 633
Robust tracking-by-detection using a detector confidence particle filter
Breitenstein, M. D., Reichlin, F., Leibe, B., Koller-Meier, E., & Van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459278 · Cited by 545
Robust Object Tracking by Hierarchical Association of Detection Responses
Huang, C., Wu, B., & Nevatia, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88688-4_58 · Cited by 497
Coupled Detection and Trajectory Estimation for Multi-Object Tracking
Leibe, B., Schindler, K., & Van Gool, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408936 · Cited by 272
Markov chain Monte Carlo data association for general multiple-target tracking problems
Oh, S., Russell, S., & Sastry, S. (2004) · 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601) · doi:10.1109/cdc.2004.1428740 · Cited by 241
Tracking many objects with many sensors
Pasula, H., Russell, S., Ostland, M., & Ritov, Y. (1999) · Cited by 165
fastHOG – a real-time GPU implementation of HOG
Prisacariu, V. A., & Reid, I. D. (2011) · Cited by 152
Self-citationMultiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association
Yu, Q., Medioni, G., & Cohen, I. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.382991 · Cited by 132
A Stochastic Graph Evolution Framework for Robust Multi-target Tracking
Bi, S., Jeng, T.-Y., Staudt, E., & Roy–Chowdhury, A. K. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_44 · Cited by 120
Multiple object tracking using flow linear programming
Berclaz, J., Fleuret, F., & Fua, P. (2009) · IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS) · doi:10.1109/pets-winter.2009.5399488 · Cited by 116
Guiding Visual Surveillance by Tracking Human Attention
Benfold, B., & Reid, I. D. (2009) · British Machine Vision Conference (BMVC) · doi:10.5244/c.23.14 · Cited by 104
Self-citationCascaded Confidence Filtering for Improved Tracking-by-Detection
Stalder, S., Gräbner, H., & Gool, L. V. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_27 · Cited by 94
Simultaneous Localisation and Mapping in Dynamic Environments (SLAMIDE) with Reversible Data Associa
Bibby, C., & Reid, I. D. (2007) · Robotics: Science and Systems · doi:10.15607/rss.2007.iii.014 · Cited by 78
Self-citationLearning to associate: HybridBoosted multi-target tracker for crowded scene
Li, Y., Huang, C., & Nevatia, R. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206735 · Cited by 68
Markov chain Monte Carlo data association for target tracking
Bergman, N., & Arnaud Doucet, B. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2000.859057 · Cited by 47
Multi-target Data Association by Tracklets with Unsupervised Parameter Estimation
Ge, W., & Collins, R. T. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.93 · Cited by 41
Multiple Human Tracking in High-Density Crowds
Ali, I., & Dailey, M. N. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04697-1_50 · Cited by 28
Automatic Player Detection, Labeling and Tracking in Broadcast Soccer Video
Liu, J., Tong, X., Li, W., Wang, T. D., Zhang, Y., Wang, H., Yang, B., Sun, L., & Yang, S. (2007) · British Machine Vision Conference (BMVC) · doi:10.5244/c.21.3 · Cited by 28
Cited by
Visual Tracking: An Experimental Survey
(2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.230 · Cited by 1,513
Multiple object tracking: A literature review
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., & Kim, T.-K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103448 · Cited by 1,099
UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking
Wen, L., Du, D., Cai, Z., Lei, Z., Chang, M.-C., Qi, H., Lim, J., Yang, M.-H., & Lyu, S. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.102907 · Cited by 804
MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
Leal-Taixé, L., Milan, A., Reid, I. D., Roth, S., & Schindler, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.01942 · Cited by 743
Self-citationHuman motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Metadata
- Authors
- Benfold, B., & Reid, I.
- Affiliations
- University of Oxford (GB)
- ORCID
- Reid, Ian
- Venue
- CVPR 2011
- Publisher
- IEEE
- Published
- 2011-06
- Pages
- 3457-3464
- OpenAlex
- W2061773916
- Cited by
- 651
- References
- 22
- Citations by year
- 2026: 10 · 2025: 15 · 2024: 12 · 2023: 20 · 2022: 37 · 2021: 45 · 2020: 42 · 2019: 53 · 2018: 60 · 2017: 53 · 2016: 79 · 2015: 66 · 2014: 72 · 2013: 53 · 2012: 28
- FWCI
- 37.96
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- multi-target tracking, pedestrian tracking, surveillance video, data association, sliding window, minimum description length, real-time tracking, crowd tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.