The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
A Novel Performance Evaluation Methodology for Single-Target Trackers
Open accessAbstract
This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Bar‐Shalom, Y., Li, X. ‐., & Kirubarajan, T. (2002) · doi:10.1002/0471221279 · Cited by 6,782
Clustering by Passing Messages Between Data Points
Frey, B. J., & Dueck, D. (2007) · Science · doi:10.1126/science.1136800 · Cited by 6,758
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753
Yilmaz, A., Javed, O., & Shah, M. (2006) · ACM Computing Surveys · doi:10.1145/1177352.1177355 · Cited by 4,792
Comaniciu, D., Ramesh, V., & Meer, P. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1195991 · Cited by 4,560
Online Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138
The FERET evaluation methodology for face-recognition algorithms
Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.879790 · Cited by 3,692
Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes
Anderson, T. W., & Darling, D. A. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729437 · Cited by 3,542
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406
Incremental Learning for Robust Visual Tracking
Ross, D. A., Lim, J., Lin, R.-S., & Yang, M.-H. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0075-7 · Cited by 3,061
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,541
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440
Exploiting the Circulant Structure of Tracking-by-Detection with Kernels
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_50 · Cited by 2,310
A Survey on Visual Surveillance of Object Motion and Behaviors
Hu, W., Tan, T., Wang, L., & Maybank, S. (2004) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2004.829274 · Cited by 2,132
Accurate Scale Estimation for Robust Visual Tracking
Danelljan, M., Häger, G., Shahbaz Khan, F., & Felsberg, M. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.65 · Cited by 2,087
Robust object tracking with online multiple instance learning
Babenko, B., Yang, M.-H., & Belongie, S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.226 · Cited by 2,007
Struck: Structured output tracking with kernels
Hare, S., Saffari, A., & Torr, P. H. S. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126251 · Cited by 1,831
The Visual Analysis of Human Movement: A Survey
Gavrila, D. M. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0716 · Cited by 1,782
A Survey of Computer Vision-Based Human Motion Capture
Moeslund, T. B., & Granum, E. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0897 · Cited by 1,768
Visual Tracking: An Experimental Survey
(2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.230 · Cited by 1,513
Adaptive Color Attributes for Real-Time Visual Tracking
Danelljan, M., Khan, F. S., Felsberg, M., & Van De Weijer, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.143 · Cited by 1,500
Real-Time Compressive Tracking
Zhang, K., Zhang, L., & Yang, M.-H. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_62 · Cited by 1,427
Robust Fragments-based Tracking using the Integral Histogram
Adam, A., Rivlin, E., & Shimshoni, I. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.256 · Cited by 1,397
A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration
Li, Y., & Zhu, J. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16181-5_18 · Cited by 1,396
Color-Based Probabilistic Tracking
Pérez, P., Hue, C., Vermaak, J., & Gangnet, M. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47969-4_44 · Cited by 1,389
Statistical Comparisons of Classifiers over Multiple Data Sets
DemšarJanez (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248548 · Cited by 1,074
Multicamera People Tracking with a Probabilistic Occupancy Map
Fleuret, F., Berclaz, J., Lengagne, R., & Fua, P. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1174 · Cited by 821
Changedetection.net: A new change detection benchmark dataset
Goyette, N., Jodoin, P.-M., Porikli, F., Konrad, J., & Ishwar, P. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6238919 · Cited by 807
Self-citationMOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
Leal-Taixé, L., Milan, A., Reid, I. D., Roth, S., & Schindler, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.01942 · Cited by 743
A survey of appearance models in visual object tracking
Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., & Hengel, A. V. D. (2013) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2508037.2508039 · Cited by 705
The Visual Object Tracking VOT2015 Challenge Results
(2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.79 · Cited by 666
The Visual Object Tracking VOT2014 Challenge Results
Kristan, M., Pflugfelder, R., Leonardis, A., Matas, J., Čehovin, L., Nebehay, G., Vojíř, T., Fernández, G., Lukežič, A., Dimitriev, A., et al. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16181-5_14 · Cited by 588
Self-citationStatistics for Engineers and Scientists
Navidi, W. C. (2004) · Cited by 583
Template matching using fast normalized cross correlation:
Briechle, K., & Hanebeck, U. D. (2001) · SPIE Proceedings · doi:10.1117/12.421129 · Cited by 539
Distribution fields for tracking
Sevilla-Lara, L., & Learned-Miller, E. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247891 · Cited by 513
Kasturi, R., Goldgof, D., Soundararajan, P., Manohar, V., Garofolo, J., Bowers, R., Boonstra, M., Korzhova, V., & Zhang, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.57 · Cited by 497
The Visual Object Tracking VOT2013 Challenge Results
Kristan, M., Pflugfelder, R., Leonardis, A., Matas, J., Porikli, F., Cehovin, L., Nebehay, G., Fernandez, G., Vojir, T., Gatt, A., et al. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.20 · Cited by 308
Self-citationReal-time visual tracking using compressive sensing
Li, H., Shen, C., & Shi, Q. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995483 · Cited by 297
The FERET evaluation methodology for face-recognition algorithms
Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (1998) · doi:10.6028/nist.ir.6264 · Cited by 273
Kwon, J., & Lee, K. M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206502 · Cited by 217
Robust scale-adaptive mean-shift for tracking
Vojir, T., Noskova, J., & Matas, J. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.03.025 · Cited by 209
Self-citationTools and techniques for video performance evaluation
Doermann, D., & Mihalcik, D. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.902888 · Cited by 190
Robust Visual Tracking Using an Adaptive Coupled-Layer Visual Model
Cehovin, L., Kristan, M., & Leonardis, A. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.145 · Cited by 187
Self-citationNUS-PRO: A New Visual Tracking Challenge
Li, A., Lin, M., Wu, Y., Yang, M.-H., & Yan, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2417577 · Cited by 176
Visual Object Tracking Performance Measures Revisited
Čehovin, L., Leonardis, A., & Kristan, M. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2520370 · Cited by 167
Self-citationConsensus-based matching and tracking of keypoints for object tracking
Nebehay, G., & Pflugfelder, R. P. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836013 · Cited by 149
Self-citationAdaptive Appearance Modeling for Video Tracking: Survey and Evaluation
Salti, S., Cavallaro, A., & Stefano, L. D. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2206035 · Cited by 141
PixelTrack: A Fast Adaptive Algorithm for Tracking Non-rigid Objects
Duffner, S., & García, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.308 · Cited by 122
PETS Metrics: On-Line Performance Evaluation Service
Young, D. P., & Ferryman, J. M. (2006) · IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS) · doi:10.1109/vspets.2005.1570931 · Cited by 113
Robust Deformable and Occluded Object Tracking With Dynamic Graph
Cai, Z., Wen, L., Lei, Z., Vasconcelos, N. M., & Li, S. Z. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2364919 · Cited by 93
Is my new tracker really better than yours?
Zajc, L. Č., Kristan, M., & Leonardis, A. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836055 · Cited by 81
Self-citationEnhanced Distribution Field Tracking Using Channel Representations
Felsberg, M. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.22 · Cited by 81
The State of the Art in Multiple Object Tracking Under Occlusion in Video Sequences
Gabriel, P. F., Verly, J., Piater, J., & Genon, A. (2003) · Cited by 74
Pang, Y., & Ling, H. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.346 · Cited by 73
Closed-world tracking of multiple interacting targets for indoor-sports applications
Kristan, M., Perš, J., Perše, M., & Kovačič, S. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2008.01.009 · Cited by 64
Self-citationColor features for tracking non-rigid objects
Nummiaro, K., Koller-Meier, E., & Gool, L. V. (2003) · Cited by 61
A Two-Stage Dynamic Model for Visual Tracking
Kristan, M., Kovačič, S., Leonardis, A., & Perš, J. (2010) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2041662 · Cited by 58
Self-citationMATRIOSKA: A Multi-level Approach to Fast Tracking by Learning
Maresca, M. E., & Petrosino, A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41184-7_43 · Cited by 56
Online Empirical Evaluation of Tracking Algorithms
Wu, H., Sankaranarayanan, A. C., & Chellappa, R. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.135 · Cited by 56
Long-Term Tracking through Failure Cases
Lebeda, K., Hadfield, S., Matas, J., & Bowden, R. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.26 · Cited by 43
Self-citationClustering Local Motion Estimates for Robust and Efficient Object Tracking
Maresca, M. E., & Petrosino, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_17 · Cited by 41
Adaptive Online Performance Evaluation of Video Trackers
SanMiguel, J. C., Cavallaro, A., & Martínez, J. M. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2182520 · Cited by 40
Nam, H., Hong, S., & Han, B. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_8 · Cited by 36
A Protocol for Evaluating Video Trackers Under Real-World Conditions
Nawaz, T. H., & Cavallaro, A. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2228497 · Cited by 35
Karasulu, B., & Korukoglu, S. (2011) · Multimedia tools and applications · doi:10.1007/s11042-010-0591-2 · Cited by 32
Thirteen Hard Cases in Visual Tracking
Chu, D. M., & Smeulders, A. W. M. (2010) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2010.85 · Cited by 20
The VOT2013 challenge: overview and additional results
Kristan, M., Pflugfelder, R. P., Leonardis, A., Matas, J., Porikli, F., Zajc, L. Č., Nebehay, G., Fernández, G. J., & Vojíř, T. (2014) · ANU Open Research (Australian National University) · Cited by 16
Self-citationWeighted Update and Comparison for Channel-Based Distribution Field Tracking
Öfjäll, K., & Felsberg, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_15 · Cited by 7
Visual tracking with dual modeling
Yi, K. M., Jeong, H., Kim, S. W., & Choi, J. Y. (2012) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1145/2425836.2425843 · Cited by 6
Cited by
Staple: Complementary Learners for Real-Time Tracking
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., & Torr, P. H. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.156 · Cited by 1,776
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking
Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., & Ling, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00552 · Cited by 1,742
“Currently, the popular dense benchmarks contain OTB [52, 53], TC-128 [35], VOT [25], NUS-PRO [28], UAV [39], NfS [14] and GOT-10k [22].”
BackgroundInfluentialFast Online Object Tracking and Segmentation: A Unifying Approach
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00142 · Cited by 1,554
TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild
Müller, M., Bibi, A., Giancola, S., Alsubaihi, S., & Ghanem, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_19 · Cited by 1,192
Discriminative Correlation Filter with Channel and Spatial Reliability
Lukezic, A., Vojir, T., Zajc, L. C., Matas, J., & Kristan, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.515 · Cited by 1,153
Self-citation
Metadata
- Authors
- Kristan, M., Matas, J., Leonardis, A., Vojir, T., Pflugfelder, R., Fernandez, G., Nebehay, G., Porikli, F., & Cehovin, L.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2016-11-01
- Volume
- 38
- Issue
- 11
- Pages
- 2137-2155
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2158827467
- Cited by
- 613
- References
- 75
- FWCI
- 50.43
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.