The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Online Multi-Target Tracking Using Recurrent Neural Networks
Open accessAbstract
We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and time-varying number of targets, b) a continuous state estimation of all present targets, and c) a discrete combinatorial problem of data association. Most previous methods involve complex models that require tedious tuning of parameters. Here, we propose for the first time, an end-to-end learning approach for online multi-target tracking. Existing deep learning methods are not designed for the above challenges and cannot be trivially applied to the task. Our solution addresses all of the above points in a principled way. Experiments on both synthetic and real data show promising results obtained at ~300 Hz on a standard CPU, and pave the way towards future research in this direction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751 · cited 2×
“Here, we consider a special case with N = 1 where all targets are assumed to move independently.”
BackgroundMethodGreff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 2×
“We follow some of the best practices found in the literature (Greff et al. 2015; Karpathy, Johnson, and Li 2015), such as setting the initial weights for the forget gates higher (1 in our case), and also employ a standard grid search to find the best setting for the present task.”
BackgroundMethodShow and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Bar‐Shalom, Y. (1988) · Medical Entomology and Zoology · Cited by 3,157
An algorithm for tracking multiple targets
Reid, D. (1979) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1979.1102177 · Cited by 3,059
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., & Dollár, P. (2014) · arXiv · doi:10.48550/arxiv.1405.0312 · Cited by 2,579 · cited 1×
“2014) or Microsoft COCO (Lin et al. 2014) have enabled deep learning methods to unfold their potential on tasks like image classification or pixel labelling.”
MethodMultiple Object Tracking Using K-Shortest Paths Optimization
Berclaz, J., Fleuret, F., Turetken, E., & Fua, P. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.21 · Cited by 1,015
Global data association for multi-object tracking using network flows
Zhang, L., Li, Y., & Nevatia, R. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587584 · Cited by 925
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845
Globally-optimal greedy algorithms for tracking a variable number of objects
Pirsiavash, H., Ramanan, D., & Fowlkes, C. C. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995604 · Cited by 790
MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
Leal-Taixé, L., Milan, A., Reid, I. D., Roth, S., & Schindler, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.01942 · Cited by 743
Self-citationLearning to Track: Online Multi-object Tracking by Decision Making
Xiang, Y., Alahi, A., & Savarese, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.534 · Cited by 674
Learning task-dependent distributed representations by backpropagation through structure
Goller, C., & Kuchler, A. (2002) · Proceedings of International Conference on Neural Networks (ICNN'96) · doi:10.1109/icnn.1996.548916 · Cited by 581
Continuous Energy Minimization for Multitarget Tracking
Milan, A., Roth, S., & Schindler, K. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.103 · Cited by 565
Self-citationOn Sequential Monte Carlo Sampling Methods for Bayesian Filtering
Doucet, A., Godsill, S., & Andrieu, C. (2005) · Readings In Unobserved Components Models · doi:10.1093/oso/9780199278657.003.0022 · Cited by 412
Bae, S.-H., & Yoon, K.-J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.159 · Cited by 410
Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor
Choi, W. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.347 · Cited by 396
Joint Probabilistic Data Association Revisited
Rezatofighi, S. H., Milan, A., Zhang, Z., Shi, Q., Dick, A., & Reid, I. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.349 · Cited by 350
Self-citationMulti-target tracking using joint probabilistic data association
Fortmann, T., Bar-Shalom, Y., & Scheffe, M. (1980) · 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes · doi:10.1109/cdc.1980.271915 · Cited by 323
A Linear Programming Approach for Multiple Object Tracking
Jiang, H., Fels, S., & Little, J. J. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383180 · Cited by 316
Coupled Detection and Trajectory Estimation for Multi-Object Tracking
Leibe, B., Schindler, K., & Van Gool, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408936 · Cited by 272
Self-citationMulti-target Tracking by Lagrangian Relaxation to Min-cost Network Flow
Butt, A. A., & Collins, R. T. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.241 · Cited by 270
Detection- and Trajectory-Level Exclusion in Multiple Object Tracking
Milan, A., Schindler, K., & Roth, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.472 · Cited by 196
Self-citationOnline Multi-object Tracking via Structural Constraint Event Aggregation
Yoon, J. H., Lee, C.-R., Yang, M.-H., & Yoon, K.-J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.155 · Cited by 181
Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks
Ondruska, P., & Posner, I. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10413 · Cited by 175
Ivakhnenko, A. G., & Lapa, V. G. (1966) · CCM Information eBooks · Cited by 158
Learning to associate: HybridBoosted multi-target tracker for crowded scene
Li, Y., Huang, C., & Nevatia, R. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206735 · Cited by 68
End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks
Ondrúška, P., Dequaire, J., Wang, D. Z., & Posner, I. (2016) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1604.05091 · Cited by 63
Data-Driven Approximations to NP-Hard Problems
Milan, A., Rezatofighi, S. H., Garg, R., Dick, A., & Reid, I. D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10750 · Cited by 56
Self-citation
Cited by
Detection and tracking meet drones challenge
Zhu, P., Wen, L., Du, D., Bian, X., Fan, H., Hu, Q., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119563 · Cited by 1,336
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
Deep learning in video multi-object tracking: A survey
Ciaparrone, G., Luque Sánchez, F., Tabik, S., Troiano, L., Tagliaferri, R., & Herrera, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.023 · Cited by 665
“in [133], already described in section 3.”
BackgroundMethodInfluentialFeatures for Multi-target Multi-camera Tracking and Re-identification
Ristani, E., & Tomasi, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00632 · Cited by 556
Quasi-Dense Similarity Learning for Multiple Object Tracking
Pang, J., Qiu, L., Li, X., Chen, H., Li, Q., Darrell, T., & Yu, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00023 · Cited by 480
“Appearance similarity in MOT To exploit instance appearance similarity to strengthen tracking and re-identify vanished objects, some methods directly use an independent model [2,20,22,29,38,40,45,49] or add an extra embedding head to the detector for end-to-end training [26, 44, 50, 53].”
Metadata
- Authors
- Milan, A., Rezatofighi, S. H., Dick, A., Reid, I., & Schindler, K.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2017-02-12
- Volume
- 31
- Issue
- 1
- ISSN
- 2374-3468
- 2159-5399
- OpenAlex
- W2963063317
- Cited by
- 506
- References
- 38
- FWCI
- 30.10
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.