The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index.
Learning to Track with Object Permanence
Open accessAbstract
Tracking by detection, the dominant approach for online multi-object tracking, alternates between localization and association steps. As a result, it strongly depends on the quality of instantaneous observations, often failing when objects are not fully visible. In contrast, tracking in humans is underlined by the notion of object permanence: once an object is recognized, we are aware of its physical existence and can approximately localize it even under full occlusions. In this work, we introduce an end-to-end trainable approach for joint object detection and tracking that is capable of such reasoning. We build on top of the recent CenterTrack architecture, which takes pairs of frames as input, and extend it to videos of arbitrary length. To this end, we augment the model with a spatio-temporal, recurrent memory module, allowing it to reason about object locations and identities in the current frame using all the previous history. It is, however, not obvious how to train such an approach. We study this question on a new, large-scale, synthetic dataset for multi-object tracking, which provides ground truth annotations for invisible objects, and propose several approaches for supervising tracking behind occlusions. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI and MOT17 datasets thanks to its robustness to occlusions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the projection of non-linear 3D motion [50] into the 2D image domain still poses a challenging problem for many models.”
From TrackFormer: Multi-Object Tracking with Transformers (2022) · cites this paper as [50]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Only [50,53,54] which follow [66] and pretrain on 6 additional tracking datasets (6M) surpass our performance.”
From TrackFormer: Multi-Object Tracking with Transformers (2022) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
Unsupervised Domain Adaptation by BackpropagationPreprint
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 5,441
Simple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
Research works printed but not linked: 1
- 2021, Parallel domain
Cited by
Sorted by citations
Items 201–226 of 226
Adj-MOT: Multi-object Tracking by ReID with Adjacent Frame Enhancement
Dong, T., Lv, S., Zhao, G., Ying, W., & Tong, C. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650827
Multi-Object Tracking Using Video Sequences with Improved Performance and Reduced Computation Cost
Su, J., Schneider, S. C., & Yaz, E. E. (2024) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta60707.2024.10666653
Zhang, Y., & Chen, H. (2024) · International Conference on Electronic Communication and Artificial Intelligence (ICECAI) · doi:10.1109/icecai62591.2024.10675282
Fod Tracking Algorithm Based on Detection Box Matching
Liu, Z., Guo, C., Huang, X., & Ye, J. (2024) · International Conference on Next Generation Data-driven Networks (NGDN) · doi:10.1109/ngdn61651.2024.10744090
AC-SORT: Adaptive Combination of Appearance and Motion Information for Multi-Object Tracking
Peng, J., Hao, Y., & Wang, H. (2024) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv63323.2024.10821602
Preaftrack: multi-object tracking based on adaptive feature matching from detection results
Zhang, M., Hou, C., Pang, Y., Shi, J., Zhang, Y., & Huang, X. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07466-6
Nakayama, Y., Mikuriya, M., & Ogino, F. (2025) · IEEE Access · doi:10.1109/access.2025.3596264
GTRF: Global Tracking Transformers for Underwater Fish Species Monitoring
Shah, C., Ebu, I. A., Nabi, M. M., Prior, J. H., Grossi, M. D., Rowell, T. J., Wallace, F., Ball, J. E., & Moorhead, R. J., II (2025) · doi:10.23919/oceans59106.2025.11245045
InstructTrack: Language-Guided Multi-Object Tracking with Semantic-Aware Association
Zhou, Z., Wang, S., Sheng, H., Yang, D., Li, S., Yang, D., & Cui, Z. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771047
Li, B., Gong, B., Kong, S., Sun, L., Liu, J., & Gao, D. (2025) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-025-00392-6
STFATrack: Multi-Object Tracking Based on Spatio-Temporal Feature Aggregation
An, Y., Fan, X., Zhang, C., Wang, Y., Xu, B., & Wang, B. (2025) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac66630.2025.11149996
Hypergraph random field model for multiple object tracking in dense scenarios
Zhang, J., Zhang, X., Zhu, Z., & Deng, C. (2026) · Applied Intelligence · doi:10.1007/s10489-025-07063-3
Xing, H., & Torsten, B. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668552
Mixed self-attention network for multi-object tracking with appearance similarity-guided NMS
Liu, Z., Feng, T., Liu, X., & Zhang, X. (2026) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2026.104785
Inductive Graph Neural Networks for Passive Vision and Acoustic Monitoring
Prummel, W. (2024) · theses.fr (ABES) · doi:10.70675/4949fe39z2d37z4b7dza42cz1d26a43a5401
Zhang, Y., Huang, W., Yan, C., Fan, R., Wang, G., & Hwang, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462134
Language Decoupling with Fine-Grained Knowledge Guidance for Referring Multi-Object Tracking
Li, G., Zhuang, S., Jian, Y., Yan, Y., & Wang, H. B. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02193
CueTrack: Weak-Cue-Enhanced and Consistency-Aligned Framework for Robust Multi-Pedestrian Tracking
Liao, L., Wang, J., Lai, S., Zhao, B., Cai, S., & Lu, Z. (2026) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981261444352
Hierarchical Sparse Graph for Multi-Dimensional Assignment in Event-Based Tracking
Zha, M., Shang, Y., Zhang, W., Yu, Q., & Guan, B. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3704671
Learning Pseudo 3D Representation for Ego-centric 2D Multiple Object Tracking
He, J., Fan, L., & Zhang, Z. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02887-z
Detection and ReID are Not All you Need: Accelerate Multi-Object Tracking with Correlation Filter
Wang, Y., Liu, T., & Hu, Z. (2026) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc69976.2026.11560651
BeyondSight: Object Permanence for End-to-End Autonomous Driving
Papais, S., Wang, L., Jain, M. R. P. S., Rezaei, B., & Waslander, S. L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37095-2_15
ByffusionMOT: An Efficient Joint Detection and Tracker Based on Diffusion Models
Fang, Y., Hu, J., Liu, B. Q., & Song, Y. (2026) · International Conference on Image and Graphics Processing (ICIGP) · doi:10.1109/icigp68997.2026.11619680
Bilakeri, S., Manisha, & Akshatha, K. R. (2026) · The Visual Computer · doi:10.1007/s00371-026-04700-x
SOPERM-Track: Spatial Object Permanence Modeling via Height Stability and Ground Perspective
Liao, L., Su, Y., Lu, Z., Chen, F., & Weng, S. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3731058
Survey on intelligent video surveillance
Elnady, M., Abdelmunim, H. E., Abidou, D., & Elmadah, I. (2026) · Journal Of Big Data · doi:10.1186/s40537-026-01560-z
Metadata
- Authors
- Tokmakov, P., Li, J., Burgard, W., & Gaidon, A.
- Affiliations
- Toyota Motor Corporation (Switzerland) (CH)
- University of Freiburg (DE)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 10840-10849
- OpenAlex
- W3142617676
- Cited by
- 226
- References
- 74
- Citations by year
- 2026: 14 · 2025: 43 · 2024: 72 · 2023: 48 · 2022: 46 · 2021: 3
- FWCI
- 13.74
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- IEEE/CVF International Conference on Computer Vision (ICCV)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multi-target tracking, object permanence, occlusion handling, joint detection and tracking, spatiotemporal memory, end-to-end tracking, KITTI, MOT17
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
