Learning Multi-Object Tracking and Segmentation From Automatic Annotations
Open accessAbstract
In this work we contribute a novel pipeline to automatically generate training data, and to improve over state-of-the-art multi-object tracking and segmentation (MOTS) methods. Our proposed track mining algorithm turns raw street-level videos into high-fidelity MOTS training data, is scalable and overcomes the need of expensive and time-consuming manual annotation approaches. We leverage state-of-the-art instance segmentation results in combination with optical flow predictions, also trained on automatically harvested training data. Our second major contribution is MOTSNet - a deep learning, tracking-by-detection architecture for MOTS - deploying a novel mask-pooling layer for improved object association over time. Training MOTSNet with our automatically extracted data leads to significantly improved sMOTSA scores on the novel KITTI MOTS dataset (+1.9%/+7.5% on cars/pedestrians), and MOTSNet improves by +4.1% over previously best methods on the MOTSChallenge dataset. Our most impressive finding is that we can improve over previous best-performing works, even in complete absence of manually annotated MOTS training data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
In Defense of the Triplet Loss for Person Re-IdentificationPreprint
Hermans, A., Beyer, L., & Leibe, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07737 · Cited by 3,271 · see citing works
High Performance Visual Tracking with Siamese Region Proposal Network
Li, B., Yan, J., Wu, W., Zhu, Z., & Hu, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00935 · Cited by 3,023 · see citing works
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918 · see citing works
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Perazzi, F., Pont-Tuset, J., McWilliams, B., Gool, L. V., Groß, M. H., & Sorkine‐Hornung, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.85 · Cited by 2,039 · see citing works
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632 · see citing works
Fast Online Object Tracking and Segmentation: A Unifying Approach
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00142 · Cited by 1,554 · see citing works
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
Neuhold, G., Ollmann, T., Bulo, S. R., & Kontschieder, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.534 · Cited by 1,434 · see citing works
Self-citationMOT16: A Benchmark for Multi-Object Tracking
Milan, A., Leal-Taixe, L., Reid, I., Roth, S., & Schindler, K. (2016) · arXiv · doi:10.48550/arxiv.1603.00831 · Cited by 1,309 · see citing works
One-Shot Video Object Segmentation
Caelles, S., Maninis, K.-K., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.565 · Cited by 927 · see citing works
MOTChallenge 2015: Towards a Benchmark for Multi-Target TrackingPreprint
Leal-Taixé, L., Milan, A., Reid, I. D., Roth, S., & Schindler, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.01942 · Cited by 923 · see citing works
UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking
Wen, L., Du, D., Cai, Z., Lei, Z., Chang, M.-C., Qi, H., Lim, J., Yang, M.-H., & Lyu, S. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.102907 · Cited by 804 · see citing works
The 2018 DAVIS Challenge on Video Object SegmentationPreprint
Caelles, S., Montes, A., Maninis, K.-K., Chen, Y., Gool, L. V., Perazzi, F., & Pont-Tuset, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00557 · Cited by 763 · see citing works
EpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298720 · Cited by 757 · see citing works
(2018) · arXiv (Cornell University) · Cited by 679 · see citing works
Multiple Hypothesis Tracking Revisited
Kim, C., Li, F., Ciptadi, A., & Rehg, J. M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.533 · Cited by 666 · see citing works
MOTS: Multi-Object Tracking and Segmentation
Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B. B. G., Geiger, A., & Leibe, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00813 · Cited by 658 · see citing works
Chen, L., Ai, H., Zhuang, Z., & Shang, C. (2018) · 2018 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486597 · Cited by 530 · see citing works
Video Segmentation by Tracking Many Figure-Ground Segments
Li, F., Kim, T., Humayun, A., Tsai, D., & Rehg, J. M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.273 · Cited by 523 · see citing works
Disentangling Monocular 3D Object Detection
Simonelli, A., Bulo, S. R., Porzi, L., Lopez-Antequera, M., & Kontschieder, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00208 · Cited by 520 · see citing works
Self-citationYang, L., Fan, Y., & Xu, N. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00529 · Cited by 505 · see citing works
Deep Extreme Cut: From Extreme Points to Object Segmentation
Maninis, K.-K., Caelles, S., Pont-Tuset, J., & Gool, L. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00071 · Cited by 465 · see citing works
Learning object class detectors from weakly annotated video
Prest, A., Leistner, C., Civera, J., Schmid, C., & Ferrari, V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248065 · Cited by 434 · see citing works
In-place Activated BatchNorm for Memory-Optimized Training of DNNs
Bulò, S. R., Porzi, L., & Kontschieder, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00591 · Cited by 324 · see citing works
Self-citationAccurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes
Shen, S. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2237921 · Cited by 311 · see citing works
YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
Xu, N., Yang, L., Fan, Y., Yue, D., Liang, Y., Yang, J., & Huang, T. (2018) · arXiv · doi:10.48550/arxiv.1809.03327 · Cited by 301 · see citing works
Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
Chen, Y., Pont-Tuset, J., Montes, A., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00130 · Cited by 273 · see citing works
Hierarchical Discrete Distribution Decomposition for Match Density Estimation
Yin, Z., Darrell, T., & Yu, F. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00620 · Cited by 253 · see citing works
Porzi, L., Bulò, S. R., Colovic, A., & Kontschieder, P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00847 · Cited by 211 · see citing works
Self-citationMotion Segmentation & Multiple Object Tracking by Correlation Co-Clustering
Keuper, M., Tang, S., Andres, B., Brox, T., & Schiele, B. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2876253 · Cited by 201 · see citing works
A Generative Appearance Model for End-To-End Video Object Segmentation
Johnander, J., Danelljan, M., Brissman, E., Khan, F. S., & Felsberg, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00916 · Cited by 196 · see citing works
Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking
Sharma, S., Ansari, J. A., Jatavallabhula, K. M., & Krishna, K. M. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461018 · Cited by 169 · see citing works
Fusion of Head and Full-Body Detectors for Multi-object Tracking
Henschel, R. D., Leal-Taixé, L., Cremers, D., & Rosenhahn, B. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00192 · Cited by 152 · see citing works
UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and TrackingPreprint
Wen, L., Du, D., Cai, Z., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04136 · Cited by 142 · see citing works
Combined image- and world-space tracking in traffic scenes
Ošep, A., Mehner, W., Mathias, M., & Leibe, B. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989230 · Cited by 114 · see citing works
Predicting Future Instance Segmentation by Forecasting Convolutional Features
Luc, P., Couprie, C., LeCun, Y., & Verbeek, J. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_36 · Cited by 82 · see citing works
Track, Then Decide: Category-Agnostic Vision-Based Multi-Object Tracking
Ošep, A., Mehner, W., Voigtlaender, P., & Leibe, B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460975 · Cited by 66 · see citing works
Joint Optical Flow and Temporally Consistent Semantic Segmentation
Hur, J., & Roth, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46604-0_12 · Cited by 64 · see citing works
Predicting Scene Parsing and Motion Dynamics in the FuturePreprint
Jin, X., Xiao, H., Shen, X., Yan, S., Lin, Z., Chen, Y., Jie, Z., Feng, J., & Yan, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03270 · Cited by 62 · see citing works
Semi-Automatic Annotation of Objects in Visual-Thermal Video
Berg, A., Johnander, J., de Gevigney, F. D., Ahlberg, J., & Felberg, M. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00277 · Cited by 34 · see citing works
Luiten, J., Torr, P. H. S., & Leibe, B. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00088 · Cited by 29 · see citing works
Panoptic-DeepLabPreprint
Cheng, B., Collins, M. D., Zhu, Y., Liu, T., Huang, T. S., Adam, H., & Chen, L.-C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04751 · Cited by 26 · see citing works
Predicting Future Instance Segmentations by Forecasting Convolutional Features
Luc, P., Couprie, C., LeCun, Y., & Verbeek, J. J. (2018) · HAL · Cited by 24 · see citing works
Video Instance SegmentationPreprint
Yang, L., Fan, Y., & Xu, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04804 · Cited by 14 · see citing works
Joint Optical Flow and Temporally Consistent Semantic SegmentationPreprint
Hur, J., & Roth, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.07716
Flagged first, then by citations
Research works printed but not linked: 5
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. In Proceedings of the IEEE Inter- national Conference on Computer Vision, 2017.
- Aljoša Ošep, Paul Voigtlaender, Jonathon Luiten, Stefan Breuers, and Bastian Leibe. Towards large-scale video ob- ject mining. Proceedings of the ECCV 2018 Workshop on Interactive and Adaptive Learning in an Open World, 2018.
- Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- beláez, Alexander Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv:1704.00675, 2017. · arXiv
- Zhichao Yin, Trevor Darrell, and Fisher Yu. Hierarchical dis- crete distribution decomposition for match density estima- tion. In Proceedings of the IEEE International Conference on Computer Vision, 2019.
- Idil Esen Zulfikar, Jonathon Luiten, and Bastian Leibe. Un- ovost: Unsupervised offline video object segmentation and tracking for the 2019 unsupervised davis challenge. In Pro- ceedings of the 2019 DAVIS Challenge on Video Object Seg- mentation -CVPR Workshops, 2019.
Cited by
Sorted by citations
TrackFormer: Multi-Object Tracking with Transformers
Meinhardt, T., Kirillov, A., Leal-Taixe, L., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00864 · Cited by 1,081 · see citing works
“Prior attempts used category-agnostic image segmentation [30], applied Mask R-CNN [16] with 3D convolutions [51], mask pooling layers [37], or represented objects as unordered point clouds [57] and cost volumes [56].”
MethodTrack to Detect and Segment: An Online Multi-Object Tracker
Wu, J., Cao, J., Song, L., Wang, Y., Yang, M., & Yuan, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01217 · Cited by 432 · see citing works
A Survey on Deep Learning Technique for Video Segmentation
Zhou, T., Porikli, F., Crandall, D., Gool, L. V., & Wang, W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3225573 · Cited by 273 · see citing works
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Chen, L.-C., Lopes, R. G., Cheng, B., Collins, M. D., Cubuk, E. D., Zoph, B., Adam, H., & Shlens, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_40 · Cited by 165 · see citing works
ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation
Qiao, S., Zhu, Y., Adam, H., Yuille, A., & Chen, L.-C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00399 · Cited by 148 · see citing works
Multi-Object Tracking in Satellite Videos With Graph-Based Multitask Modeling
He, Q., Sun, X., Yan, Z., Li, B., & Fu, K. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3152250 · Cited by 127 · see citing works
Online multi-object tracking with unsupervised re-identification learning and occlusion estimation
Liu, Q., Chen, D., Chu, Q., Yuan, L., Liu, B., Zhang, L., & Yu, N. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.008 · Cited by 110 · see citing works
Segment as Points for Efficient Online Multi-Object Tracking and Segmentation
Xu, Z., Zhang, W., Tan, X., Yang, W., Huang, H., Wen, S., Ding, E., & Huang, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_16 · Cited by 89 · see citing works
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
Li, X., Zhang, W., Pang, J., Chen, K., Cheng, G., Tong, Y., & Loy, C. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01828 · Cited by 89 · see citing works
Cao, X., Zheng, Y., Yao, Y., Qin, H., Cao, X., & Guo, S. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3526066 · Cited by 58 · see citing works
MOPT: Multi-Object Panoptic TrackingPreprint
Hurtado, J. V., Mohan, R., Burgard, W., & Valada, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08189 · Cited by 59 · see citing works
Self-Supervised Learning to Detect Key Frames in Videos
Yan, X., Gilani, S. Z., Feng, M., Zhang, L., Qin, H., & Mian, A. S. (2020) · Sensors · doi:10.3390/s20236941 · Cited by 43 · see citing works
Chang, X., Pan, H., Sun, W., & Gao, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3056383 · Cited by 42 · see citing works
Multi-Object Tracking and Segmentation Via Neural Message Passing
Brasó, G., Cetintas, O., & Leal-Taixé, L. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01678-6 · Cited by 38 · see citing works
A Review of Multi‐Object Tracking in Recent Times
Li, S., Ren, H., Xie, X., & Cao, Y. (2025) · IET Computer Vision · doi:10.1049/cvi2.70010 · Cited by 39 · see citing works
Filter Pruning via Learned Representation Median in the Frequency Domain
Zhang, X., Xie, W., Li, Y., Lei, J., & Du, Q. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3124284 · Cited by 34 · see citing works
Learning to Associate Every Segment for Video Panoptic Segmentation
Woo, S., Kim, D., Lee, J., & Kweon, I. S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00273 · Cited by 33 · see citing works
DG-Labeler and DGL-MOTS Dataset: Boost the Autonomous Driving Perception
Cui, Y., Cao, Z., Xie, Y., Jiang, X., Tao, F., Chen, Y., Li, L., & Liu, D. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00347 · Cited by 30 · see citing works
Segment as Points for Efficient and Effective Online Multi-Object Tracking and Segmentation
Xu, Z., Yang, W., Zhang, W., Tan, X., Huang, H., & Huang, L. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3087898 · Cited by 28 · see citing works
RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation
Wan, Z., Mao, Y., Zhang, J., & Dai, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00920 · Cited by 28 · see citing works
OmniTracker: Unifying Visual Object Tracking by Tracking-With-Detection
Wang, J., Wu, Z., Chen, D., Luo, C., Dai, X., Yuan, L., & Jiang, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3529926 · Cited by 42 · see citing works
Continuous Copy-Paste for One-stage Multi-object Tracking and Segmentation
Xu, Z., Meng, A., Shi, Z., Yang, W., Chen, Z., & Huang, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01504 · Cited by 24 · see citing works
Zhang, H., Liu, J., Yu, Z., & Wang, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.039 · Cited by 23 · see citing works
3D Pose Estimation and Tracking in Handball Actions Using a Monocular Camera
Šajina, R., & Ivašić-Kos, M. (2022) · Journal of Imaging · doi:10.3390/jimaging8110308 · Cited by 24 · see citing works
SegDQ: Segmentation assisted multi-object tracking with dynamic query-based transformers
Liu, Y., Bai, T., Tian, Y., Wang, Y., Wang, J., Wang, X., & Wang, F. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.073 · Cited by 20 · see citing works
Fast forward approximation and multitask inversion of gravity anomaly based on UNet3+
Lv, M., Zhang, Y., & Liu, S. (2023) · Geophysical Journal International · doi:10.1093/gji/ggad106 · Cited by 17 · see citing works
Improving Video Segmentation via Dynamic Anchor Queries
Zhou, Y., Zhang, T., Ji, S., Yan, S., & Li, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72973-7_26 · Cited by 17 · see citing works
Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects
Maag, K., Chan, R., Uhlemeyer, S., Kowol, K., & Gottschalk, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26348-4_28 · Cited by 16 · see citing works
Pro2Diff: Proposal Propagation for Multi-Object Tracking via the Diffusion Model
Liu, H., Zhang, C., Fan, B., & Xu, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3494600 · Cited by 13 · see citing works
Tracking Multiple Deformable Objects in Egocentric Videos
Huang, M., Li, X., Hu, J. F., Peng, H., & Lyu, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00147 · Cited by 13 · see citing works
Toward High Quality Multi-Object Tracking and Segmentation Without Mask Supervision
Cheng, W., Wu, Y., Wu, Z., Ling, H., & Hua, G. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3403497 · Cited by 11 · see citing works
Enhancing Image Annotation With Object Tracking and Image Retrieval: A Systematic Review
Fernandes, R., Pessoa, A. C. P., Salgado, M., de Paiva, A. C., Paçal, İ., & Cunha, A. M. T. D. S. (2024) · IEEE Access · doi:10.1109/access.2024.3406018 · Cited by 11 · see citing works
Deep learning techniques for Video Instance Segmentation: A survey
Xu, C., Li, C., Hu, Y., Lim, C. P., & Creighton, D. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111763 · Cited by 15 · see citing works
Schindler, F., & Steinhage, V. (2022) · Ecological Informatics · doi:10.1016/j.ecoinf.2022.101794 · Cited by 11 · see citing works
Weakly Supervised Multi-Object Tracking and Segmentation
Ruiz, I., Porzi, L., Bulò, S. R., Kontschieder, P., & Serrat, J. (2021) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw52041.2021.00018 · Cited by 11 · see citing works
Self-citationTowards the Robustness of Multiple Object Tracking Systems
Xie, X., Duan, Y., Chen, S., & Xuan, J. (2022) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre55969.2022.00046 · Cited by 10 · see citing works
Prototype-guided Instance matching for multiple pedestrian tracking
Wang, Q., Zhang, W., Yang, W., Xu, C., & Cui, Z. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.068 · Cited by 10 · see citing works
SSAT: Self-Supervised Associating Network for Multiobject Tracking
Chung, T.-Y., Cho, M., Lee, H.-S., & Lee, S. (2022) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2022.3186751 · Cited by 10 · see citing works
Fast Segmentation-Based Object Tracking Model for Autonomous Vehicles
Dong, X., Niu, J., Cui, J., Fu, Z., & Ouyang, Z. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60239-0_18 · Cited by 9 · see citing works
OffsetNet: Towards Efficient Multiple Object Tracking, Detection, and Segmentation
Zhang, W., Li, J., Xia, M., Gao, X., Tan, X., Shi, Y., Huang, Z., & Li, G. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3485644 · Cited by 7 · see citing works
REAL-TIME DEEP NEURAL NETWORKS FOR MULTIPLE OBJECT TRACKING AND SEGMENTATION ON MONOCULAR VIDEO
Basharov, I., & Yudin, D. A. (2021) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xliv-2-w1-2021-15-2021 · Cited by 7 · see citing works
Automated suspect detection system and its current state of art and future research directions
Archana, M., Kavitha, S., & Vathsala, A. V. (2024) · AIP conference proceedings · doi:10.1063/5.0193831 · Cited by 5 · see citing works
Polarmask-Tracker: Lightweight Multi-Object Tracking and Segmentation Model for Edge Device
Dong, X., Ouyang, Z., Guo, Z., & Niu, J. (2021) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom52081.2021.00100 · Cited by 5 · see citing works
PointTrack++ for Effective Online Multi-Object Tracking and SegmentationPreprint
Xu, Z., Zhang, W., Tan, X., Yang, W., Su, X., Yuan, Y., Zhang, H., Wen, S., Ding, E., & Huang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01549 · Cited by 9 · see citing works
Saving costs for video data annotation in wildlife monitoring
Schindler, F., & Steinhage, V. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101418 · Cited by 4 · see citing works
Motion estimation for large displacements and deformations
Chen, Q., & Poullis, C. (2022) · Scientific Reports · doi:10.1038/s41598-022-21987-7 · Cited by 4 · see citing works
Unsupervised Spatio-temporal Latent Feature Clustering for Multiple-object Tracking and Segmentation
Siddique, A., Mozhdehi, R. J., & Medeiros, H. P. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.329 · Cited by 4 · see citing works
Self-Improving Semantic Perception for Indoor LocalisationPreprint
Blum, H., Milano, F., Zurbrügg, R., Siegward, R., Cadena, C., & Gawel, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01595 · Cited by 4 · see citing works
VGT-MOT: visibility-guided tracking for online multiple-object tracking
Wang, S., Li, W., Wang, L., Xu, L., & Deng, Q. (2023) · Machine Vision and Applications · doi:10.1007/s00138-023-01398-y · Cited by 3 · see citing works
Membership inference attack for beluga whales discrimination
Araújo, V. M., Gambs, S., Michaud, R., Lautraite, H., Schneider, L., & Chion, C. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102388 · Cited by 3 · see citing works
Blinding and blurring the multi-object tracker with adversarial perturbations
Pang, H., Ma, R., Su, J., Liu, C., Gao, Y., & Jin, Q. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106331 · Cited by 3 · see citing works
Park, J., Kim, S., Park, W.-H., Song, S., & Yoo, H. (2024) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2024.3446662 · Cited by 3 · see citing works
FAFMOTS: A Fast and Anchor Free Method for Online Joint Multi-Object Tracking and Segmentation
Li, S., Feng, W., Yang, L., Yang, W., Yang, S., & Lan, L. (2022) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct57072.2022.00098 · Cited by 3 · see citing works
Park, J., Kim, S., Park, W.-H., Song, S., & Yoo, H. (2024) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas58744.2024.10558649 · Cited by 3 · see citing works
Camera Motion Estimation from RGB-D-Inertial Scene Flow
Cerezo, S., & Civera, J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00089 · Cited by 3 · see citing works
Rolando, A., D’Uva, D., & Scandiffio, A. (2021) · FrancoAngeli srl eBooks · doi:10.3280/oa-686.23 · Cited by 3 · see citing works
Multi Object Tracking for Similar Instances: A Hybrid Architecture
Fóthi, Á., Faragó, K. B., Kopácsi, L., Milacski, Z. Á., Varga, V., & Lörincz, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63830-6_37 · Cited by 3 · see citing works
MOTS R-CNN: Cosine-margin-triplet loss for multi-object trackingPreprint
Unde, A. S., & Rameshan, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03512 · Cited by 5 · see citing works
Weakly-supervised Road Condition Classification Using Automatically Generated Labels
Zhou, W., Cruz, E., Worrall, S., Gomez‐Donoso, F., Cazorla, M., & Nebot, E. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294207 · Cited by 2 · see citing works
Entangled appearance and motion structures network for multi-object tracking and segmentation
Aryanfar, E., Shoorehdeli, M. A., & Seydi, V. (2024) · Machine Vision and Applications · doi:10.1007/s00138-024-01634-z · Cited by 2 · see citing works
Correcting Road Image Annotations
Lam, P., Chen, W., de Silva, L., & Brilakis, I. (2025) · Lecture notes in civil engineering · doi:10.1007/978-3-031-84208-5_39 · Cited by 2 · see citing works
Multi-Object Tracking with Hallucinated and Unlabeled VideosPreprint
McKee, D., Shuai, B., Berneshawi, A., Wang, M., Modolo, D., Lazebnik, S., & Tighe, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08836 · Cited by 2 · see citing works
IoT-Based Precision Litchi Tracking and Counting Method Using Gated Metrics
Lu, J., Bao, G., Deng, X., Han, X., Lan, Y., & Wu, H. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3561130 · Cited by 1 · see citing works
Autoregressive Temporal Modeling for Advanced Tracking-by-Diffusion
Nguyen, P., Madhok, R., Raj, B., & Luu, K. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02439-x · Cited by 1 · see citing works
AA-Pipe: automatic annotation pipeline for visible and thermal infrared video
Anderson, C., Schenck, E., Reinhardt, C. N., Blue, R., & Clipp, B. (2025) · Optical Engineering · doi:10.1117/1.oe.64.9.092205 · Cited by 1 · see citing works
Dual-Linear Attention Network for Multi-Object Tracking and Segmentation
Ren, Y., Wu, X., & Fu, H. (2025) · Applied Sciences · doi:10.3390/app16010065 · Cited by 1 · see citing works
Gaussian Splatting SLAM Based on Visibility Loop Closure and Depth Optimization
Xiao, Y., Wu, X., & Zhang, L. (2025) · International Conference on Mechatronics Technology and Aerospace Engineering (ICMTAE) · doi:10.1109/icmtae66890.2025.11427980 · Cited by 1 · see citing works
ICFD: An Incremental Learning Method Based on Data Feature Distribution
Zhu, Y., Tan, Y., Li, X., Wu, Q., & Ning, X. (2022) · IEEE Smart World Congress (SWC) · doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00103 · Cited by 1 · see citing works
A Multiple Object Tracker with Spatiotemporal Memory Network
Peng, X., Chi, J., Wang, Z., Yan, F., & Liu, J. (2023) · International Journal of Aerospace Engineering · doi:10.1155/2023/9959178 · Cited by 1 · see citing works
Chen, L., Liao, G., Zhu, G., & Zeng, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78312-8_17 · Cited by 1 · see citing works
Video Region Annotation with Sparse Bounding Boxes
Xu, Y., Wu, Y., Zuraimi, N. S. B., Nobuhara, S., & Nishino, K. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01719-0 · Cited by 2 · see citing works
Brand Analysis in Social Networks Using Deep Learning Techniques
Tan, F. G., & YÜKSEL, E. (2021) · European Journal of Science and Technology · doi:10.31590/ejosat.938604 · Cited by 1 · see citing works
Image Segmentation Procedure for Mapping Spatial Quality of Slow Routes
Scandiffio, A. (2022) · FrancoAngeli srl eBooks · doi:10.3280/oa-845-c215 · Cited by 1 · see citing works
MPTS-AFBP: Multi-pedestrian Tracking and Segmentation Based on Anchor-Free Detector
He, X., Cheng, K., & Meng, C. (2021) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai53619.2021.9551053
PolyTracker: Progressive Contour Regression for Multiple Object Tracking and Segmentation
Shen, S., Feng, H., Zhou, W., & Li, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18916-6_50
Cao, X., Zheng, Y., Yao, Y., Qin, H., Cao, X., & Guo, S. (2023) · doi:10.36227/techrxiv.24071700
Robust Environmental Perception of Multi-object Tracking
Pan, H., Wang, J., Yu, X., Sun, W., & Gao, H. (2023) · Recent advancements in connected autonomous vehicle technologies · doi:10.1007/978-981-99-7790-1_6
Wen, C., Gao, Y., & Li, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446257
Learning Appearance and Motion Cues for Panoptic Tracking
Hurtado, J. V., Marvi, S., Mohan, R., & Valada, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246265
Chen, L., Liao, G., Lou, J. J., Xu, F., Hu, B., Chen, L., & Zeng, H. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247101
Chin, Y.-T., Huang, T.-C., Wu, S.-J., & Liao, W.-H. (2025) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan66881.2025.11208111
- Authors
- Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulo, S. R., & Kontschieder, P.
- Affiliations
- Graz University of Technology (AT)
- Universitat Autònoma de Barcelona (ES)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 6845-6854
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2993027547
- Cited by
- 82
- References
- 53
- Citations by year
- 2025: 14 · 2024: 13 · 2023: 10 · 2022: 18 · 2021: 19 · 2020: 8
- FWCI
- 4.24
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Agencia Estatal de Investigación
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- multi-target tracking, instance segmentation, automatic training data generation, optical flow, tracking-by-detection, object association
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
