Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Open accessAbstract
Simultaneous tracking of multiple persons in real-world environments is an active research field and several approaches have been proposed, based on a variety of features and algorithms. Recently, there has been a growing interest in organizing systematic evaluations to compare the various techniques. Unfortunately, the lack of common metrics for measuring the performance of multiple object trackers still makes it hard to compare their results. In this work, we introduce two intuitive and general metrics to allow for objective comparison of tracker characteristics, focusing on their precision in estimating object locations, their accuracy in recognizing object configurations and their ability to consistently label objects over time. These metrics have been extensively used in two large-scale international evaluations, the 2006 and 2007 CLEAR evaluations, to measure and compare the performance of multiple object trackers for a wide variety of tracking tasks. Selected performance results are presented and the advantages and drawbacks of the presented metrics are discussed based on the experience gained during the evaluations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The MOTChallenge benchmark implements all CLEAR MOT evaluation metrics.
“However, the MOTChallenge benchmark implements all CLEAR MOT [5] evaluation metrics.”
From TrackFormer: Multi-Object Tracking with Transformers (2022) · cites this paper as [5]
Queries are able to discriminate between features from the previous and current frames by applying a temporal feature encoding.
“Queries are able to discriminate between features from the two frames by applying a temporal feature encoding as in [56].”
From TrackFormer: Multi-Object Tracking with Transformers (2022) · cites this paper as [56]
Cited by
Sorted by citations
Simple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286 · see citing works
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433 · see citing works
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00252 · Cited by 3,437 · see citing works
You'll never walk alone: Modeling social behavior for multi-target tracking
Pellegrini, S., Ess, A., Schindler, K., & van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459260 · Cited by 1,683 · see citing works
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking
Zhang, Y., Wang, C., Wang, X., Zeng, W., & Liu, W. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01513-4 · Cited by 1,861 · see citing works
MOT16: A Benchmark for Multi-Object Tracking
Milan, A., Leal-Taixe, L., Reid, I., Roth, S., & Schindler, K. (2016) · arXiv · doi:10.48550/arxiv.1603.00831 · Cited by 1,309 · see citing works
Towards Real-Time Multi-Object Tracking
Wang, Z., Zheng, L., Liu, Y., Li, Y., & Wang, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_7 · Cited by 1,195 · see citing works
StrongSORT: Make DeepSORT Great Again
Du, Y., Zhao, Z., Song, Y., Zhao, Y., Su, F., Gong, T., & Meng, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3240881 · Cited by 1,191 · see citing works
“We use the metrics MOTA, IDs, IDF1, HOTA, AssA, DetA and FPS to evaluate tracking performance [2, 40, 29].”
MethodVirtualWorlds as Proxy for Multi-object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.470 · Cited by 1,079 · see citing works
The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking
Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., & Tian, Q. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_23 · Cited by 1,099 · see citing works
Multiple Object Tracking Using K-Shortest Paths Optimization
Berclaz, J., Fleuret, F., Turetken, E., & Fua, P. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.21 · Cited by 1,015 · see citing works
TrackFormer: Multi-Object Tracking with Transformers
Meinhardt, T., Kirillov, A., Leal-Taixe, L., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00864 · Cited by 1,081 · see citing works
“[2] However, the MOTChallenge benchmark implements all CLEAR MOT [2] evaluation metrics.”
MethodMultiple Hypothesis Tracking Revisited
Kim, C., Li, F., Ciptadi, A., & Rehg, J. M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.533 · Cited by 666 · see citing works
Deep learning in video multi-object tracking: A survey
Ciaparrone, G., Luque Sánchez, F., Tabik, S., Troiano, L., Tagliaferri, R., & Herrera, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.023 · Cited by 665 · see citing works
MOTS: Multi-Object Tracking and Segmentation
Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B. B. G., Geiger, A., & Leibe, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00813 · Cited by 658 · see citing works
Stable multi-target tracking in real-time surveillance video
Benfold, B., & Reid, I. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995667 · Cited by 651 · see citing works
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · see citing works
Features for Multi-target Multi-camera Tracking and Re-identification
Ristani, E., & Tomasi, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00632 · Cited by 556 · see citing works
Chen, L., Ai, H., Zhuang, Z., & Shang, C. (2018) · 2018 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486597 · Cited by 530 · see citing works
Learning by Tracking: Siamese CNN for Robust Target Association
Leal-Taixe, L., Canton-Ferrer, C., & Schindler, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.59 · Cited by 444 · see citing works
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking
Dendorfer, P., Os̆ep, A., Milan, A., Schindler, K., Cremers, D., Reid, I., Roth, S., & Leal-Taixé, L. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01393-0 · Cited by 374 · see citing works
TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
Pang, B., Li, Y., Zhang, Y., Li, M., & Lu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00634 · Cited by 303 · see citing works
The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes
Patil, A., Malla, S., Gang, H., & Chen, Y.-T. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793925 · Cited by 263 · see citing works
Joint Monocular 3D Vehicle Detection and Tracking
Hu, H.-N., Cai, Q.-Z., Wang, D., Lin, J., Sun, M., Kraehenbuehl, P., Darrell, T., & Yu, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00549 · Cited by 249 · see citing works
Robust Multi-Modality Multi-Object Tracking
Zhang, W., Zhou, H., Sun, S., Wang, Z., Shi, J., & Loy, C. C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00245 · Cited by 250 · see citing works
Learning to Track with Object Permanence
Tokmakov, P., Li, J., Burgard, W., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01068 · Cited by 226 · see citing works
No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles Using Cameras and LiDARs
Rangesh, A., & Trivedi, M. M. (2019) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2019.2938110 · Cited by 158 · see citing works
GSM: Graph Similarity Model for Multi-Object Tracking
Liu, Q., Chu, Q., Liu, B., & Yu, N. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/74 · Cited by 123 · see citing works
A Systematic Review of Drone Based Road Traffic Monitoring System
Bisio, I., Garibotto, C., Haleem, H., Lavagetto, F., & Sciarrone, A. (2022) · IEEE Access · doi:10.1109/access.2022.3207282 · Cited by 121 · see citing works
“In addition, the litera434 ture proposes the following two standard MOT evaluation 435 metrics [61].”
MethodLearning Multi-Object Tracking and Segmentation From Automatic Annotations
Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulo, S. R., & Kontschieder, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00688 · Cited by 82 · see citing works
Orientation- and Scale-Invariant Multi-Vehicle Detection and Tracking from Unmanned Aerial Videos
Wang, J., Simeonova, S., & Shahbazi, M. (2019) · Remote Sensing · doi:10.3390/rs11182155 · Cited by 50 · see citing works
An efficient edge artificial intelligence multipedestrian tracking method with rank constraint
Yang, H., Wen, J., Wu, X., He, L., & Mumtaz, S. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2897128 · Cited by 38 · see citing works
- Authors
- Bernardin, K., & Stiefelhagen, R.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- ORCID
- Stiefelhagen, Rainer
- Venue
- EURASIP Journal on Image and Video Processing
- Published
- 2008
- Volume
- 2008
- Pages
- 1-10
- ISSN
- 1687-5176 (ISSN-L)
- 1687-5281
- OpenAlex
- W2124781496
- Cited by
- 2,918
- References
- 0
- Citations by year
- 2026: 253 · 2025: 327 · 2024: 340 · 2023: 305 · 2022: 277 · 2021: 243 · 2020: 207 · 2019: 169 · 2018: 155 · 2017: 124 · 2016: 143 · 2015: 117 · 2014: 85 · 2013: 66 · 2012: 58
- FWCI
- 23.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- EURASIP Journal on Image and Video Processing (publishedVersion)
- Repository KITopen (Karlsruhe Institute of Technology) (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- KITopen (other-oa)
- Funders
- European Commission
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Target Tracking and Data Fusion in Sensor Networks
- Keywords
- multiple object tracking, tracking performance evaluation, human tracking, object localization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
