Content-Based Video Copy Detection Benchmarking at TRECVID
The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Abstract
This article presents an overview of the video copy detection benchmark which was run over a period of 4 years (2008--2011) as part of the TREC Video Retrieval (TRECVID) workshop series. The main contributions of the article include i) an examination of the evolving design of the evaluation framework and its components (system tasks, data, measures); ii) a high-level overview of results and best-performing approaches; and iii) a discussion of lessons learned over the four years. The content-based copy detection (CCD) benchmark worked with a large collection of synthetic queries, which is atypical for TRECVID, as was the use of a normalized detection cost framework. These particular evaluation design choices are motivated and appraised.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Livescu, K., Bezman, A., Borges, N., Yung, L.-Y., Çetin, Ö., Frankel, J., King, S., Magimai.-Doss, M., Chi, X., & Lavoie, L. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,145
An efficient parts-based near-duplicate and sub-image retrieval system
Ke, Y., Sukthankar, R., & Huston, L. (2004) · Proceedings of the 12th annual ACM international conference on Multimedia · doi:10.1145/1027527.1027729 · Cited by 361
Practical elimination of near-duplicates from web video search
Wu, X., Hauptmann, A. G., & Ngo, C.-W. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291280 · Cited by 340
Video copy detection: a comparative study
Law-To, J., Chen, L., Joly, A., Laptev, I., Buisson, O., Gouet-Brunet, V., Boujemaa, N., & Stentiford, F. (2007) · Proceedings of the 6th ACM international conference on Image and video retrieval · doi:10.1145/1282280.1282336 · Cited by 273 · cited 8×
“The query creation procedure followed during TRECVID 2008-2011 was built on the work done during the first live benchmark initiative for video copy detection that took place during CIVR2007 and supported by the MUSCLE network of excellence.”
BackgroundMethodInfluentialContent-Based Copy Retrieval Using Distortion-Based Probabilistic Similarity Search
Joly, A., Buisson, O., & Frelicot, C. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.886278 · Cited by 261 · cited 1×
“These transformations used in MUSCLE – similarly in [Meng et al. 2003],[Ke and Sukthankar 2004],[Chiu et al. 2006], and [Joly et al. 2007] – seem to represent a variety of plausible effects that might realistically follow from re-encoding for alternate platforms/devices, editing for inclusion in…”
MethodNear-duplicate video retrieval: Current research and future trends
Liu, J., Huang, Z., Cai, H., Shen, H. T., Ngo, C. W., & Wang, W. (2013) · ACM Computing Surveys · doi:10.1145/2501654.2501658 · Cited by 179 · cited 4×
“A more comprehensive introduction into system architectures for content based copy detection is provided in [Liu et al. 2013]”
BackgroundMethodInfluentialAn Image-Based Approach to Video Copy Detection With Spatio-Temporal Post-Filtering
Douze, M., Jegou, H., & Schmid, C. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2046265 · Cited by 165 · cited 1×
“For details see [Douze et al. 2008] and [Douze et al. 2010].”
BackgroundProceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315129 · Cited by 143
Audio Fingerprinting: Combining Computer Vision & Data Stream Processing
Baluja, S., & Covell, M. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366210 · Cited by 117 · cited 1×
“Since detection of untransformed audio copies was relatively easy [Baluja and Covell 2007], and the primary interest of the TRECVID community was in video analysis, it was decided to model the required task with video-only queries.”
Method
Research works printed but not linked: 2
- Jonathan G. Fiscus and George R . Doddington . 2002 . Topic Detection and Tracking Evaluation Overview. Kluwer , Norwell, MA, 17--31. http://dl.acm.org/citation.cfm?id=772260.772263. Jonathan G. Fiscus and George R. Doddington. 2002. Topic Detection and Tracking Evaluation Overview. Kluwer, Norwell, MA, 17--31. http://dl.acm.org/citation.cfm?id=772260.772263.
- J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http://wwwrocq.inria.fr/imedia/civr-bench/. J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http://wwwrocq.inria.fr/imedia/civr-bench/. · DOI · Paper page
Cited by
Comprehensive Feature-Based Robust Video Fingerprinting Using Tensor Model
Nie, X., Yin, Y., Sun, J., Liu, J., & Cui, C. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2629758 · Cited by 56
“org); another is a standard database collected from the TRECVID CCD task [31].”
Joint Multi-View Hashing for Large-Scale Near-Duplicate Video Retrieval
Nie, X., Jing, W., Cui, C., Zhang, C., Zhu, L., & Yin, Y. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2913383 · Cited by 52
Robust Image Fingerprinting Based on Feature Point Relationship Mining
Nie, X., Li, X., Chai, Y., Cui, C., Xi, X., & Yin, Y. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2790953 · Cited by 38
An image-based near-duplicate video retrieval and localization using improved Edit distance
Liu, H., Zhao, Q., Wang, H., Lv, P. M., & Chen, Y. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-4176-6 · Cited by 26
“What’s more, for each type of original videos, there are also eight transformations applied, as listed in Table 1 [2].”
BackgroundMethodAdvance on large scale near-duplicate video retrieval
Shen, L., Hong, R., & Hao, Y. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-019-8229-7 · Cited by 25
The 2023 video similarity dataset and challenge
Pizzi, E., Kordopatis-Zilos, G., Patel, H., Postelnicu, G. M., Ravindra, S. N., Gupta, A., Papadopoulos, S., Tolias, G., & Douze, M. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.103997 · Cited by 21
Visual Interestingness Prediction: A Benchmark Framework and Literature Review
Constantin, M., Ștefan, L.-D., Ionescu, B., Duong, N. Q. K., Demarty, C.-H., & Sjöberg, M. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01443-1 · Cited by 27
Partial-copy detection of non-simulated videos using learning at decision level
Guzman-Zavaleta, Z. J., & Feregrino-Uribe, C. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6345-2 · Cited by 14
Fast Audio Fingerprinting System Using GPU and a Clustering-Based Technique
Ouali, C., Dumouchel, P., & Gupta, V. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2541303 · Cited by 10
“TRECVID 2009 and 2010 are two datasets provided by NIST [41] for the task of video copy detection.”
MethodA robust and low-cost video fingerprint extraction method for copy detection
Guzman-Zavaleta, Z. J., Feregrino-Uribe, C., Morales‐Sandoval, M., & Menendez-Ortiz, A. F.-U. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-4168-6 · Cited by 9
“Hence, state-of-the-art methods focus on robustness more than efficiency, which is not suitable for applications with real-time requirements or in high volume databases [1, 13, 42, 46]).”
BackgroundJiang, D., & Kim, J. (2018) · Applied Sciences · doi:10.3390/app8101735 · Cited by 7
“Video searching (VS) has been a challenging research topic since the mid-1990s, and video copy detection (VCD) also started at that time [1].”
BackgroundEfficient copy detection for compressed digital videos by spatial and temporal feature extraction
Su, P.-C., & Wu, C.-S. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-015-3132-1 · Cited by 7
“Many methods rely on the extraction of interest points for generating the video summary or the near-duplicate video retrieval [1, 16].”
MethodTwo-layer video fingerprinting strategy for near-duplicate video detection
Nie, X., Jing, W., Lin, Y., Cui, C., & Yin, Y. (2017) · IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew.2017.8026322 · Cited by 6
Guzman-Zavaleta, Z. J., & Feregrino-Uribe, C. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166047 · Cited by 6
The 2023 Video Similarity Dataset and ChallengePreprint
Pizzi, E., Kordopatis-Zilos, G., Patel, H., Postelnicua, G., Ravindra, S. N., Gupta, A., Papadopoulos, S., Tolias, G., & Douze, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4597445 · Cited by 5
Content-Based Multimedia Copy Detection
Ouali, C., Dumouchel, P., & Gupta, V. (2015) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism.2015.40 · Cited by 4
“A detailed description of the evaluation framework and the TRECVID datasets can be found in [10].”
MethodDomain transfer multi-instance dictionary learning
Wang, K., Liu, J., & González, D. (2016) · Neural Computing and Applications · doi:10.1007/s00521-016-2406-5 · Cited by 4
“The first data set used is the TRECVID data set [1, 26].”
MethodWary, A., & Neelima, A. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08412-4 · Cited by 4
SFD: Similar Frame Dataset for Content-Based Video Retrieval
Han, C., Meng, G., & Huo, C. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10647446 · Cited by 3
BAF: An audio fingerprinting dataset for broadcast monitoring
Cortès, G., Ciurana, A., Molina, E., Miron, M., Meyers, O., Six, J., & Serra, X. (2022) · Ghent University Academic Bibliography (Ghent University) · doi:10.5281/zenodo.7316811 · Cited by 3
Secure Multimedia Content Distribution for M2M Communication
Ye, C., Xiong, Z., Ding, Y., Zhang, X., Wang, G., & Xu, F. (2016) · International Journal of Security and Its Applications · doi:10.14257/ijsia.2016.10.4.26 · Cited by 3
Robust video fingerprints using positions of salient regions
Ouali, C., Dumouchel, P., & Gupta, V. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952715 · Cited by 2
“[14] M.-C. Yeh, C.-Y. Hsu, and C.-S. Lu, "NTNU-Academia Sinica at TRECVID 2010 content based copy detection," 2010 TREC Video Retrieval Evaluation Notebook Papers. p. National Institute of Standards and Technology (NIST).”
MethodInfluentialEncoder settings impact on intra-prediction-based descriptors for video retrieval
Rouhi, A. H., & Thom, J. A. (2017) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2017.12.009 · Cited by 2
Park, B., & Kim, Y. (2026) · Journal of Software Forensics · doi:10.29056/jsf.2026.06.08 · Cited by 1
Block based video alignment with linear time and space complexity
Kappeler, A., Iliadis, M., Wang, H., & Katsaggelos, A. K. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532975 · Cited by 1
“Furthermore they are usually evaluated on short clips such as trecvid [13] and not designed to align two long videos.”
BackgroundQuality assurance in large collections of video sequences
Polok, L., Klicnar, L., Beran, V., Smrž, P., & Zemčík, P. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351471 · Cited by 1
“The proposed methods were tested on a subset of TRECVID 2008 dataset [18], especially the BBC rushes.”
MethodNear-Duplicate Web Video Retrieval and Localization Using Improved Edit Distance
Liu, H., Zhao, Q., Wang, H., & Zhang, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45814-4_12 · Cited by 1
A simple approach towards efficient partial-copy video detection
Guzman-Zavaleta, Z. J., & Feregrino-Uribe, C. (2016) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2016.7813396 · Cited by 1
“Moreover, in [6], [18]–[20], it is also reported that current methods are not focused on applications where false alarms are not tolerated nor focused on generating compact descriptors for improving their search in databases.”
BackgroundMethodResultVideo copy detection using motion co-occurrence feature
Arabacı, M. A., & Esen, E. (2015) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2015.7130243
“Önerilen yöntem, içerik tabanlı kopya bulma probleminde yaygın olarak kullanılan TRECVID 2009 veriseti ile test edilmiştir [10].”
An Approach to Video Search Engine
Sharma, T., & Mr.Gopinath, M. P. (2017) · International Journal of Engineering and Technology · doi:10.21817/ijet/2017/v9i2/170902176
Hash length: a neglected element
Qi, H., Li, J., Wu, Q., Wan, W., & Sun, J. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6221-0
“After these processing, the CC_WEB_VIDEO Dataset with 51,802 videos in 24 sets and TRECVID 2009 Dataset with 20,210 videos in 97 sets are used in our experiments.”
MethodInfluentialBalancing Efficiency and Accuracy: An Analysis of Sampling for Video Copy Detection
Chen, X., & Satoh, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-2054-8_9
Hu, Z., Li, X., Zheng, Y., & Shen, W. (2026) · IEEE Access · doi:10.1109/access.2026.3707321
Metadata
- Authors
- Awad, G., Over, P., & Kraaij, W.
- Affiliations
- NOAA Climate Program Office (US)
- National Institute of Standards and Technology (US)
- Radboud University Nijmegen (NL)
- Venue
- ACM Transactions on Information Systems
- Published
- 2014-06
- Volume
- 32
- Issue
- 3
- Pages
- 1-40
- ISSN
- 1046-8188 (ISSN-L)
- 1558-2868
- OpenAlex
- W2141622653
- Cited by
- 30
- References
- 31
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 1 · 2021: 1 · 2020: 2 · 2019: 1 · 2018: 4 · 2017: 4 · 2016: 10 · 2015: 4
- FWCI
- 5.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Information Systems (publishedVersion, public-domain)
- TNO Repository (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- video copy detection, content-based video retrieval, TRECVID, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.