Content-Based Video Copy Detection Benchmarking at TRECVID
The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Abstract
This article presents an overview of the video copy detection benchmark which was run over a period of 4 years (2008--2011) as part of the TREC Video Retrieval (TRECVID) workshop series. The main contributions of the article include i) an examination of the evolving design of the evaluation framework and its components (system tasks, data, measures); ii) a high-level overview of results and best-performing approaches; and iii) a discussion of lessons learned over the four years. The content-based copy detection (CCD) benchmark worked with a large collection of synthetic queries, which is atypical for TRECVID, as was the use of a normalized detection cost framework. These particular evaluation design choices are motivated and appraised.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Livescu, K., Bezman, A., Borges, N., Yung, L.-Y., Çetin, Ö., Frankel, J., King, S., Magimai.-Doss, M., Chi, X., & Lavoie, L. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,145
An efficient parts-based near-duplicate and sub-image retrieval system
Ke, Y., Sukthankar, R., & Huston, L. (2004) · Proceedings of the 12th annual ACM international conference on Multimedia · doi:10.1145/1027527.1027729 · Cited by 361
Practical elimination of near-duplicates from web video search
Wu, X., Hauptmann, A. G., & Ngo, C.-W. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291280 · Cited by 340
Video copy detection: a comparative study
Law-To, J., Chen, L., Joly, A., Laptev, I., Buisson, O., Gouet-Brunet, V., Boujemaa, N., & Stentiford, F. (2007) · Proceedings of the 6th ACM international conference on Image and video retrieval · doi:10.1145/1282280.1282336 · Cited by 273 · cited 8×
“The query creation procedure followed during TRECVID 2008-2011 was built on the work done during the first live benchmark initiative for video copy detection that took place during CIVR2007 and supported by the MUSCLE network of excellence.”
BackgroundMethodInfluentialContent-Based Copy Retrieval Using Distortion-Based Probabilistic Similarity Search
Joly, A., Buisson, O., & Frelicot, C. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.886278 · Cited by 261 · cited 1×
“These transformations used in MUSCLE – similarly in [Meng et al. 2003],[Ke and Sukthankar 2004],[Chiu et al. 2006], and [Joly et al. 2007] – seem to represent a variety of plausible effects that might realistically follow from re-encoding for alternate platforms/devices, editing for inclusion in…”
MethodNear-duplicate video retrieval: Current research and future trends
Liu, J., Huang, Z., Cai, H., Shen, H. T., Ngo, C. W., & Wang, W. (2013) · ACM Computing Surveys · doi:10.1145/2501654.2501658 · Cited by 179 · cited 4×
“A more comprehensive introduction into system architectures for content based copy detection is provided in [Liu et al. 2013]”
BackgroundMethodInfluentialAn Image-Based Approach to Video Copy Detection With Spatio-Temporal Post-Filtering
Douze, M., Jegou, H., & Schmid, C. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2046265 · Cited by 165 · cited 1×
“For details see [Douze et al. 2008] and [Douze et al. 2010].”
BackgroundProceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315129 · Cited by 143
Audio Fingerprinting: Combining Computer Vision & Data Stream Processing
Baluja, S., & Covell, M. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366210 · Cited by 117 · cited 1×
“Since detection of untransformed audio copies was relatively easy [Baluja and Covell 2007], and the primary interest of the TRECVID community was in video analysis, it was decided to model the required task with video-only queries.”
MethodAlbiol, A., Ch, M. J., Albiol, F. A., & Torres, L. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326601 · Cited by 77 · cited 1×
“…(perhaps embedded in other non-reference video) is a very important problem in various application settings, such as: — Tracking the exact number of broadcasts of a certain commercial (airing a commercial is costly, so monitoring is a business goal) [Sadlier et al. 2002] , [Albiol et al. 2004].”
BackgroundAutomatic TV advertisement detection from MPEG bitstream
Sadlier, D. A., Marlow, S., O'Connor, N. E., & Murphy, N. A. (2002) · Pattern Recognition · doi:10.1016/s0031-3203(01)00251-5 · Cited by 73 · cited 3×
“Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax + 1 (212) 869-0481, or permissions@acm.org. c (cid:13) 2099 ACM 1046-8188/2099/00-ART99 $15.00 DOI: http://dx.doi.org/10.1145/0000000.0000000 — Improving video search by suppressing or clustering…”
BackgroundInfluentialPractical Online Near-Duplicate Subsequence Detection for Continuous Video Streams
Huang, Z. H., Shen, H. T., Shao, J., Cui, B., & Zhou, X. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2050737 · Cited by 56 · cited 2×
“Following an advertisement campaign of competitors as it appears in a number of versions tailored to different timeslots, with different local information overlays, etc. [Huang et al. 2010].”
BackgroundInfluentialInternet image archaeology: automatically tracing the manipulation history of photographs on the web
Kennedy, L., & Chang, S.-F. (2008) · Proceedings of the 16th ACM international conference on Multimedia · doi:10.1145/1459359.1459406 · Cited by 50 · cited 1×
“The authors are unable find any systematic empirical studies describing the nature and number of actual transformations in video copies as defined in this paper; [Kennedy and Chang 2008] analyze Internet image edit differences in real data to create ground truth but do not provide statistics.”
BackgroundA Time Warping Based Approach for Video Copy Detection
Chiu, C., Li, C.-H., Wang, H.-A., Chen, C., & Chien, L. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.187 · Cited by 44 · cited 2×
“2003],[Ke and Sukthankar 2004],[Chiu et al. 2006], and [Joly et al.”
BackgroundMethodEnhancing DPF for near-replica image recognition
Meng, Y., Chang, E., & Li, B. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211498 · Cited by 44 · cited 2×
“These transformations used in MUSCLE – similarly in [Meng et al. 2003],[Ke and Sukthankar 2004],[Chiu et al.”
MethodEffective and scalable video copy detection
Liu, Z., Liu, T., Gibbon, D. C., & Shahraray, B. (2010) · Multimedia Information Retrieval · doi:10.1145/1743384.1743409 · Cited by 37 · cited 1×
“For details see [Liu et al. 2009] and [Liu et al. 2010].”
BackgroundAT&T Research at TRECVID 2009 Content-based Copy Detection
Liu, Z., Shahraray, B., & Liu, T. (2009) · TRECVID · Cited by 33
A survey of the image copy detection
Wan, Y., Yuan, Q., Ji, S., He, L., & Wang, Y. (2008) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/iccis.2008.4670942 · Cited by 30 · cited 1×
“It also makes no attempt to support research in the use of watermarks to detect copies but rather reflects a scenario in which no mark was added to the original before copies could be made, “the case for a large part of the existing content” according to [Wan et al. 2008].”
BackgroundCRIM’s content-based audio copy detection system for TRECVID 2009
Gupta, V., Boulianne, G., & Cardinal, P. (2010) · Multimedia Tools and Applications · doi:10.1007/s11042-010-0608-x · Cited by 25
CRIM's content-based copy detection system for TRECVID
Héritier, M., Gupta, V., Gagnon, L., Boulianne, G., Foucher, S., & Cardinal, P. (2009) · TRECVID · Cited by 24
Video Copy Detection Using a Soft Cascade of Multimodal Features
Jiang, M., Tian, Y., & Huang, T. (2012) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2012.189 · Cited by 20 · cited 1×
“For details see [Jian et al. 2011] and [Jiang et al. 2012].”
BackgroundA multimodal video copy detection approach with sequential pyramid matching
Tian, Y., Jiang, M., Mou, L., Rang, X., & Huang, T. (2011) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2011.6116504 · Cited by 18
Effective and Efficient Image Copy Detection Based on GPU
Xie, H., Gao, K., Zhang, Y., Li, J., Liu, Y., & Ren, H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35740-4_26 · Cited by 12
Content-based video copy detection using nearest-neighbor mapping
Gupta, V., Varcheie, P. D. Z., Gagnon, L., & Boulianne, G. (2012) · International Conference on Information Science, Signal Processing and their Applications (ISSPA) · doi:10.1109/isspa.2012.6310685 · Cited by 11
CRIM AT TRECVID-2011: CONTENT-BASED COPY DETECTION USING NEAREST-NEIGHBOR MAPPING
Gupta, V., Varcheie, P. D. Z., Varcheie, Z., Gagnon, L., & Boulianne, G. (2011) · Cited by 9
YouTube or YouLose? Can YouTube Survive a Copyright Infringement Lawsuit
Breen, J. C. (2007) · bepress Legal Repository · Cited by 6
Content-based video copy detection
Barrios, J. M. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631539 · Cited by 5
Evaluating detection of near duplicate video segments
Werner Bailer, A. (2010) · ACM International Conference on Image and Video Retrieval (CIVR) · doi:10.1145/1816041.1816072 · Cited by 3 · cited 1×
“(Although NDCR via its parameterization supports a variety of application scenarios, comparison of re-sults across studies beyond TRECVID would require comparable parameter settings [Bailer 2010].)”
MethodVisual Tools for Assisting Child Pornography Investigators
Eendebak, P. T., Kraaij, W., Raaijmakers, S. A., Ranguelova, E., de Rooij, O., Thean, A. H. C., & Worring, M. (2008) · Cited by 2
Self-citation
Research works printed but not linked: 2
- Jonathan G. Fiscus and George R . Doddington . 2002 . Topic Detection and Tracking Evaluation Overview. Kluwer , Norwell, MA, 17--31. http://dl.acm.org/citation.cfm?id=772260.772263. Jonathan G. Fiscus and George R. Doddington. 2002. Topic Detection and Tracking Evaluation Overview. Kluwer, Norwell, MA, 17--31. http://dl.acm.org/citation.cfm?id=772260.772263.
- J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http://wwwrocq.inria.fr/imedia/civr-bench/. J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http://wwwrocq.inria.fr/imedia/civr-bench/. · DOI · Paper page
Cited by
Comprehensive Feature-Based Robust Video Fingerprinting Using Tensor Model
Nie, X., Yin, Y., Sun, J., Liu, J., & Cui, C. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2629758 · Cited by 56
“org); another is a standard database collected from the TRECVID CCD task [31].”
Joint Multi-View Hashing for Large-Scale Near-Duplicate Video Retrieval
Nie, X., Jing, W., Cui, C., Zhang, C., Zhu, L., & Yin, Y. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2913383 · Cited by 52
Robust Image Fingerprinting Based on Feature Point Relationship Mining
Nie, X., Li, X., Chai, Y., Cui, C., Xi, X., & Yin, Y. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2790953 · Cited by 38
An image-based near-duplicate video retrieval and localization using improved Edit distance
Liu, H., Zhao, Q., Wang, H., Lv, P. M., & Chen, Y. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-4176-6 · Cited by 26
“What’s more, for each type of original videos, there are also eight transformations applied, as listed in Table 1 [2].”
BackgroundMethodAdvance on large scale near-duplicate video retrieval
Shen, L., Hong, R., & Hao, Y. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-019-8229-7 · Cited by 25
Metadata
- Authors
- Awad, G., Over, P., & Kraaij, W.
- Affiliations
- NOAA Climate Program Office (US)
- National Institute of Standards and Technology (US)
- Radboud University Nijmegen (NL)
- Venue
- ACM Transactions on Information Systems
- Published
- 2014-06
- Volume
- 32
- Issue
- 3
- Pages
- 1-40
- ISSN
- 1046-8188 (ISSN-L)
- 1558-2868
- OpenAlex
- W2141622653
- Cited by
- 30
- References
- 31
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 1 · 2021: 1 · 2020: 2 · 2019: 1 · 2018: 4 · 2017: 4 · 2016: 10 · 2015: 4
- FWCI
- 5.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Information Systems (publishedVersion, public-domain)
- TNO Repository (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- video copy detection, content-based video retrieval, TRECVID, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.