Video Copy Detection Using Spatio-Temporal CNN Features
Open accessThe DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index; none of the 7 sources its claims cite is checked for support yet.
Abstract
To protect the copyright of digital videos, video copy detection has become a hot topic in the field of digital copyright protection. Since a video sequence generally contains a large amount of data, to achieve efficient and effective copy detection, the key issue is to extract compact and discriminative video features. To this end, we propose a video copy detection scheme using spatio-temporal convolutional neural network (CNN) features. First, we divide each video sequence into multiple video clips and sample the frames of each video clip. Second, the sampled frames of each video clip are fed into a pre-trained CNN model to generate the corresponding convolutional feature maps (CFMs). Third, based on the generated CFMs, we extract the CNN features on the spatial and temporal domains of each video clip, i.e., the spatio-temporal CNN features. Finally, video copy detection is efficiently and effectively implemented based on the extracted spatio-temporal CNN features. The experiments on the commonly used video dataset, i.e., TRECVID 2008, demonstrate that the proposed method performs well in aspects of both accuracy and efficiency and shows superiority to several other copy detection methods using the state-of-the-art features.
Overview
CoChat Summary
TL;DR
This paper proposes video copy detection using spatio-temporal CNN features extracted at video-clip level from AlexNet convolutional feature maps, combining a 256-dimensional spatial feature with temporal features based on centroid coordinates. On TRECVID 2008, it reports higher mAP than sum-pooling and max-pooling baselines, with slightly higher time and memory costs.
Key findings
- On TRECVID 2008, the proposed method achieves mAP 0.653, compared with 0.648 for sum-pooling and 0.6431 for max-pooling; average time per query is 0.87 s versus 0.56 s and 0.58 s, and index size is 6.63 MB versus 6.27 MB and 6.27 MB.
- The spatial CNN feature is 256-dimensional, generated from the last convolutional layer’s CFMs via sum-pooling on a fused image of each sampled video clip.
- Temporal CNN features are extracted from pairwise distance and angle relationships among frame CFM centroids, complementing the spatial feature and improving discriminability.
- Parameter experiments select L = 7, b = 10, and weighting factors α = 0.9 and β = 0.9 as giving optimal mAP; smaller L and b improve accuracy but increase time cost.
Methods
- Each video is divided into clips of L frames, frames are sparsely sampled at interval b, and sampled frames are fused into one image for spatial feature extraction using a pre-trained AlexNet model’s last convolutional layer CFMs and sum-pooling.
- For temporal features, each sampled frame is fed into the CNN, an average CFM is computed per frame, centroid coordinates are calculated, and pairwise distances and angles among centroids form two temporal features; an inverted index is built with BOW/K-means, storing spatial features and centroid coordinates.
- Evaluation uses TRECVID 2008, with 31 original videos as queries and 310 videos generated by 10 common modifications; performance is measured by PR curve, mAP, average time per query, and index memory.
Limitations
- The paper states that its method requires slightly higher memory space and time cost than the two compared pooling-based methods.
- Evident from the text: evaluation is limited to TRECVID 2008 with 31 original videos and 310 copies; no cross-dataset or larger-scale validation is reported.
- Evident from the text: the comparison table includes only sum-pooling and max-pooling variants within the proposed framework, not a broad set of external state-of-the-art methods.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 6 Not checked · 22 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“According to the two gures, when the value of is 0.9 and is 0.9, the optimal mAP value is obtained.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The examples demonstrate that our method can achieve desirable accuracy for copy detection.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental results demonstrate that the proposed copy detection method based on the spatial-temporal CNN features can ef ciently and effectively detect video copies from datasets.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From the gure, it is clear that smaller L and b lead to better detection performance.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, too large or too small leads to inferior performance.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
5 more claims
Create a free account to see all 30 claims and every source verdict.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 10
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ A. GROUPING AND SAMPLING VIDEO FRAMES · p. 3
“Then, each video clip Ci is sampled to Si D f s ( i ; 1) ; s ( i ; 2) ; ; s ( i ; T ) g with the sampling interval b, where T D b L = b c means the number of frames in the sampled video clip Si .”
§ B. PARAMETER SELECTION · p. 6
“Therefore, to achieve a good tradeoff between accuracy, memory consumption, and time cost, we set L and b to 7 and 10, respectively, which are used in the following experiments.”
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 2×
“One of most typical hand-crafted features used for copy detection is high-dimensional scale invariant feature transform (SIFT) feature [27], which is usually combined with bag-of-visual-word (BOW) model to build an inverted index structure to speed up the feature matching process between videos.”
MethodSpeeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 1×
“In earlier works, video copy detection methods mostly use handcrafted features including SIFT [27] and its extensions such as principal component analysis on SIFT (PCA-SIFT) feature [28] and speeded-up robust feature (SURF) [29].”
MethodORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · cited 1×
“combine SURF features with Oriented Fast and Rotated Brief (ORB) features [30] for copy detection.”
MethodPCA-SIFT: a more distinctive representation for local image descriptors
Ke, Y., & Sukthankar, R. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315206 · Cited by 3,237
Recent Progress on Generative Adversarial Networks (GANs): A Survey
Pan, Z., Yu, W., Yi, X., Khan, A., Yuan, F., & Zheng, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2905015 · Cited by 701
Aggregating Local Deep Features for Image Retrieval
Yandex, A. B., & Lempitsky, V. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.150 · Cited by 674
From generic to specific deep representations for visual recognition
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301270 · Cited by 394
Exploiting local features from deep networks for image retrieval
Ng, J. Y.-H., Yang, F., & Davis, L. S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301272 · Cited by 344
Effective and Efficient Global Context Verification for Image Copy Detection
Zhou, Z., Wang, Y., Wu, Q. M. J., Yang, C.-N., & Sun, X. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2601065 · Cited by 336
Self-citation[Paper] Visual Instance Retrieval with Deep Convolutional Networks
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.251 · Cited by 276 · cited 1×
“Several meth-ods [31]–[34] have demonstrated that the features extracted from the last convolutional layer show superior performance than the features from the fully connected layers.”
BackgroundAggregating Deep Convolutional Features for Image Retrieval
Babenko, A., & Lempitsky, V. (2015) · arXiv · doi:10.48550/arxiv.1510.07493 · Cited by 224 · cited 1×
“Then, the sum-pooling operation [36] is implemented on each generated CFM to generate the spatial CNN feature of the video clip S i , denoted as SF i = { sf ( i , 1) , sf ( i , 2) , · · · , sf ( i , K ) } , K = 256.”
MethodSpatiotemporal sequence matching for efficient video copy detection
Kim, C., & Vasudev, B. (2005) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2004.836751 · Cited by 223
Time-aware distributed service recommendation with privacy-preservation
Qi, L., Wang, R., Hu, C., Li, S., He, Q., & Xu, X. (2019) · Information Sciences · doi:10.1016/j.ins.2018.11.030 · Cited by 210 · cited 1×
“With the rapid development of multimedia technology and the wide use of Internet, the number of digital multimedia files increase exponentially on the Internet, and many security issues of multimedia data have arisen [1]–[12].”
BackgroundQi, L., Zhang, X., Dou, W., Hu, C., Yang, C., & Chen, J. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.02.050 · Cited by 192
Self-citationQi, L., Chen, Y., Yuan, Y., Fu, S., Zhang, X., & Xu, X. (2020) · World Wide Web · doi:10.1007/s11280-019-00684-y · Cited by 192
Gong, W., Qi, L., & Xu, Y. (2018) · Wireless Communications and Mobile Computing · doi:10.1155/2018/3075849 · Cited by 166
An Image-Based Approach to Video Copy Detection With Spatio-Temporal Post-Filtering
Douze, M., Jegou, H., & Schmid, C. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2046265 · Cited by 165 · cited 1×
“[16] extract SIFT features from video frames and match individual video frames based on their SIFT features and the BOW model to detect video copies.”
MethodA fusion steganographic algorithm based on Faster R-CNN
Meng, R. (2018) · doi:10.3970/cmc.2018.055.001 · Cited by 149
Effective and Efficient Image Copy Detection with Resistance to Arbitrary Rotation
Zhou, Z., Yang, C., Chen, B., Sun, X., Liu, Q., & Wu, Q. M. J. (2016) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2015edp7341 · Cited by 114 · cited 2×
“There are two ways for detecting illegal copies: digital watermarking and content-based copy detection [1]-[4].”
BackgroundMethodSelf-citationExploiting Local Features from Deep Networks for Image RetrievalPreprint
Ng, J. Y.-H., Yang, F., & Davis, L. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.05133 · Cited by 95
Lei, J., Duan, J., Wu, F., et al. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2617332 · Cited by 94
Dynamic Mobile Crowdsourcing Selection for Electricity Load Forecasting
Qi, L., Dou, W., Wang, W., Li, G., Yu, H., & Wan, S. (2018) · IEEE Access · doi:10.1109/access.2018.2866641 · Cited by 91
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., & Kompatsiaris, Y. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-51811-4_21 · Cited by 83 · cited 1×
“Kordopatis-Zilos et al. [25] extract frame-level descriptors by applying max pooling on the activations of multiple convolutional layers of CNN. Zhang et al. [26] directly use the famous CNN model, i.e. , Alexnet’s output of the sixth full connected (FC6) layer as the keyframe-level representation,…”
MethodIntegrating SIFT and CNN Feature Matching for Partial-Duplicate Image Detection
Zhou, Z., Wu, Q. M. J., Wan, S., Sun, W., & Sun, X. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2909936 · Cited by 57
Self-citationSecuring display path for security-sensitive applications on mobile devices
(2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · doi:10.3970/cmc.2018.055.017 · Cited by 52
Verifiable Diversity Ranking Search Over Encrypted Outsourced Data
Liu, Y., Peng, H., & Wang, J. (2018) · doi:10.3970/cmc.2018.055.037 · Cited by 52
A robust hashing algorithm based on SURF for video copy detection
Yang, G., Chen, N., & Jiang, Q. (2012) · Computers & Security · doi:10.1016/j.cose.2011.11.004 · Cited by 48
Video Sequence Matching Based on the Invariance of Color Correlation
Lei, Y., Luo, W., Wang, Y., & Huang, J. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2201670 · Cited by 43 · cited 2×
“Most of the existing content-based copy detection methods can be roughly categorized into handcrafted feature-based [17]–[21] and convolutional neural network (CNN) feature-based methods [22]–[26].”
BackgroundCompact Deep Invariant Descriptors for Video Retrieval
Lou, Y., Bai, Y., Lin, J., Wang, S., Chen, J., Chandrasekhar, V., Duan, L.-Y., Huang, T., Kot, A. C., & Gao, W. (2017) · 2017 Data Compression Conference (DCC) · doi:10.1109/dcc.2017.31 · Cited by 40 · cited 1×
“[22] propose a Nested Invariance Pooling (NIP) method to obtain compact and robust CNN descriptors.”
MethodSelf-citationKim, B. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2007.913748 · Cited by 39
Can an element form a two-dimensional nanosheet of type 15 pentagons?
Liu, L., Kankam, I., & Zhuang, H. L. (2018) · Computational Materials Science · doi:10.1016/j.commatsci.2018.07.031 · Cited by 38
Two-class 3D-CNN classifiers combination for video copy detection
Li, J., Zhang, H., Wan, W., & Sun, J. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6047-9 · Cited by 29 · cited 1×
“[23] use a 3D-CNN model to extract features from video streams directly, and then convert multi-classification”
MethodShape-Preserving Object Depth Control for Stereoscopic Images
Lei, J., Peng, B., Zhang, C., et al. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2749146 · Cited by 24
Multiple Distance-Based Coding: Toward Scalable Feature Matching for Large-Scale Web Image Search
Zhou, Z., Wu, Q. M. J., & Sun, X. (2019) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2919570 · Cited by 24 · cited 1×
“There are two ways for detecting illegal copies: digital watermarking and content-based copy detection [13]–[16].”
BackgroundSelf-citationLow complexity versatile video coding for traffic surveillance system
Pan, Z., Qin, H., Yi, X., Zheng, Y., & Khan, A. (2019) · International Journal of Sensor Networks · doi:10.1504/ijsnet.2019.10020953 · Cited by 20 · cited 1×
“In addition, content-based copy detection techniques can be also applied in many other emerging applications [5]-[14].”
MethodVideo Copy Detection Based on Deep CNN Features and Graph-Based Sequence Matching
Zhang, X., Xie, Y., Luan, X., He, J., Zhang, L., & Wu, L. (2018) · Wireless Personal Communications · doi:10.1007/s11277-018-5450-x · Cited by 12
Robust video copy detection in large-scale TV streams using local features and CFAR based threshold
Özbulak, G., Kahraman, F., & Baykut, S. (2016) · IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1109/icdsp.2016.7868529 · Cited by 9 · cited 1×
“In [19], Özbulak et al. combine SURF features with Oriented Fast and Rotated Brief (ORB) features [30] for copy detection.”
BackgroundContent-based Video Copy Detection using Binary Object Fingerprints
Liu, M., Po, L.-M., Zhou, C. S., Yuen, W. Y. F., Cheung, H. K., Wong, P. H. W., Luk, H.-T., & Lau, K.-W. (2018) · IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) · doi:10.1109/icspcc.2018.8567827 · Cited by 5 · cited 3×
“Most of the existing content-based copy detection methods can be roughly categorized into handcrafted feature-based [15]-[19] and convolutional neural network (CNN) featurebased methods [20]-[24].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- 2008, Trec Video Retrieval Evaluation
Cited by
Cheng, H., Yu, H., Zeng, P., Osipov, E., Li, S., & Vyatkin, V. V. (2020) · Sensors · doi:10.3390/s20195659 · Cited by 56
“The Conv layer is the key functional block of a CNN, which convolutes a filter matrix with values from a receptive field of neurons [37] and finally extracts representative features from input data.”
BackgroundGanga, B., Lata, B. T., & Venugopal, K. R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127932 · Cited by 55
Learning Spatiotemporal Features for Esophageal Abnormality Detection From Endoscopic Videos
Ghatwary, N., Zolgharni, M., Janan, F., & Ye, X. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2995193 · Cited by 34
“Zhou et al. [19] suggested a video copy detection system utilizing spatiotemporal features by dividing the video sequence into multiple video clips and extracting CNN features from the spatial and temporal domains of each video clip.”
Nayak, A. A., Venugopala, P. S., Sarojadevi, H., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17386-3 · Cited by 9
“In [39], spatiotemporal features and features extracted using CNN approach play a crucial role in enhancing accuracy and efficiency.”
CNN Feature-Based Image Copy Detection with Contextual Hash Embedding
Zhou, Z., Wang, M., Cao, Y., & Su, Y. (2020) · Mathematics · doi:10.3390/math8071172 · Cited by 10
“In another work [29], the spatial-temporal CNN features are generated and matched for video copy detection.”
MethodSelf-citation
Metadata
- Authors
- Zhou, Z., Chen, J., Yang, C.-N., & Sun, X.
- Affiliations
- Nanjing University of Information Science and Technology (CN)
- National Dong Hwa University (TW)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 100658-100665
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2964069226
- Semantic Scholar
- 01a443ccd8e13d930a7bf97a9b34cfbce9a246e7
- Cited by
- 18
- References
- 40
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 5 · 2023: 1 · 2022: 3 · 2021: 1 · 2020: 3
- FWCI
- 0.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61602253, U1836208, U1536206, U1836110, 61672294)
- National Key Research and Development Program of China (2018YFB1003205)
- Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund
- Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET) fund, China
- Ministry of Science and Technology (MOST) through Pervasive Artificial Intelligence Research (PAIR) Labs, Taiwan (108-2634-F-259-001)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- video copy detection, spatio-temporal features, convolutional neural network features, video copyright protection, video fingerprinting, frame sampling, feature extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.