CoChat
Sign Up
Cited by 84Anomaly Detection Techniques and ApplicationsHuman Pose and Action RecognitionVideo Surveillance and Tracking Methods

Deep Video Anomaly Detection: Opportunities and Challenges

Open access
Ren, Jing, Xia, Feng, Liu, Yemeng, Lee, Ivan·2021·2021 International Conference on Data Mining Workshops (ICDMW)
DOI 10.1109/icdmw53433.2021.00125
Open this paper in CoChat — free accountOpen PDF

Abstract

Anomaly detection is a popular and vital task in various research contexts, which has been studied for several decades. To ensure the safety of people’s lives and assets, video surveillance has been widely deployed in various public spaces, such as crossroads, elevators, hospitals, banks, and even in private homes. Deep learning has shown its capacity in a number of domains, ranging from acoustics, images, to natural language processing. However, it is non-trivial to devise intelligent video anomaly detection systems cause anomalies significantly differ from each other in different application scenarios. There are numerous advantages if such intelligent systems could be realised in our daily lives, such as saving human resources in a large degree, reducing financial burden on the government, and identifying the anomalous behaviours timely and accurately. Recently, many studies on extending deep learning models for solving anomaly detection problems have emerged, resulting in beneficial advances in deep video anomaly detection techniques. In this paper, we present a comprehensive review of deep learning-based methods to detect the video anomalies from a new perspective. Specifically, we summarise the opportunities and challenges of deep learning models on video anomaly detection tasks, respectively. We put forth several potential future research directions of intelligent video anomaly detection system in various application domains. Moreover, we summarise the characteristics and technical problems in current deep learning methods for video anomaly detection.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icdmw53433.2021.00125

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Hybrid models are easier to scale up and use less computing resources.

    “Hybrid models are easier to scale up and use less computing resources (Ren etal. 2021; Peng etal. 2022).”

    From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Ren et al. 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 71

  1. [1][1] B. Ramachandra, M. Jones, and R. R. Vatsavai, 'A survey of singlescene video anomaly detection,' IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020. -
  2. [2][2] J. Ren, F. Xia, X. Chen, J. Liu, M. Hou, A. Shehzad, N. Sultanova, and X. Kong, 'Matching algorithms: Fundamentals, applications and challenges,' IEEE Transactions on Emerging Topics in Computational Intelligence , vol. 5, no. 3, pp. 332-350, 2021. -
  3. [3][3] L. Wan, F. Xia, X. Kong, C.-H. Hsu, R. Huang, and J. Ma, 'Deep matrix factorization for trust-aware recommendation in social networks,' IEEE Transactions on Network Science and Engineering , vol. 8, no. 1, pp. 511-528, 2020. -
  4. [4][4] L. Wan, Y. Sun, I. Lee, W. Zhao, and F. Xia, 'Industrial pollution areas detection and location via satellite-based IIoT,' IEEE Transactions on Industrial Informatics , vol. 17, no. 3, pp. 1785-1794, 2020. -
  5. [5][5] G. Pang, C. Shen, L. Cao, and A. V. D. Hengel, 'Deep learning for anomaly detection: A review,' ACM Computing Surveys (CSUR) , vol. 54, no. 2, pp. 1-38, 2021. -
  6. [6][6] S. C. Wong, V. Stamatescu, A. Gatt, D. Kearney, I. Lee, and M. D. McDonnell, 'Track everything: Limiting prior knowledge in online multi-object recognition,' IEEE Transactions on Image Processing , vol. 26, no. 10, pp. 4669-4683, 2017. -
  7. [7][7] X. Kong, X. Liu, B. Jedari, M. Li, L. Wan, and F. Xia, 'Mobile crowdsourcing in smart cities: Technologies, applications, and future challenges,' IEEE Internet of Things Journal , vol. 6, no. 5, pp. 80958113, 2019. -
  8. [8][8] X. Kong, K. Ma, S. Hou, D. Shang, and F. Xia, 'Human interactive behavior: A bibliographic review,' IEEE Access , vol. 7, pp. 4611-4628, 2018. -
  9. [9][9] K. Xu, T. Sun, and X. Jiang, 'Video anomaly detection and localization based on an adaptive intra-frame classification network,' IEEE Transactions on Multimedia , vol. 22, no. 2, pp. 394-406, 2019. -
  10. [10][10] B. R. Kiran, D. M. Thomas, and R. Parakkal, 'An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos,' Journal of Imaging , vol. 4, no. 2, p. 36, 2018. -
  11. [11][11] A. B. Mabrouk and E. Zagrouba, 'Abnormal behavior recognition for intelligent video surveillance systems: A review,' Expert Systems with Applications , vol. 91, pp. 480-491, 2018. -
  12. [12][12] K. Pawar and V. Attar, 'Deep learning approaches for video-based anomalous activity detection,' World Wide Web , vol. 22, no. 2, pp. 571601, 2019. -
  13. [13][13] H. Yao and X. Hu, 'A survey of video violence detection,' CyberPhysical Systems , pp. 1-24, 2021. -
  14. [15][15] J. J. P. Suarez and P. C. Naval Jr, 'A survey on deep learning techniques for video anomaly detection,' arXiv preprint arXiv:2009.14146 , 2020. - · arXiv
  15. [16][16] M. Su, C. Zhang, Y. Tong, B. Liang, S. Ma, and J. Wang, 'Deep learning in video violence detection,' in 2021 International Conference on Computer Technology and Media Convergence Design (CTMCD) . IEEE, 2021, pp. 268-272. -
  16. [17][17] S. Roshan, G. Srivathsan, K. Deepak, and S. Chandrakala, 'Violence detection in automated video surveillance: Recent trends and comparative studies,' The Cognitive Approach in Cloud Computing and Internet of Things Technologies for Surveillance Tracking Systems , pp. 157-171, 2020. -
  17. [18][18] M. Ramzan, A. Abid, H. U. Khan, S. M. Awan, A. Ismail, M. Ahmed, M. Ilyas, and A. Mahmood, 'A review on state-of-the-art violence detection techniques,' IEEE Access , vol. 7, pp. 107 560-107 575, 2019. -
  18. [19][19] B. Mohammadi, M. Fathy, and M. Sabokrou, 'Image/video deep anomaly detection: A survey,' arXiv preprint arXiv:2103.01739 , 2021. - · arXiv
  19. [20][20] Z. Ning, X. Wang, J. J. Rodrigues, and F. Xia, 'Joint computation offloading, power allocation, and channel assignment for 5G-enabled traffic management systems,' IEEE Transactions on Industrial Informatics , vol. 15, no. 5, pp. 3058-3067, 2019. -
  20. [21][21] L. Zhu, F. R. Yu, Y. Wang, B. Ning, and T. Tang, 'Big data analytics in intelligent transportation systems: A survey,' IEEE Transactions on Intelligent Transportation Systems , vol. 20, no. 1, pp. 383-398, 2018. -
  21. [22][22] F. Xia, J. Wang, X. Kong, Z. Wang, J. Li, and C. Liu, 'Exploring human mobility patterns in urban scenarios: A trajectory data perspective,' IEEE Communications Magazine , vol. 56, no. 3, pp. 142-149, 2018. -
  22. [24][24] X. Kong, X. Song, F. Xia, H. Guo, J. Wang, and A. Tolba, 'Lotad: Longterm traffic anomaly detection based on crowdsourced bus trajectory data,' World Wide Web , vol. 21, no. 3, pp. 825-847, 2018. -
  23. [25][25] A. S. El-Wakeel, A. Osman, N. Zorba, H. S. Hassanein, and A. Noureldin, 'Robust positioning for road information services in challenging environments,' IEEE Sensors Journal , vol. 20, no. 6, pp. 3182-3195, 2019. -
  24. [26][26] W. Wang, F. Xia, H. Nie, Z. Chen, Z. Gong, X. Kong, and W. Wei, 'Vehicle trajectory clustering based on dynamic representation learning of internet of vehicles,' IEEE Transactions on Intelligent Transportation Systems , vol. 22, no. 6, pp. 3567-3576, 2020. -
  25. [27][27] C. Chen, B. Liu, S. Wan, P. Qiao, and Q. Pei, 'An edge traffic flow detection scheme based on deep learning in an intelligent transportation system,' IEEE Transactions on Intelligent Transportation Systems , vol. 22, no. 3, pp. 1840-1852, 2021. -
  26. [28][28] A. B. Ahmad and T. Tsuji, 'Traffic monitoring system based on deep learning and seismometer data,' Applied Sciences , vol. 11, no. 10, p. 4590, 2021. -
  27. [29][29] F. Xia, J. Wang, X. Kong, D. Zhang, and Z. Wang, 'Ranking station importance with human mobility patterns using subway network datasets,' IEEE Transactions on Intelligent Transportation Systems , vol. 21, no. 7, pp. 2840-2852, 2019. -
  28. [30][30] Y. Li, J. Wu, X. Bai, X. Yang, X. Tan, G. Li, S. Wen, H. Zhang, and E. Ding, 'Multi-granularity tracking with modularlized components for unsupervised vehicles anomaly detection,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , 2020, pp. 586-587. -
  29. [32][32] J. Hu, L. Shen, and G. Sun, 'Squeeze-and-excitation networks,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7132-7141. -
  30. [33][33] A. Aboah, 'A vision-based system for traffic anomaly detection using deep learning and decision trees,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 4207-4212. -
  31. [34][34] P. Stapleton and J. Blanchard, 'Remote proctoring: Expanding reliability and trust,' in Proceedings of the 52nd ACM Technical Symposium on Computer Science Education , 2021, pp. 1243-1243. -
  32. [35][35] Y. Atoum, L. Chen, A. X. Liu, S. D. Hsu, and X. Liu, 'Automated online exam proctoring,' IEEE Transactions on Multimedia , vol. 19, no. 7, pp. 1609-1624, 2017. -
  33. [36][36] R. Bawarith, A. Basuhail, A. Fattouh, and S. Gamalel-Din, 'E-exam cheating detection system,' International Journal of Advanced Computer Science and Applications , vol. 8, no. 4, pp. 176-181, 2017. -
  34. [37][37] L. C. O. Tiong and H. J. Lee, 'E-cheating prevention measures: Detection of cheating at online examinations using deep learning approach-a case study,' arXiv preprint arXiv:2101.09841 , 2021. - · arXiv
  35. [38][38] S. Pandya, H. Ghayvat, K. Kotecha, M. Awais, S. Akbarzadeh, P. Gope, S. C. Mukhopadhyay, and W. Chen, 'Smart home anti-theft system: a novel approach for near real-time monitoring and smart home security for wellness protocol,' Applied System Innovation , vol. 1, no. 4, p. 42, 2018. -
  36. [39][39] A. Yang, C. Zhang, Y. Chen, Y. Zhuansun, and H. Liu, 'Security and privacy of smart home systems based on the internet of things and stereo matching algorithms,' IEEE Internet of Things Journal , vol. 7, no. 4, pp. 2521-2530, 2019. -
  37. [40][40] S. W. Yahaya, A. Lotfi, and M. Mahmud, 'Towards a data-driven adaptive anomaly detection system for human activity,' Pattern Recognition Letters , vol. 145, pp. 200-207, 2021. -
  38. [41][41] K. I. Withanage, I. Lee, R. Brinkworth, S. Mackintosh, and D. Thewlis, 'Fall recovery subactivity recognition with RGB-D cameras,' IEEE transactions on industrial informatics , vol. 12, no. 6, pp. 2312-2320, 2016. -
  39. [42][42] A. Markovitz, G. Sharir, I. Friedman, L. Zelnik-Manor, and S. Avidan, 'Graph embedded pose clustering for anomaly detection,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 10 539-10 547. -
  40. [43][43] R. Morais, V. Le, T. Tran, B. Saha, M. Mansour, and S. Venkatesh, 'Learning regularity in skeleton trajectories for anomaly detection in videos,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 11 996-12 004. -
  41. [44][44] M. LEE, 'Does population aging contribute to increased fiscal spending?' The Journal of Business Economics and Environmental Studies , vol. 9, no. 4, pp. 23-28, 2019. -
  42. [45][45] P. Visutsak and M. Daoudi, 'The smart home for the elderly: Perceptions, technologies and psychological accessibilities: The requirements analysis for the elderly in Thailand,' in 2017 XXVI International Conference on Information, Communication and Automation Technologies (ICAT) . IEEE, 2017, pp. 1-6. -
  43. [47][47] S. Jayashri et al. , 'Video analytics on social distancing and detecting mask,' Turkish Journal of Computer and Mathematics Education (TURCOMAT) , vol. 12, no. 9, pp. 2916-2921, 2021. -
  44. [48][48] Y. C. Hou, M. Z. Baharuddin, S. Yussof, and S. Dzulkifly, 'Social distancing detection with deep learning model,' in 2020 8th International Conference on Information Technology and Multimedia (ICIMU) . IEEE, 2020, pp. 334-338. -
  45. [49][49] K. Bhambani, T. Jain, and K. A. Sultanpure, 'Real-time face mask and social distancing violation detection system using yolo,' in 2020 IEEE Bangalore Humanitarian Technology Conference (B-HTC) . IEEE, 2020, pp. 1-6. -
  46. [50][50] F. Zuo, J. Gao, A. Kurkcu, H. Yang, K. Ozbay, and Q. Ma, 'Referencefree video-to-real distance approximation-based urban social distancing analytics amid COVID-19 pandemic,' Journal of Transport & Health , vol. 21, p. 101032, 2021. -
  47. [51][51] S. Saponara, A. Elhanashi, and A. Gagliardi, 'Implementing a real-time, AI-based, people detection and social distancing measuring system for COVID-19,' Journal of Real-Time Image Processing , pp. 1-11, 2021. -
  48. [53][53] Z. Zhang, C. Conly, and V. Athitsos, 'A survey on vision-based fall detection,' in Proceedings of the 8th ACM international conference on PErvasive technologies related to assistive environments , 2015, pp. 1-7. -
  49. [54][54] A. Prati, C. Shan, and K.-K. Wang, 'Sensors, vision and networks: From video surveillance to activity recognition and health monitoring,' Journal of Ambient Intelligence and Smart Environments , vol. 11, pp. 5-22, 01 2019. -
  50. [55][55] L. Cattani, D. Alinovi, G. Ferrari, R. Raheli, E. Pavlidis, C. Spagnoli, and F. Pisani, 'Monitoring infants by automatic video processing: A unified approach to motion analysis,' Computers in Biology & Medicine , vol. 80, pp. 158-165, 2017. -
  51. [56][56] S. Yu, F. Xia, Y. Sun, T. Tang, X. Yan, and I. Lee, 'Detecting outlier patterns with query-based artificially generated searching conditions,' IEEE Transactions on Computational Social Systems , vol. 8, no. 1, pp. 134-147, 2020. -
  52. [58][58] Y. He, J. Guo, and X. Zheng, 'From surveillance to digital twin: Challenges and recent advances of signal processing for industrial internet of things,' IEEE Signal Processing Magazine , vol. 35, no. 5, pp. 120-129, 2018. -
  53. [61][61] G. Yiping, L. Xinyu, and L. Gao, 'A deep lifelong learning method for digital twin-driven defect recognition with novel classes,' Journal of Computing and Information Science in Engineering , vol. 21, no. 3, p. 031004, 2021. -
  54. [62][62] K.-T. Nguyen, D.-T. Dinh, M. N. Do, and M.-T. Tran, 'Anomaly detection in traffic surveillance videos with gan-based future frame prediction,' in Proceedings of the 2020 International Conference on Multimedia Retrieval , 2020, pp. 457-463. -
  55. [63][63] F. Dong, Y. Zhang, and X. Nie, 'Dual discriminator generative adversarial network for video anomaly detection,' IEEE Access , vol. 8, pp. 88 170-88 176, 2020. -
  56. [64][64] C. Zhou and R. C. Paffenroth, 'Anomaly detection with robust deep autoencoders,' in Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining , 2017, pp. 665-674. -
  57. [65][65] M. Ribeiro, A. E. Lazzaretti, and H. S. Lopes, 'A study of deep convolutional auto-encoders for anomaly detection in videos,' Pattern Recognition Letters , vol. 105, pp. 13-22, 2018. -
  58. [66][66] D. Chen, P. Wang, L. Yue, Y. Zhang, and T. Jia, 'Anomaly detection in surveillance video based on bidirectional prediction,' Image and Vision Computing , vol. 98, p. 103915, 2020. -
  59. [67][67] K. Deepak, S. Chandrakala, and C. K. Mohan, 'Residual spatiotemporal autoencoder for unsupervised video anomaly detection,' Signal, Image and Video Processing , vol. 15, no. 1, pp. 215-222, 2021. -
  60. [68][68] J. An and S. Cho, 'Variational autoencoder based anomaly detection using reconstruction probability,' Special Lecture on IE , vol. 2, no. 1, pp. 1-18, 2015. -
  61. [69][69] T. Wang, M. Qiao, Z. Lin, C. Li, H. Snoussi, Z. Liu, and C. Choi, 'Generative neural networks for anomaly detection in crowded scenes,' IEEE Transactions on Information Forensics and Security , vol. 14, no. 5, pp. 1390-1399, 2018. -
  62. [70][70] M. Sabokrou, M. Khalooei, M. Fathy, and E. Adeli, 'Adversarially learned one-class classifier for novelty detection,' in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 3379-3388. -
  63. [71][71] P. Wu, J. Liu, and F. Shen, 'A deep one-class neural network for anomalous event detection in complex scenes,' IEEE transactions on neural networks and learning systems , vol. 31, no. 7, pp. 2609-2622, 2019. -
  64. [72][72] S. M. Erfani, S. Rajasegarar, S. Karunasekera, and C. Leckie, 'Highdimensional and large-scale anomaly detection using a linear one-class SVM with deep learning,' Pattern Recognition , vol. 58, pp. 121-134, 2016. -
  65. [73][73] J. T. Zhou, J. Du, H. Zhu, X. Peng, Y. Liu, and R. S. M. Goh, 'Anomalynet: An anomaly detection network for video surveillance,' IEEE Transactions on Information Forensics and Security , vol. 14, no. 10, pp. 2537-2550, 2019. -
  66. [74][74] Y. Su, J. Li, A. Plaza, A. Marinoni, P. Gamba, and S. Chakravortty, 'DAEN: Deep autoencoder networks for hyperspectral unmixing,' IEEE Transactions on Geoscience and Remote Sensing , vol. 57, no. 7, pp. 4309-4321, 2019. -
  67. [75][75] T. S. Buda, B. Caglayan, and H. Assem, 'Deepad: A generic framework based on deep learning for time series anomaly detection,' in PacificAsia Conference on Knowledge Discovery and Data Mining . Springer, 2018, pp. 577-588. -
  68. [76][76] D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. v. d. Hengel, 'Memorizing normality to detect anomaly: Memoryaugmented deep autoencoder for unsupervised anomaly detection,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 1705-1714. -
  69. [77][77] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, 'Generative adversarial networks,' Communications of the ACM , vol. 63, no. 11, pp. 139-144, 2020. -
  70. [78][78] L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. M¨ uller, and M. Kloft, 'Deep one-class classification,' in International conference on machine learning . PMLR, 2018, pp. 4393-4402. -
  71. [79][79] R. Chalapathy and S. Chawla, 'Deep learning for anomaly detection: A survey,' arXiv preprint arXiv:1901.03407 , 2019. · arXiv

Cited by

Sorted by citations

Authors
Ren, J., Xia, F., Liu, Y., & Lee, I.
Affiliations
  • Federation University (AU)
  • Dalian University of Technology (CN)
  • University of South Australia (AU)
ORCID
Venue
2021 International Conference on Data Mining Workshops (ICDMW)
Publisher
IEEE
Published
2021-12
Pages
959-966
ISSN
2375-9259 (ISSN-L)
DOI
10.1109/icdmw53433.2021.00125
OpenAlex
W3206170757
Cited by
84
References
7
Citations by year
2026: 10 · 2025: 19 · 2024: 13 · 2023: 27 · 2022: 15
FWCI
6.35
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE ... International Conference on Data Mining workshops (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, other-oa)
  • FedUni ResearchOnline (Federation University Australia) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
Keywords
video anomaly detection, deep learning, video surveillance, surveillance security, research challenges

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review