Agent based dynamic thresholding for adaptive anomaly detection using reinforcement learning
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Reinforcement learning approaches to dynamic thresholding [22] and adaptive cost policies have been proposed conceptually but lack empirical validation inside live systems.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [18]
In the citing paper
“Earlier ANN-based alert triage experiments hinted at SIEM alignment [5], yet no study has documented a calibrated CNN-based insider detector operating live within the SIEM loop.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [18]
In the citing paper
“(iv) SIEM-integrated, calibrated evaluation remains rare [5,22,32].”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [18]
In the citing paper
“Despite this, nearly all prior CNN-based insider detectors stop at offline evaluation.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [18]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2×
“Specifically, we apply the popular DRL algorithm DQN [25] to solve the MDP given its efficiency in the anomaly detection task.”
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 2×
“The application of anomaly detection has been explored extensively across a variety of research fields and application domains including data mining, network intrusion detection, medical and public health, and energy management [4].”
Support Vector Data Description
Tax, D. M. J., & Duin, R. P. W. (2004) · Machine Learning · doi:10.1023/b:mach.0000008084.60811.49 · Cited by 3,512
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
Isolation-Based Anomaly Detection
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2012) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/2133360.2133363 · Cited by 2,063 · cited 1×
“Anomalies are often characterized by their rarity and distinctiveness, while simultaneously providing critical information [2].”
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219845 · Cited by 1,771 · cited 1×
“[20] presented an unsupervised and nonparametric thresholding approach for spacecraft anomaly detection without statistical assumptions about prediction errors.”
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., et al. (2020) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · doi:10.48550/arxiv.2007.02500 · Cited by 1,718
Deep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240 · cited 4×
“They have difficulty adapting to non-stationary and evolving time series data [1].”
InfluentialUSAD: UnSupervised Anomaly Detection on Multivariate Time Series
Audibert, J., Michiardi, P., Guyard, F., Marti, S., & Zuluaga, M. A. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403392 · Cited by 1,094 · cited 5×
“In the literature, many existing anomaly detection approaches use static or expert-defined thresholds [3, 5, 6].”
InfluentialA Dataset to Support Research in the Design of Secure Water Treatment Systems
Goh, J., Adepu, S., Junejo, K. N., & Mathur, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-71368-7_8 · Cited by 1,045 · cited 1×
“It consists of 11-day continuous operations where 7 days are under normal scenarios and 4 days are under attack scenarios [32].”
SWaT: a water treatment testbed for research and training on ICS security
Mathur, A. P., & Tippenhauer, N. O. (2016) · 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater) · doi:10.1109/cyswater.2016.7469060 · Cited by 878 · cited 1×
“Secure Water Treatment (SWaT) 4 is an operational testbed for a real-world water treatment system producing filtered water, which is widely used in the field of cyber and physical system security [31].”
Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
Xu, H., Feng, Y., Chen, J., Wang, Z., Qiao, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., et al. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3185996 · Cited by 808 · cited 3×
“This is for the consideration that in real-world applications, we are more concerned about the evaluation metrics of contiguous abnormal segments rather than the point-wise metrics [29].”
Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines
Choi, K., Yi, J., Park, C., & Yoon, S. (2021) · IEEE Access · doi:10.1109/access.2021.3107975 · Cited by 588 · cited 2×
“AE-based anomaly detection methods have been investigated in many studies such as [1, 17, 30].”
Anomaly Detection in Streams with Extreme Value Theory
Siffer, A., Fouque, P.-A., Termier, A., & Largouet, C. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098144 · Cited by 553 · cited 3×
“[7] proposed a statistical method based on EVT and introduced two algorithms for thresholding in stationary and drifting contexts, respectively.”
InfluentialWADI: a water distribution testbed for research in the design of secure cyber physical systems
Ahmed, C. M., Palleti, V. R., & Mathur, A. P. (2017) · Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks · doi:10.1145/3055366.3055375 · Cited by 550 · cited 1×
“Water Distribution (WADI) is a testbed conducted for the secure water distribution system which is physically connected to the SWaT testbed [33].”
Robust Deep Learning Methods for Anomaly Detection
Chalapathy, R., Khoa, N. L. D., & Chawla, S. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3406704 · Cited by 502
Enhancing one-class support vector machines for unsupervised anomaly detection
Amer, M., Goldstein, M., & Abdennadher, S. (2013) · Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description · doi:10.1145/2500853.2500857 · Cited by 427 · cited 1×
“In the past, commonly used anomaly detection methods included classification-based methods such as SVM [10–12], clustering-based methods such as K-means [13, 14], and distance-based methods such as KNN [15, 16].”
Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning
Mousavi, S. S., Schukat, M., & Howley, E. (2017) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2017.0153 · Cited by 354
Self-citationAnomaly Detection for Time Series Using VAE-LSTM Hybrid Model
Lin, S., Clark, R., Birke, R., Schonborn, S., Trigoni, N., & Roberts, S. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053558 · Cited by 312
Chen, C., Modares, H., Xie, K., Lewis, F. L., Wan, Y., & Xie, S. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2019.2905215 · Cited by 243 · cited 1×
“RL has been successfully employed to provide adaptive optimal control [24] in various domains.”
LSTM-Based VAE-GAN for Time-Series Anomaly Detection
Niu, Z., Yu, K., & Wu, X. (2020) · Sensors · doi:10.3390/s20133738 · Cited by 237 · cited 3×
“In the literature, many existing anomaly detection approaches use static or expert-defined thresholds [3, 5, 6].”
On the Time Series $K$-Nearest Neighbor Classification of Abnormal Brain Activity
Chaovalitwongse, W. A., Fan, Y.-J., & Sachdeo, R. C. (2007) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2007.897589 · Cited by 222 · cited 1×
“In the past, commonly used anomaly detection methods included classification-based methods such as SVM [10–12], clustering-based methods such as K-means [13, 14], and distance-based methods such as KNN [15, 16].”
GERBIL: General Entity Annotator Benchmarking Framework
Usbeck, R., Röder, M., Ngonga Ngomo, A.-C., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., et al. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741626 · Cited by 190
Recent Advances in Anomaly Detection Methods Applied to Aviation
Basora, L., Olive, X., & Dubot, T. (2019) · Aerospace · doi:10.3390/aerospace6110117 · Cited by 177
Su, M.-Y. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.08.137 · Cited by 176
Designing an adaptive production control system using reinforcement learning
Kuhnle, A., Kaiser, J.-P., Theiß, F., Stricker, N., & Lanza, G. (2021) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-020-01612-y · Cited by 162 · cited 1×
“An RL agent is capable of providing advantageous control for complex dynamic systems [8].”
Ahn, K. U., & Park, C. S. (2019) · Science and Technology for the Built Environment · doi:10.1080/23744731.2019.1680234 · Cited by 121
A DQN-based intelligent control method for heavy haul trains on long steep downhill section
Liu, W., Su, S., Tang, T., & Wang, X. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103249 · Cited by 107 · cited 1×
“DQN provides a powerful and flexible framework for solving optimal control problems, making it a popular choice for many researchers and practitioners [25–28].”
Graph neural network approach for anomaly detection
Xie, L., Pi, D., Zhang, X., Chen, J., Luo, Y. P., & Yu, W. (2021) · Measurement · doi:10.1016/j.measurement.2021.109546 · Cited by 101 · cited 1×
“[21] proposed a thresholding method by analyzing periodic time windows and using the moving weighted average to calculate thresholds.”
Policy-based reinforcement learning for time series anomaly detection
Yu, M., & Sun, S. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103919 · Cited by 80 · cited 1×
“[22] proposed a policy-based anomaly detector PTAD which adopted a DRL algorithm, i.e., asynchronous advantage actor-critic (A3C), and embedded it with LSTM components.”
A Dynamic Deep Reinforcement Learning-Bayesian Framework for Anomaly Detection
Watts, J., van Wyk, F., Rezaei, S., Wang, Y., Masoud, N., & Khojandi, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3200906 · Cited by 56 · cited 1×
“[23] modeled anomaly detection in the transportation domain as a partial observable MDP and used convolutional neural network (CNN) and A3C to solve it, which can generate dynamic classi-fication thresholds.”
Research and Implementation of an Anomaly Detection Model Based on Clustering Analysis
Han, L. (2010) · doi:10.1109/iptc.2010.94 · Cited by 34 · cited 1×
“In the past, commonly used anomaly detection methods included classification-based methods such as SVM [10–12], clustering-based methods such as K-means [13, 14], and distance-based methods such as KNN [15, 16].”
A Mixed Unsupervised Clustering-Based Intrusion Detection Model
Zhang, C., Zhang, G., & Sun, S. (2009) · International Conference on Genetic and Evolutionary Computing · doi:10.1109/wgec.2009.72 · Cited by 26 · cited 1×
“In the past, commonly used anomaly detection methods included classification-based methods such as SVM [10–12], clustering-based methods such as K-means [13, 14], and distance-based methods such as KNN [15, 16].”
Research works printed but not linked: 3
- Yang X, Howley E, Schukat M (2023) Adt: agent-based dynamic thresholding for anomaly detection. Adaptive and learning agents workshop
- Ma J, Perkins S (2003) Time-series novelty detection using one-class support vector machines. Proceed Int Joint Conf Neural Netw 3:1741–1745
- Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International conference on learning representations
Cited by
Putrada, A. G., & Ilhami, D. A. S. (2024) · IEEE Student Conference on Research and Development (SCOReD) · doi:10.1109/scored64708.2024.10872727 · Cited by 8
A Federated Learning Approach for Efficient Anomaly Detection in Electric Power Steering Systems
Kea, K., Han, Y., & Min, Y.-J. (2024) · IEEE Access · doi:10.1109/access.2024.3397000 · Cited by 12
Raja, P., Dhanalakshmi, B., Rajan, T., Ravi, V., & Alghamdi, N. S. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-025-02134-1 · Cited by 5
Early Detection of DMA-Level Leaks in Water Networks Using Robust Regression Ensemble Framework
Chatterjee, S., Ghumkar, S., Ahbab, M. M., Ramachandran, A., Tenbrinck, D., Maier, A., Semmelmann, K., & Bayer, S. (2026) · Water · doi:10.3390/w18050563 · Cited by 4
Automated Machine Learning Configuration to Learn Intrusion Detectors on Attack-Free Datasets
Anser, O., François, J., & Chrisment, I. (2024) · Conference on Local Computer Networks · doi:10.1109/lcn60385.2024.10639690 · Cited by 7
Metadata
- Authors
- Yang, X., Howley, E., & Schukat, M.
- Affiliations
- Ollscoil na Gaillimhe – University of Galway (IE)
- Venue
- Neural Computing and Applications
- Published
- 2024-12-03
- Volume
- 37
- Issue
- 23
- Pages
- 18775-18791
- ISSN
- 0941-0643 (ISSN-L)
- 1433-3058
- OpenAlex
- W4404949567
- Cited by
- 11
- References
- 32
- Citations by year
- 2026: 7 · 2025: 3 · 2024: 1
- FWCI
- 1.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Neural Computing and Applications (publishedVersion)
- Funders
- College of Science and Engineering Postgraduate Research Scholarship of the University of Galway
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- anomaly detection, dynamic thresholding, adaptive anomaly detection, reinforcement learning, deep Q-network, Markov decision process, binary thresholding, time series anomaly detection, non-stationary data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.