Anomaly Detection in Microservice-Based Systems
Open accessAbstract
Currently, distributed software systems have evolved at an unprecedented pace. Modern software-quality requirements are high and require significant staff support and effort. This study investigates the use of a supervised machine learning model, a Multi-Layer Perceptron (MLP), for anomaly detection in microservices. The study covers the creation of a microservices infrastructure, the development of a fault injection module that simulates application-level and service-level anomalies, the creation of a system monitoring dataset, and the creation and validation of the MLP model to detect anomalies. The results indicate that the MLP model effectively detects anomalies in both domains with higher accuracy, precision, recovery, and F1 score on the service-level anomaly dataset. The potential for more effective distributed system monitoring and management automation is highlighted in this study by focusing on service-level metrics such as service response times. This study provides valuable information about the effectiveness of supervised machine learning models in detecting anomalies across distributed software systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568 · see citing works
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835 · see citing works
Bishop, C. (1996) · Taylor & Francis eBooks · doi:10.1201/9781420050646.ptb6 · Cited by 1,607 · see citing works
Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
Xu, H., Feng, Y., Chen, J., Wang, Z., Qiao, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., et al. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3185996 · Cited by 1,001 · see citing works
Survey on Anomaly Detection using Data Mining Techniques
Agrawal, S., & Agrawal, J. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.08.220 · Cited by 598 · see citing works
Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
Fu, Q., Lou, J., Wang, Y., & Jiang, L. (2009) · Proceedings · doi:10.1109/icdm.2009.60 · Cited by 555 · see citing works
Machine Learning Techniques for Anomaly Detection: An Overview
Omar, S., Ngadi, A., & Jebur, H. (2013) · International Journal of Computer Applications · doi:10.5120/13715-1478 · Cited by 349 · see citing works
Temporal causal modeling with graphical granger methods
Arnold, A. O., Liu, Y., & Abe, N. (2007) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281203 · Cited by 294 · see citing works
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Liu, D., Xiang, Q., & He, C. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338961 · Cited by 277 · see citing works
Liu, D., Zhao, Y., Xu, H., Sun, Y., Pei, D., Luo, J., Jing, X., & Mei, F. (2015) · ACM Internet Measurement Conference (IMC) · doi:10.1145/2815675.2815679 · Cited by 257 · see citing works
Unsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks
Liu, P., Xu, H., Ouyang, Q., Jiao, R., Chen, Z., Zhang, S., Yang, J., Mo, L., Zeng, J., Xue, W., et al. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00014 · Cited by 252 · see citing works
Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network
Lu, S., Wei, X., Li, Y., & Wang, L. (2018) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/datacom/cyberscitec.2018.00037 · Cited by 228 · see citing works
Orrù, P. F., Zoccheddu, A., Sassu, L., Mattia, C., Cozza, R., & Arena, S. (2020) · Sustainability · doi:10.3390/su12114776 · Cited by 228 · see citing works
Z-Score Normalization, Hubness, and Few-Shot Learning
Fei, N., Gao, Y., Lu, Z., & Xiang, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00021 · Cited by 209 · see citing works
Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments
Lin, J., Chen, P., & Zheng, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03596-9_1 · Cited by 204 · see citing works
Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection
Brown, A., Tuor, A. R., Hutchinson, B., & Nichols, N. (2018) · Proceedings of the First Workshop on Machine Learning for Computing Systems · doi:10.1145/3217871.3217872 · Cited by 203 · see citing works
Inferring models of concurrent systems from logs of their behavior with CSight
Beschastnikh, I., Brun, Y., Ernst, M. D., & Krishnamurthy, A. (2014) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2568225.2568246 · Cited by 182 · see citing works
Anomaly detection using random forest: A performance revisited
Primartha, R., & Tama, B. A. (2017) · International Conference on Data and Software Engineering · doi:10.1109/icodse.2017.8285847 · Cited by 179 · see citing works
Oliveira, T. P., Barbar, J. S., & Soares, A. S. (2015) · International Journal of Big Data Intelligence · doi:10.1504/ijbdi.2016.073903 · Cited by 154 · see citing works
Anomaly Detection Using Program Control Flow Graph Mining From Execution Logs
Nandi, A., Mandal, A., Atreja, S., Dasgupta, G. B., & Bhattacharya, S. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939712 · Cited by 154 · see citing works
Chen, P., Qi, Y., Zheng, P., & Hou, D. (2014) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2014.6848128 · Cited by 144 · see citing works
Using Hidden Semi-Markov Models for Effective Online Failure Prediction
Salfner, F., & Malek, M. (2007) · Proceedings/Proceedings - Symposium on Reliable Distributed Systems · doi:10.1109/srds.2007.35 · Cited by 135 · see citing works
All Eyes on You: Distributed Multi-Dimensional IoT Microservice Anomaly Detection
Pahl, M., & Aubet, F.-X. (2018) · Conference on Network and Service Management · Cited by 133 · see citing works
Mobile Network Traffic Prediction Using MLP, MLPWD, and SVM
Nikravesh, A. Y., Ajila, S. A., Lung, C., & Ding, W. (2016) · IEEE International Congress on Big Data (BigData Congress) · doi:10.1109/bigdatacongress.2016.63 · Cited by 120 · see citing works
Failure prediction based on log files using Random Indexing and Support Vector Machines
Fronza, I., Sillitti, A., Succi, G., Terho, M., & Vlasenko, J. (2012) · Journal of Systems and Software · doi:10.1016/j.jss.2012.06.025 · Cited by 117 · see citing works
MLP Neural Network Based Gas Classification System on Zynq SoC
Zhai, X., Ali, A. A. S., Amira, A., et al. (2016) · IEEE Access · doi:10.1109/access.2016.2619181 · Cited by 116 · see citing works
Anomaly Detection and Classification using Distributed Tracing and Deep Learning
Nedelkoski, S., Cardoso, J., & Kao, O. (2019) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2019.00038 · Cited by 91 · see citing works
LogSed: Anomaly Diagnosis through Mining Time-Weighted Control Flow Graph in Logs
Jia, T., Yang, L., Chen, P., Li, Y., Meng, F., & Xu, J. (2017) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2017.64 · Cited by 86 · see citing works
CloudPD: Problem determination and diagnosis in shared dynamic clouds
Sharma, B., Jayachandran, P., Verma, A., & Das, C. R. (2013) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2013.6575298 · Cited by 79 · see citing works
Gonzalez-Cuautle, D., Hernandez-Suarez, A., Sanchez-Perez, G., et al. (2020) · Applied Sciences · doi:10.3390/app10030794 · Cited by 74 · see citing works
Chen, P., Qi, Y., & Hou, D. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2607739 · Cited by 73 · see citing works
Shan, H., Chen, Y., Li, X., Zhang, Y., Xiao, X., He, X., Li, M., & Ding, W. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313653 · Cited by 73 · see citing works
Anomaly Detection and Diagnosis for Container-Based Microservices with Performance Monitoring
Du, Q., Xie, T., & He, Y. T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05063-4_42 · Cited by 71 · see citing works
Detecting anomalies in microservices with execution trace comparison
Meng, L., Ji, F., Sun, Y., & Wang, T. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.040 · Cited by 70 · see citing works
Machine Learning Techniques for Network Anomaly Detection: A Survey
Eltanbouly, S., Bashendy, M., Al‐Naimi, N., et al. (2020) · IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT) · doi:10.1109/iciot48696.2020.9089465 · Cited by 69 · see citing works
Anomaly detection and diagnosis for cloud services: Practical experiments and lessons learned
Sauvanaud, C., Kaâniche, M., Kanoun, K., Lazri, K., & Silvestre, G. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2018.01.039 · Cited by 68 · see citing works
Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network
Serpen, G., & Gao, Z. (2014) · Procedia Computer Science · doi:10.1016/j.procs.2014.09.078 · Cited by 67 · see citing works
Moghanian, S., Saravi, F. B., Javidi, G., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3040740 · Cited by 55 · see citing works
Workflow-Aware Automatic Fault Diagnosis for Microservice-Based Applications With Statistics
Wang, T., Zhang, W., Xu, J., & Gu, Z. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3022028 · Cited by 54 · see citing works
Jin, M., Lv, A., Zhu, Y., Wen, Z., Zhong, Y., Zhao, Z., Wu, J., Li, H., He, H., & Chen, F. (2020) · IEEE Access · doi:10.1109/access.2020.3044610 · Cited by 50 · see citing works
Self-Supervised Anomaly Detection from Distributed Traces
Bogatinovski, J., Nedelkoski, S., Cardoso, J., & Kao, O. (2020) · 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc48980.2020.00054 · Cited by 50 · see citing works
Detection of Diabetes Using Multilayer Perceptron
Mohapatra, S. K., Swain, J. K., & Mohanty, M. N. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-981-13-2182-5_11 · Cited by 49 · see citing works
Anomaly detection in cyber security attacks on networks using MLP deep learning
Toe, T. T., Chiew, G., Franco, E. J., et al. (2018) · International Conference on Smart Computing and Electronic Enterprise (ICSCEE) · doi:10.1109/icscee.2018.8538395 · Cited by 48 · see citing works
Predicting failures in multi-tier distributed systems
Mariani, L., Pezzè, M., Riganelli, O., & Xin, R. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.110464 · Cited by 46 · see citing works
Multilayer perceptron neural network technique for fraud detection
Mubarek, A. M., & Adalı, E. (2017) · doi:10.1109/ubmk.2017.8093417 · Cited by 45 · see citing works
Mishra, M. K., & Dash, R. (2014) · doi:10.1109/icit.2014.25 · Cited by 44 · see citing works
Multi-layer perceptron for network intrusion detection
Rosay, A., Riou, K., Carlier, F., et al. (2021) · Annals of Telecommunications · doi:10.1007/s12243-021-00852-0 · Cited by 38 · see citing works
Detecting Anomalous Behavior of Black-Box Services Modeled with Distance-Based Online Clustering
Gulenko, A., Schmidt, F., Acker, A., Wallschläger, M., Kao, O., & Liu, F. (2018) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2018.00134 · Cited by 36 · see citing works
Chen, H., Chen, P., & Yu, G. (2020) · IEEE Access · doi:10.1109/access.2020.2977464 · Cited by 36 · see citing works
PAD: Performance Anomaly Detection in Multi-server Distributed Systems
Peiris, M., Hill, J. H., Thelin, J., Bykov, S., Kliot, G., & Konig, C. (2014) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2014.107 · Cited by 33 · see citing works
Bucchiarone, A., Dragoni, N., Dustdar, S., et al. (2019) · Springer eBooks · doi:10.1007/978-3-030-31646-4 · Cited by 33 · see citing works
TaskInsight: A Fine-Grained Performance Anomaly Detection and Problem Locating System
Zhang, X., Meng, F., Chen, P., & Xu, J. (2016) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2016.0136 · Cited by 27 · see citing works
Li, W., Tordsson, J., Elmroth, E., & Kao, O. (2021) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) · doi:10.1109/acsos52086.2021.00029 · Cited by 27 · see citing works
Detection of Performance Anomalies in Web-Based Applications
Magalhães, J. P., & Silva, L. M. (2010) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2010.15 · Cited by 25 · see citing works
An Improved LOF Outlier Detection Algorithm
Xu, S., Liu, H., Duan, L., et al. (2021) · IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) · doi:10.1109/icaica52286.2021.9498181 · Cited by 16 · see citing works
Tansig activation function (of MLP network) for cardiac abnormality detection
Adnan, J., Daud, N. G. N., Ishak, M. T., et al. (2018) · AIP conference proceedings · doi:10.1063/1.5022900 · Cited by 10 · see citing works
Leveraging Deep Learning to Improve Performance Predictability in Cloud Microservices with Seer
Gan, Y., Zhang, Y., Hu, K., Cheng, D., He, Y., Pancholi, M., & Delimitrou, C. (2019) · ACM SIGOPS Operating Systems Review · doi:10.1145/3352020.3352026 · Cited by 8 · see citing works
Yagoub, I., Khan, M. A., & Li, J. (2018) · doi:10.1109/icabcd.2018.8465400 · Cited by 7 · see citing works
Zang, X., Chen, W., Zou, J., Zhou, S., Lisong, H., & Ruigang, L. (2018) · 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2) · doi:10.1109/ei2.2018.8582217 · Cited by 7 · see citing works
Detection of Performance Anomalies in Cloud Services: A Correlation Analysis Approach
Abdelrahman, G. M. (2016) · International Journal Of Mechanical Engineering And Information Technology · doi:10.18535/ijmeit/v4i9.06 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Newman, S. Building Microservices; O'Reilly Media, Inc.: Sebastopol, CA, USA, 2021.
- Mazzara, M.; Bucchiarone, A.; Dragoni, N.; Rivera, V. Size matters: Microservices research and applications. Microservices: Science and Engineering; Springer: Cham, Switzerland, 2020; pp. 29-42.
- Liu, D.; Zhao, Y.; Xu, H.; Sun, Y.; Pei, D.; Luo, J.; Jing, X.; Feng, M. Opprentice: Towards practical and automatic anomaly detection through machine learning. In Proceedings of the 2015 Internet Measurement Conference, Tokyo, Japan, 28-30 October 2015; pp. 211-224.
- Akkaya, B.; Çolako glu, N. Comparison of Multi-Class Classification Algorithms on Early Diagnosis of Heart Diseases. In Proceedings of the ISBIS Young Business and Industrial Statisticians Workshop on Recent Advances in Data Science and Business Analytics, Istanbul, Turkey, 25-28 September 2019.
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016.
- Sridharan, C. Distributed Systems Observability; O'Reilly Media, Inc.: Sebastopol, CA, USA, 2018.
Cited by
Sorted by citations
Toward Context-Aware Anomaly Detection for AIOps in Microservices Using Dynamic Knowledge Graphs
Moens, P., Steenwinckel, B., Ongenae, F., Volckaert, B., & Hoecke, S. V. (2026) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2026.3652304 · Cited by 6 · see citing works
“Nobre et al. [27] investigates a supervised deep learning approach to anomaly detection in microservice-based systems.”
- Authors
- Nobre, J., Pires, E. J. S., & Reis, A.
- Affiliations
- Universidade Aberta (PT)
- University of Trás-os-Montes and Alto Douro (PT)
- Corresponding author
- Reis, Arsénio
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2023-07-05
- Volume
- 13
- Issue
- 13
- Pages
- 7891
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4383345785
- Cited by
- 34
- References
- 62
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 6 · 2023: 2
- FWCI
- 3.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- anomaly detection, microservices, distributed systems, multilayer perceptron, fault injection, system monitoring, service response time, supervised learning, software quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review