An Anomaly Detection Algorithm for Microservice Architecture Based on Robust Principal Component Analysis
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Microservice architecture (MSA) is a new software architecture, which divides a large single application and service into dozens of supporting microservices. With the increasingly popularity of MSA, the security issues of MSA get a lot of attention. In this paper, we propose an algorithm for mining causality and the root cause. Our algorithm consists of two parts: invocation chain anomaly analysis based on robust principal component analysis (RPCA) and a single indicator anomaly detection algorithm. The single indicator anomaly detection algorithm is composed of Isolation Forest (IF) algorithm, One-Class Support Vector Machine (SVM) algorithm, Local Outlier Factor (LOF) algorithm, and$3\sigma $principle. For general and network time-consuming anomaly in the process of the MSA, we formulate different anomaly time-consuming detection strategies. We select a batch of sample data and three batches of test data of the 2020 International AIOps Challenge to debug our algorithm. According to the scoring criteria of the competition organizers, our algorithm has an average score of 0.8304 (The full score is 1) in the four batches of data. Our proposed algorithm has higher accuracy than some traditional machine learning algorithms in anomaly detection.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“many systems monitor CPU and memory utilization [93,94], network traffic [95], cache metrics, and I/O operations [14,44,96]”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [94]
In the citing paper
“Based on the analysis of the individual root-cause identification methods, we can determine the top five methods: • Multilayer Perceptron (MLP) [137] (Precision: 99.8%, Recall: 98%, F1-Score: 99%); • Metric causality graph [97] (Precision: 97%); • Robust Principal Component Analysis (RPCA) [94] (Precision: 90%); • Spectrum-Based Fault Localization (SBFL) [81] (Precision: 82%, Recall: 93%, F1-Score”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [94]
In the citing paper
“Anomaly detection models, such as AEs and Isolation Forests [56], often attempt to learn the normal behavior from training data.”
From Toward Context-Aware Anomaly Detection for AIOps in Microservices Using Dynamic Knowledge Graphs (2026) · cites this paper as [56]
Citations
Cites
Robust principal component analysis?
Candès, E. J., Li, X., Ma, Y., & Wright, J. (2011) · Journal of the ACM · doi:10.1145/1970392.1970395 · Cited by 6,921
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 1×
“Under this random segmentation strategy, the anomalous point w have a short path usually and we define L ( w ) = 1 [39].”
BackgroundLOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335388 · Cited by 4,149
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332
Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
Fu, Q., Lou, J.-G., Wang, Y., & Li, J. (2009) · 2009 Ninth IEEE International Conference on Data Mining · doi:10.1109/icdm.2009.60 · Cited by 542
Generic and Scalable Framework for Automated Time-series Anomaly Detection
Laptev, N., Amizadeh, S., & Flint, I. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788611 · Cited by 425
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2887384 · Cited by 410
Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery
Vaswani, N., Bouwmans, T., Javed, S., & Narayanamurthy, P. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2018.2826566 · Cited by 356
Singing-voice separation from monaural recordings using robust principal component analysis
Huang, P.-S., Chen, S. D., Smaragdis, P., & Hasegawa-Johnson, M. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6287816 · Cited by 325
On the Applications of Robust PCA in Image and Video Processing
Bouwmans, T., Javed, S., Zhang, H., Lin, Z., & Otazo, R. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2853589 · Cited by 255
Opprentice: Towards Practical and Automatic Anomaly Detection Through Machine Learning
Liu, D., Zhao, Y., Xu, H., Sun, Y., Pei, D., Luo, J., Jing, X., & Feng, M. (2015) · Proceedings of the 2015 Internet Measurement Conference · doi:10.1145/2815675.2815679 · Cited by 252
Using an Ensemble of One-Class SVM Classifiers to Harden Payload-based Anomaly Detection Systems
Perdisci, R., Gu, G., & Lee, W. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.165 · Cited by 239
Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments
Lin, J., Chen, P., & Zheng, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-03596-9_1 · Cited by 192
CloudRanger: Root Cause Identification for Cloud Native Systems
Wang, P., Xu, J., Ma, M., Lin, W., Pan, D., Wang, Y., & Chen, P. (2018) · 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) · doi:10.1109/ccgrid.2018.00076 · Cited by 181 · cited 1×
“proposed a dynamic causal relationship analysis approach to construct impact graphs amongst applications without given the topology [14].”
BackgroundDiagnosing root causes of intermittent slow queries in cloud databases
Ma, M., Yin, Z., Zhang, S., Wang, S., Zheng, C., Jiang, X., Hu, H., Luo, C., Li, Y., Qiu, N., et al. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3389133.3389136 · Cited by 176
Chen, P., Qi, Y., Zheng, P., & Hou, D. (2014) · IEEE INFOCOM 2014 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2014.6848128 · Cited by 140
Root cause detection in a service-oriented architecture
Kim, M., Sumbaly, R., & Shah, S. (2013) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2494232.2465753 · Cited by 138
AutoMAP: Diagnose Your Microservice-based Web Applications Automatically
Ma, M., Xu, J., Wang, Y., Chen, P., Zhang, Z., & Wang, P. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380111 · Cited by 129
Graph-based root cause analysis for service-oriented and microservice architectures
Brandón, Á., Solé, M., Huélamo, A., Solans, D., Pérez, M. S., & Muntés-Mulero, V. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2019.110432 · Cited by 121 · cited 1×
“It can expand a single component rather than the entire application stack to meet the service level agreement [1], [2].”
BackgroundRobust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank MatricesPreprint
Wright, J. N., Ganesh, A., Rao, S., & Ma, Y. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0905.0233 · Cited by 118
Least squares one-class support vector machine
Choi, Y.-S. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.05.007 · Cited by 107
Recovery of Low-Rank Plus Compressed Sparse Matrices With Application to Unveiling Traffic Anomalies
Mardani, M., Mateos, G., & Giannakis, G. B. (2013) · IEEE Transactions on Information Theory · doi:10.1109/tit.2013.2257913 · Cited by 99
Rethinking PCA for Modern Data Sets: Theory, Algorithms, and Applications [Scanning the Issue]
Vaswani, N., Chi, Y., & Bouwmans, T. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2853498 · Cited by 97
Qiu, J., Du, Q., Yin, K., Zhang, S.-L., & Qian, C. (2020) · Applied Sciences · doi:10.3390/app10062166 · Cited by 96 · cited 1×
“Currently, most studies of root cause analysis (RCA) rely mainly on monitoring data, including log data, service dependency data, and invocation chain data [7]–[11].”
BackgroundRobust and Rapid Adaption for Concept Drift in Software System Anomaly Detection
Ma, M., Zhang, S., Pei, D., Huang, X., & Dai, H. (2018) · 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2018.00013 · Cited by 92
Chen, P., Qi, Y., & Hou, D. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2607739 · Cited by 85
G-RCA: a generic root cause analysis platform for service quality management in large IP networks
Yan, H., Breslau, L., Ge, Z., Massey, D., Pei, D., & Yates, J. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2012.2188837 · Cited by 72
Robust principal component analysis
Partridge, M., & Jabri, M. A. (2002) · IEEE Workshop on Neural Networks for Signal Processing (NNSP) · doi:10.1109/nnsp.2000.889420 · Cited by 59
Lightweight and Adaptive Service API Performance Monitoring in Highly Dynamic Cloud Environment
Xu, J., Wang, Y., Chen, P., & Wang, P. (2017) · IEEE International Conference on Services Computing · doi:10.1109/scc.2017.80 · Cited by 32
Mining causality graph for automatic web-based service diagnosis
Nie, X., Zhao, Y., Sui, K., Pei, D., Chen, Y., & Qu, X. (2016) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/pccc.2016.7820614 · Cited by 30
Lin, W., Ma, M., Pan, D., & Wang, P. (2018) · 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC) · doi:10.1109/pccc.2018.8711092 · Cited by 29
Detecting Large-Scale System Problems by Mining Console Logs
Xu, W., & Huang, L. (2010) · Cited by 28
Ensemble tensor decomposition for infrared thermography cracks detection system
Song, J., Gao, B., Woo, W. L., & Tian, G. Y. (2020) · Infrared Physics & Technology · doi:10.1016/j.infrared.2020.103203 · Cited by 24
Ding, Y., & Liu, J. (2017) · IEEE Conference on Energy Internet and Energy System Integration (EI2) · doi:10.1109/ei2.2017.8245663 · Cited by 21
Malicious Corruption-Resilient Wide-Area Oscillation Monitoring using Online Robust PCA
Mahapatra, K., & Chaudhuri, N. R. (2018) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm.2018.8586404 · Cited by 11
Anomaly Detection Based on Mining Six Local Data Features and BP Neural Network
Zhang, Y., Zhu, Y., Li, X., Wang, X., & Guo, X. (2019) · Symmetry · doi:10.3390/sym11040571 · Cited by 9
Self-citationResearch and Application of Microservice in Power Grid Dispatching Control System
Huang, L., Zhuang, W., Sun, M., & Zhang, H. (2020) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec48623.2020.9084931 · Cited by 8
Li, F., & Akagi, M. (2018) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553584 · Cited by 6
Q., J. M. Á., Sanabria-Ordoñez, J. A., & Melo, J. I. G. (2019) · IEEE Latin American Test Symposium (LATS) · doi:10.1109/latw.2019.8704556 · Cited by 6
Midiag: A Sequential Trace-Based Fault Diagnosis Framework for Microservices
Meng, L., Sun, Y., & Zhang, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59592-0_9 · Cited by 5
Extra-adaptive robust online subspace tracker for anomaly detection from streaming networks
Amoozegar, M., Bidgoli, B. M., Rezghi, M., & Fanaee‐T, H. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103741 · Cited by 3
Research works printed but not linked: 2
- wang, 2017, Fault diagnosis for microservices with execution trace monitoring, J Softw, 28, 1435
- zhao, 2020, Exploration of research on microservice fault diagnosis technology, J Netw New Media, 9, 57
Cited by
Soldani, J., & Brogi, A. (2023) · ACM Computing Surveys · doi:10.1145/3501297 · Cited by 256
“Jin et al. [35] allows detecting performance anomalies in multi-service applications based on the offline analysis of the traces collected through distributed tracing, by also considering the performance metrics of application services that can be directly collected from their runtime environment (e.g., CPU and memory c…”
BackgroundMethodInfluentialRobust Failure Diagnosis of Microservice System Through Multimodal Data
Zhang, S., Jin, P., Lin, Z., Sun, Y., Zhang, B., Xia, S., Li, Z., Zhong, Z., Ma, M., Jin, W., et al. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3290018 · Cited by 118
Microservice security: a systematic literature review
Berardi, D., Giallorenzo, S., Mauro, J., Melis, A., Montesi, F., & Prandini, M. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.779 · Cited by 94
On Microservice Analysis and Architecture Evolution: A Systematic Mapping Study
Bushong, V., Abdelfattah, A. S., Maruf, A. A., Das, D., Lehman, A., Jaroszewski, E., Coffey, M., Cerny, T., Frajták, K., Tisnovsky, P., et al. (2021) · Applied Sciences · doi:10.3390/app11177856 · Cited by 116
Rehman, Z., Tariq, N., Moqurrab, S. A., Yoo, J., & Srivastava, G. (2023) · Expert Systems · doi:10.1111/exsy.13467 · Cited by 88
Metadata
- Authors
- Jin, M., Lv, A., Zhu, Y., Wen, Z., Zhong, Y., Zhao, Z., Wu, J., Li, H., He, H., & Chen, F.
- Affiliations
- South China University of Technology (CN)
- JOYY Inc., Guangzhou, China
- School of Management Administration, Guangdong Industry Polytechnic, Guangzhou, China
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 226397-226408
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3113082031
- Cited by
- 50
- References
- 41
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 10 · 2023: 10 · 2022: 12 · 2021: 3
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61802129)
- Natural Science Foundation of Guangdong Province, China (2018A030310381)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- microservices, anomaly detection, robust principal component analysis, root cause analysis, Isolation Forest, one-class support vector machine, local outlier factor, AIOps, microservices security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.