Adaptive performance anomaly detection for online service systems via pattern sketching
Open accessThe DOI matches its Crossref record; 31 of the 34 references it cites were found in a registry or an index.
Abstract
To ensure the performance of online service systems, their status is closely monitored with various software and system metrics. Performance anomalies represent the performance degradation issues (e.g., slow response) of the service systems. When performing anomaly detection over the metrics, existing methods often lack the merit of interpretability, which is vital for engineers and analysts to take remediation actions. Moreover, they are unable to effectively accommodate the ever-changing services in an online fashion. To address these limitations, in this paper, we propose ADSketch, an interpretable and adaptive performance anomaly detection approach based on pattern sketching. ADSketch achieves interpretability by identifying groups of anomalous metric patterns, which represent particular types of performance issues. The underlying issues can then be immediately recognized if similar patterns emerge again. In addition, an adaptive learning algorithm is designed to embrace unprecedented patterns induced by service updates or user behavior changes. The proposed approach is evaluated with public data as well as industrial data collected from a representative online service system in Huawei Cloud. The experimental results show that ADSketch outperforms state-of-the-art approaches by a significant margin, and demonstrate the effectiveness of the online algorithm in new pattern discovery. Furthermore, our approach has been successfully deployed in industrial practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies also highlight the importance of correlating system-level and application-level metrics [103,104], enabling earlier anomaly identification and more accurate root-cause localization.”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [104]
Citations
Cites
Clustering by Passing Messages Between Data Points
Frey, B. J., & Dueck, D. (2007) · Science · doi:10.1126/science.1136800 · Cited by 6,758
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., & Pei, D. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330672 · Cited by 1,804 · cited 3×
“, [29, 32], essentially corresponds to the process adopted in RQ1 (i.”
BackgroundMethodSelf-citationDetecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219845 · Cited by 1,771 · cited 3×
“Among them, LSTM [15, 37] and Donut [34] achieve comparable performance compared to that of ADSketch (i.”
BackgroundMethodA Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder
Park, D., Hoshi, Y., & Kemp, C. C. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2801475 · Cited by 1,115
Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
Xu, H., Feng, Y., Chen, J., Wang, Z., Qiao, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., et al. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3185996 · Cited by 808
Yeh, C.-C. M., Zhu, Y., Ulanova, L., Begum, N., Ding, Y., Dau, H. A., Silva, D. F., Mueen, A., & Keogh, E. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0179 · Cited by 653
Time-Series Anomaly Detection Service at Microsoft
Ren, H., Xu, B., Wang, Y., Yi, C., Huang, C., Kou, X., Xing, T., Yang, M., Tong, J., & Zhang, Q. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330680 · Cited by 623
Anomaly Detection in Streams with Extreme Value Theory
Siffer, A., Fouque, P.-A., Termier, A., & Largouet, C. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098144 · Cited by 553
Log clustering based problem identification for online service systems
Lin, Q., Zhang, H., Lou, J.-G., Zhang, Y., & Chen, X. (2016) · Proceedings of the 38th International Conference on Software Engineering Companion · doi:10.1145/2889160.2889232 · Cited by 427
Self-citationFast Shapelets: A Scalable Algorithm for Discovering Time Series Shapelets
Rakthanmanon, T., & Keogh, E. (2013) · Proceedings of the 2013 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972832.74 · Cited by 388
Loda: Lightweight on-line detector of anomalies
Pevný, T. (2016) · Machine Learning · doi:10.1007/s10994-015-5521-0 · Cited by 357
AIOps: Real-World Challenges and Research Innovations
Dang, Y., Lin, Q., & Huang, P. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00023 · Cited by 238
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J., & Wang, B. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201224 · Cited by 226
Identifying impactful service system problems via log analysis
He, S., Lin, Q., Lou, J.-G., Zhang, H., Lyu, M. R., & Zhang, D. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236083 · Cited by 201
Self-citationDisk aware discord discovery: finding unusual time series in terabyte sized datasets
Yankov, D., Keogh, E., & Rebbapragada, U. D. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-008-0131-9 · Cited by 159
Gray Failure: The Achilles' Heel of Cloud-Scale Systems
Huang, P., Guo, C., Zhou, L., Lorch, J. R., Dang, Y., Chintalapati, M., & Yao, R. (2017) · Proceedings of the 16th Workshop on Hot Topics in Operating Systems · doi:10.1145/3102980.3103005 · Cited by 147
Diagnosing root causes of intermittent slow queries in cloud databases
Ma, M., Yin, Z., Zhang, S., Wang, S., Zheng, C., Jiang, X., Hu, H., Luo, C., Li, Y., Qiu, N., et al. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3389133.3389136 · Cited by 134
Graph-based root cause analysis for service-oriented and microservice architectures
Brandón, Á., Solé, M., Huélamo, A., Solans, D., Pérez, M. S., & Muntés-Mulero, V. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2019.110432 · Cited by 121 · cited 1×
“[4] manually defined a set of features collected from metrics, logs, and anomalies to characterize failures.”
Matrix Profile XI: SCRIMP++: Time Series Motif Discovery at Interactive Speeds
Zhu, Y., Yeh, C.-C. M., Zimmerman, Z., Kamgar, K., & Keogh, E. (2018) · 2018 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2018.00099 · Cited by 118
Predicting Node failure in cloud service systems
Lin, Q., Hsieh, K., Dang, Y., Zhang, H., Sui, K., Xu, Y., Lou, J.-G., Li, C., Wu, Y., Yao, R., et al. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236060 · Cited by 111
Self-citationFarshchi, M., Schneider, J.-G., Weber, I., & Grundy, J. (2015) · 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2015.7381796 · Cited by 104
Towards intelligent incident management: why we need it and how we make it
Chen, Z., Kang, Y., Li, L., Zhang, X., Zhang, H., Xu, H., Zhou, Y., Yang, L., Sun, J., Xu, Z., et al. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417055 · Cited by 99
Self-citationSoftware analytics for incident management of online services: An experience report
Lou, J.-G., Lin, Q., Ding, R., Fu, Q., Zhang, D., & Xie, T. (2013) · 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693105 · Cited by 63
LeSiNN: Detecting Anomalies by Identifying Least Similar Nearest Neighbours
Pang, G., Ting, K. M., & Albrecht, D. (2015) · 2015 IEEE International Conference on Data Mining Workshop (ICDMW) · doi:10.1109/icdmw.2015.62 · Cited by 56
Predicting Performance Anomalies in Software Systems at Run-time
Zhao, G., Hassan, S., Zou, Y., Truong, D., & Corbin, T. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3440757 · Cited by 42
DISCRETE TIME REPRESENTATION OF CONTINUOUS TIME ARMA PROCESSES
Chambers, M. J., & Thornton, M. A. (2011) · Econometric Theory · doi:10.1017/s0266466611000181 · Cited by 35
Toward Adaptive Disk Failure Prediction via Stream Mining
Han, S., Lee, P. P. C., Shen, Z., He, C., Liu, Y., & Huang, T. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00044 · Cited by 31
Performance Issues? Hey DevOps, Mind the Uncertainty
Trubiani, C., Jamshidi, P., Cito, J., Shang, W., Jiang, Z. M., & Borg, M. (2018) · IEEE Software · doi:10.1109/ms.2018.2875989 · Cited by 20
Mercer, R., Alaee, S., Abdoli, A., Singh, S., Murillo, A. C., & Keogh, E. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00151 · Cited by 20
Disk aware discord discovery: finding unusual time series in terabyte sized datasets
YankovDragomir, KeoghEamonn, & RebbapragadaUmaa (2008) · Knowledge and Information Systems · doi:10.5555/1473147.1473152 · Cited by 9
Research works printed but not linked: 6
- 2018. KPI Anomaly Detection Competition. Retrieved April, 2021 from http://iops.ai/competition_detail/?competition_id=5&flag=1
- 2018. KPI Anomaly Detection Dataset. Retrieved April, 2021 from http://iops.ai/dataset_detail/?id=10
- Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. 2004. Convex optimization. Cambridge university press.
- Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In 2008 eighth ieee international conference on data mining. IEEE, 413--422.
- Chang Lou, Peng Huang, and Scott Smith. 2020. Understanding, detecting and localizing partial failures in large system software. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). 559--574.
- Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations.
Cited by
Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data
Lee, C., Yang, T., Chen, Z., Su, Y., & Lyu, M. R. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00150 · Cited by 149
“Previous anomaly detection approaches are usually based on system logs [58]–[62] or KPIs [63]–[67], or both [68], targeting traditional distributed systems without complex invocation relationships.”
Self-citationTFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysis
Zhang, C., Zhou, T., Wen, Q., & Sun, L. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557470 · Cited by 138
Actionable and interpretable fault localization for recurring failures in online service systems
Li, Z., Zhao, N., Li, M., Lu, X., Wang, L., Chang, D., Nie, X., Cao, L., Zhang, W., Sui, K., et al. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549092 · Cited by 70
Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention
Lee, C., Yang, T., Chen, Z., Su, Y., Yang, Y., & Lyu, M. R. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00148 · Cited by 68
Self-citationTimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models
Si, H., Li, J., Pei, C., Cui, H., Yang, J., Sun, Y., Zhang, S., Li, J., Zhang, H., Han, J., et al. (2024) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre62328.2024.00017 · Cited by 58
Self-citation
Metadata
- Authors
- Chen, Z., Liu, J., Su, Y., Zhang, H., Ling, X., Yang, Y., & Lyu, M. R.
- Affiliations
- Chinese University of Hong Kong (HK)
- Sun Yat-sen University (CN)
- University of Newcastle Australia (AU)
- Huawei Technologies (China) (CN)
- Venue
- Proceedings of the 44th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2022-05-21
- Pages
- 61-72
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4226375347
- Cited by
- 52
- References
- 34
- Citations by year
- 2026: 8 · 2025: 20 · 2024: 12 · 2023: 9 · 2022: 3
- FWCI
- 10.66
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- NOVA (University of Newcastle Australia) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Research Grants Council of the Hong Kong Special Administrative Region (CUHK 14210920)
- Australian Research Council (ARC) Discovery Projects (DP200102940 and DP220103044)
- Guangdong Key Research Program (No. 2020B010165002)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Keywords
- performance anomaly detection, online service systems, interpretable anomaly detection, adaptive learning, system metrics, performance degradation, service updates
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.