CoChat
Sign Up
Cited by 32Software System Performance and ReliabilityAnomaly Detection Techniques and ApplicationsCloud Computing and Resource Management

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

MonitorAssistant: Simplifying Cloud Service Monitoring via Large Language Models

Open access
Yu, Zhaoyang, Ma, Minghua, Zhang, Chaoyun, Qin, Si, Kang, Yu, Bansal, Chetan, Rajmohan, Saravan, Dang, Yingnong, Pei, Changhua, Pei, Dan, Lin, Qingwei, Zhang, Dongmei·2024·Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering
DOI 10.1145/3663529.3663826
Open this paper in CoChat — free account

Abstract

In large-scale cloud service systems, monitoring metric data and conducting anomaly detection is an important way to maintain reliability and stability. However, great disparity exists between academic approaches and industrial practice to anomaly detection. Industry predominantly uses simple, efficient methods due to better interpretability and ease of implementation. In contrast, academically favor deep-learning methods, despite their advanced capabilities, face practical challenges in real-world applications. To address these challenges, this paper introduces MonitorAssistant, an end-to-end practical anomaly detection system via Large Language Models. MonitorAssistant automates model configuration recommendation achieving knowledge inheritance and alarm interpretation with guidance-oriented anomaly reports, facilitating a more intuitive engineer-system interaction through natural language. By deploying MonitorAssistant in Microsoft's cloud service system, we validate its efficacy and practicality, marking a significant advancement in the field of practical anomaly detection for large-scale cloud services.

Overview

Integrity signals

Registry

Crossref · 10.1145/3663529.3663826

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In incident management, LLM agents help develop systems capable of recommending investigative queries, summarising monitoring signals, and identifying potential root causes.

    “In incident management, this helps in developing systems capable of recommending investigative queries, summarising monitoring signals, and identifying potential root causes [7], [8], [10].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [7]

  • By combining language inference with operational tools, language models can support event workflows, including monitoring and diagnostics.

    “Recent research demonstrates that by combining language inference with operational tools, language models can support event workflows, including monitoring and diagnostics.[7], [8], [9], [10].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [7]

  • The useful foundation for LLMs in operations is the interface between the model and the operational system: what the model may observe, which tools it may call, what evidence it must collect, which checks it must pass, and how unsafe or uncertain actions are stopped.

    “For this reason, the useful foundation is not a general account of prompting or language modelling.It is the interface between the model and the operational system: what the model may observe, which tools it may call, what evidence it must collect, which checks it must pass, and how unsafe or uncertain actions are stopped [9], [7], [8], [67], [31], [74].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [7]

  • The reference architecture condenses the safe-autonomy commitments that recur across incident assistants, tool-grounded diagnosis systems, and verification-heavy NetOps workflows.

    “It condenses the safe-autonomy commitments that recur across incident assistants, tool-grounded diagnosis systems, and verification-heavy NetOps workflows [8], [7], [10], [9], [1], [3], [47], [51], [76], [67], [31], [74].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [7]

  • Permissions, verification checks, approval paths, and rollback plans must be part of the architecture from the start.

    “permissions, verification checks, approval paths, and rollback plans must be part of the architecture from the start [9], [7], [8], [76].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [7]

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and Shyamal Anadkat. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  2. Toufique Ahmed Supriyo Ghosh Chetan Bansal Thomas Zimmermann Xuchao Zhang and Saravan Rajmohan. 2023. Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models. arXiv preprint arXiv:2301.03797. · DOI · Paper page
  3. 10.1109/ASE.2019.00042 · DOI
  4. 10.1145/3324884.3416624 · DOI
  5. Yinfang Chen Huaibing Xie Minghua Ma Yu Kang Xin Gao Liu Shi Yunjie Cao Xuedong Gao Hao Fan and Ming Wen. 2024. Automatic Root Cause Analysis via Large Language Models for Cloud Incidents. · DOI · Paper page
  6. 10.1145/3368089.3409768 · DOI · Paper page
  7. 10.14778/3632093.3632101 · DOI · Paper page
  8. Zhuangbin Chen, Yu Kang, Liqun Li, Xu Zhang, Hongyu Zhang, Hui Xu, Yangfan Zhou, Li Yang, Jeffrey Sun, Zhangwei Xu, Yingnong Dang, Feng Gao, Pu Zhao, Bo Qiao, Qingwei Lin, Dongmei Zhang, and Michael R. Lyu. 2020. Towards intelligent incident management: why we need it and how we make it. In ESEC/SIGSOFT FSE. ACM, 1487–1497.
  9. Ruomeng Ding, Chaoyun Zhang, Lu Wang, Yong Xu, Minghua Ma, Xiaomin Wu, Meng Zhang, Qingjun Chen, Xin Gao, and Xuedong Gao. 2023. TraceDiag: Adaptive, Interpretable, and Efficient Root Cause Analysis on Large-Scale Microservice Systems. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1762–1773.
  10. 10.1007/s11023-020-09548-1 · DOI · Paper page
  11. Vaibhav Ganatra, Anjaly Parayil, Supriyo Ghosh, Yu Kang, Minghua Ma, Chetan Bansal, Suman Nath, and Jonathan Mace. 2023. Detection Is Better Than Cure: A Cloud Incidents Perspective. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1891–1902.
  12. 10.1145/3219819.3219845 · DOI · Paper page
  13. Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. 2019. Deep learning for time series classification: a review. Data mining and knowledge discovery, 33, 4 (2019), 917–963.
  14. Yuxuan Jiang, Chaoyun Zhang, Shilin He, Zhihao Yang, Minghua Ma, Si Qin, Yu Kang, Yingnong Dang, Saravan Rajmohan, Qingwei Lin, and Dongmei Zhang. 2023. Xpert: Empowering Incident Management with Query Recommendations via Large Language Models. arxiv:2312.11988. · arXiv
  15. Haozhe Li, Minghua Ma, Yudong Liu, Si Qin, Bo Qiao, Randolph Yao, Harshwardhan Chaturvedi, Tri Tran, Murali Chintalapati, and Saravan Rajmohan. 2023. CODEC: Cost-Effective Duration Prediction System for Deadline Scheduling in the Cloud. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). 298–308.
  16. 10.1109/ICSE-SEIP58684.2023.00018 · DOI
  17. Qingwei Lin, Tianci Li, Pu Zhao, Yudong Liu, Minghua Ma, Lingling Zheng, Murali Chintalapati, Bo Liu, Paul Wang, and Hongyu Zhang. 2023. Edits: An easy-to-difficult training strategy for cloud failure prediction. In Companion Proceedings of the ACM Web Conference 2023. 371–375.
  18. 10.1145/2815675.2815679 · DOI
  19. Jinyang Liu, Wenwei Gu, Zhuangbin Chen, Yichen Li, Yuxin Su, and Michael R Lyu. 2024. MTAD: Tools and Benchmarks for Multivariate Time Series Anomaly Detection. arXiv preprint arXiv:2401.06175. · arXiv
  20. 10.1109/ISSRE59848.2023.00045 · DOI
  21. Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang and Jie Tang. 2023. GPT understands too. AI Open. · DOI
  22. H Zare Moayedi and MA Masnadi-Shirazi. 2008. Arima model for network traffic prediction and anomaly detection. In 2008 international symposium on information technology. 4, 1–6.
  23. 10.1145/2723372.2737793 · DOI
  24. 10.14778/3529337.3529354 · DOI
  25. Xiaoting Qin, Minghua Ma, Yuheng Zhao, Jue Zhang, Chao Du, Yudong Liu, Anjaly Parayil, Chetan Bansal, Saravan Rajmohan, and Íñigo Goiri. 2023. How different are the cloud workloads? characterizing large-scale private and public cloud workloads. In 2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 522–530.
  26. Thanawin Rakthanmanon and Eamonn Keogh. 2013. Fast shapelets: A scalable algorithm for discovering time series shapelets. In proceedings of the 2013 SIAM International Conference on Data Mining. 668–676.
  27. 10.1145/3292500.3330680 · DOI · Paper page
  28. 10.5555/3615924.3615949 · DOI
  29. Zexin Wang, Changhua Pei, Minghua Ma, Xin Wang, Zhihan Li, Dan Pei, Saravan Rajmohan, Dongmei Zhang, Qingwei Lin, Haiming Zhang, Jianhui Li, and Gaogang Xie. 2024. Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective. In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024. ACM, 3096–3105.
  30. Canhua Wu, Nengwen Zhao, Lixin Wang, Xiaoqin Yang, Shining Li, Ming Zhang, Xing Jin, Xidao Wen, Xiaohui Nie, Wenchi Zhang, Kaixin Sui, and Dan Pei. 2021. Identifying Root-Cause Metrics for Incident Diagnosis in Online Service Systems. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). 91–102.
  31. 10.1145/3178876.3185996 · DOI
  32. Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2022. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In ICLR. OpenReview.net.
  33. 10.1109/ICSE-SEIP58684.2023.00026 · DOI
  34. 10.1145/1557019.1557122 · DOI
  35. Zhaoyang Yu, Qianyu Ouyang, Changhua Pei, Xin Wang, Wenxiao Chen, Liangfei Su, Huai Jiang, Xuanrun Wang, Jianhui Li, and Dan Pei. 2024. Causality Enhanced Graph Representation Learning for Alert-Based Root Cause Analysis. In 2024 IEEE/ACM 24rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid).
  36. Zhaoyang Yu, Changhua Pei, Shenglin Zhang, Xidao Wen, Jianhui Li, Gaogang Xie, and Dan Pei. 2023. AutoKAD: Empowering KPI Anomaly Detection with Label-Free Deployment. In ISSRE. IEEE, 13–23.
  37. Zhaoyang Yu, Shenglin Zhang, Mingze Sun, Yingke Li, Yankai Zhao, Xiaolei Hua, Lin Zhu, Xidao Wen, and Dan Pei. 2024. Supervised Fine-Tuning for Unsupervised KPI Anomaly Detection for Mobile Web Systems. In Proceedings of the ACM on Web Conference 2024. 2859–2869.
  38. 10.1109/ICSE-SEIP58684.2023.00029 · DOI
  39. Xuchao Zhang Supriyo Ghosh Chetan Bansal Rujia Wang Minghua Ma Yu Kang and Saravan Rajmohan. 2024. Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4. arxiv:2401.13810. · DOI
  40. Chenyu Zhao, Minghua Ma, Zhenyu Zhong, Shenglin Zhang, Zhiyuan Tan, Xiao Xiong, LuLu Yu, Jiayi Feng, Yongqian Sun, and Yuzhi Zhang. 2023. Robust multimodal failure detection for microservice systems. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5639–5649.
  41. 10.1145/3377813.3381363 · DOI
  42. 10.1145/3468264.3468543 · DOI
  43. Zhenyu Zhong Qiliang Fan Jiacheng Zhang Minghua Ma Shenglin Zhang Yongqian Sun Qingwei Lin Yuzhi Zhang and Dan Pei. 2023. A Survey of Time Series Anomaly Detection Methods in the AIOps Domain. arXiv preprint arXiv:2308.00393. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Yu, Z., Ma, M., Zhang, C., Qin, S., Kang, Y., Bansal, C., Rajmohan, S., Dang, Y., Pei, C., Pei, D., Lin, Q., & Zhang, D.
Affiliations
  • Tsinghua University (CN)
  • Microsoft (United States) (US)
  • Microsoft Research Asia (China) (CN)
  • Chinese Academy of Sciences (CN)
ORCID
Venue
Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering
Publisher
ACM
Published
2024-07-10
Pages
38-49
DOI
10.1145/3663529.3663826
OpenAlex
W4400484838
Cited by
32
References
2
Citations by year
2026: 16 · 2025: 10 · 2024: 6
FWCI
14.01
Type
conference paper
Language
en
Open access
Licence
Locations
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion, cc-by)
Funders
National Natural Science Foundation of China (62072264)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
Keywords
cloud service monitoring, anomaly detection, large language models, knowledge inheritance, Microsoft cloud services

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review