LLexus: an AI agent system for incident management
Open accessAbstract
When operating a software service on a cloud, the complexity of keeping multiple distributed components responsive is a significant challenge for engineering teams. Engineers frequently rely on Troubleshooting Guides (TSGs) to navigate how to mitigate performance or outage incidents. However, the effectiveness of TSGs is often hindered by their length, implicit reliance on tribal knowledge, and the variable quality of their content. This paper introduces LLexus, an agent-based AI system to automate the execution of TSGs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Defences against telemetry manipulation include cross-checking telemetry, validating formats, tracking provenance, and refusing action when evidence is too weak.
“Defences include cross-checking telemetry, validating formats, tracking provenance, and refusing action when evidence is too weak [83], [30], [31].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [83]
Section 4.3 (page unavailable) describes validating tool choices and checking that plan inputs can be sourced from previous steps, the incident, or predefined values. Section 4 also describes flagging invalid responses and limiting retries. These are safeguards for generated troubleshooting plans, not stated defences against telemetry manipulation; the paper does not establish cross-checking telemetry, validating telemetry formats, or refusing action specifically because telemetry evidence is weak. (LLM, conf=0.76)
Experimental — accuracy not yet measuredRisks from tool misuse, prompt injection through operational artefacts, drift between verified and realised state, and rollback failure become more serious once the agent is allowed to propose or execute actions.
“Hallucination or story-mode reasoning can affect every rung, but tool misuse, prompt injection through operational artefacts, drift between verified and realised state, and rollback failure become more serious once the agent is allowed to propose or execute actions [81], [95], [83], [65], [102].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [83]
The Overview of LLexus (section 3) warns that generic tools such as SSH would give the system unrestricted access to production systems, and Multi-Step Plan Generation (section 4) validates tool selection to avoid hallucination-related errors. These passages support concern about tool use when actions are executed, but the paper does not address prompt injection through operational artefacts, drift between verified and realised state, or rollback failure. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIn AIOps, the more common failure is accelerating a bad mitigation, or delaying a correct mitigation through repeated refusal or mis-triage.
“In AIOps, the more common failure is accelerating a bad mitigation, or delaying a correct mitigation through repeated refusal or mis-triage [83], [94].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [83]
Section 3.2.1 says nondeterministic incident investigation can take unexpected paths and produce incorrect answers. Section 6.2 gives an example of a generated plan that incorrectly escalated an issue. The paper does not establish that these are the more common AIOps failures, or describe accelerating a bad mitigation or delaying a correct one through repeated refusal. (LLM, conf=0.76)
Experimental — accuracy not yet measuredAn agent may still act wrongly, even when it is grounded, if the evidence on which it relies is false or incomplete.
“An agent may still act wrongly, even when it is grounded, if the evidence on which it relies is false or incomplete [83].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [83]
In adversarial settings, compromised devices may inject misleading logs, counters, or protocol events.
“In adversarial settings, compromised devices may inject misleading logs, counters, or protocol events [83], [80].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [83]
The Network Link case study describes engineers examining metrics and logs to diagnose unexpected behavior, but does not discuss compromised devices or deliberately misleading data. Searches for adversarial settings, falsified telemetry, counters, and protocol events found no passage addressing the claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · see citing works
On the Planning Abilities of Large Language Models - A Critical InvestigationPreprint
Valmeekam, K., Marquez, M., Sreedharan, S., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.15771 · Cited by 525 · see citing works
VAL: automatic plan validation, continuous effects and mixed initiative planning using PDDL
Howey, R., Long, D., & Fox, M. (2005) · 16th IEEE International Conference on Tools with Artificial Intelligence · doi:10.1109/ictai.2004.120 · Cited by 194 · see citing works
Automatic Root Cause Analysis via Large Language Models for Cloud Incidents
Chen, Y., Xie, H., Ma, M., Kang, Y., Gao, X., Shi, L., Cao, Y., Gao, X., Fan, H., Wen, M., et al. (2024) · Proceedings of the Nineteenth European Conference on Computer Systems · doi:10.1145/3627703.3629553 · Cited by 190 · see citing works
Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models
Ahmed, T., Ghosh, S., Bansal, C., Zimmermann, T., Zhang, X., & Rajmohan, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00149 · Cited by 148 · see citing works
Exploring LLM-Based Agents for Root Cause Analysis
Roy, D., Zhang, X., Bhave, R., Bansal, C., Las-Casas, P., Fonseca, R., & Rajmohan, S. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663841 · Cited by 113 · see citing works
Self-citationAn Empirical Investigation of Incident Triage for Online Service Systems
Chen, J., He, X., Lin, Q., Xu, Y. D., Zhang, H., Hao, D., Gao, F., Xu, Z., Dang, Y., & Zhang, D. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00020 · Cited by 102 · see citing works
On the Planning Abilities of Large Language Models - A Critical Investigation
Valmeekam, K., Marquez, M., Sreedharan, S., & Subbarao Kambhampati, B. (2023) · neural information processing systems · doi:10.52202/075280-3320 · Cited by 98 · see citing works
Continuous Incident Triage for Large-Scale Online Service Systems
Chen, J., He, X., Lin, Q., Zhang, H., Hao, D., Gao, F., Xu, Z., Dang, Y., & Zhang, D. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00042 · Cited by 86 · see citing works
Bansal, C., Renganathan, S., Asudani, A., Midy, O., & Janakiraman, M. (2020) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3377813.3381353 · Cited by 45 · see citing works
Xpert: empowering incident management with query recommendations via large language models
Jiang, Y., Zhang, C., He, S., Yang, Z., Ma, M., Qin, S., Kang, Y., Dang, Y., Rajmohan, S., Lin, Q., et al. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639081 · Cited by 45 · see citing works
Mining Root Cause Knowledge from Cloud Service Incident Investigations for AIOps
Saha, A., & Hoi, S. C. H. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3510457.3513030 · Cited by 44 · see citing works
Neural Knowledge Extraction From Cloud Service Incidents
Shetty, M., Bansal, C., Kumar, S., Rao, N., Nagappan, N., & Zimmermann, T. (2021) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00031 · Cited by 24 · see citing works
AutoTSG: learning and synthesis for incident troubleshooting
Shetty, M., Bansal, C., Upadhyayula, S. P., Radhakrishna, A., & Gupta, A. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3558958 · Cited by 13 · see citing works
SoftNER: Mining knowledge graphs from cloud incidents
Shetty, M., Bansal, C., Kumar, S., Rao, N., & Nagappan, N. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10159-w · Cited by 12 · see citing works
Understanding and predicting incident mitigation time
Wang, W., Chen, J., Yang, L., Zhang, H., & Wang, Z. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107119 · Cited by 11 · see citing works
Xpert: Empowering Incident Management with Query Recommendations via Large Language ModelsPreprint
Jiang, Y., Zhang, C., He, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11988 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- Kaikai An, Fangkai Yang, Liqun Li, Zhixing Ren, Hao Huang, Lu Wang, Pu Zhao, Yu Kang, Hua Ding, Qingwei Lin, Saravan Rajmohan, and Qi Zhang. Nissist: An incident mitigation copilot based on trou- bleshooting guides. arXiv preprint arXiv:2402.17531, 2024. · arXiv
- Chetan Bansal, Sundararajan Renganathan, Ashima Asudani, Olivier Midy, and Mathru Janakiraman. Decaf: Diagnosing and triaging per- formance issues in large-scale cloud services. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Soft- ware Engineering in Practice, pages 201-210, 2020.
- Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early exper- iments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. · arXiv
- Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauff- mann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harki- rat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tau- man Kalai, Yin Tat Lee, and Yuanzhi Li. Textbooks are all you need. arXiv preprint arXiv:2306.11644, 2023. · arXiv
- Jiajun Jiang, Weihai Lu, Junjie Chen, Qingwei Lin, Pu Zhao, Yu Kang, Hongyu Zhang, Yingfei Xiong, Feng Gao, Zhangwei Xu, Yingnong Dang, and Dongmei Zhang. How to mitigate the incident? an effective troubleshooting guide recommendation technique for online service systems. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, page 14101420, New York, NY, USA, 2020. Association for Computing Machinery.
- Shinji Kikuchi. Prediction of workloads in incident management based on incident ticket updating history. In Proceedings of the 8th In- ternational Conference on Utility and Cloud Computing, UCC '15, page 333340. IEEE Press, 2015.
- Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023. · arXiv
- Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. Can large language models really improve by self-critiquing their own plans? arXiv preprint arXiv:2310.08118, 2023. · arXiv
Cited by
Sorted by citations
A Survey of AIOps in the Era of Large Language Models
Zhang, L., Jia, T., Jia, M., Wu, Y., Liu, A., Yang, Y., Wu, Z., Hu, X., Yu, P., & Li, Y. (2026) · ACM Computing Surveys · doi:10.1145/3746635 · Cited by 49 · see citing works
AI-Augmented SOC: A Survey of LLMs and Agents for Security Automation
Srinivas, S., Kirk, B., Zendejas, J., Espino, M., Boskovich, M., Bari, A., Dajani, K., & Alzahrani, N. (2025) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp5040095 · Cited by 39 · see citing works
Implementing Agentic AI Into ERP Software
Sarferaz, S. (2025) · IEEE Access · doi:10.1109/access.2025.3621887 · Cited by 15 · see citing works
Between Promise and Pain: The Reality of Automating Failure Analysis in Microservices with LLMs
Cornacchia, A., Alabdulaal, I., Saghier, I., Mirdad, A., Fayoumi, O., & Canini, M. (2025) · ACM SIGOPS Asia-Pacific Workshop on Systems (APSys) · doi:10.1145/3725783.3764388 · Cited by 4 · see citing works
The Dual Role of Large Language Models in Network Security: Survey and Research Trends
Liu, H., Xue, J., Zhao, S., Liu, Y., & Lu, Z. (2025) · ACM Workshop on Wireless Security and Machine Learning (WiseML) · doi:10.1145/3733965.3733972 · Cited by 3 · see citing works
A Context-Aware, AI-Driven Load Balancing Framework for Incident Escalation in SOCs
Abuaziz, A., & Çeliktaş, B. (2025) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas66241.2025.11101733 · Cited by 1 · see citing works
Intent-based System Design and Operation
Anand, V., Li, Y., Kumbhare, A. G., Irvene, C., Bansal, C., Somashekar, G., Mace, J., Las-Casas, P., Bianchini, R., & Fonseca, R. (2025) · Workshop on Practical Adoption Challenges of ML for Systems (PACMI) · doi:10.1145/3766882.3767182 · Cited by 1 · see citing works
Self-citationKikuta, D., Ikeuchi, H., & Tajiri, K. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00331 · Cited by 1 · see citing works
Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety
Bilal, M., Crowcroft, J., Wang, R., Xu, X., & Dustdar, S. (2027) · Computer Science Review · doi:10.1016/j.cosrev.2026.101075
“…k : budgets and success criteria Copilot (read-only) Search, retrieval, summarisation, and query suggestion; no propose or execute tools [8], [7], [67] Provenance-linked observations;cited sourcesnippets; freshnessmarkers whereavailable[8], [7],[25] Redaction, provenance rules, citation…”
InfluentialAI Powered Server Log Management Software
Rekha, D. S. B. N. S., Sai, G. R., Swaroop, K., Kiran, V. R. P., Shekhar, K. S. C., & Hota, C. S. (2025) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces67310.2025.11337167
Artificial Intelligence Technologies in Solving Information Security Problems
Potienko, D. A., Chmykhalo, D. S., & Легонько, О. Л. (2026) · Herald of Dagestan State Technical University Technical Sciences · doi:10.21822/2073-6185-2026-53-1-157-169
Intelligent Multi-Agent Orchestration Framework for Enterprise Financial Systems
Biru, S. (2026) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn70047.2026.11594167
Li, Z., Wang, N., Liu, J., Zhang, Y., Tong, F., Chen, Z., Li, C., Liu, M., Zhang, X., Wu, Y., et al. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786903
Leveraging LLMs for Alert Summarization and Mitigation Plan Generation
Sarda, K., Ji, H., Zaki, A. M., Watts, I., Shwartz, L., & Litoiu, M. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805255
Design and Practice of an AIOps Framework for Large-Scale Campus Networks
Lai, Q., Liu, W., Chen, J., Fu, Z., & Ma, H. (2026) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms69089.2026.11668372
HERO: Hypothesis-Centered Root-Cause Analysis for Microservice Incidents
Lyu, J., Yi, J., Liang, S., Chen, P., & Pan, L. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837509
- Authors
- Las-Casas, P., Kumbhare, A. G., Fonseca, R., & Agarwal, S.
- Affiliations
- Microsoft (United States) (US)
- Microsoft
- Venue
- ACM SIGOPS Operating Systems Review
- Published
- 2024-08-14
- Volume
- 58
- Issue
- 1
- Pages
- 23-36
- ISSN
- 0163-5980 (ISSN-L)
- OpenAlex
- W4401560363
- Cited by
- 14
- References
- 19
- Citations by year
- 2026: 6 · 2025: 8
- FWCI
- 2.34
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGOPS Operating Systems Review (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Keywords
- incident management, troubleshooting guide, AI agents, distributed systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
