CoChat

Root Cause Analysis for Microservice System based on Causal Inference: How Far Are We?

Open access
Pham, Luan, Ha, Huong, Zhang, Hongyu · 2024 · Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Abstract

Microservice architecture has become a popular architecture adopted by many cloud applications. However, identifying the root cause of a failure in microservice systems is still a challenging and time-consuming task. In recent years, researchers have introduced various causal inference-based root cause analysis methods to assist engineers in identifying the root causes. To gain a better understanding of the current status of causal inference-based root cause analysis techniques for microservice systems, we conduct a comprehensive evaluation of nine causal discovery methods and twenty-one root cause analysis methods. Our evaluation aims to understand both the effectiveness and efficiency of causal inference-based root cause analysis methods, as well as other factors that affect their performance. Our experimental results and analyses indicate that no method stands out in all situations; each method tends to either fall short in effectiveness, efficiency, or shows sensitivity to specific parameters. Notably, the performance of root cause analysis methods on synthetic datasets may not accurately reflect their performance in real systems. Indeed, there is still a large room for further improvement. Furthermore, we also suggest possible future work based on our findings.

Overview

Integrity signals

Registry

Crossref · 10.1145/3691620.3695065

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

4 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 37 references

Research works printed but not linked: 23

  1. 2023. The Istio service mesh. Retrieved Oct 10, 2023 from https://istio.io/
  2. 2023. An open-source monitoring and alerting toolkit. Retrieved Oct 10, 2023 from https://prometheus.io/
  3. Devansh Arpit Matthew Fernandez Itai Feigenbaum Weiran Yao Chenghao Liu Wenzhuo Yang Paul Josel Shelby Heinecke Eric Hu Huan Wang Stephen Hoi Caiming Xiong Kun Zhang and Juan Carlos Niebles. 2023. Salesforce CausalAI Library: A Fast and Scalable framework for Causal Analysis of Time Series and Tabular Data. (2023). arXiv:arXiv preprint arXiv:2301.10859 [cs.LG] · arXiv
  4. Charles K. Assaad, Emilie Devijver, and Eric Gaussier. 2022. Survey and Evaluation of Causal Discovery Methods for Time Series. Journal of Artificial Intelligence Research 73 (2022).
  5. Konstantina Biza, Ioannis Tsamardinos, and Sofia Triantafillou. 2020. Tuning causal discovery algorithms. In International Conference on Probabilistic Graphical Models. PMLR, 17--28.
  6. Junjie Chen, Xiaoting He, Qingwei Lin, Yong Xu, Hongyu Zhang, Dan Hao, Feng Gao, Zhangwei Xu, Yingnong Dang, and Dongmei Zhang. 2019. An Empirical Investigation of Incident Triage for Online Service Systems. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 111--120.
  7. Yujun Chen, Xian Yang, Qingwei Lin, Hongyu Zhang, Feng Gao, Zhangwei Xu, Yingnong Dang, Dongmei Zhang, Hang Dong, Yong Xu, Hao Li, and Yu Kang. 2019. Outage Prediction and Diagnosis for Cloud Service Systems. In The World Wide Web Conference (San Francisco, CA, USA) (WWW '19). Association for Computing Machinery, New York, NY, USA, 2659--2665.
  8. Clark Glymour, Kun Zhang, and Peter Spirtes. 2019. Review of Causal Discovery Methods Based on Graphical Models. Frontiers in Genetics 10 (2019).
  9. Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. 2022. Root Cause Analysis of Failures in Microservices through Causal Discovery. In Advances in Neural Information Processing Systems (NeurIPS'22), Vol. 35. 31158--31170.
  10. Amin Jaber, Murat Kocaoglu, Karthikeyan Shanmugam, and Elias Bareinboim. 2020. Causal Discovery from Soft Interventions with Unknown Targets: Characterization and Learning. In Advances in Neural Information Processing Systems (NeurIPS'20), Vol. 33. 9551--9561.
  11. Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, and Michael R Lyu. 2023. Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data. arXiv preprint arXiv:2302.05092 (2023). · arXiv
  12. Jinjin Lin, Pengfei Chen, and Zibin Zheng. 2018. Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments. In Service-Oriented Computing. 3--20.
  13. Chenghao Liu, Wenzhuo Yang, Himanshu Mittal, Manpreet Singh, Doyen Sahoo, and Steven CH Hoi. 2023. PyRCA: A Library for Metric-based Root Cause Analysis. arXiv preprint arXiv:2306.11417 (2023). · arXiv
  14. Meng Ma, Weilan Lin, Disheng Pan, and Ping Wang. 2019. MS-Rank: Multi-Metric and Self-Adaptive Root Cause Diagnosis for Microservice Applications. In IEEE International Conference on Web Services (ICWS'19). 60--67.
  15. Vineet K Raghu, Allen Poon, and Panayiotis V Benos. 2018. Evaluation of causal structure learning methods on mixed data types. In Proceedings of 2018 ACM SIGKDD Workshop on Causal Discovery. PMLR, 48--65.
  16. Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. 2019. Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances 5, 11 (2019).
  17. 10.5555/1248547.1248619 · doi:10.5555/1248547.1248619
  18. Julien Siebert. 2023. Applications of statistical causal inference in software engineering. Information and Software Technology 159 (2023).
  19. Jacopo Soldani and Antonio Brogi. 2022. Anomaly Detection and Failure Root Cause Analysis in (Micro) Service-Based Cloud Applications: A Survey. Comput. Surveys 55, 3 (2022).
  20. Matthew J. Vowels, Necati Cihan Camgoz, and Richard Bowden. 2022. D'Ya Like DAGs? A Survey on Structure Learning and Causal Discovery. Comput. Surveys 55, 4, Article 82 (2022).
  21. Ping Wang, Jingmin Xu, Meng Ma, Weilan Lin, Disheng Pan, Yuan Wang, and Pengfei Chen. 2018. CloudRanger: Root Cause Identification for Cloud Native Systems. In 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID'18). 492--502.
  22. Qing Wang, Larisa Shwartz, Genady Ya. Grabarnik, Vijay Arya, and Karthikeyan Shanmugam. 2021. Detecting Causal Structure on Cloud Application Microservices Using Granger Causality Models. In IEEE 14th International Conference on Cloud Computing (CLOUD'21). 558--565.
  23. Li Wu, Johan Tordsson, Jasmin Bogatinovski, Erik Elmroth, and Odej Kao. 2021. MicroDiag: Fine-grained Performance Diagnosis for Microservice Systems. In 2021 IEEE/ACM International Workshop on Cloud Intelligence. 31--36.

Cited by

Metadata

Authors
Pham, L., Ha, H., & Zhang, H.
Affiliations
  • RMIT University (AU)
  • Chongqing University (CN)
Venue
Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering
Publisher
ACM
Published
2024-10-27
Pages
706-715
OpenAlex
W4403536391
Cited by
38
References
36
Citations by year
2026: 29 · 2025: 8 · 2024: 1
FWCI
16.64
Type
conference paper
Language
en
Open access
Locations
  • IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Australian Research Council Discovery Project (DP220103044)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
root cause analysis, microservices, causal inference, causal discovery, fault diagnosis, cloud applications, method evaluation, synthetic dataset, performance sensitivity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.