DyCause: Crowdsourcing to Diagnose Microservice Kernel Failure
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Today many web applications in the cloud (apps) are built based on microservices. However, as the anomaly propagates in a highly dynamic and complex way, troubleshooting them becomes full of challenges. Existing diagnostic methods are mostly designed based on monitoring metrics retrieved from the microservice system kernel. Therefore, application owners and even site reliability engineers (SREs) cannot effectively resort to those methods when the microservice systems lack such a comprehensive monitoring infrastructure. In this article, we develop DyCause, a crowdsourcing solution to the asymmetric diagnostic information problem. Our solution collects the operational status of kernel services collaboratively from the user space and initiates diagnosis on demand. Without the requirement of any architectural or functional infrastructure, it is both fast and lightweight to deploy DyCause in a microservice system. In order to discover the fine-grained dynamic causalities between services during the anomaly, we also design an efficient algorithm based on statistical analysis. Based on this algorithm, we can also analyze the anomaly propagation paths within the microservice system and generate a better interpretable diagnosis. In our evaluation, we test DyCause in a controlled simulation environment and a real-world cloud system. Our results have shown that DyCause has the best accuracy and efficiency among several state-of-the-art methods and is more robust in terms of parameters.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“DyCause [65,134] applies Granger causality and Extreme Value Theory to capture latency fluctuations and extreme event structures across APIs.”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [134]
Citations
Cites
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Limiting forms of the frequency distribution of the largest or smallest member of a sample
Fisher, R. A., & Tippett, L. H. C. (1928) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100015681 · Cited by 3,310
Sur La Distribution Limite Du Terme Maximum D'Une Serie Aleatoire
Gnedenko, B. (1943) · The Annals of Mathematics · doi:10.2307/1968974 · Cited by 2,258
Residual Life Time at Great Age
Balkema, A. A., & de Haan, L. (1974) · The Annals of Probability · doi:10.1214/aop/1176996548 · Cited by 1,476
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332 · cited 1×
“Most of them first model the microservice system as static structures such as a hybrid graph and Markov models and then detect problems from the deviations of previous structures [8], [9], [10], [11].”
BackgroundPinpoint: problem determination in large, dynamic Internet services
Chen, M. Y., Kiciman, E., Fratkin, E., Fox, A., & Brewer, E. (2003) · Proceedings International Conference on Dependable Systems and Networks · doi:10.1109/dsn.2002.1029005 · Cited by 835
Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
Xu, H., Feng, Y., Chen, J., Wang, Z., Qiao, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., et al. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3185996 · Cited by 808
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture
Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2016) · IEEE Software · doi:10.1109/ms.2016.64 · Cited by 735 · cited 1×
“W ITH the benefits of high reusability, fast evolution, and easy scaling, web application developments have embraced the microservice architecture [1].”
MethodLogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs
Meng, W., Liu, Y., Zhu, Y., Zhang, S., Pei, D., Liu, Y., Chen, Y., Zhang, R., Tao, S., Sun, P., et al. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/658 · Cited by 708 · cited 1×
“Some methods such as LogAD [12] and LogAnomaly [13] mine rich information from unstructured logs and detect anomaly log patterns.”
MethodEstimating high-dimensional directed acyclic graphs with the PC-algorithm
Kalisch, M., & Bühlmann, P. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.math/0510436 · Cited by 591
Anomaly Detection in Streams with Extreme Value Theory
Siffer, A., Fouque, P.-A., Termier, A., & Largouet, C. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098144 · Cited by 553
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2887384 · Cited by 410 · cited 1×
“These incidents not only cause economic losses for online business services but also degrades the reputation of cloud platforms [6].”
BackgroundVillamizar, M., Garces, O., Castro, H., Verano, M., Salamanca, L., Casallas, R., & Gil, S. (2015) · 2015 10th Computing Colombian Conference (10CCC) · doi:10.1109/columbiancc.2015.7333476 · Cited by 308
Structured comparative analysis of systems logs to diagnose performance problems
Nagaraj, K., Killian, C. E., & Neville, J. (2012) · Purdue e-Pubs (Purdue University System) · Cited by 289
MicroRCA: Root Cause Localization of Performance Issues in Microservices
Wu, L., Tordsson, J., Elmroth, E., & Kao, O. (2020) · NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium · doi:10.1109/noms47738.2020.9110353 · Cited by 265
Performance modeling and system management for multi-component online services
Stewart, C., & Shen, K. (2005) · doi:10.5555/1251203.1251209 · Cited by 219
(2015) · doi:10.1145/2815400.2815415 · Cited by 202
Detailed diagnosis in enterprise networks
Kandula, S., Mahajan, R., Verkaik, P., Agarwal, S., Padhye, J., & Bahl, P. (2009) · Proceedings of the ACM SIGCOMM 2009 conference on Data communication · doi:10.1145/1592568.1592597 · Cited by 194
Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments
Lin, J., Chen, P., & Zheng, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-03596-9_1 · Cited by 192
CloudRanger: Root Cause Identification for Cloud Native Systems
Wang, P., Xu, J., Ma, M., Lin, W., Pan, D., Wang, Y., & Chen, P. (2018) · 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) · doi:10.1109/ccgrid.2018.00076 · Cited by 181 · cited 8×
“…microservice system, only a few apps will be affected by an anomaly in the kernel, which means that we should perform a diagnosis when the app has [20] Metrics PC algorithm Random walk MicroCause [26] Metrics Path condition time series algorithm Random walk MonitorRank [22] Topology, Metrics…”
BackgroundMethodInfluentialSelf-citationThe mystery machine: end-to-end performance analysis of large-scale internet services
Chow, M., Meisner, D., Flinn, J., Peek, D., & Wenisch, T. F. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685066 · Cited by 166
Localizing Failure Root Causes in a Microservice through Causality Inference
Meng, Y., Zhang, S., Sun, Y., Zhang, R., Hu, Z., Zhang, Y., Jia, C., Wang, Z., & Pei, D. (2020) · 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS) · doi:10.1109/iwqos49365.2020.9213058 · Cited by 161
(2017) · doi:10.1145/3132747.3132749 · Cited by 144
Chen, P., Qi, Y., Zheng, P., & Hou, D. (2014) · IEEE INFOCOM 2014 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2014.6848128 · Cited by 140
Circuit Breakers, Discovery, and API Gateways in MicroservicesPreprint
Montesi, F., & Weber, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05830 · Cited by 140
Sieve: actionable insights from monitored metrics in distributed systems
Thalheim, J., Rodrigues, A., Akkus, I. E., Bhatotia, P., Chen, R., Viswanath, B., Jiao, L., & Fetzer, C. (2017) · Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference · doi:10.1145/3135974.3135977 · Cited by 106
Robust and Rapid Adaption for Concept Drift in Software System Anomaly Detection
Ma, M., Zhang, S., Pei, D., Huang, X., & Dai, H. (2018) · 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2018.00013 · Cited by 92
Self-citationAn Approach for Anomaly Diagnosis Based on Hybrid Graph Model with Logs for Distributed Services
Jia, T., Chen, P., Yang, L., Li, Y., Meng, F., & Xu, J. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.12 · Cited by 78
An empirical investigation of practical log anomaly detection for online service systems
Zhao, N., Wang, H., Li, Z., Peng, X., Wang, G., Pan, Z., Wu, Y., Feng, Z., Wen, X., Zhang, W., et al. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3473933 · Cited by 77 · cited 1×
“Some methods such as LogAD [12] and LogAnomaly [13] mine rich information from unstructured logs and detect anomaly log patterns.”
MethodRoot cause detection in a service-oriented architecture
Kim, M., Sumbaly, R., & Shah, S. (2013) · Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems · doi:10.1145/2465529.2465753 · Cited by 73
Marwede, N., Rohr, M., van Hoorn, A., & Hasselbring, W. (2009) · 2009 13th European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2009.15 · Cited by 63
Architecture and dependability of large-scale internet services
Oppenheimer, D., & Patterson, D. A. (2002) · IEEE Internet Computing · doi:10.1109/mic.2002.1036037 · Cited by 53
Samir, A., & Pahl, C. (2019) · 2019 7th International Conference on Future Internet of Things and Cloud (FiCloud) · doi:10.1109/ficloud.2019.00036 · Cited by 49
Faster, deeper, easier: crowdsourcing diagnosis of microservice kernel failure from user space
Pan, Y., Ma, M., Jiang, X., & Wang, P. (2021) · Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3460319.3464805 · Cited by 42 · cited 1×
“This paper is extended from our previous work [37] with the following improvements.”
ResultSelf-citationCTF: Anomaly Detection in High-Dimensional Time Series with Coarse-to-Fine Model Transfer
Sun, M., Su, Y., Zhang, S., Cao, Y., Liu, Y., Pei, D., Wu, W., Zhang, Y., Liu, X., & Tang, J. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488755 · Cited by 40
Automating chaos experiments in production
Basiri, A., Hochstein, L., Jones, N., & Tucker, H. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00012 · Cited by 37
Performance troubleshooting in data centers: an annotated bibliography?
Wang, C., Kavulya, S. P., Tan, J., Hu, L., Kutare, M., Kasick, M., Schwan, K., Narasimhan, P., & Gandhi, R. (2013) · ACM SIGOPS Operating Systems Review · doi:10.1145/2553070.2553079 · Cited by 36
Lin, W., Ma, M., Pan, D., & Wang, P. (2018) · 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC) · doi:10.1109/pccc.2018.8711092 · Cited by 29 · cited 6×
“In order to run the baselines [18], [22], [23] that need such information, we make use of one state-of-the-art causal discovery algorithm (PC algorithm) to estimate a dependency graph, which is a commonly used way in many previous diagnosis methods [19], [20].”
BackgroundMethodInfluentialSelf-citationCloudRaid: hunting concurrency bugs in the cloud via log-mining
Lu, J., Li, F., Li, L., & Feng, X. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236071 · Cited by 28
Discovery of Causal Time Intervals
Li, Z., Zheng, G., Agarwal, A., Xue, L., & Lauvaux, T. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974973.90 · Cited by 21
InvarNet-X: A Black-Box Invariant-Based Approach to Diagnosing Big Data Systems
Chen, P., Qi, Y., & Hou, D. (2015) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2015.2497143 · Cited by 4
Cited by
Causal reasoning in Software Quality Assurance: A systematic review
Giamattei, L., Guerriero, A., Pietrantuono, R., & Russo, S. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107599 · Cited by 35
MicroIRC: Instance-level Root Cause Localization for Microservice Systems
Zhu, Y., Wang, J., Li, B., Zhao, Y., Zhang, Z., Xiong, Y., & Chen, S. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112145 · Cited by 29
Root Cause Localization for Microservice Systems in Cloud-edge Collaborative EnvironmentsPreprint
Zhu, Y., Wang, J., Li, B., Tang, X., Li, H., Zhang, N., & Zhao, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.13604 · Cited by 16
SoK: Microservice Architectures from a Dependability PerspectivePreprint
Kažemaks, D., & Decouchant, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.03392 · Cited by 5
Anomaly detection and root-cause identification in microservices: a survey
Barata, L. M., Sequeira, S., Lopes, E., Inácio, P. R. M., & Freire, M. M. (2026) · Cluster Computing · doi:10.1007/s10586-026-06095-9 · Cited by 4
“DyCause [65, 134] applies Granger causality and Extreme Value Theory to capture latency fluctuations and extreme event struc - tures across APIs.”
Metadata
- Authors
- Pan, Y., Ma, M., Jiang, X., & Wang, P.
- Affiliations
- Peking University (CN)
- Venue
- IEEE Transactions on Dependable and Secure Computing
- Published
- 2023-11
- Volume
- 20
- Issue
- 6
- Pages
- 4763-4777
- ISSN
- 1545-5971 (ISSN-L)
- 1941-0018
- 2160-9209
- OpenAlex
- W4313591680
- Cited by
- 11
- References
- 41
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 3
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Dependable and Secure Computing (publishedVersion)
- Funders
- National Natural Science Foundation of China (62072006, 92167104)
- Science and Technology on Communication Networks Laboratory (6142104200103)
- ByteDance University Research Project
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- microservices, fault diagnosis, crowdsourcing, anomaly propagation, interpretable diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.