Faster, deeper, easier: crowdsourcing diagnosis of microservice kernel failure from user space
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
With the widespread use of cloud-native architecture, increasing web applications (apps) choose to build on microservices. Simultaneously, troubleshooting becomes full of challenges owing to the high dynamics and complexity of anomaly propagation. Existing diagnostic methods rely heavily on monitoring metrics collected from the kernel side of microservice systems. Without a comprehensive monitoring infrastructure, application owners and even cloud operators cannot resort to these kernel-space solutions. This paper summarizes several insights on operating a top commercial cloud platform. Then, for the first time, we put forward the idea of user-space diagnosis for microservice kernel failures. To this end, we develop a crowdsourcing solution - DyCause, to resolve the asymmetric diagnostic information problem. DyCause deploys on the application side in a distributed manner. Through lightweight API log sharing, apps collect the operational status of kernel services collaboratively and initiate diagnosis on demand. Deploying DyCause is fast and lightweight as we do not have any architectural and functional requirements for the kernel. To reveal more accurate correlations from asymmetric diagnostic information, we design a novel statistical algorithm that can efficiently discover the time-varying causalities between services. This algorithm also helps us build the temporal order of the anomaly propagation. Therefore, by using DyCause, we can obtain more in-depth and interpretable diagnostic clues with limited indicators. We apply and evaluate DyCause on both a simulated test-bed and a real-world cloud system. Experimental results verify that DyCause running in the user-space outperforms several state-of-the-art algorithms running in the kernel on accuracy. Besides, DyCause shows superior advantages in terms of algorithmic efficiency and data sensitivity. Simply put, DyCause produces a significantly better result than other baselines when analyzing much fewer or sparser metrics. To conclude, DyCause is faster to act, deeper in analysis, and easier to deploy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“DyCause [65,134] applies Granger causality and Extreme Value Theory to capture latency fluctuations and extreme event structures across APIs.”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [65]
In the citing paper
“Local dependency graphs have also been used for backward BFS traversal and anomaly chain reconstruction [65].”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [65]
Citations
Cites
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332 · cited 1×
“Most of them such as [11, 26, 28, 34] describe the system performance through various models and detect anomalies by detecting violations of these standards.”
BackgroundPinpoint: problem determination in large, dynamic Internet services
Chen, M. Y., Kiciman, E., Fratkin, E., Fox, A., & Brewer, E. (2003) · Proceedings International Conference on Dependable Systems and Networks · doi:10.1109/dsn.2002.1029005 · Cited by 835 · cited 1×
“Besides, another category of diagnosis approaches [5, 6, 8, 19, 24] digs through system logs and tracings [1].”
BackgroundMicroservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture
Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2016) · IEEE Software · doi:10.1109/ms.2016.64 · Cited by 735 · cited 1×
“The microservice architecture is becoming prevalent in web application developments for its high reusability, fast evolution, and easy scaling features [3].”
BackgroundMobile Crowd Sensing and Computing: The Review of an Emerging Human-Powered Sensing Paradigm
Guo, B., Wang, Z., Yu, Z., Wang, Y., Yen, N. Y., Huang, R., & Zhou, X. (2015) · ACM Computing Surveys · doi:10.1145/2794400 · Cited by 719
Anomaly Detection in Streams with Extreme Value Theory
Siffer, A., Fouque, P.-A., Termier, A., & Largouet, C. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098144 · Cited by 553 · cited 5×
“Our study in this paper adopts SPOT for the preliminary anomaly detector as the first step.”
MethodInfluentialZhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2887384 · Cited by 410 · cited 1×
“These failures have caused extensive impacts on online businesses and severe economic losses [35].”
BackgroundVillamizar, M., Garces, O., Castro, H., Verano, M., Salamanca, L., Casallas, R., & Gil, S. (2015) · 2015 10th Computing Colombian Conference (10CCC) · doi:10.1109/columbiancc.2015.7333476 · Cited by 308 · cited 1×
“This loosely-coupled paradigm enables independent refactoring and dynamic scaling for both microservices and applications [30].”
MicroRCA: Root Cause Localization of Performance Issues in Microservices
Wu, L., Tordsson, J., Elmroth, E., & Kao, O. (2020) · NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium · doi:10.1109/noms47738.2020.9110353 · Cited by 265 · cited 3×
“Previous diagnostic methods [7, 14, 15, 17, 18, 22, 29, 32, 33] either assume a static correlation within a given interval or use static causal discovery algorithms to build the dependencies [13].”
BackgroundMethod(2015) · doi:10.1145/2815400.2815415 · Cited by 202
Detailed diagnosis in enterprise networks
Kandula, S., Mahajan, R., Verkaik, P., Agarwal, S., Padhye, J., & Bahl, P. (2009) · Proceedings of the ACM SIGCOMM 2009 conference on Data communication · doi:10.1145/1592568.1592597 · Cited by 194 · cited 9×
“For example, NetMedic [14] leverages component templates to construct the dependency graph of traditional software and estimates dependency strength using history states.”
BackgroundMethodInfluentialMicroscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments
Lin, J., Chen, P., & Zheng, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-03596-9_1 · Cited by 192 · cited 4×
“Previous diagnostic methods [7, 14, 15, 17, 18, 22, 29, 32, 33] either assume a static correlation within a given interval or use static causal discovery algorithms to build the dependencies [13].”
BackgroundMethodInfluentialCloudRanger: Root Cause Identification for Cloud Native Systems
Wang, P., Xu, J., Ma, M., Lin, W., Pan, D., Wang, Y., & Chen, P. (2018) · 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) · doi:10.1109/ccgrid.2018.00076 · Cited by 181 · cited 14×
“As for recent methods that use causal discovery algorithms, we select CloudRanger [32] and MicroCause [23] as our baselines.”
BackgroundMethodSelf-citationLocalizing Failure Root Causes in a Microservice through Causality Inference
Meng, Y., Zhang, S., Sun, Y., Zhang, R., Hu, Z., Zhang, Y., Jia, C., Wang, Z., & Pei, D. (2020) · 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS) · doi:10.1109/iwqos49365.2020.9213058 · Cited by 161 · cited 12×
“As the PCTS algorithm can not handle this sub-dataset, MicroCause produces no valid results.”
BackgroundMethodInfluential(2017) · doi:10.1145/3132747.3132749 · Cited by 144 · cited 1×
“For instance, frameworks such as [12, 21] extend the traditional tracing to a dynamic way that help diagnose problems, such as software bugs, misconfigurations, and limping hardware.”
BackgroundChen, P., Qi, Y., Zheng, P., & Hou, D. (2014) · IEEE INFOCOM 2014 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2014.6848128 · Cited by 140 · cited 3×
“Previous diagnostic methods [7, 14, 15, 17, 18, 22, 29, 32, 33] either assume a static correlation within a given interval or use static causal discovery algorithms to build the dependencies [13].”
BackgroundMethodAutoMAP: Diagnose Your Microservice-based Web Applications Automatically
Ma, M., Xu, J., Wang, Y., Chen, P., Zhang, Z., & Wang, P. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380111 · Cited by 129 · cited 1×
“As anomaly can propagate indirectly in microservice architecture [20], the app owner could not tell whether these services are the root cause or affected by others from a single user’s perspective.”
BackgroundSelf-citationSieve: actionable insights from monitored metrics in distributed systems
Thalheim, J., Rodrigues, A., Akkus, I. E., Bhatotia, P., Chen, R., Viswanath, B., Jiao, L., & Fetzer, C. (2017) · Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference · doi:10.1145/3135974.3135977 · Cited by 106 · cited 4×
“Previous diagnostic methods [7, 14, 15, 17, 18, 22, 29, 32, 33] either assume a static correlation within a given interval or use static causal discovery algorithms to build the dependencies [13].”
BackgroundMethodInfluentialAn Approach for Anomaly Diagnosis Based on Hybrid Graph Model with Logs for Distributed Services
Jia, T., Chen, P., Yang, L., Li, Y., Meng, F., & Xu, J. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.12 · Cited by 78 · cited 1×
“Most of them such as [11, 26, 28, 34] describe the system performance through various models and detect anomalies by detecting violations of these standards.”
BackgroundRoot cause detection in a service-oriented architecture
Kim, M., Sumbaly, R., & Shah, S. (2013) · Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems · doi:10.1145/2465529.2465753 · Cited by 73 · cited 11×
“Many other approaches such as [7, 15, 17, 22, 33] analyze service calling records to construct a calling dependency graph.”
BackgroundMethodInfluentialMarwede, N., Rohr, M., van Hoorn, A., & Hasselbring, W. (2009) · 2009 13th European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2009.15 · Cited by 63 · cited 8×
“Many other approaches such as [7, 15, 17, 22, 33] analyze service calling records to construct a calling dependency graph.”
BackgroundMethodInfluentialArchitecture and dependability of large-scale internet services
Oppenheimer, D., & Patterson, D. A. (2002) · IEEE Internet Computing · doi:10.1109/mic.2002.1036037 · Cited by 53
Samir, A., & Pahl, C. (2019) · 2019 7th International Conference on Future Internet of Things and Cloud (FiCloud) · doi:10.1109/ficloud.2019.00036 · Cited by 49 · cited 1×
“Most of them such as [11, 26, 28, 34] describe the system performance through various models and detect anomalies by detecting violations of these standards.”
BackgroundAutomating chaos experiments in production
Basiri, A., Hochstein, L., Jones, N., & Tucker, H. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00012 · Cited by 37 · cited 1×
“In such an architecture, users can develop applications by modularly combining microservices through lightweight web protocols [4].”
Performance troubleshooting in data centers: an annotated bibliography?
Wang, C., Kavulya, S. P., Tan, J., Hu, L., Kutare, M., Kasick, M., Schwan, K., Narasimhan, P., & Gandhi, R. (2013) · ACM SIGOPS Operating Systems Review · doi:10.1145/2553070.2553079 · Cited by 36 · cited 1×
“Many works focus on the primary problem of troubleshooting microservice system failures: anomaly detection [31].”
BackgroundLin, W., Ma, M., Pan, D., & Wang, P. (2018) · 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC) · doi:10.1109/pccc.2018.8711092 · Cited by 29 · cited 6×
“RankScore extends the simple form proposed in [18, 22] to support multiple root causes.”
BackgroundMethodSelf-citationCloudRaid: hunting concurrency bugs in the cloud via log-mining
Lu, J., Li, F., Li, L., & Feng, X. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236071 · Cited by 28 · cited 1×
“Besides, another category of diagnosis approaches [5, 6, 8, 19, 24] digs through system logs and tracings [1].”
BackgroundDiscovery of Causal Time Intervals
Li, Z., Zheng, G., Agarwal, A., Xue, L., & Lauvaux, T. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974973.90 · Cited by 21 · cited 6×
“However, in the real world, causality does not always exist or may only be significant for a short period of time [16].”
BackgroundMethodInfluentialInvarNet-X: A Black-Box Invariant-Based Approach to Diagnosing Big Data Systems
Chen, P., Qi, Y., & Hou, D. (2015) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2015.2497143 · Cited by 4 · cited 1×
“Besides, another category of diagnosis approaches [5, 6, 8, 19, 24] digs through system logs and tracings [1].”
Background
Research works printed but not linked: 5
- (Accessed : December 2 , 2020 ). The OpenTracing Project . https://opentracing.io/ (Accessed: December 2, 2020). The OpenTracing Project. https://opentracing.io/
- Michael Chow , David Meisner , Jason Flinn , Daniel Peek , and Thomas F. Wenisch . 2014. The mystery machine: End-to-end performance analysis of large-scale internet services . In 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI ’14 , Broomfield, CO, USA , October 6-8, 2014 . USENIX Association, 217–231. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/chow Michael Chow, David Meisner, Jason Flinn, Daniel Peek, and Thomas F. Wenisch. 2014. The mystery machine: End-to-end performance analysis of large-scale internet services. In 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI ’14, Broomfield, CO, USA, October 6-8, 2014. USENIX Association, 217–231. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/chow
- Markus Kalisch and Peter Bühlmann . 2007 . Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J . Mach. Learn. Res. , 8, Mar (2007), 613–636. http://dl.acm.org/citation.cfm?id=1314520 Markus Kalisch and Peter Bühlmann. 2007. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res., 8, Mar (2007), 613–636. http://dl.acm.org/citation.cfm?id=1314520
- Karthik Nagaraj , Charles Edwin Killian , and Jennifer Neville . 2012 . Structured comparative analysis of systems logs to diagnose performance problems . In Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2012 , San Jose, CA, USA , April 25-27, 2012. USENIX Association, 353–366. https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/nagaraj Karthik Nagaraj, Charles Edwin Killian, and Jennifer Neville. 2012. Structured comparative analysis of systems logs to diagnose performance problems. In Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2012, San Jose, CA, USA, April 25-27, 2012. USENIX Association, 353–366. https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/nagaraj
- Christopher Stewart and Kai Shen . 2005 . Performance modeling and system management for multi-component online services . In 2nd Symposium on Networked Systems Design and Implementation (NSDI 2005), May 2-4, 2005, Boston, Massachusetts, USA, Proceedings. USENIX, 71–84 . http://www.usenix.org/events/nsdi05/tech/stewart.html Christopher Stewart and Kai Shen. 2005. Performance modeling and system management for multi-component online services. In 2nd Symposium on Networked Systems Design and Implementation (NSDI 2005), May 2-4, 2005, Boston, Massachusetts, USA, Proceedings. USENIX, 71–84. http://www.usenix.org/events/nsdi05/tech/stewart.html
Cited by
DeepTraLog: trace-log combined microservice anomaly detection through graph-based deep learning
Zhang, C., Peng, X., Sha, C., Zhang, K., Fu, Z., Wu, X., Lin, Q., & Zhang, D. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510180 · Cited by 207
“In microservice systems, traces are widely used in anomaly detection and root cause analysis [10, 19, 20, 25, 26, 28].”
MethodYu, G., Chen, P., Li, Y., Chen, H., Li, X., & Zheng, Z. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616249 · Cited by 140
“Over the years, many metrics-based approaches [6, 26, 35, 62, 64, 65], trace-based approaches [8, 9, 13, 22, 32, 67, 68, 71], and log-based approaches [3, 20, 36, 45, 52, 56] have been devoted to au-”
BackgroundRobust Failure Diagnosis of Microservice System Through Multimodal Data
Zhang, S., Jin, P., Lin, Z., Sun, Y., Zhang, B., Xia, S., Li, Z., Zhong, Z., Ma, M., Jin, W., et al. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3290018 · Cited by 118
Self-citationPham, L., Ha, H., & Zhang, H. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660805 · Cited by 48
Failure Diagnosis in Microservice Systems: A Comprehensive Survey and Analysis
Zhang, S., Xia, S., Fan, W., Shi, B., Xiong, X., Zhong, Z., Ma, M., Sun, Y., & Pei, D. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715005 · Cited by 57
Self-citation
Metadata
- Authors
- Pan, Y., Ma, M., Jiang, X., & Wang, P.
- Affiliations
- Peking University (CN)
- Venue
- Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 646-657
- OpenAlex
- W3179172661
- Cited by
- 42
- References
- 29
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 14 · 2023: 7 · 2022: 5
- FWCI
- 6.01
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Keywords
- microservices, crowdsourcing, anomaly propagation, cloud-native applications, distributed diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.