Workflow-Aware Automatic Fault Diagnosis for Microservice-Based Applications With Statistics
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Microservice architectures bring many benefits, e.g., faster delivery, improved scalability, and greater autonomy, so they are widely adopted to develop and operate Internet-based applications. How to effectively diagnose the faults of applications with lots of dynamic microservices has become a key to guarantee applications' performance and reliability. As a microservice performs various behaviors in different workflows of processing requests, existing approaches often cannot accurately locate the root cause of an application with interactive microservices in a dynamic deployment environment. We propose a workflow-aware automatic fault diagnosis approach for microservice-based applications with statistics. We characterize traces across microservices with calling trees, and then learn trace patterns as baselines. For the faults affecting the workflows of processing requests, we estimate the workflows' anomaly degrees, and then locate the microservices causing anomalies by comparing the difference between current traces and learned baselines with tree edit distance. For performance anomalies causing significantly increased response time, we employ principal component analysis to extract suspicious microservices with large fluctuation in response time. Finally, we evaluate our approach on three typical microservice-based applications with a series of experiments. The results show that our approach can accurately locate the microservices causing anomalies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Open standards like OpenTracing underpin many of these systems [71,89,90], enabling language-agnostic instrumentation and widespread adoption in industry (e.g., Apple, Uber).”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [89]
Citations
Cites
Sparse Principal Component Analysis
Zou, H., Hastie, T., & Tibshirani, R. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x113430 · Cited by 3,152 · cited 2×
“For PCA, the computational complexity is O ( min ( n 3 , p 3 )) [19], where p is the number of features and n is the number of dimensions.”
BackgroundMethodJolliffe, I. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat06472 · Cited by 1,221
Newman, S. (2015) · CERN Document Server (European Organization for Nuclear Research) · Cited by 794
A workload characterization study of the 1998 World Cup Web site
Arlitt, M., & Jin, T. (2000) · IEEE Network · doi:10.1109/65.844498 · Cited by 766
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2887384 · Cited by 410 · cited 1×
“[31] conduct an empirical study to investigate the effectiveness of existing industrial debugging practices and whether they can be fur-”
BackgroundAutomatic web news extraction using tree edit distance
Reis, D. C., Golgher, P. B., Silva, A. S., & Laender, A. F. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988740 · Cited by 342 · cited 2×
“We adopt the existing tree edit distance algorithm RTDM [12] to calculate the distance between the calling tree T i and the calling tree T j , which is the number of operations (i.e., add, delete, and modify) transforming T i to T j .”
BackgroundMethodZhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Liu, D., Xiang, Q., & He, C. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338961 · Cited by 277 · cited 1×
“MEPFL [32] trains prediction models with the defined features of system trace logs, and then predicts latent error and fault localization for microservice applications with the system trace logs collected from the executions of the target application and its faulty versions.”
BackgroundService Function Chain Orchestration Across Multiple Domains: A Full Mesh Aggregation Approach
Sun, G., Li, Y., Liao, D., & Chang, V. (2018) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2018.2861717 · Cited by 254 · cited 1×
“Microservices cooperate with each other to complete various tasks with complex interactions, and thus processing a single request often spans multiple stateless microservices [36].”
BackgroundMicroscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments
Lin, J., Chen, P., & Zheng, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-03596-9_1 · Cited by 192 · cited 1×
“We choose this fault as analyzed in [10], and deploy a microservice with a service interface injectedMethod() for suspending the execution called by “AccountServiceImp.create(User user)” method.”
MethodPerformance Anomaly Detection and Bottleneck Identification
Ibidunmoye, O., Hernández-Rodriguez, F., & Elmroth, E. (2015) · ACM Computing Surveys · doi:10.1145/2791120 · Cited by 173 · cited 2×
“Performance anomalies commonly manifest as spikes in end-to-end latency (i.e., anomalous response time of requests), when the service interface of a microser-vice is invoked [18].”
BackgroundTPC-W: a benchmark for e-commerce
Menasce, D. A. (2002) · IEEE Internet Computing · doi:10.1109/mic.2002.1003136 · Cited by 161 · cited 1×
“To show real application scenarios, we simulate workloads complying with the TPC-W specification [11] of industrial standard e-commerce benchmarks.”
MethodAnomaly Detection Using Program Control Flow Graph Mining From Execution Logs
Nandi, A., Mandal, A., Atreja, S., Dasgupta, G. B., & Bhattacharya, S. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939712 · Cited by 154 · cited 3×
“Nandi et al. [8] template sequences from logs to form a control flow graph spanning distributed components, and then flag deviations from the expected behavior of run-time logs.”
BackgroundMethodOnline and Scalable Unsupervised Network Anomaly Detection Method
Dromard, J., Roudiere, G., & Owezarski, P. (2017) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2016.2627340 · Cited by 142 · cited 1×
“Dromard et al. [37] present an online and real-time unsupervised network anomaly detection algorithm, which relies on a discrete time-sliding window to update the feature space and an incremental grid clustering to detect the anomalies.”
Gremlin: Systematic Resilience Testing of Microservices
Heorhiadi, V., Rajagopalan, S., Jamjoom, H., Reiter, M. K., & Sekar, V. (2016) · 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2016.11 · Cited by 134 · cited 2×
“Many existing fault diagnosis techniques [24] resort to static system behavior models in identifying anomalies based on plenty of historical trace data.”
BackgroundPerformance Modeling for Cloud Microservice Applications
Jindal, A., Podolskiy, V., & Gerndt, M. (2019) · ACM/SPEC International Conference on Performance Engineering (ICPE) · doi:10.1145/3297663.3310309 · Cited by 134 · cited 1×
“Since the procedure of proceeding the same requests includes a similar sequence of microservices in a workflow, the response time is about the sum of every microservice’s processing time [21].”
AutoMAP: Diagnose Your Microservice-based Web Applications Automatically
Ma, M., Xu, J., Wang, Y., Chen, P., Zhang, Z., & Wang, P. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380111 · Cited by 129 · cited 1×
“MS-Rank [33] proposes a composite impact graph construction algorithm to discover causal relationships between microservices, designs a diagnostic algorithm to identify the root cause services, and then establishes a self-adaptive mechanism to update the confidence of different metrics according to…”
BackgroundSelf-citationCloudTPS: Scalable Transactions for Web Applications in the Cloud
Wei, Z., Pierre, G., & Chi, C. (2011) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2011.18 · Cited by 110 · cited 1×
“Many causes (e.g., software flaws, system configurations, runtime environment) can trigger applications’ faults, which manifest as anomalous behaviors (e.g., service failure, response delay) [16].”
BackgroundCloudSeer: Workflow Monitoring of Cloud Infrastructures via Interleaved Logs
Yu, X., Joshi, P., Xu, J., Jin, G., Zhang, H., & Jiang, G. (2016) · Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/2872362.2872407 · Cited by 93 · cited 3×
“We diagnose two types of silent faults [7] including the faults affecting the workflows of processing requests and the faults causing significant fluctuation in response time.”
BackgroundMethodSelf-citationAutomated anomaly detection and performance modeling of enterprise applications
Cherkasova, L., Ozonat, K., Mi, N., Symons, J., & Smirni, E. (2009) · ACM Transactions on Computer Systems · doi:10.1145/1629087.1629089 · Cited by 91 · cited 1×
“Cherkasova et al. [13] also discussed that the time of processing a request did not vary widely.”
BackgroundRoot Cause Analysis of Anomalies of Multitier Services in Public Clouds
Weng, J., Wang, J. H., Yang, J., & Yang, Y. (2018) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2018.2843805 · Cited by 79
Adaptive Anomaly Detection in Performance Metric Streams
Ibidunmoye, O., Rezaie, A.-R., & Elmroth, E. (2017) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2017.2750906 · Cited by 71 · cited 1×
“Ibidunmoye et al. [30] propose two unsupervised incremental techniques following a two-step strategy, which estimates an underlying temporal property of the stream via adaptive learning, and then applies statistically robust control charts to recognize deviations.”
MethodFD4C: Automatic Fault Diagnosis Framework for Web Applications in Cloud Computing
Wang, T., Zhang, W., Ye, C., Wei, J., Zhong, H., & Huang, T. (2016) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2015.2430834 · Cited by 68 · cited 4×
“Furthermore, a series of experimental results in our previous works [2], [23] also showed that the response time of a request was stable, when the physical resources did not reach to bottlenecks.”
BackgroundMethodResultInfluentialSelf-citationFailure Diagnosis for Distributed Systems using Targeted Fault Injection
Pham, C., Wang, L., Tak, B., Baset, S., Tang, C., Kalbarczyk, Z., & Iyer, R. (2016) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2016.2575829 · Cited by 65 · cited 1×
“We inject 42 faults [9] (i.e., physical resources, application server, database, application) listed in Table IV, and then employ popular methods [22] to inject these faults to validate our fault diagnosis.”
MethodFailure Analysis and Prediction for Big-Data Systems
Rosà, A., Chen, L. Y., & Binder, W. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2543718 · Cited by 53
Pivot Tracing: Dynamic Causal Monitoring for Distributed Systems
Mace, J., Roelke, R., & Fonseca, R. (2017) · ACM Transactions on Computer Systems · doi:10.1145/3208104 · Cited by 51
Dean, D. J., Nguyen, H., Wang, P., Gu, X., Sailer, A., & Kochut, A. (2015) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2015.2444392 · Cited by 41
Workload-aware anomaly detection for Web applications
Wang, T., Wei, J., Zhang, W., Zhong, H., & Huang, T. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.03.060 · Cited by 40 · cited 3×
“Furthermore, a series of experimental results in our previous works [2], [23] also showed that the response time of a request was stable, when the physical resources did not reach to bottlenecks.”
BackgroundMethodResultSelf-citationJiang, M., Munawar, M. A., Reidemeister, T., & Ward, P. A. S. (2011) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2011.16 · Cited by 37 · cited 1×
“We choose this fault as analyzed in [20], and use a TC (Traffic Control) tool in Linux to cause 25% packet loss of networks.”
MethodWorkload-Adaptive Configuration Tuning for Hierarchical Cloud Schedulers
Han, R., Liu, C. H., Zong, Z., Chen, L. Y., Liu, W., Wang, S., & Zhan, J. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2923197 · Cited by 31 · cited 1×
“Han et al. [38] analyze patterns of unsuccessful jobs and tasks, focusing on their inter-dependencies, uncover their root causes by inspecting key workload and system attributes, and develop three on-line prediction models to classify jobs into four classes upon arrival time.”
BackgroundmilliScope: A Fine-Grained Monitoring Framework for Performance Debugging of n-Tier Web Services
Lai, C.-A., Kimball, J., Zhu, Τ., Wang, Q., & Pu, C. (2017) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.228 · Cited by 30 · cited 1×
“Lai et al. [26] collaborate multiple monitors for event and resource monitoring to reconstruct the flow of each client request, and then profile execution performance to identify the execution boundary in a distributed system.”
BackgroundFCatch: Automatically Detecting Time-of-fault Bugs in Cloud Systems
Liu, H., Wang, X., Li, G., Lu, S., Ye, F., & Tian, C. (2018) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3173162.3177161 · Cited by 27
Statistical Analysis of Latency Through Semantic Profiling
Huang, J. M., Mozafari, B., & Wenisch, T. F. (2017) · European Conference on Computer Systems · doi:10.1145/3064176.3064179 · Cited by 22 · cited 1×
“Huang et al. [27] propose a profiler to identify the dominant sources of response time variance in a semantic context, which employs a variance tree to analyze the thread interleaving and deconstructs overall response time variance into the variance and covariance of functions’ execution time.”
BackgroundAnomaly Detection in Complex Real World Application Systems
Gow, R., Rabhi, F., & Venugopal, S. (2017) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2017.2771403 · Cited by 16 · cited 1×
“Gow et al. [29] detail a system performance measurement method to detect slowdown event and characterize application behaviors, which enables business to manage the stability and performance of the end-user experience to support productivity and effectiveness of business processes.”
MethodJanevski, N., & Goševa-Popstojanova, K. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.3 · Cited by 13 · cited 1×
“Since a microservice performs various behaviors in different work-flows [17], the proposed approach detects anomalies in the condition of a learned specific workflow pattern to improve the diagnosis precision.”
BackgroundDetecting performance anomaly with correlation analysis for Internetware
Wang, T., Wei, J., Qin, F., Zhang, W., Zhong, H., & Huang, T. (2013) · Science China Information Sciences · doi:10.1007/s11432-013-4906-6 · Cited by 8 · cited 2×
“In the future work, we plan to introduce our previous works [2], [6], [23] based on physical metrics in our approach by combine tracing execution with physical resources to improve its effect.”
BackgroundMethodSelf-citationA Visit to the Jungle of Terminology
Avižienis, A. (2017) · International Conference on Dependable Systems and Networks workshops · doi:10.1109/dsn-w.2017.32 · Cited by 7 · cited 1×
“In most cases, a fault that is the root cause of an error remains dormant for some time and, once it becomes active, it causes an incorrect system state that is an error called as the manifestation of the fault [34].”
BackgroundA Data Set for User Request Trace-Oriented Monitoring and its Applications
Zhou, J., Chen, Z., Wang, J., Zheng, Z., & Lyu, M. R.-T. (2015) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2015.2491286 · Cited by 6 · cited 1×
“We inject 42 faults [9] (i.e., physical resources, application server, database, application) listed in Table IV, and then employ popular methods [22] to inject these faults to validate our fault diagnosis.”
MethodCloudSeer: Workflow Monitoring of Cloud Infrastructures via Interleaved Logs
Yu, X., Joshi, P., Xu, J., Jin, G., Zhang, H., & Jiang, G. (2016) · ACM SIGOPS Operating Systems Review · doi:10.1145/2954680.2872407 · Cited by 5
Self-citationWang, J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-813304-0.00004-9 · Cited by 4
An Endpoint Communication Profiling Tool for Distributed Computing Frameworks
Rosà, A., Chen, L. Y., & Binder, W. (2016) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2016.67 · Cited by 3 · cited 1×
“Current approaches [4] generally focus on precisely and efficiently collecting traces, and require operators to manually set anomaly alarms according to domain knowledge.”
Background
Cited by
Soldani, J., & Brogi, A. (2023) · ACM Computing Surveys · doi:10.1145/3501297 · Cited by 256
“Meng et al. [54] and Wang et al. [88] enable detecting functional anomalies by collecting traces while testing an application in a pre-production environment and by building a set of representative call trees from collected traces.”
BackgroundMethodInfluentialIdentifying bad software changes via multimodal anomaly detection for online service systems
Zhao, N., Chen, J., Yu, Z., Wang, H., Li, J., Qiu, B., Xu, H., Zhang, W., Sui, K., & Pei, D. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3468543 · Cited by 89
“The first system is Train-Ticket [65], an open-source microservice system, which has been widely applied in the literature [31, 42, 68, 82].”
InfluentialSelf-citationAnomaly detection in microservice environments using distributed tracing data analysis and NLP
Kohyarnejadfard, I., Aloise, D., Azhari, S. V., & Dagenais, M. (2022) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-022-00296-4 · Cited by 63
TraceGra: A trace-based anomaly detection for microservice using graph deep learning
Chen, J., Liu, F., Jiang, J., Zhong, G., Xu, D., Tan, Z., & Shi, S. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.03.028 · Cited by 55
Panahandeh, M., Hamou-Lhadj, A., Hamdaqa, M., & Miller, J. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111917 · Cited by 49
“Studies have shown that the analysis of distributed traces is useful for program comprehension [10], [11], fault diagnosis [12], [13], root cause analysis [14], [15], and anomaly detection [16], [17], the focus of this paper.”
BackgroundMethod
Metadata
- Authors
- Wang, T., Zhang, W., Xu, J., & Gu, Z.
- Affiliations
- Chinese Academy of Sciences (CN)
- University College Dublin (IE)
- Xiaomi (China) (CN)
- Venue
- IEEE Transactions on Network and Service Management
- Published
- 2020-12
- Volume
- 17
- Issue
- 4
- Pages
- 2350-2363
- ISSN
- 1932-4537 (ISSN-L)
- 2373-7379
- OpenAlex
- W3084013135
- Cited by
- 54
- References
- 40
- Citations by year
- 2026: 2 · 2025: 11 · 2024: 9 · 2023: 12 · 2022: 15 · 2021: 4 · 2020: 1
- FWCI
- 3.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Network and Service Management (publishedVersion)
- Funders
- National Key R&D Program of China (2017YFB1400804)
- National Natural Science Foundation of China (61872344)
- Beijing Natural Science Foundation (4182070)
- Youth Innovation Promotion Association of Chinese Academy of Sciences Fund (2018144)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Keywords
- automated fault diagnosis, root cause localization, tree edit distance, principal component analysis, performance anomalies, anomaly detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.