From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index; 19 of the 28 sources its claims cite were checked for support.
Abstract
In modern infrastructure dominated by cloudnative architectures and microservices, massive heterogeneous logs have become core assets for Artificial Intelligence for IT Operations (AIOps) and security analysis. However, traditional data mining methods are limited by their “correlation, not causation” nature. When dealing with Advanced Persistent Threats (APTs) and non-linear cascading failures, these methods face challenges such as high false alarm rates, lack of interpretability, and difficulty in supporting automated intervention. This paper systematically reconstructs the technical framework of log-based causal analysis for security and operations, advocating for a paradigm shift from passive “correlation discovery” to active “causal intervention”. We analyze four cutting-edge technical routes: explicit dependency mining based on provenance graphs, implicit structure discovery based on statistics, causal representation based on deep learning, and Large Language Model (LLM)-driven semantic fusion and agent architectures. This paper reveals the cognitive disputes, theoretical challenges, and technical difficulties existing models face when moving toward rigorous causal inference. Finally, we propose the future visions of “Causal-Native Logging” and “Neuro-symbolic Agents”.
Overview
CoChat Summary
TL;DR
This survey reconstructs the technical framework of log-based causal analysis for security and AIOps, arguing for a paradigm shift from passive "correlation discovery" to active "causal intervention." It reviews four technical routes—provenance-graph dependency mining, statistical causal discovery, deep representation learning, and LLM-driven semantic fusion—and proposes "Causal-Native Logging" and "Neuro-symbolic Agents" as future directions.
Key findings
- The paper organizes the field into four routes: (1) explicit dependency mining via provenance graphs (BackTracker, SLEUTH, CAPTAIN, Kairos, NODLINK, HOLMES, OmegaLog, SPARSE); (2) statistical causal discovery (PC, FCI, RFCI, PC-Stable, PCMCI, GES/XGES, MMHC, H2PC, LiNGAM/DirectLiNGAM/LvLiNGAM/PNL, NOTEARS, SDCD); (3) deep representation learning (DeepLog, LogBERT, ATLAS, AIRTAG, TRACE, Log2Graph, Madeline, SHIL, MAGIC, Trace2vec, GRAIN); and (4) LLM semantic fusion (LogPPT, LogLLM, OMNISEC, LLM-LADE, TAMO, PROVSEEK, LogReasoner).
- Each route carries a characteristic failure mode: provenance graphs suffer "dependency explosion" and a semantic gap; statistical methods face an "identifiability chasm" and sensitivity to distributional assumptions; deep learning struggles to move from correlation induction to counterfactual reasoning; LLM agents are vulnerable to "causal hallucinations" and lack auditable reasoning chains (Table I).
- The paper identifies three shared problem classes: cognitive disputes (conflating correlation proxies with counterfactual causation; the assumption-strength vs. conclusion-certainty trade-off; misaligning task performance with intrinsic causal capability), theoretical challenges (identifiability from purely observational data, lack of auditability in black-box reasoning, absence of evaluation benchmarks for causal understanding), and technical difficulties (completeness vs. interpretability, an "efficiency-scale-quality" impossible triangle, and robustness degradation under concept drift).
- Proposed future directions: "Causal-Native Logging" (injecting logical clocks such as Lamport timestamps, global tracking IDs, and causal parent pointers at event generation), a unified entity-event causal graph with multi-view mining across security and AIOps, and trustworthy neuro-symbolic agents combining neural perception with symbolic constraint engines.
Methods
- Design: a narrative survey/review paper, not an empirical study. It synthesizes prior literature across four technical routes and organizes challenges into a taxonomy (Fig. 2).
- Data or sample: no dataset or experimental sample; the "data" are cited works (43 references) spanning USENIX Security, IEEE S&P, NDSS, ACSAC, and journal venues.
- Main analysis: qualitative comparison of the four routes along dimensions of core mechanism, primary input, main advantages, core challenges, and representative methods (Table I), followed by a structured discussion of open problems and future architecture proposals (Figs. 1 and 3).
Limitations
- The paper states no explicit limitations section; it is a survey and does not report its own empirical constraints.
- Evident from the text: the comparison in Table I is qualitative and no quantitative benchmark or evaluation is performed, so the relative performance of the surveyed methods is not assessed.
- Evident from the text: the proposed future directions (Causal-Native Logging, unified causal graphs, neuro-symbolic agents) are visions without implementation or validation, and the paper itself notes that evaluation benchmarks for causal understanding capabilities are absent.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
14 Valid · 10 Not valid · 7 Not checked · 26 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Although subsequent improvements exist, such as RFCI[14] (sacrificing information for speed), PC-Stable[15] (eliminating order dependency), and PCMCI[16] for time-series data, they have consistently failed to overcome the fundamental flaw that CI tests are highly prone to statistical errors under high-dimensional, sparse logs.”
What the sources say
- Partly supportedcites [14] matched to “Learning high-dimensional directed acyclic graphs with latent and selection variables” (2012) sourceExperimental — accuracy not yet measured
The Introduction (§1.3; page not supplied) says RFCI is faster than FCI, may be less informative, and avoids some low-power conditional-independence tests in small samples. But §4 and the Discussion (§6) prove consistency for RFCI in sparse high-dimensional settings; the paper does not assess PC-Stable, PCMCI, or sparse logs, and does not establish that these methods consistently fail because of testing errors. (LLM, conf=0.88)
- Not matchedcites [15] Colombo, 2014, Order-independent constraint-based causal structure learning[J], J. Mach. Learn. Res.
We searched by its title and found no matching paper.
- Disputedcites [16] matched to “Detecting and quantifying causal associations in large nonlinear time series datasets” (2019) sourceExperimental — accuracy not yet measured
The paper does identify PCMCI as a method for time-series data, but it contradicts the claim that PCMCI consistently fails to address CI-test errors in high-dimensional, sparse data. The Theoretical properties of PCMCI section says the method exploits network sparsity and controls false positives; the Discussion and conclusion reports higher detection power for datasets with dozens to hundreds of variables and correctly controlled false-positive rates. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“MAGIC[34] proposed a masked graph representation learning framework, self-supervisedly learning the deep topological patterns of normal system behaviors through random masking and reconstruction on benign provenance graphs.”
What the sources say
- Supportedcites [34] Jia, 2024, {MAGIC}: Detecting advanced persistent threats via masked graph representation learning[C] source
The Introduction and §3 describe MAGIC as a self-supervised masked graph representation approach trained on benign audit logs to model benign system behaviors. Sections 4.2.1 and 4.2.3 describe randomly masking nodes and learning through feature and graph-structure reconstruction, supporting the claim about learning structural patterns from benign provenance graphs. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“NOTEARS[25] innovatively utilizes algebraic features to reconstruct discrete acyclic graph searches into a continuous optimization problem, allowing direct solving using gradient descent.”
What the sources say
- Partly supportedcites [25] Zheng, 2018, Dags with no tears: Continuous optimization for structure learning[J], Advances in neural information processing systems sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says an algebraic characterization of DAGs led to a continuous optimization program, and §4.3 (p. 8) describes solving the differentiable program with standard optimizers. The paper identifies L-BFGS-B, a gradient-based quasi-Newton method, as a natural choice; it does not say the program is solved by gradient descent specifically. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“OMNISEC[39] adopts a RAG architecture, fusing external threat intelligence and benign behavior knowledge bases into provenance graphs to assist LLMs in judging suspicious behaviors.”
What the sources say
- Partly supportedcites [39] Cheng, 2025, Omnisec: LLM-driven provenance-based intrusion detection via retrieval-augmented behavior prompting[J], arXiv preprint arXiv sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says OMNISEC builds benign-behavior and threat-intelligence knowledge bases and uses RAG to help an LLM decide whether abnormal behavior is an attack. Section 4.3 (p. 5) describes these as external knowledge bases whose retrieved content informs the model; it does not say their contents are fused into the provenance graphs. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To overcome the limitation of provenance graph-based methods that only capture explicit relationships, statistical causal discovery techniques have been introduced into the field of log analysis.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
8 more claims
Create a free account to see all 57 claims and every source verdict.
- It may state two separate claims as one. 28
- Probably not a substantial claim. 32
Citations
Cites
Causation, Prediction, and Search
Spirtes, P., Glymour, C., & Scheines, R. (2001) · doi:10.7551/mitpress/1754.001.0001 · Cited by 4,521
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
The max-min hill-climbing Bayesian network structure learning algorithm
Tsamardinos, I., Brown, L. E., & Aliferis, C. F. (2006) · Machine Learning · doi:10.1007/s10994-006-6889-7 · Cited by 1,822
DAGs with NO TEARS: Continuous Optimization for Structure Learning
Zheng, X., Aragam, B., Ravikumar, P., & Xing, E. P. (2018) · arXiv · doi:10.48550/arxiv.1803.01422 · Cited by 1,700
Detecting and quantifying causal associations in large nonlinear time series datasets
Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., & Sejdinovic, D. (2019) · Science Advances · doi:10.1126/sciadv.aau4996 · Cited by 979 · cited 2×
“Although subsequent improvements exist, such as RFCI[14] (sacrificing information for speed), PC-Stable[15] (eliminating order dependency), and PCMCI[16] for time-series data, they have consistently failed to overcome the fundamental flaw that CI tests are highly prone to statistical errors under…”
HOLMES: Real-Time APT Detection through Correlation of Suspicious Information Flows
Milajerdi, S. M., Gjomemo, R., Eshete, B., Sekar, R., & Venkatakrishnan, V. N. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00026 · Cited by 573
LogBERT: Log Anomaly Detection via BERT
Guo, H., Yuan, S., & Wu, X. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534113 · Cited by 454 · cited 1×
“LogBERT[28] innovatively introduced Masked Language Modeling (MLM), treating logs as word sequences and utilizing BERT's powerful attention mechanism to learn long-distance contextual dependencies.”
Learning high-dimensional directed acyclic graphs with latent and selection variables
Colombo, D., Maathuis, M. H., Kalisch, M., & Richardson, T. S. (2012) · The Annals of Statistics · doi:10.1214/11-aos940 · Cited by 384
King, S. T., & Chen, P. M. (2003) · ACM SIGOPS Operating Systems Review · doi:10.1145/1165389.945467 · Cited by 285
King, S. T., & Chen, P. M. (2003) · Proceedings of the nineteenth ACM symposium on Operating systems principles · doi:10.1145/945445.945467 · Cited by 232
Research works printed but not linked: 7
- 2025, M-trends 2025[R/OL]
- Gunawat, 2025, Applying Causal Inference AI Models to Root Cause Analysis (RCA) in DevOps[J], International Journal for Research Trends and Innovation, 10, 2456
- Alsaheel, 2021, {ATLAS}: A sequence-based learning approach for attack investigation[C], 3005
- Colombo, 2014, Order-independent constraint-based causal structure learning[J], J. Mach. Learn. Res., 15, 3741
- Chickering, 2002, Optimal structure identification with greedy search[J], Journal of machine learning research, 3, 507
- Shimizu, 2006, A linear non-Gaussian acyclic model for causal discovery[J], Journal of Machine Learning Research, 7
- Ding, 2023, {AIRTAG}: Towards automated attack investigation by unsupervised learning with log texts[C], 373
Metadata
- Authors
- Zhu, Y., Li, X., Suo, J., Yin, Y., & Huang, C.
- Affiliations
- Guangxi Normal University (CN)
- Venue
- 2026 International Conference on AI in Education Technology and Applications (AIETA)
- Publisher
- IEEE
- Published
- 2026-03-27
- Pages
- 9-16
- OpenAlex
- W7160636270
- Semantic Scholar
- 3078564efee433bf1d963164389f7161e6e3d250
- Cited by
- 0
- References
- 23
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on AI in Education Technology and Applications (AIETA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Information Systems › Digital and Cyber Forensics
- Keywords
- AIOps, cybersecurity, advanced persistent threats, causal inference, correlation versus causation, provenance graphs, cascading failures, large language models, semantic fusion, automated intervention, cloud-native infrastructure, microservices, false alarm reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps. https://app.cochat.ai/papers/10.1109/aieta69357.2026.11508946
- The paper
- Zhu, Y., Li, X., Suo, J., Yin, Y., & Huang, C. (2026). From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps. 2026 International Conference on AI in Education Technology and Applications (AIETA). https://doi.org/10.1109/aieta69357.2026.11508946
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.