The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Sensitivity of PCA for traffic anomaly detection
Abstract
Detecting anomalous traffic is a crucial part of managing IP networks. In recent years, network-wide anomaly de-tection based on Principal Component Analysis (PCA) has emerged as a powerful method for detecting a wide vari-ety of anomalies. We show that tuning PCA to operate effectively in practice is difficult and requires more robust techniques than have been presented thus far. We analyze a week of network-wide traffic measurements from two IP backbones (Abilene and Geant) across three different traffic aggregations (ingress routers, OD flows, and input links), and conduct a detailed inspection of the feature time se-ries for each suspected anomaly. Our study identifies and evaluates four main challenges of using PCA to detect traf-fic anomalies: (i) the false positive rate is very sensitive to small differences in the number of principal components in the normal subspace, (ii) the effectiveness of PCA is sensi-tive to the level of aggregation of the traffic measurements, (iii) a large anomaly may inadvertently pollute the normal subspace, (iv) correctly identifying which flow triggered the anomaly detector is an inherently challenging problem.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554 · cited 1×
“Two techniques that have been used include (i) determining topk by visually inspecting the scree plot — a method referred to as Cattell’s Scree Test [5] in the statistics literature, and (ii) retaining the first k principal components that do not contain a 3σ deviation from the mean when the traffic matrix is projecte…”
MethodAnalysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0071325 · Cited by 9,633
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
Schölkopf, B., Smola, A., & Müller, K.-R. (1998) · Neural Computation · doi:10.1162/089976698300017467 · Cited by 8,084
The Varimax Criterion for Analytic Rotation in Factor Analysis
Kaiser, H. F. (1958) · Psychometrika · doi:10.1007/bf02289233 · Cited by 7,580
Mixtures of Probabilistic Principal Component Analyzers
Tipping, M. E., & Bishop, C. M. (1999) · Neural Computation · doi:10.1162/089976699300016728 · Cited by 1,873
Mining anomalies using traffic feature distributions
Lakhina, A., Crovella, M., & Diot, C. (2005) · Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications · doi:10.1145/1080091.1080118 · Cited by 1,054
Self-citationDiagnosing network-wide traffic anomalies
Lakhina, A., Crovella, M., & Diot, C. (2004) · Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications · doi:10.1145/1015467.1015492 · Cited by 1,039
Self-citationPrediction by Supervised Principal Components
Bair, E., Hastie, T., Paul, D., & Tibshirani, R. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000628 · Cited by 817
A signal analysis of network traffic anomalies
Barford, P., Kline, J., Plonka, D., & Ron, A. (2002) · Proceedings of the second ACM SIGCOMM Workshop on Internet measurment - IMW '02 · doi:10.1145/637201.637210 · Cited by 787
Fast accurate computation of large-scale IP traffic matrices from link loads
Zhang, Y., Roughan, M., Duffield, N., & Greenberg, A. (2003) · Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems · doi:10.1145/781027.781053 · Cited by 505
Structural analysis of network traffic flows
Lakhina, A., Papagiannaki, K., Crovella, M., Diot, C., Kolaczyk, E. D., & Taft, N. (2004) · Proceedings of the joint international conference on Measurement and modeling of computer systems · doi:10.1145/1005686.1005697 · Cited by 451
Self-citationCharacterization of network-wide anomalies in traffic flows
Lakhina, A., Crovella, M., & Diot, C. (2004) · Proceedings of the 4th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1028788.1028813 · Cited by 421
Self-citationMontanelli, R. G., & Humphreys, L. G. (1976) · Psychometrika · doi:10.1007/bf02293559 · Cited by 268
Combining Filtering and Statistical Methods for Anomaly Detection
Soule, A., Salamatian, K., & Taft, N. (2005) · Cited by 258
Self-citationDetection and identification of network anomalies using sketch subspaces
Li, X., Bian, F., Crovella, M., Diot, C., Govindan, R., Iannaccone, G., & Lakhina, A. (2006) · Proceedings of the 6th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1177080.1177099 · Cited by 232
Self-citationZhang⋆, Y., Ge, Z., Greenberg, A. G., & Roughan, M. (2005) · doi:10.5555/1251086.1251116 · Cited by 209
MIND: A Distributed Multi-Dimensional Indexing System for Network Diagnosis
Li, X., Bian, F., Zhang, H., Diot, C., Govindan, R., Hong, W., & Iannaccone, G. (2006) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2006.205 · Cited by 29
Self-citation
Cited by
Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., & Tygar, J. D. (2011) · Proceedings of the 4th ACM workshop on Security and artificial intelligence · doi:10.1145/2046684.2046692 · Cited by 1,194
Map-matching for low-sampling-rate GPS trajectories
Lou, Y., Zhang, C., Zheng, Y., Xie, X., Wang, W., & Huang, Y. (2009) · Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems · doi:10.1145/1653771.1653820 · Cited by 797
Anomaly Detection for IoT Time-Series Data: A Survey
Cook, A. A., Misirli, G., & Fan, Z. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2958185 · Cited by 790
Classification-Reconstruction Learning for Open-Set Recognition
Yoshihashi, R., Shao, W., Kawakami, R., You, S., Iida, M., & Naemura, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00414 · Cited by 472
A comprehensive survey on network anomaly detection
Fernandes, G., Rodrigues, J. J. P. C., Carvalho, L. F., Al-Muhtadi, J. F., & Proença, M. L. (2019) · Telecommunication Systems · doi:10.1007/s11235-018-0475-8 · Cited by 382
Metadata
- Authors
- Ringberg, H., Soule, A., Rexford, J., & Diot, C.
- Venue
- Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
- Publisher
- ACM
- Published
- 2007-06-12
- Pages
- 109-120
- OpenAlex
- W2118931532
- Cited by
- 353
- References
- 17
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.