CoChat

Evaluating model drift in machine learning algorithms

Nelson, Kevin, Corbin, George, Anania, Mark, Kovacs, Matthew, Tobias, Jeremy, Blowers, Misty · 2015 · 2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA)

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.

Abstract

Machine learning is rapidly emerging as a valuable technology thanks to its ability to learn patterns from large data sets and solve problems that are impossible to model using conventional programming logic. As machine learning techniques become more mainstream, they are being applied to a wider range of application domains. These algorithms are now trusted to make critical decisions in secure and adversarial environments such as healthcare, fraud detection, and network security, in which mistakes can be incredibly costly. They are also a critical component to most modern autonomous systems. However, the data driven approach utilized by these machine learning methods can prove to be a weakness if the data on which the models rely are corrupted by either nefarious or accidental means. Models that utilize on-line learning or periodic retraining to learn new patterns and account for data distribution changes are particularly susceptible to corruption through model drift. In modeling this type of scenario, specially crafted data points are added to the training set over time to adversely influence the system, inducing model drift which leads to incorrect classifications. Our work is focused on exploring the resistance of various machine learning algorithms to such an approach. In this paper we present an experimental framework designed to measure the susceptibility of anomaly detection algorithms to model drift. We also exhibit our preliminary results using various machine learning algorithms commonly found in intrusion detection research.

Overview

Integrity signals

Registry

Crossref · 10.1109/cisda.2015.7208643

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 17 references

Research works printed but not linked: 1

  1. biggio, 2013, Evasion atacks against machine learning at test time, Machine Learning and Knowledge Discovery in Databases, 387

Cited by

Metadata

Authors
Nelson, K., Corbin, G., Anania, M., Kovacs, M., Tobias, J., & Blowers, M.
Affiliations
United States Air Force Research Laboratory (US)
Venue
2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA)
Publisher
IEEE
Published
2015-05
Pages
1-8
OpenAlex
W1487801014
Cited by
31
References
17
Citations by year
2026: 3 · 2025: 8 · 2024: 6 · 2023: 7 · 2022: 4 · 2021: 1 · 2020: 1 · 2019: 1
FWCI
0.00
Type
conference paper
Language
en
Open access
closed
Locations
IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
Keywords
model drift, anomaly detection, intrusion detection, machine learning robustness, data poisoning, online learning, distribution shift, classification accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.