Certified Data Removal from Machine Learning Models

Model manipulation methods usually give a verifiability guarantee using theory-based and information bound-based methods.

“Significantly, model manipulation methods usually give a verifiability guarantee using theory-based and information bound-based methods [40, 41, 43].”

Approximate unlearning is usually guaranteed by differential privacy techniques, such as (ε, δ)-certified unlearning.

“This approximation is usually guaranteed by differential privacy techniques, such as ( ε , δ ) -certified unlearning [41, 42].”

The above methods mainly focused on the core problem of empirical risk minimization, where the goal is to find approximate minimizers of the empirical loss on the remaining training dataset after unlearning samples.

“The above methods mainly focused on the core problem of empirical risk minimization, where the goal is to find approximate minimizers of the empirical loss on the remaining training dataset after unlearning samples [41, 51].”

Unlearning methods with a theoretical guarantee are usually based on rich assumptions and can rarely be applied to complex models, since complex deep models usually make those assumptions invalid.

“Meanwhile, unlearning methods with a theoretical guarantee are usually based on rich assumptions and can rarely be applied to complex models, since complex deep models usually make those assumptions invalid [41, 53].”

Most of the above schemes focus on relatively simple machine learning problems, such as linear logistic regression, or complex models with special assumptions.

“Second, most of the above schemes focus on relatively simple machine learning problems, such as linear logistic regression, or complex models with special assumptions [40, 41, 43, 51].”

Some methods provide a certified unlearning definition ensuring that the unlearned model cannot be distinguished from a model trained on the remaining dataset from scratch.

“Some methods provide a certified unlearning definition [41, 53], which ensures that the unlearned model cannot be distinguished from a model trained on the remaining dataset from scratch.”

Guo et al.'s unlearning algorithms are designed for linear models with strongly convex regularization.

“For example, Guo et al.'s [41] unlearning algorithms are designed for linear models with strongly convex regularization.”

Based on the guarantee provided by certified unlearning, Guo et al. [41], Warnecke et al. [53], and Sekhari et al. [42] limit the distribution similarity between the unlearned model and the retrained model.

“Based on the guarantee provided by certified unlearning , they limit the distribution similarity between the unlearned model and the retrained model.”

Guo et al. [41], Warnecke et al. [53], and Sekhari et al. [42] provide a method of theoretical verification to verify the effectiveness of their proposed unlearning schemes.

“Guo et al. [41], Warnecke et al. [53], and Sekhari et al. [42] provide a method of theoretical verification to verify the effectiveness of their proposed unlearning schemes.”

Guo et al. [41] hide information about the unlearned samples using a loss perturbation technique at the time of training.

“For example, Guo et al. [41] hide information about the unlearned samples using a loss perturbation technique [118] at the time of training.”

The loss perturbation technique involves perturbing the empirical risk through a random linear term.

“The technique involves perturbing the empirical risk through a random linear term.”