CoChat
Sign Up
PreprintCited by 2Adversarial Robustness in Machine LearningExplainable Artificial Intelligence (XAI)Machine Learning and Data Classification

Edit at your own risk: evaluating the robustness of edited models to distribution shifts

Open access
Brown, Davis, Godfrey, Charles, Nizinski, Cody, Tu, Jonathan, Kvinge, Henry·2023·arXiv
DOI 10.48550/arxiv.2303.00046arXiv 2303.00046
Open this paper in CoChat — free accountOpen PDF

Abstract

The current trend toward ever-larger models makes standard retraining procedures an ever-more expensive burden. For this reason, there is growing interest in model editing, which enables computationally inexpensive, interpretable, post-hoc model modifications. While many model editing techniques are promising, research on the properties of edited models is largely limited to evaluation of validation accuracy. The robustness of edited models is an important and yet mostly unexplored topic. In this paper, we employ recently developed techniques from the field of deep learning robustness to investigate both how model editing affects the general robustness of a model, as well as the robustness of the specific behavior targeted by the edit. We find that edits tend to reduce general robustness, but that the degree of degradation depends on the editing algorithm and layers chosen. Motivated by these observations we introduce a new model editing algorithm, 1-layer interpolation (1-LI), which uses weight-space interpolation to navigate the trade-off between editing task accuracy and general robustness.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2303.00046

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 32

  1. [Bar+19] Andrei Barbu et al. 'ObjectNet: A Large-Scale Bias-Controlled Dataset for Pushing the Limits of Object Recognition Models'. In: Advances in Neural Information Processing Systems . Vol. 32. Curran Associates, Inc., 2019.
  2. [Bau+20] David Bau et al. Rewriting a Deep Generative Model . July 2020. arXiv: 2007.15646 [cs] . · arXiv
  3. [Bom+21] Rishi Bommasani et al. 'On the Opportunities and Risks of Foundation Models'. In: ArXiv (2021). URL: https://crfm.stanford.edu/assets/report.pdf .
  4. [GDG19] Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain . Mar. 2019. DOI: 10.48550/ arXiv.1708.06733 . arXiv: 1708.06733 [cs] . (Visited on 05/11/2023). · arXiv
  5. [Gei+13] Andreas Geiger et al. 'Vision meets Robotics: The KITTI Dataset'. In: International Journal of Robotics Research (IJRR) (2013).
  6. [Gei+18] Robert Geirhos et al. 'ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness'. In: arXiv preprint arXiv:1811.12231 (2018). · arXiv
  7. [Ghi+17] Golnaz Ghiasi et al. Exploring the Structure of a Real-Time, Arbitrary Neural Artistic Stylization Network . Aug. 2017. DOI: 10.48550/arXiv.1705.06830 . arXiv: 1705. 06830 [cs] . · DOI
  8. [GVS14] Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. 'Qualitatively characterizing neural network optimization problems'. In: arXiv preprint arXiv:1412.6544 (2014). · arXiv
  9. [Has+21] Peter Hase et al. 'Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs'. In: arXiv preprint arXiv:2111.13654 (2021). · arXiv
  10. [HD19] Dan Hendrycks and Thomas Dietterich. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations . Mar. 2019. DOI: 10.48550/arXiv.1903. 12261 . arXiv: 1903.12261 [cs, stat] . · DOI
  11. [Hen+21a] Dan Hendrycks et al. Natural Adversarial Examples . Mar. 2021. DOI: 10.48550/ arXiv.1907.07174 . arXiv: 1907.07174 [cs, stat] . · arXiv
  12. [Hen+21b] Dan Hendrycks et al. 'The Many Faces of Robustness: A Critical Analysis of Out-ofDistribution Generalization'. In: ICCV (2021).
  13. [Hen+22] Dan Hendrycks et al. Unsolved Problems in ML Safety . June 2022. arXiv: 2109.13916 [cs] . · arXiv
  14. [Ilh+22] Gabriel Ilharco et al. Patching Open-Vocabulary Models by Interpolating Weights . Aug. 2022. arXiv: 2208.05592 [cs] . · arXiv
  15. [Kia+22] Bobak Kiani et al. projUNN: Efficient Method for Training Deep Networks with Unitary Matrices . Oct. 2022. arXiv: 2203.05483 [quant-ph] . · arXiv
  16. [Koh+21] Pang Wei Koh et al. 'WILDS: A Benchmark of in-the-Wild Distribution Shifts'. In: International Conference on Machine Learning (ICML) . 2021.
  17. [Koh72] Teuvo Kohonen. 'Correlation Matrix Memories'. In: IEEE Transactions on Computers C-21.4 (Apr. 1972), pp. 353-359. ISSN: 1557-9956. DOI: 10.1109/TC.1972. 5008975 . · DOI
  18. [Koh77] Teuvo Kohonen. Associative Memory . Ed. by King Sun Fu et al. Vol. 17. Communication and Cybernetics. Berlin, Heidelberg: Springer, 1977. DOI: 10.1007/978-3-64296384-1 . · DOI
  19. [Lee+23] Yoonho Lee et al. 'Surgical Fine-Tuning Improves Adaptation to Distribution Shifts'. In: The Eleventh International Conference on Learning Representations . 2023. URL: https://openreview.net/forum?id=APuPRxjHvZ .
  20. [Luc+21] James Lucas et al. 'Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes'. In: ArXiv (Apr. 2021).
  21. [Men+22a] Kevin Meng et al. Locating and Editing Factual Associations in GPT . June 2022. arXiv: 2202.05262 [cs] . · arXiv
  22. [Men+22b] Kevin Meng et al. Mass-Editing Memory in a Transformer . Oct. 2022. DOI: 10.48550/ arXiv.2210.07229 . arXiv: 2210.07229 [cs] . · arXiv
  23. [Mil+21] John Miller et al. Accuracy on the Line: On the Strong Correlation Between Out-ofDistribution and In-Distribution Generalization . Oct. 2021. arXiv: 2107.04649 [cs, stat] . · arXiv
  24. [Mit+22a] Eric Mitchell et al. Fast Model Editing at Scale . June 2022. DOI: 10.48550/arXiv. 2110.11309 . arXiv: 2110.11309 [cs] . · DOI
  25. [Mit+22b] Eric Mitchell et al. Memory-Based Model Editing at Scale . June 2022. DOI: 10.48550/ arXiv.2206.06520 . arXiv: 2206.06520 [cs] . · arXiv
  26. [Net+11] Yuval Netzer et al. 'Reading Digits in Natural Images with Unsupervised Feature Learning'. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011 . 2011. URL: http://ufldl.stanford.edu/housenumbers/nips2011\_ housenumbers.pdf .
  27. [Pas+19] Adam Paszke et al. 'PyTorch: An Imperative Style, High-Performance Deep Learning Library'. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems . Red Hook, NY, USA: Curran Associates Inc., 2019.
  28. [Rad+21] Alec Radford et al. 'Learning Transferable Visual Models From Natural Language Supervision'. In: International Conference on Machine Learning . 2021.
  29. [Sag+22] Shiori Sagawa et al. 'Extending the WILDS Benchmark for Unsupervised Adaptation'. In: International Conference on Learning Representations (ICLR . 2022.
  30. [San+21] Shibani Santurkar et al. 'Editing a Classifier by Rewriting Its Prediction Rules'. In: Advances in Neural Information Processing Systems . Vol. 34. Curran Associates, Inc., 2021, pp. 23359-23373.
  31. [Tao+20] Rohan Taori et al. 'Measuring Robustness to Natural Distribution Shifts in Image Classification'. In: Advances in Neural Information Processing Systems . V ol. 33. Curran Associates, Inc., 2020, pp. 18583-18599.
  32. [Wor+22] Mitchell Wortsman et al. Robust Fine-Tuning of Zero-Shot Models . June 2022. arXiv: 2109.01903 [cs] . · arXiv

Cited by

Sorted by citations

Related

Authors
Brown, D., Godfrey, C., Nizinski, C., Tu, J., & Kvinge, H.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2303.00046
arXiv
2303.00046
OpenAlex
W4322825541
Cited by
1
References
0
Citations by year
2023: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
model editing, robustness, distribution shift, deep learning robustness, interpretability, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review