Does learning require memorization? a short tale about a long tail
Open accessAbstract
State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels. This tendency to memorize seemingly useless training data labels is not explained by existing theoretical analyses. Memorization of the training data also presents significant privacy risks when the training data contains sensitive personal information and thus it is important to understand whether such memorization is necessary for accurate learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Complex models like neural networks are likely more susceptible to leakage due to their ability to memorize data.
“Yet, complex models like neural networks are likely more susceptible to leakage due to their ability to memorize data52,53.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [53]
The Introduction says deep neural networks can memorize training examples, and the Abstract says memorization poses significant privacy risks when training data contain sensitive personal information. The paper does not establish the comparative claim that complex models are more susceptible to leakage than other models. No page numbers are provided in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466 · see citing works
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936 · see citing works
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856 · see citing works
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283 · see citing works
Boosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P. L., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,541 · see citing works
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · see citing works
Estimation of Dependences Based on Empirical Data
Vapnik, V. N. (2006) · Information science and statistics · doi:10.1007/0-387-34239-7 · Cited by 2,252 · see citing works
How to construct random functions
Goldreich, O., Goldwasser, S., & Micali, S. M. (1986) · Journal of the ACM · doi:10.1145/6490.6503 · Cited by 2,108 · see citing works
Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
Bartlett, P. L., & Mendelson, S. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1_15 · Cited by 1,989 · see citing works
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · see citing works
The iNaturalist Species Classification and Detection Dataset
Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., & Belongie, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00914 · Cited by 1,451 · see citing works
A Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307 · see citing works
Learning Imbalanced Datasets with Label-Distribution-Aware Margin LossPreprint
Cao, K., Wei, C., Gaidon, A., Aréchiga, N., & Ma, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07413 · Cited by 1,080 · see citing works
Learning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849 · see citing works
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep LearningPreprint
Neyshabur, B., Tomioka, R., & Srebro, N. (2014) · International Conference on Learning Representations · doi:10.48550/arxiv.1412.6614 · Cited by 742 · see citing works
Exploring Generalization in Deep Learning
Neyshabur, B., Bhojanapalli, S., McAllester, D., & Srebro, N. (2017) · neural information processing systems · Cited by 670 · see citing works
(2000) · Applied Physics Letters · doi:10.1162/153244302760200704 · Cited by 668 · see citing works
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629 · see citing works
Spectrally-normalized margin bounds for neural networksPreprint
Bartlett, P. L., Foster, D. J., & Telgarsky, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08498 · Cited by 609 · see citing works
Benign overfitting in linear regression
Bartlett, P. L., Long, P. M., Lugosi, G., & Tsigler, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907378117 · Cited by 554 · see citing works
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning
Cui, Y., Song, Y., Sun, C., Howard, A., & Belongie, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00432 · Cited by 522 · see citing works
Train faster, generalize better: stability of stochastic gradient descent
Hardt, M., Recht, B., & Singer, Y. (2016) · International Conference on Machine Learning · Cited by 516 · see citing works
Surprises in high-dimensional ridgeless least squares interpolation
Hastie, T. J., Andrea Montanari, A., Rosset, S., & Tibshirani, R. J. (2022) · The Annals of Statistics · doi:10.1214/21-aos2133 · Cited by 510 · see citing works
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016) · arXiv · doi:10.48550/arxiv.1610.05755 · Cited by 492 · see citing works
Wang, Y.-X., Ramanan, D., & Hebert, M. (2017) · neural information processing systems · Cited by 468 · see citing works
Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm
Needell, D., Srebro, N., & Ward, R. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0864-7 · Cited by 431 · see citing works
Rademacher penalties and structural risk minimization
Koltchinskii, V. I. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.930926 · Cited by 364 · see citing works
Train faster, generalize better: Stability of stochastic gradient descentPreprint
Hardt, M., Recht, B., & Singer, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01240 · Cited by 358 · see citing works
Learnability, Stability and Uniform Convergence
Shalev‐Shwartz, S., Shamir, O., Srebro, N., & Sridharan, K. (2010) · Cited by 304 · see citing works
Preserving Statistical Validity in Adaptive Data Analysis
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A. (2015) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/2746539.2746580 · Cited by 258 · see citing works
Self-citationExplaining the Success of AdaBoost and Random Forests as Interpolating ClassifiersPreprint
Wyner, A. J., Olson, M., Bleich, J., & Mease, D. (2015) · arXiv (Cornell University) · doi:10.5555/3122009.3153004 · Cited by 244 · see citing works
Shalev‐Shwartz, S., & Srebro, N. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390273 · Cited by 241 · see citing works
Differential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219 · see citing works
Mukherjee, S., Niyogi, P., Poggio, T., & Rifkin, R. M. (2006) · Advances in Computational Mathematics · doi:10.1007/s10444-004-7634-z · Cited by 198 · see citing works
Just interpolate: Kernel “Ridgeless” regression can generalizePreprint
Liang, T., & Rakhlin, A. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 190 · see citing works
Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics
Wu, X., Li, F., Kumar, A., Chaudhuri, K., Jha, S., & Naughton, J. F. (2017) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3035918.3064047 · Cited by 182 · see citing works
Babbar, R., & Schölkopf, B. (2017) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3018661.3018741 · Cited by 180 · see citing works
Stochastic Convex Optimization.
Shalev‐Shwartz, S., Shamir, O., Srebro, N., & Sridharan, K. (2009) · Conference on Learning Theory · Cited by 169 · see citing works
Understanding Membership Inferences on Well-Generalized Learning Models
Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Gunter, C. A., & Chen, K. (2018) · arXiv · doi:10.48550/arxiv.1802.04889 · Cited by 166 · see citing works
The Devil is in the Tails: Fine-grained Classification in the WildPreprint
Horn, G. V., & Perona, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01450 · Cited by 155 · see citing works
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence EstimationPreprint
Feldman, V., & Zhang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03703 · Cited by 146 · see citing works
Self-citationOverfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
Belkin, M., Hsu, D., & Mitra, P. P. (2018) · Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) · doi:10.48550/arxiv.1806.05161 · Cited by 139 · see citing works
Capturing Long-Tail Distributions of Object Subcategories
Zhu, X., Anguelov, D., & Ramanan, D. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.122 · Cited by 135 · see citing works
Harmless Interpolation of Noisy Data in Regression
Muthukumar, V., Vodrahalli, K., Subramanian, V., et al. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849614 · Cited by 135 · see citing works
Characterising Bias in Compressed ModelsPreprint
Hooker, S., Moorosi, N., Clark, G., Samy Bengio, A., & Denton, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03058 · Cited by 132 · see citing works
Data scarcity, robustness and extreme multi-label classification
Babbar, R., & Schölkopf, B. (2019) · Machine Learning · doi:10.1007/s10994-019-05791-5 · Cited by 131 · see citing works
What Do Compressed Deep Neural Networks Forget?Preprint
Hooker, S., Courville, A. C., Clark, G., Dauphin, Y., & Frome, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05248 · Cited by 125 · see citing works
Distribution-free inequalities for the deleted and holdout error estimates
Devroye, L., & Wagner, T. J. (1979) · IEEE Transactions on Information Theory · doi:10.1109/tit.1979.1056032 · Cited by 120 · see citing works
Ma, S., Bassily, R., & Belkin, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06559 · Cited by 120 · see citing works
Does data interpolation contradict statistical optimality?Preprint
Belkin, M., Rakhlin, A., & Tsybakov, A. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09471 · Cited by 113 · see citing works
Geometry of Optimization and Implicit Regularization in Deep LearningPreprint
Neyshabur, B., Tomioka, R., Salakhutdinov, R., & Srebro, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.03071 · Cited by 98 · see citing works
Towards Demystifying Membership Inference AttacksPreprint
Truex, S., Liu, L., Gürsoy, M. E., Yu, L., & Wei, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09173 · Cited by 84 · see citing works
Towards Measuring Membership PrivacyPreprint
Long, Y., Bindschaedler, V., & Gunter, C. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09136 · Cited by 72 · see citing works
Sample Complexity Bounds for Differentially Private Learning.
Chaudhuri, K., & Hsu, D. (2011) · PubMed · Cited by 71 · see citing works
High probability generalization bounds for uniformly stable algorithms with nearly optimal ratePreprint
Feldman, V., & Vondrák, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10710 · Cited by 70 · see citing works
Self-citationSTABILITY RESULTS IN LEARNING THEORY
Rakhlin, A., Mukherjee, S., & Poggio, T. (2005) · Analysis and Applications · doi:10.1142/s0219530505000650 · Cited by 63 · see citing works
Rates of Convergence for Nearest Neighbor ClassificationPreprint
Chaudhuri, K., & Dasgupta, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.0067 · Cited by 63 · see citing works
Surprises in High-Dimensional Ridgeless Least Squares InterpolationPreprint
Hastie, T., Andrea Montanari, A., Rosset, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08560 · Cited by 63 · see citing works
Competitive distribution estimation: why is Good-Turing good
Orlitsky, A., & Suresh, A. T. (2015) · Cited by 52 · see citing works
A New Perspective on an Old Perceptron Algorithm
Shalev‐Shwartz, S., & Singer, Y. (2005) · Lecture notes in computer science · doi:10.1007/11503415_18 · Cited by 45 · see citing works
Instance optimal learning of discrete distributions
Valiant, G., & Valiant, P. (2016) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/2897518.2897641 · Cited by 42 · see citing works
Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes BackPreprint
Feldman, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04414 · Cited by 30 · see citing works
Self-citationDistribution Density, Tails, and Outliers in Machine Learning: Metrics and ApplicationsPreprint
Carlini, N., Erlingsson, Ú., & Papernot, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13427 · Cited by 20 · see citing works
Preserving Statistical Validity in Adaptive Data AnalysisPreprint
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2664 · Cited by 20 · see citing works
Self-citationFeldman, V. (2019) · Open Collections · doi:10.14288/1.0377608 · Cited by 16 · see citing works
Self-citationPrototypical Examples in Deep Learning: Metrics, Characteristics, and Utility
Carlini, N., Erlingsson, Ú., & Papernot, N. (2018) · Cited by 14 · see citing works
arXiv adds category for applied physics
Grant, A. (2017) · Physics Today · doi:10.1063/pt.5.9095 · Cited by 14 · see citing works
DNN or k-NN: That is the Generalize vs. Memorize QuestionPreprint
Cohen, G., Sapiro, G., & Giryes, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06822 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- Rohit Babbar and Bernhard Schölkopf. 2017. Dismec: Distributed sparse machines for extreme multi-label classification. In Proceedings of the tenth ACM international conference on web search and data mining. ACM, 721-729.
- Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. 2018. Reconcil- ing modern machine learning and the bias-variance trade-off. arXiv preprint arXiv:1812.11118 (2018). · arXiv
- Mikhail Belkin, Daniel Hsu, and Ji Xu. 2019. Two models of double descent for weak features. arXiv preprint arXiv:1903.07571 (2019). · arXiv
- Mikhail Belkin, Siyuan Ma, and Soumik Mandal. 2018. To Understand Deep Learning We Need to Understand Kernel Learning. In ICML (Proceedings of Machine Learning Research), Vol. 80. PMLR, 541-549. http://proceedings.mlr. press/v80/belkin18a.html
- Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. JMLR 2 (2002), 499-526.
- Vitaly Feldman. 2019. Does Learning Require Memorization? A Short Tale about a Long Tail. CoRR abs/1906.05271 (2019). arXiv:1906.05271 http://arxiv.org/abs/ 1906.05271 · arXiv
- Yuanzhi Li, Tengyu Ma, and Hongyang Zhang. 2018. Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations. In Conference On Learning Theory. 2-47.
- Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian J. Goodfellow, and Kunal Talwar. 2017. Semi-supervised Knowledge Transfer for Deep Learning from Pri- vate Training Data. In Proceedings of the 5th International Conference on Learning Representations (ICLR).
- Alexander Rakhlin and Xiyu Zhai. 2019. Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenon. In COLT, Vol. 99. PMLR, 2595-2623. http://proceedings.mlr.press/v99/rakhlin19a.html
- Robert E Schapire. 2013. Explaining adaboost. In Empirical inference. Springer, 37-52.
- Nathan Srebro, Karthik Sridharan, and Ambuj Tewari. 2010. Smoothness, Low Noise and Fast Rates. In NIPS. 2199-2207. http://papers.nips.cc/paper/3894- smoothness-low-noise-and-fast-rates.pdf
- Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In ICLR. https://openreview.net/forum?id=Sy8gdB9xx
Cited by
Sorted by citations
Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387 · see citing works
Data leakage inflates prediction performance in connectome-based machine learning models
Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., & Scheinost, D. (2024) · Nature Communications · doi:10.1038/s41467-024-46150-w · Cited by 236 · see citing works
The Curious Case of Hallucinations in Neural Machine Translation
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.92 · Cited by 142 · see citing works
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.06500 · Cited by 101 · see citing works
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning
Razeghi, Y., Logan IV, R. L., Gardner, M., & Singh, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.59 · Cited by 89 · see citing works
Subpopulation Data Poisoning Attacks
Jagielski, M., Severi, G., Pousette Harger, N., & Oprea, A. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3485368 · Cited by 86 · see citing works
Generative Ai in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024) · doi:10.2139/ssrn.4821952 · Cited by 57 · see citing works
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51 · see citing works
On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
Thudi, A., Jia, H., Shumailov, I., & Papernot, N. (2021) · arXiv · doi:10.48550/arxiv.2110.11891 · Cited by 48 · see citing works
Fooling MOSS Detection with Pretrained Language Models
Biderman, S., & Raff, E. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557079 · Cited by 37 · see citing works
Sankar, B. S., Gilliland, D., Rincon, J., Hermjakob, H., Yan, Y., Adam, I., Lemaster, G., Wang, D., Watson, K., Bui, A., et al. (2024) · Bioengineering · doi:10.3390/bioengineering11100984 · Cited by 26 · see citing works
On the foundations of Earth foundation models
Zhu, X. X., Xiong, Z., Wang, Y., Stewart, A. J., Heidler, K., Wang, Y., Yuan, Z., Dujardin, T., Xu, Q., & Shi, Y. (2026) · Communications Earth & Environment · doi:10.1038/s43247-025-03127-x · Cited by 26 · see citing works
Esmradi, A., Yip, D. W., & Chan, C. F. (2023) · arXiv · doi:10.48550/arxiv.2312.10982 · Cited by 24 · see citing works
Esmradi, A., Yip, D. W., & Chan, C. F. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-97-1274-8_6 · Cited by 24 · see citing works
Neural networks memorise personal information from one sample
Hartley, J., Sanchez, P. P., Haider, F., & Tsaftaris, S. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-48034-3 · Cited by 10 · see citing works
- Authors
- Feldman, V.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Feldman, Vitaly
- ORCID
- Feldman, Vitaly
- Venue
- Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing
- Publisher
- ACM
- Published
- 2020-06-22
- Pages
- 954-959
- OpenAlex
- W2950181023
- Cited by
- 287
- References
- 76
- Citations by year
- 2026: 33 · 2025: 47 · 2024: 54 · 2023: 42 · 2022: 40 · 2021: 51 · 2020: 17 · 2019: 3
- FWCI
- 14.33
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Symposium on Theory of Computing (STOC) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- memorization, image recognition, privacy risks, training data memorization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
