Carlini, Nicholas

Towards Evaluating the Robustness of Neural Networks

Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP)

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 8,859

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020) · arXiv

Machine Learning and AlgorithmsDomain Adaptation and Few-Shot Learning

Cited by 2,892

Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 2,172

Extracting Training Data from Large Language Models

Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv

Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in Data

Cited by 727

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks

Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv

Privacy-Preserving Technologies in DataAdversarial Robustness in Machine Learning

Cited by 629

On Adaptive Attacks to Adversarial Example Defenses

Tramer, F., Carlini, N., Brendel, W., & Madry, A. (2020) · arXiv

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 486

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Merrill, M. A., Shaw, A. G., Carlini, N., Li, B., Raj, H., Bercovich, I., Shi, L., Shin, J. Y., Walshe, T., Buchanan, E. K., et al. (2026) · arXiv

AI-based Problem Solving and PlanningArtificial Intelligence Applications

Cited by 392

Deduplicating Training Data Makes Language Models Better

Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingNatural Language Processing Techniques

Cited by 304

Measuring Robustness to Natural Distribution Shifts in Image Classification

Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., & Schmidt, L. (2020) · arXiv

Adversarial Robustness in Machine LearningDomain Adaptation and Few-Shot Learning

Cited by 206

Quantifying Memorization Across Neural Language Models

Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., & Zhang, C. (2022) · arXiv

Topic ModelingPrivacy-Preserving Technologies in Data

Cited by 204

Universal and Transferable Adversarial Attacks on Aligned Language Models

Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023) · arXiv

Adversarial Robustness in Machine LearningSpeech Recognition and Synthesis

Cited by 197

Evading Deepfake-Image Detectors with White- and Black-Box Attacks

Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Digital Media Forensic DetectionAdversarial Robustness in Machine Learning

Cited by 170

Poisoning Web-Scale Training Datasets is Practical

Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2024) · 2024 IEEE Symposium on Security and Privacy (SP)

Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in Data

Cited by 138

Poisoning Web-Scale Training Datasets is Practical

Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2023) · arXiv

Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in Data

Cited by 138

Scalable Extraction of Training Data from (Production) Language Models

Nasr, M., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Wallace, E., Tramèr, F., & Lee, K. (2023) · arXiv

Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in Data

Cited by 96

Unsolved Problems in ML Safety

Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv

Ethics and Social Impacts of AISafety Systems Engineering in Autonomy

Cited by 95

Identifying and Mitigating the Security Risks of Generative AI

Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security

Adversarial Robustness in Machine LearningInformation and Cyber Security

Cited by 57

ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases

Zhong, Z., Raghunathan, A., & Carlini, N. (2025) · arXiv

Software Engineering ResearchSoftware Testing and Debugging Techniques

Cited by 54

Counterfactual Memorization in Neural Language Models

Zhang, C., Ippolito, D., Lee, K., Jagielski, M., Tramèr, F., & Carlini, N. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 54

Preventing generation of verbatim memorization in language models gives a false sense of privacy

Ippolito, D., Tramer, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette Choo, C., & Carlini, N. (2023) · Proceedings of the 16th International Natural Language Generation Conference

Privacy-Preserving Technologies in DataAdversarial Robustness in Machine Learning

Cited by 28

Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards

Huang, Y., Nasr, M., Angelopoulos, A., Carlini, N., Chiang, W.-L., Choquette-Choo, C. A., Ippolito, D., Jagielski, M., Lee, K., Liu, K. Z., et al. (2025) · arXiv

Adversarial Robustness in Machine LearningSpam and Phishing Detection

Cited by 17

Defeating Prompt Injections by Design

Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F. (2026) · 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 15

Deduplicating Training Data Makes Language Models Better

Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 11

Increasing confidence in adversarial robustness evaluations

Zimmermann, R. S., Brendel, W., Tramer, F., & Carlini, N. (2022) · Advances in Neural Information Processing Systems 35

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 8