Durmus, Esin

On the Opportunities and Risks of Foundation Models

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv

Ethics and Social Impacts of AIArtificial Intelligence Applications

Cited by 2,676

Benchmarking Large Language Models for News Summarization

Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2023) · arXiv

Topic ModelingAdvanced Text Analysis Techniques

Cited by 415

Benchmarking Large Language Models for News Summarization

Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2024) · Transactions of the Association for Computational Linguistics

Topic ModelingAdvanced Text Analysis Techniques

Cited by 415

Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale

Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., Hashimoto, T., Jurafsky, D., Zou, J., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency

Ethics and Social Impacts of AIHate Speech and Cyberbullying Detection

Cited by 342

FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization

Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Topic ModelingAdvanced Text Analysis Techniques

Cited by 306

Holistic Evaluation of Language Models

Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 255

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Aremu, A., Bosselut, A., Chandu, K. R., Clinciu, M.-A., Das, D., Dhole, K., et al. (2021) · Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)

Natural Language Processing TechniquesTopic Modeling

Cited by 159

Whose Opinions Do Language Models Reflect?

Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023) · arXiv

Sentiment Analysis and Opinion MiningEthics and Social Impacts of AI

Cited by 114

Evaluating Human-Language Model Interaction

Lee, M., Srivastava, M., Hardy, A., Thickstun, J., Durmus, E., Paranjape, A., Gerard-Ursin, I., Li, X. L., Ladhak, F., Rong, F., et al. (2022) · arXiv

Speech and dialogue systemsNatural Language Processing Techniques

Cited by 61

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Anuoluwapo, A., Bosselut, A., Chandu, K. R., Clinciu, M., Das, D., Dhole, K. D., et al. (2021) · arXiv

Natural Language Processing TechniquesTopic Modeling

Cited by 49

Towards Measuring the Representation of Subjective Global Opinions in Language Models

Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al. (2023) · arXiv

Ethics and Social Impacts of AITopic Modeling

Cited by 48

Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization

Ladhak, F., Durmus, E., He, H., Cardie, C., & McKeown, K. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingAdvanced Text Analysis Techniques

Cited by 46

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., et al. (2026) · arXiv

Explainable Artificial Intelligence (XAI)Topic Modeling

Cited by 36

Measuring Faithfulness in Chain-of-Thought Reasoning

Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., et al. (2023) · arXiv

Topic ModelingExplainable Artificial Intelligence (XAI)

Cited by 33

When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization

Ladhak, F., Durmus, E., Suzgun, M., Zhang, T., Jurafsky, D., McKeown, K., & Hashimoto, T. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

Topic ModelingNatural Language Processing Techniques

Cited by 30

Studying Large Language Model Generalization with Influence Functions

Grosse, R., Bae, J., Anil, C., Elhage, N., Tamkin, A., Tajdini, A., Steiner, B., Li, D., Durmus, E., Perez, E., et al. (2023) · arXiv

Explainable Artificial Intelligence (XAI)Adversarial Robustness in Machine Learning

Cited by 26

Many-shot Jailbreaking

Anil, C., Durmus, E., Panickssery, N., Sharma, M., Benton, J., Kundu, S., Batson, J., Tong, M., Mu, J., Ford, D., et al. (2024) · Advances in Neural Information Processing Systems 37

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 25

Opportunities and Risks of LLMs for Scalable Deliberation with Polis

Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv

Ethics and Social Impacts of AIPolitical Theory and Democracy

Cited by 20

Spurious Correlations in Reference-Free Evaluation of Text Generation

Durmus, E., Ladhak, F., & Hashimoto, T. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Natural Language Processing TechniquesTopic Modeling

Cited by 16

Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations

Handa, K., Tamkin, A., McCain, M., Huang, S., Durmus, E., Heck, S., Mueller, J., Hong, J., Ritchie, S., Belonax, T., et al. (2025) · arXiv

Labor market dynamics and wage inequalityArtificial Intelligence Applications

Cited by 11

Evaluating and Mitigating Discrimination in Language Model Decisions

Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., & Ganguli, D. (2023) · arXiv

Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)

Cited by 11

Question Decomposition Improves the Faithfulness of Model-Generated Reasoning

Radhakrishnan, A., Nguyen, K., Chen, A., Chen, C., Denison, C., Hernandez, D., Durmus, E., Hubinger, E., Kernion, J., Lukošiūtė, K., et al. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 8

Specific versus General Principles for Constitutional AI

Kundu, S., Bai, Y., Kadavath, S., Askell, A., Callahan, A., Chen, A., Goldie, A., Balwit, A., Mirhoseini, A., McLean, B., et al. (2023) · arXiv

Ethics and Social Impacts of AISpeech and dialogue systems

Cited by 7

Sabotage Evaluations for Frontier Models

Benton, J., Wagner, M., Christiansen, E., Anil, C., Perez, E., Srivastav, J., Durmus, E., Ganguli, D., Kravec, S., Shlegeris, B., et al. (2024) · arXiv

Adversarial Robustness in Machine LearningEthics and Social Impacts of AI

Cited by 2