Heidecke, Johannes

HealthBench: Evaluating Large Language Models Towards Improved Human Health

Arora, R. K., Wei, J., Hicks, R. S., Bowman, P., Quiñonero-Candela, J., Tsimpourlas, F., Sharman, M., Shah, M., Vallone, A., Beutel, A., et al. (2025) · arXiv

Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Cited by 654

PaperBench: Evaluating AI's Ability to Replicate AI Research

Starace, G., Jaffe, O., Sherburn, D., Aung, J., Chan, J. S., Maksin, L., Dias, R., Mays, E., Kinsella, B., Thompson, W., et al. (2025) · arXiv

Artificial Intelligence ApplicationsSoftware Engineering Research

Cited by 310

GPT-4o System Card

OpenAI, :, Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., et al. (2024) · arXiv

Multimodal Machine Learning ApplicationsTopic Modeling

Cited by 180

OpenAI o1 System Card

OpenAI, :, Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., et al. (2024) · arXiv

Adversarial Robustness in Machine LearningEthics and Social Impacts of AI

Cited by 50

OpenAI GPT-5 System Card

Singh, A., Fry, A., Perelman, A., Tart, A., Ganesh, A., El-Kishky, A., McLaughlin, A., Low, A., Ostrow, A., Ananthram, A., et al. (2026) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 26

Rule Based Rewards for Language Model Safety

Mu, T., Helyar, A., Heidecke, J., Achiam, J., Vallone, A., Kivlichan, I., Lin, M., Beutel, A., Schulman, J., & Weng, L. (2024) · Advances in Neural Information Processing Systems 37

Ethics and Social Impacts of AIAdversarial Robustness in Machine Learning

Cited by 14

The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., & Beutel, A. (2024) · arXiv

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 14

AI-based Clinical Decision Support for Primary Care: A Real-World Study

Korom, R., Kiptinness, S., Adan, N., Said, K., Ithuli, C., Rotich, O., Kimani, B., King'ori, I., Kamau, S., Atemba, E., et al. (2025) · arXiv

Artificial Intelligence in Healthcare and EducationElectronic Health Records Systems

Cited by 10

SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?

Miserendino, S., Wang, M., Patwardhan, T., & Heidecke, J. (2025) · arXiv

Software Engineering ResearchSoftware Engineering Techniques and Practices

Cited by 5

Persona Features Control Emergent Misalignment

Wang, M., la Tour, T. D., Watkins, O., Makelov, A., Chi, R. A., Miserendino, S., Wang, J., Rajaram, A., Heidecke, J., Patwardhan, T., et al. (2025) · arXiv

Adversarial Robustness in Machine LearningEthics and Social Impacts of AI

Cited by 2