Callahan, Alison

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., et al. (2024) · JAMA

Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Cited by 738

Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review

Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., et al. (2025) · JAMA

Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Cited by 662

A Nationwide Network of Health AI Assurance Laboratories

Shah, N. H., Halamka, J. D., Saria, S., Pencina, M., Tazbaz, T., Tripathi, M., Callahan, A., Hildahl, H., & Anderson, B. (2024) · JAMA

Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AI

Cited by 114

MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records

Fleming, S. L., Lozano, A., Haberkorn, W. J., Jindal, J. A., Reis, E., Thapa, R., Blankemeier, L., Genkins, J. Z., Steinberg, E., Nayak, A., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence

Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Cited by 66

MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records

Fleming, S. L., Lozano, A., Haberkorn, W. J., Jindal, J. A., Reis, E. P., Thapa, R., Blankemeier, L., Genkins, J. Z., Steinberg, E., Nayak, A., et al. (2023) · arXiv

Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Cited by 65

Association of Hemoglobin A 1c Levels With Use of Sulfonylureas, Dipeptidyl Peptidase 4 Inhibitors, and Thiazolidinediones in Patients With Type 2 Diabetes Treated With Metformin: Analysis From the Observational Health Data Sciences and Informatics Initiative

Vashisht, R., Jung, K., Schuler, A., Banda, J. M., Park, R. W., Jin, S., Li, L., Dudley, J. T., Johnson, K. W., Shervey, M. M., et al. (2018) · JAMA Network Open

Diabetes Treatment and ManagementDiabetes Management and Research

Cited by 61

Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems

Callahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst

Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AI

Cited by 61

Feasibility and evaluation of a large-scale external validation approach for patient-level prediction in an international data network: Validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation

Reps, J. M., Williams, R. D., You, S. C., Falconer, T., Minty, E., Callahan, A., Ryan, P. B., Park, R. W., Lim, H.-S., & Rijnbeek, P. (2020) · BMC Medical Research Methodology

Atrial Fibrillation Management and OutcomesMachine Learning in Healthcare

Cited by 40

Performing an Informatics Consult: Methods and Challenges

Schuler, A., Callahan, A., Jung, K., & Shah, N. H. (2018) · Journal of the American College of Radiology

Electronic Health Records SystemsArtificial Intelligence in Healthcare and Education

Cited by 37

APLUS: A Python library for usefulness simulations of machine learning models in healthcare

Wornow, M., Gyang Ross, E., Callahan, A., & Shah, N. H. (2023) · Journal of Biomedical Informatics

Artificial Intelligence in Healthcare and EducationArtificial Intelligence in Healthcare

Cited by 26

Using Aggregate Patient Data at the Bedside via an On-Demand Consultation Service

Callahan, A., Gombar, S., Cahan, E. M., Jung, K., Steinberg, E., Polony, V., Morse, K., Tibshirani, R., Hastie, T., Harrington, R., et al. (2021) · NEJM Catalyst

Electronic Health Records SystemsMachine Learning in Healthcare

Cited by 24

ACE: the Advanced Cohort Engine for searching longitudinal patient records

Callahan, A., Polony, V., Posada, J. D., Banda, J. M., Gombar, S., & Shah, N. H. (2021) · Journal of the American Medical Informatics Association

Electronic Health Records SystemsMachine Learning in Healthcare

Cited by 22

BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing

Fries, J. A., Weber, L., Seelam, N., Altay, G., Datta, D., Garda, S., Kang, M., Su, R., Kusa, W., Cahyawijaya, S., et al. (2022) · arXiv

Biomedical Text Mining and OntologiesTopic Modeling

Cited by 17

Why and How to Monitor Deployed AI Systems in Health Care

Keyes, T., Callahan, A., Pandya, A. S., Ambers, N., Banda, J. M., Fuentes, M., Lugtu, C., Masariya, P., Nallan, S., O’Brien, C., et al. (2026) · NEJM Catalyst

Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AI

Cited by 10

Monitoring Deployed AI Systems in Health Care

Keyes, T., Callahan, A., Pandya, A. S., Ambers, N., Banda, J. M., Fuentes, M., Lugtu, C., Masariya, P., Nallan, S., O'Brien, C., et al. (2025) · arXiv

Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AI

Cited by 3