From Deployment Challenges to Safer Generative AI: Evaluating and Improving LLMs for Health Systems

Monday, November 3, 2025 – 2:00 pm CT

About the Webinar:

Large language models (LLMs) are increasingly being used in healthcare, often built into electronic medical records and decision-support tools as pre-trained components. Our analyses of their real-world use in both private and safety-net health systems show consistent patterns of where and why they fail to generalize or perform robustly. Building on these findings, we introduce approaches to reduce factual errors and improve model performance in clinical tasks, along with evaluation methods for assessing performance after deployment in real healthcare settings. Together, these efforts show that realizing the potential of LLMs in healthcare requires not only better model design but also systematic, context-aware evaluation to ensure they work safely and effectively across diverse health systems.

About the Presenter:

Dr. Rumi Chunara develops computational and statistical approaches to acquire, integrate, and use data to improve population health. At NYU, she directs the Center for Health Data Science, advancing methods for public and population health while building knowledge ecosystems that connect disciplines and contexts. Previously, Dr. Chunara was a Postdoctoral Fellow and Instructor at HealthMap and the Children’s Hospital Informatics Program at Harvard Medical School. She earned her PhD from the Harvard-MIT Division of Health Sciences and Technology, her SM from MIT and her BSc from Caltech. Her honors include the NSF CAREER Award, Facebook Research Award, Gates Foundation Grand Challenges Award, MIT Technology Review Top 35 Innovators Under 35, and a Max Planck Society Sabbatical Award.. More about Rumi Chunara.