Designing Reinforcement Learning Algorithms for Mobile Health
November 21, 2022
Anna Trella
PhD Candidate in Computer Science
Harvard University
About the Webinar:
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To allow domain experts to have confidence that the RL algorithm they deploy can learn effectively in these challenging environments, we offer 1) a trustworthy and generalizable framework for designing and comprehensively evaluating RL algorithms in a principled manner and 2) reward design that generalizes the bandit algorithm to consider the impact of the current decision on the future. Further, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the framework. We illustrate the use of the framework and reward design for developing an RL algorithm for Oralytics, a mobile health study aiming to improve users’ tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in early 2023.
About Anna Trella:
Anna L. Trella is a Ph.D. candidate in computer science at Harvard University advised by Finale Doshi-Velez and Susan A. Murphy. Her research interests include developing and deploying reinforcement learning systems in mobile health that improve patient health outcomes and allow domain experts to answer scientific questions. Before coming to Harvard, she was a technical lead and software development engineer at Amazon, developing backend, desktop, mobile web, and mobile app (native and react native) platforms for the configurable, contextual, and personalized navigation experience for Wholefoods Market, Prime Now, and Amazon Fresh. More about Anna Trella.