Harnessing real world behavior data to optimize treatment delivery
3 - (Sym 120) Assessing Personalization in Digital Health
Sunday, November 20, 2022
10:00 AM – 11:30 AM EST
Location: Broadhurst/Belasco, 5th Floor
Keywords: Technology / Mobile Health, Treatment Development, Translational Research Recommended Readings: Areán, P. A., Pratap, A., Hsin, H., Huppert, T. K., Hendricks, K. E., Heagerty, P. J., ... & Comtois, K. A. (2021). Perceived utility and characterization of personal google search histories to detect data patterns proximal to a suicide attempt in individuals who previously attempted suicide: pilot cohort study. Journal of medical Internet research, 23(5), e27918. Evans, H., Lakshmi, U., Watson, H., Ismail, A., Sherrill, A. M., Kumar, N., & Arriaga, R. I. (2020, April). Understanding the Care Ecologies of Veterans with PTSD. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-15). Liu, T., Meyerhoff, J., Eichstaedt, J. C., Karr, C. J., Kaiser, S. M., Kording, K. P., ... & Ungar, L. H. (2022). The relationship between text message sentiment and self-reported depression. Journal of affective disorders, 302, 7-14.
Artificial Intelligence algorithms, such as reinforcement learning algorithms, provide an attractive suite of online learning methods for personalizing intervention delivery in Digital Health. However, after an algorithm has been run in a clinical study, how do we assess whether personalization occurred? We might find individuals for whom it appears that the algorithm has indeed learned and in which settings the individual is more responsive to a particular intervention message. But could this have happened completely by chance? Consider the clinical trial, HeartSteps in which we implemented a Bayesian online reinforcement learning algorithm in order to personalize intervention delivery. In this talk we apply classical meta-analysis methods from the clinical trials literature to assess, using HeartSteps data, whether individuals respond on average to the interventions as well as to assess treatment effect heterogeneity. Further we use bootstrap methods from statistics to conduct exploratory analyses concerning whether for a given individual, the reinforcement algorithm does indeed personalize intervention delivery.