Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference最新文献

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care. 支持口腔自我护理的在线强化学习算法的奖励设计

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2023-06-27 DOI: 10.1609/aaai.v37i13.26866

Anna L Trella, Kelly W Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A Murphy

{"title":"Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care.","authors":"Anna L Trella, Kelly W Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A Murphy","doi":"10.1609/aaai.v37i13.26866","DOIUrl":"10.1609/aaai.v37i13.26866","url":null,"abstract":"<p><p>While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.</p>","PeriodicalId":74524,"journal":{"name":"Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference","volume":"37 13","pages":"15724-15730"},"PeriodicalIF":0.0,"publicationDate":"2023-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10457015/pdf/nihms-1851571.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10119179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care 支持口腔自我护理的在线强化学习算法的奖励设计

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2022-08-15 DOI: 10.48550/arXiv.2208.07406

Anna L. Trella, Kelly W. Zhang, I. Nahum-Shani, V. Shetty, F. Doshi-Velez, S. Murphy

{"title":"Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care","authors":"Anna L. Trella, Kelly W. Zhang, I. Nahum-Shani, V. Shetty, F. Doshi-Velez, S. Murphy","doi":"10.48550/arXiv.2208.07406","DOIUrl":"https://doi.org/10.48550/arXiv.2208.07406","url":null,"abstract":"While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.","PeriodicalId":74524,"journal":{"name":"Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference","volume":"37 13 1","pages":"15724-15730"},"PeriodicalIF":0.0,"publicationDate":"2022-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49335687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

THink: Inferring Cognitive Status from Subtle Behaviors. 思考：从细微行为推断认知状态

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2014-07-01

Randall Davis, David J Libon, Rhoda Au, David Pitman, Dana L Penney

引用次数: 0

Statistical Relational Learning to Predict Primary Myocardial Infarction from Electronic Health Records 从电子健康记录中预测原发性心肌梗死的统计关系学习

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2012-07-22 DOI: 10.1609/aaai.v26i2.18981

Jeremy C. Weiss, Sriraam Natarajan, P. Peissig, C. McCarty, David Page

引用次数: 21

Statistical Relational Learning to Predict Primary Myocardial Infarction from Electronic Health Records. 从电子健康记录中预测原发性心肌梗死的统计关系学习。

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2012-01-01

Jeremy C Weiss, David Page, Peggy L Peissig, Sriraam Natarajan, Catherine McCarty

引用次数: 0

Application of the Actor-Critic Architecture to Functional Electrical Stimulation Control of a Human Arm. Actor-Critic架构在人体手臂功能性电刺激控制中的应用。

Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference Pub Date : 2009-01-01

Philip Thomas, Michael Branicky, Antonie van den Bogert, Kathleen Jagodnik

引用次数: 0