Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT

IF 3.6 2区 医学 Q1 HEALTH CARE SCIENCES & SERVICES
Nils Hentati Isacsson , Kirsten Zantvoort , Erik Forsell , Magnus Boman , Viktor Kaldo
{"title":"Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT","authors":"Nils Hentati Isacsson ,&nbsp;Kirsten Zantvoort ,&nbsp;Erik Forsell ,&nbsp;Magnus Boman ,&nbsp;Viktor Kaldo","doi":"10.1016/j.invent.2024.100773","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><p>Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment.</p></div><div><h3>Methods</h3><p>We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods.</p></div><div><h3>Results</h3><p>The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %).</p></div><div><h3>Conclusion</h3><p>Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.</p></div>","PeriodicalId":48615,"journal":{"name":"Internet Interventions-The Application of Information Technology in Mental and Behavioural Health","volume":null,"pages":null},"PeriodicalIF":3.6000,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2214782924000666/pdfft?md5=1990aa321535e76c003880576de54367&pid=1-s2.0-S2214782924000666-main.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Internet Interventions-The Application of Information Technology in Mental and Behavioural Health","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214782924000666","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Objective

Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment.

Methods

We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods.

Results

The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %).

Conclusion

Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.

充分利用时间序列症状数据:基于互联网的 CBT 症状预测机器学习研究
目的预测哪些人在早期无法从基于互联网的认知行为疗法(ICBT)中获得足够的益处,有助于更好地分配有限的心理保健资源。治疗期间症状的重复测量是预测疗效的最有力指标,我们希望通过以下数据来研究明确考虑时间依赖性的方法是否优于不考虑时间依赖性的方法:(a) 治疗前只有两个时间点的数据;(b) 治疗前时间点和初始治疗期间三个时间点的数据、我们使用 1) 常用的时间无关方法(即线性回归和随机森林模型)和 2) 时间相关方法(即多层次模型回归、混合效应随机森林和长短期记忆模型)来预测治疗期间的症状,包括最终结果。结果模型预测治疗后结果的均方根误差(RMSE)为14%-12%,平衡准确率为67%-74%。与时间相关的模型没有更高的准确率。使用初始治疗期(b)的数据,而不是仅使用治疗前(a)的数据,预测结果的均方误差(RMSE)增加了 1.3 个百分点(12% 至 10.7%),BACC 增加了 6 个百分点(69% 至 75%)。为了更好地理解模型复杂性、数据集长度和宽度与当前预测任务之间的相互作用,有必要开展进一步的研究。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.50
自引率
9.30%
发文量
94
审稿时长
6 weeks
期刊介绍: Official Journal of the European Society for Research on Internet Interventions (ESRII) and the International Society for Research on Internet Interventions (ISRII). The aim of Internet Interventions is to publish scientific, peer-reviewed, high-impact research on Internet interventions and related areas. Internet Interventions welcomes papers on the following subjects: • Intervention studies targeting the promotion of mental health and featuring the Internet and/or technologies using the Internet as an underlying technology, e.g. computers, smartphone devices, tablets, sensors • Implementation and dissemination of Internet interventions • Integration of Internet interventions into existing systems of care • Descriptions of development and deployment infrastructures • Internet intervention methodology and theory papers • Internet-based epidemiology • Descriptions of new Internet-based technologies and experiments with clinical applications • Economics of internet interventions (cost-effectiveness) • Health care policy and Internet interventions • The role of culture in Internet intervention • Internet psychometrics • Ethical issues pertaining to Internet interventions and measurements • Human-computer interaction and usability research with clinical implications • Systematic reviews and meta-analysis on Internet interventions
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信