决策可靠性与强化学习计算参数

Anahit Mkrtchian, Vincent Valton, Jonathan P. Roiser
{"title":"决策可靠性与强化学习计算参数","authors":"Anahit Mkrtchian, Vincent Valton, Jonathan P. Roiser","doi":"10.5334/cpsy.86","DOIUrl":null,"url":null,"abstract":"Computational models can offer mechanistic insight into cognition and therefore have the potential to transform our understanding of psychiatric disorders and their treatment. For translational efforts to be successful, it is imperative that computational measures capture individual characteristics reliably. To date, this issue has received little consideration. Here we examine the reliability of reinforcement learning and economic models derived from two commonly used tasks. Healthy individuals (N=50) completed a restless four-armed bandit and a calibrated gambling task twice, two weeks apart. Reward and punishment processing parameters from the reinforcement learning model showed fair-to-good reliability, while risk/loss aversion parameters from a prospect theory model exhibited good-to-excellent reliability. Both models were further able to predict future behaviour above chance within individuals. This prediction was better when based on participants’ own model parameters than other participants’ parameter estimates. These results suggest that reinforcement learning, and particularly prospect theory parameters, can be measured reliably to assess learning and decision-making mechanisms, and that these processes may represent relatively distinct computational profiles across individuals. Overall, these findings indicate the translational potential of clinically-relevant computational parameters for precision psychiatry.","PeriodicalId":72664,"journal":{"name":"Computational psychiatry (Cambridge, Mass.)","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2023-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"Reliability of Decision-Making and Reinforcement Learning Computational Parameters\",\"authors\":\"Anahit Mkrtchian, Vincent Valton, Jonathan P. Roiser\",\"doi\":\"10.5334/cpsy.86\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Computational models can offer mechanistic insight into cognition and therefore have the potential to transform our understanding of psychiatric disorders and their treatment. For translational efforts to be successful, it is imperative that computational measures capture individual characteristics reliably. To date, this issue has received little consideration. Here we examine the reliability of reinforcement learning and economic models derived from two commonly used tasks. Healthy individuals (N=50) completed a restless four-armed bandit and a calibrated gambling task twice, two weeks apart. Reward and punishment processing parameters from the reinforcement learning model showed fair-to-good reliability, while risk/loss aversion parameters from a prospect theory model exhibited good-to-excellent reliability. Both models were further able to predict future behaviour above chance within individuals. This prediction was better when based on participants’ own model parameters than other participants’ parameter estimates. These results suggest that reinforcement learning, and particularly prospect theory parameters, can be measured reliably to assess learning and decision-making mechanisms, and that these processes may represent relatively distinct computational profiles across individuals. Overall, these findings indicate the translational potential of clinically-relevant computational parameters for precision psychiatry.\",\"PeriodicalId\":72664,\"journal\":{\"name\":\"Computational psychiatry (Cambridge, Mass.)\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-02-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computational psychiatry (Cambridge, Mass.)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.5334/cpsy.86\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computational psychiatry (Cambridge, Mass.)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.5334/cpsy.86","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2

摘要

计算模型可以提供对认知的机械洞察,因此有可能改变我们对精神疾病及其治疗的理解。为了使翻译工作取得成功,计算度量可靠地捕获个体特征是必要的。到目前为止,这个问题几乎没有得到考虑。在这里,我们检验了强化学习和经济模型的可靠性,这些模型来源于两个常用的任务。健康个体(N=50)完成了两次不安分的四臂强盗和校准的赌博任务,间隔两周。强化学习模型的奖惩加工参数具有从一般到优良的信度,前景理论模型的风险/损失厌恶加工参数具有从优良到优良的信度。这两个模型都能进一步预测个体的未来行为。这种预测是基于参与者自己的模型参数比其他参与者的参数估计更好。这些结果表明,可以可靠地测量强化学习,特别是前景理论参数,以评估学习和决策机制,并且这些过程可能代表个体之间相对不同的计算特征。总的来说,这些发现表明了精确精神病学的临床相关计算参数的转化潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Reliability of Decision-Making and Reinforcement Learning Computational Parameters
Computational models can offer mechanistic insight into cognition and therefore have the potential to transform our understanding of psychiatric disorders and their treatment. For translational efforts to be successful, it is imperative that computational measures capture individual characteristics reliably. To date, this issue has received little consideration. Here we examine the reliability of reinforcement learning and economic models derived from two commonly used tasks. Healthy individuals (N=50) completed a restless four-armed bandit and a calibrated gambling task twice, two weeks apart. Reward and punishment processing parameters from the reinforcement learning model showed fair-to-good reliability, while risk/loss aversion parameters from a prospect theory model exhibited good-to-excellent reliability. Both models were further able to predict future behaviour above chance within individuals. This prediction was better when based on participants’ own model parameters than other participants’ parameter estimates. These results suggest that reinforcement learning, and particularly prospect theory parameters, can be measured reliably to assess learning and decision-making mechanisms, and that these processes may represent relatively distinct computational profiles across individuals. Overall, these findings indicate the translational potential of clinically-relevant computational parameters for precision psychiatry.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
4.30
自引率
0.00%
发文量
0
审稿时长
17 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信