运动技能习得中的强化学习:利用奖励积极性了解短期和长期行为适应的内在机制。

IF 2.6 3区 医学 Q2 BEHAVIORAL SCIENCES
Frontiers in Behavioral Neuroscience Pub Date : 2024-10-30 eCollection Date: 2024-01-01 DOI:10.3389/fnbeh.2024.1466970
Mariane F B Bacelar, Keith R Lohse, Juliana O Parma, Matthew W Miller
{"title":"运动技能习得中的强化学习:利用奖励积极性了解短期和长期行为适应的内在机制。","authors":"Mariane F B Bacelar, Keith R Lohse, Juliana O Parma, Matthew W Miller","doi":"10.3389/fnbeh.2024.1466970","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition.</p><p><strong>Methods: </strong>In the present study, we explored a 134-person dataset consisting of learners' feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions.</p><p><strong>Results: </strong>Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners' performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test.</p><p><strong>Discussion: </strong>Together, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon.</p>","PeriodicalId":12368,"journal":{"name":"Frontiers in Behavioral Neuroscience","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11557390/pdf/","citationCount":"0","resultStr":"{\"title\":\"Reinforcement learning in motor skill acquisition: using the reward positivity to understand the mechanisms underlying short- and long-term behavior adaptation.\",\"authors\":\"Mariane F B Bacelar, Keith R Lohse, Juliana O Parma, Matthew W Miller\",\"doi\":\"10.3389/fnbeh.2024.1466970\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Introduction: </strong>According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition.</p><p><strong>Methods: </strong>In the present study, we explored a 134-person dataset consisting of learners' feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions.</p><p><strong>Results: </strong>Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners' performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test.</p><p><strong>Discussion: </strong>Together, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon.</p>\",\"PeriodicalId\":12368,\"journal\":{\"name\":\"Frontiers in Behavioral Neuroscience\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2024-10-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11557390/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Frontiers in Behavioral Neuroscience\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.3389/fnbeh.2024.1466970\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/1/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"Q2\",\"JCRName\":\"BEHAVIORAL SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers in Behavioral Neuroscience","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3389/fnbeh.2024.1466970","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/1/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"BEHAVIORAL SCIENCES","Score":null,"Total":0}
引用次数: 0

摘要

引言根据强化学习理论,人类会根据实际结果与预期结果之间的差异(即预测误差)来调整自己的行为,其主要目的是通过行动获得最大回报。尽管强化学习为我们理解如何习得运动技能提供了一个强有力的理论框架,但很少有研究对强化学习预测及其在运动技能习得中的内在机制进行调查:在本研究中,我们探索了一个由134人组成的数据集,其中包括学习者在练习阶段和延迟保持测试期间的反馈诱发大脑活动(奖励积极性;RewP)和运动准确性,以研究这些变量是否与强化学习预测相互影响:结果表明,RewP与试验准确性之间存在非线性关系,这种关系受学习者成绩水平的调节。具体来说,与成绩差的学习者相比,成绩好的学习者对违反奖励预期的情况更敏感,这可能是因为他们对技能有了更强的表征,能够依赖更稳定的结果预测。此外,与我们的预测相反,习得过程中的平均RewP并不能预测延迟保留测试的成绩:总之,这些研究结果支持使用强化学习模型来理解短期行为适应,并强调了运动技能巩固过程的复杂性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Reinforcement learning in motor skill acquisition: using the reward positivity to understand the mechanisms underlying short- and long-term behavior adaptation.

Introduction: According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition.

Methods: In the present study, we explored a 134-person dataset consisting of learners' feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions.

Results: Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners' performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test.

Discussion: Together, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Frontiers in Behavioral Neuroscience
Frontiers in Behavioral Neuroscience BEHAVIORAL SCIENCES-NEUROSCIENCES
CiteScore
4.70
自引率
3.30%
发文量
506
审稿时长
6-12 weeks
期刊介绍: Frontiers in Behavioral Neuroscience is a leading journal in its field, publishing rigorously peer-reviewed research that advances our understanding of the neural mechanisms underlying behavior. Field Chief Editor Nuno Sousa at the Instituto de Pesquisa em Ciências da Vida e da Saúde (ICVS) is supported by an outstanding Editorial Board of international experts. This multidisciplinary open-access journal is at the forefront of disseminating and communicating scientific knowledge and impactful discoveries to researchers, academics, clinicians and the public worldwide. This journal publishes major insights into the neural mechanisms of animal and human behavior, and welcomes articles studying the interplay between behavior and its neurobiological basis at all levels: from molecular biology and genetics, to morphological, biochemical, neurochemical, electrophysiological, neuroendocrine, pharmacological, and neuroimaging studies.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信