The scope and limits of fine-grained image and category information in the ventral visual pathway.

IF 4.4 2区 医学 Q1 NEUROSCIENCES
Markus W Badwal, Johanna Bergmann, Johannes Roth, Christian F Doeller, Martin N Hebart
{"title":"The scope and limits of fine-grained image and category information in the ventral visual pathway.","authors":"Markus W Badwal, Johanna Bergmann, Johannes Roth, Christian F Doeller, Martin N Hebart","doi":"10.1523/JNEUROSCI.0936-24.2024","DOIUrl":null,"url":null,"abstract":"<p><p>Humans can easily abstract incoming visual information into discrete semantic categories. Previous research employing functional MRI (fMRI) in humans has identified cortical organizing principles that allow not only for coarse-scale distinctions such as animate versus inanimate objects but also more fine-grained distinctions at the level of individual objects. This suggests that fMRI carries rather fine-grained information about individual objects. However, most previous work investigating fine-grained category representations either additionally included coarse-scale category comparisons of objects, which confounds fine-grained and coarse-scale distinctions, or only used a single exemplar of each object, which confounds visual and semantic information. To address these challenges, here we used multisession human fMRI (female and male) paired with a broad yet homogenous stimulus class of 48 terrestrial mammals, with 2 exemplars per mammal. Multivariate decoding and representational similarity analysis (RSA) revealed high image-specific reliability in low- and high-level visual regions, indicating stable representational patterns at the image level. In contrast, analyses across exemplars of the same animal yielded only small effects in the lateral occipital complex (LOC), indicating rather subtle category effects in this region. Variance partitioning with a deep neural network and shape model showed that across exemplar effects in EVC were largely explained by low-level visual appearance, while representations in LOC appeared to also contain higher category-specific information. These results suggest that representations typically measured with fMRI are dominated by image-specific visual or coarse-grained category information but indicate that commonly employed fMRI protocols may reveal subtle yet reliable distinctions between individual objects.<b>Significance Statement</b> While it has been suggested that functional MRI (fMRI) responses in ventral visual cortex carry fine-grained information about individual objects, much previous research has confounded fine-grained with coarse-scale category information or only used individual visual exemplars, which potentially confounds semantic and visual object information. Here we address these challenges in a multisession fMRI study where participants viewed a highly homogenous stimulus set of 48 land mammals with 2 exemplars per animal. Our results reveal a strong dominance of image-specific effects and additionally indicate subtle yet reliable category-specific effects in lateral occipital complex, underscoring the capacity of commonly employed fMRI protocols to uncover fine-grained visual information.</p>","PeriodicalId":50114,"journal":{"name":"Journal of Neuroscience","volume":null,"pages":null},"PeriodicalIF":4.4000,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Neuroscience","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1523/JNEUROSCI.0936-24.2024","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

Humans can easily abstract incoming visual information into discrete semantic categories. Previous research employing functional MRI (fMRI) in humans has identified cortical organizing principles that allow not only for coarse-scale distinctions such as animate versus inanimate objects but also more fine-grained distinctions at the level of individual objects. This suggests that fMRI carries rather fine-grained information about individual objects. However, most previous work investigating fine-grained category representations either additionally included coarse-scale category comparisons of objects, which confounds fine-grained and coarse-scale distinctions, or only used a single exemplar of each object, which confounds visual and semantic information. To address these challenges, here we used multisession human fMRI (female and male) paired with a broad yet homogenous stimulus class of 48 terrestrial mammals, with 2 exemplars per mammal. Multivariate decoding and representational similarity analysis (RSA) revealed high image-specific reliability in low- and high-level visual regions, indicating stable representational patterns at the image level. In contrast, analyses across exemplars of the same animal yielded only small effects in the lateral occipital complex (LOC), indicating rather subtle category effects in this region. Variance partitioning with a deep neural network and shape model showed that across exemplar effects in EVC were largely explained by low-level visual appearance, while representations in LOC appeared to also contain higher category-specific information. These results suggest that representations typically measured with fMRI are dominated by image-specific visual or coarse-grained category information but indicate that commonly employed fMRI protocols may reveal subtle yet reliable distinctions between individual objects.Significance Statement While it has been suggested that functional MRI (fMRI) responses in ventral visual cortex carry fine-grained information about individual objects, much previous research has confounded fine-grained with coarse-scale category information or only used individual visual exemplars, which potentially confounds semantic and visual object information. Here we address these challenges in a multisession fMRI study where participants viewed a highly homogenous stimulus set of 48 land mammals with 2 exemplars per animal. Our results reveal a strong dominance of image-specific effects and additionally indicate subtle yet reliable category-specific effects in lateral occipital complex, underscoring the capacity of commonly employed fMRI protocols to uncover fine-grained visual information.

腹侧视觉通路中细粒度图像和类别信息的范围和局限。
人类可以轻松地将接收到的视觉信息抽象为离散的语义类别。此前对人类进行的功能性核磁共振成像(fMRI)研究发现,大脑皮层的组织原则不仅允许粗略区分有生命和无生命的物体,还允许在单个物体的层次上进行更精细的区分。这表明,fMRI 可携带有关单个物体的精细信息。然而,之前大多数研究细粒度类别表征的工作要么额外包含了物体的粗尺度类别比较,从而混淆了细粒度和粗尺度的区分;要么只使用了每个物体的单个示例,从而混淆了视觉和语义信息。为了应对这些挑战,我们在这里使用了多期人类 fMRI(女性和男性)与广泛但同质的刺激类别(48 种陆生哺乳动物)配对,每种哺乳动物使用 2 个示例。多变量解码和表征相似性分析(RSA)显示,在低级和高级视觉区域中,特定图像的可靠性很高,这表明图像水平上的表征模式是稳定的。与此相反,对同一动物的不同示例进行的分析仅在侧枕复合体(LOC)中产生了微小的影响,这表明该区域存在相当微妙的类别效应。利用深度神经网络和形状模型进行的方差划分表明,EVC 中的跨范例效应主要是由低级视觉外观解释的,而 LOC 中的表征似乎也包含较高的特定类别信息。这些结果表明,通常用fMRI测量的表征是由图像特异性视觉或粗粒度类别信息主导的,但也表明,常用的fMRI方案可能会揭示单个物体之间微妙而可靠的区别。意义声明 虽然有人认为,腹侧视觉皮层的功能磁共振成像(fMRI)反应携带着关于单个物体的细粒度信息,但之前的许多研究都将细粒度和粗粒度类别信息混为一谈,或者只使用单个视觉范例,这可能会混淆语义和视觉物体信息。在这里,我们通过一项多期 fMRI 研究来解决这些难题。在这项研究中,参与者观看了由 48 种陆地哺乳动物组成的高度同质化的刺激集,每种动物有 2 个范例。我们的研究结果表明,图像特异性效应占主导地位,此外,外侧枕叶复合体还显示出微妙但可靠的类别特异性效应,这突出表明常用的 fMRI 方案有能力揭示细粒度的视觉信息。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Neuroscience
Journal of Neuroscience 医学-神经科学
CiteScore
9.30
自引率
3.80%
发文量
1164
审稿时长
12 months
期刊介绍: JNeurosci (ISSN 0270-6474) is an official journal of the Society for Neuroscience. It is published weekly by the Society, fifty weeks a year, one volume a year. JNeurosci publishes papers on a broad range of topics of general interest to those working on the nervous system. Authors now have an Open Choice option for their published articles
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信