解释人工智能:理解遗产点云的深度学习模型

Q2 Environmental Science
F. Matrone, A. Felicetti, M. Paolanti, R. Pierdicca
{"title":"解释人工智能:理解遗产点云的深度学习模型","authors":"F. Matrone, A. Felicetti, M. Paolanti, R. Pierdicca","doi":"10.5194/isprs-annals-x-m-1-2023-207-2023","DOIUrl":null,"url":null,"abstract":"Abstract. Deep Learning has been pivotal in many real-world applications (e.g., autonomous driving, medicine and retail). With the wide availability of consumer-grade depth sensors, acquiring 3D data has become more affordable and effective, and many 3D datasets are currently publicly available. 3D data provides a great opportunity for a better comprehension of the surrounding environment for machines. There is a growing need for innovative methods for the treatment and analysis of point clouds and for their classification. The complex hidden layers, which are at the basis of deep neural networks (DNNs), make it difficult to interpret these models, that up to a few years ago DNNs were considered and treated as black box operators. Still, with their increasing popularity, making them explainable and interpretable has become mandatory. A lot of efforts were devoted to developing an Explainable Artificial Intelligence (XAI) framework for explaining DNNs decisions with 2D data, while only a few studies have attempted to investigate the explainability of 3D DNNs and, even more, heritage scenarios. To overcome these limitations, it was proposed the BubblEX framework: a novel multimodal fusion framework to learn the 3D point features. In our work, BubblEX has been exploited to understand the decisions taken by DNNs for heritage point clouds. The approach has been applied to a Digital Cultural Heritage Dataset, which is publicly available: the ArCH (Architectural Cultural Heritage) Dataset.\n","PeriodicalId":30633,"journal":{"name":"ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences","volume":"114 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2023-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"EXPLAINING AI: UNDERSTANDING DEEP LEARNING MODELS FOR HERITAGE POINT CLOUDS\",\"authors\":\"F. Matrone, A. Felicetti, M. Paolanti, R. Pierdicca\",\"doi\":\"10.5194/isprs-annals-x-m-1-2023-207-2023\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract. Deep Learning has been pivotal in many real-world applications (e.g., autonomous driving, medicine and retail). With the wide availability of consumer-grade depth sensors, acquiring 3D data has become more affordable and effective, and many 3D datasets are currently publicly available. 3D data provides a great opportunity for a better comprehension of the surrounding environment for machines. There is a growing need for innovative methods for the treatment and analysis of point clouds and for their classification. The complex hidden layers, which are at the basis of deep neural networks (DNNs), make it difficult to interpret these models, that up to a few years ago DNNs were considered and treated as black box operators. Still, with their increasing popularity, making them explainable and interpretable has become mandatory. A lot of efforts were devoted to developing an Explainable Artificial Intelligence (XAI) framework for explaining DNNs decisions with 2D data, while only a few studies have attempted to investigate the explainability of 3D DNNs and, even more, heritage scenarios. To overcome these limitations, it was proposed the BubblEX framework: a novel multimodal fusion framework to learn the 3D point features. In our work, BubblEX has been exploited to understand the decisions taken by DNNs for heritage point clouds. The approach has been applied to a Digital Cultural Heritage Dataset, which is publicly available: the ArCH (Architectural Cultural Heritage) Dataset.\\n\",\"PeriodicalId\":30633,\"journal\":{\"name\":\"ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences\",\"volume\":\"114 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-06-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.5194/isprs-annals-x-m-1-2023-207-2023\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Environmental Science\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.5194/isprs-annals-x-m-1-2023-207-2023","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Environmental Science","Score":null,"Total":0}
引用次数: 0

摘要

摘要深度学习在许多现实世界的应用中都是至关重要的(例如,自动驾驶、医药和零售)。随着消费级深度传感器的广泛使用,获取3D数据变得更加实惠和有效,许多3D数据集目前都是公开可用的。3D数据为机器更好地理解周围环境提供了一个很好的机会。越来越需要创新的方法来处理和分析点云并对其进行分类。复杂的隐藏层是深度神经网络(dnn)的基础,这使得解释这些模型变得困难,直到几年前,dnn还被认为是黑箱算子。尽管如此,随着它们越来越受欢迎,使它们变得可解释和可解释已成为强制性的。许多研究都致力于开发一个可解释的人工智能(XAI)框架,用于用2D数据解释dnn的决策,而只有少数研究试图调查3D dnn的可解释性,甚至更多,传统场景。为了克服这些局限性,提出了一种新的多模态融合框架——BubblEX框架来学习三维点特征。在我们的工作中,我们利用BubblEX来理解dnn对传统点云所做的决定。该方法已应用于一个公开的数字文化遗产数据集:ArCH(建筑文化遗产)数据集。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
EXPLAINING AI: UNDERSTANDING DEEP LEARNING MODELS FOR HERITAGE POINT CLOUDS
Abstract. Deep Learning has been pivotal in many real-world applications (e.g., autonomous driving, medicine and retail). With the wide availability of consumer-grade depth sensors, acquiring 3D data has become more affordable and effective, and many 3D datasets are currently publicly available. 3D data provides a great opportunity for a better comprehension of the surrounding environment for machines. There is a growing need for innovative methods for the treatment and analysis of point clouds and for their classification. The complex hidden layers, which are at the basis of deep neural networks (DNNs), make it difficult to interpret these models, that up to a few years ago DNNs were considered and treated as black box operators. Still, with their increasing popularity, making them explainable and interpretable has become mandatory. A lot of efforts were devoted to developing an Explainable Artificial Intelligence (XAI) framework for explaining DNNs decisions with 2D data, while only a few studies have attempted to investigate the explainability of 3D DNNs and, even more, heritage scenarios. To overcome these limitations, it was proposed the BubblEX framework: a novel multimodal fusion framework to learn the 3D point features. In our work, BubblEX has been exploited to understand the decisions taken by DNNs for heritage point clouds. The approach has been applied to a Digital Cultural Heritage Dataset, which is publicly available: the ArCH (Architectural Cultural Heritage) Dataset.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences
ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences Environmental Science-Environmental Science (miscellaneous)
CiteScore
2.00
自引率
0.00%
发文量
0
审稿时长
16 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信