Transactions of The Japanese Society for Artificial Intelligence最新文献_第4页

Empirical Analysis of Methods for Evaluating Faithfulness of Explanations by Feature Attribution 特征归因解释可信度评价方法的实证分析

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-11-01 DOI: 10.1527/tjsai.38-6_c-n22

Yuya Asazuma, Kazuaki Hanawa, Kentaro Inui

{"title":"Empirical Analysis of Methods for Evaluating Faithfulness of Explanations by Feature Attribution","authors":"Yuya Asazuma, Kazuaki Hanawa, Kentaro Inui","doi":"10.1527/tjsai.38-6_c-n22","DOIUrl":"https://doi.org/10.1527/tjsai.38-6_c-n22","url":null,"abstract":"Many high-performance machine learning models in the real world exhibit the black box problem. This issue is widely recognized as needing output reliability and model transparency. XAI (Explainable AI) represents a research field that addresses this issue. Within XAI, feature attribution methods, which clarify the importance of features irrespective of the task or model type, have become a central focus. Evaluating their efficacy based on empirical evidence is essential when proposing new methods. However, extensive debate exists regarding the properties that importance should be possessed, and a consensus on specific evaluation methods remains elusive. Given this context, many existing studies adopt their evaluation techniques, leading to fragmented discussions. This study aims to ”evaluate the evaluation methods,” focusing mainly on the faithfulness metric, deemed especially significant in evaluation criteria. We conducted empirical experiments related to existing evaluation techniques. The experiments approached the topic from two angles: correlation-based comparative evaluations and property verification using random sequences. In the former experiment, we investigated the correlation between faithfulness evaluation tests using numerous models and feature attribution methods. As a result, we found that very few test combinations exhibited high correlation, and many combinations showed low or no correlation. In the latter experiment, we observed that the measured faithfulness varied depending on the model and dataset by using random sequences instead of feature attribution methods to verify the properties of the faithfulness tests.","PeriodicalId":23256,"journal":{"name":"Transactions of The Japanese Society for Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135161382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domains 领域提示学习方法使CLIP有效适应未知领域

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-11-01 DOI: 10.1527/tjsai.38-6_b-mc2

Xin Zhang, Shixiang Shane Gu, Yutaka Matsuo, Yusuke Iwasawa

{"title":"Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domains","authors":"Xin Zhang, Shixiang Shane Gu, Yutaka Matsuo, Yusuke Iwasawa","doi":"10.1527/tjsai.38-6_b-mc2","DOIUrl":"https://doi.org/10.1527/tjsai.38-6_b-mc2","url":null,"abstract":"Domain generalization (DG) is a difficult transfer learning problem aiming to learn a generalizable model for unseen domains. Recent foundation models (FMs) are robust to many distribution shifts and, therefore, should substantially improve the performance of DG. In this work, we study generic ways to adopt contrastive languageimage pre-training (CLIP), a visual-language foundation model, for DG problems in image classification. While empirical risk minimization (ERM) greatly improves the accuracy with bigger backbones and training datasets using standard DG benchmarks, fine-tuning FMs is not practical in many real-world situations. We propose Domain Prompt Learning (DPL) as a novel approach for domain inference in the form of conditional prompt generation. DPL achieved a significant accuracy improvement with only training a lightweight prompt generator (a three-layer MLP), whose parameter is of equivalent scale to the classification projector in the previous DG literature. Combining DPL with CLIP provides surprising performance, raising the accuracy of zero-shot CLIP from 73.7% to 79.3% on several standard datasets, namely PACS, VLCS, OfficeHome, and TerraIncognita. We hope the simplicity and success of our approach lead to broader adoption and analysis of foundation models in the domain generalization field.","PeriodicalId":23256,"journal":{"name":"Transactions of The Japanese Society for Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135161241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

Detecting Fraudulent Cryptocurrencies Using Natural Language Processing Techniques 使用自然语言处理技术检测欺诈性加密货币

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-09-01 DOI: 10.1527/tjsai.38-5_e-n34

Mitsuaki Ueno, Takashi Sano, H. Honda, Shugo Nakamura

引用次数: 0

設計根拠に注目した鉄鋼材料設計知識モデルの記述と活用关注设计依据的钢铁材料设计知识模型的描述和活用

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-09-01 DOI: 10.1527/tjsai.38-5_c-mc1

Yoshinobu Kitamura, Junta Fujikawa, Masaaki Imazono, Kazuya Asano, Toru Inazumi, Taro Kizu, Yoshimasa Funakawa, Mayumi Ojima, Yukinori Iizuka

引用次数: 0

Diverse Pose Generation for Multi-Joint Characters Conditional on Body Position using IMLE 基于IMLE的基于体位条件的多关节角色多姿态生成

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-09-01 DOI: 10.1527/tjsai.38-5_b-mb2

Kazuma Sasaki, Shinya Kitaoka, Yuri Odagiri

引用次数: 0

Model-Based Reinforcement Learning using Model Mediator in Dynamic Multi-Agent Environment 动态多智能体环境中基于模型的模型中介强化学习

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-09-01 DOI: 10.1527/tjsai.38-5_a-mb1

Shota Imai, Yusuke Iwasawa, Yutaka Matsuo

引用次数: 0

Estimating Counterfactual Outcomes of Time-varying Treatments using Deep Gaussian Process 用深度高斯过程估计时变处理的反事实结果

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-09-01 DOI: 10.1527/tjsai.38-5_d-mc3

Y. Norimatsu

引用次数: 0

A Proposal of Response Continuity Prediction Model for Attentive Listening Agents 一种专注听力主体反应连续性预测模型的提出

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-07-01 DOI: 10.1527/tjsai.38-4_c-mb5

Xing Yan, Yasuharu Den

引用次数: 0

日本語文法誤り訂正における事前学習済みモデルを用いたデータ増強日语语法纠错中使用事先学习完毕模型的数据增强

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-07-01 DOI: 10.1527/tjsai.38-4_a-l41

Hideyoshi Kato, Masaaki Okabe, Michiharu Kitano, Hiroshi Yadohisa

引用次数: 0

The Effect of Increasing Number of Topics to Polarization and Echo Chambers on SNS 极化和回声室主题数量增加对SNS的影响

Transactions of The Japanese Society for Artificial Intelligence Pub Date : 2023-07-01 DOI: 10.1527/tjsai.38-4_b-n11

T. Nagura, E. Akiyama

引用次数: 0