ATA: An Abstract-Train-Abstract approach for explanation-friendly deep reinforcement learning

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks Pub Date : 2025-06-21 DOI:10.1016/j.neunet.2025.107749

Shi Peng , Si Liu , Dapeng Zhi , Peixin Wang , Chenyang Xu , Cheng Chen , Min Zhang

{"title":"ATA: An Abstract-Train-Abstract approach for explanation-friendly deep reinforcement learning","authors":"Shi Peng , Si Liu , Dapeng Zhi , Peixin Wang , Chenyang Xu , Cheng Chen , Min Zhang","doi":"10.1016/j.neunet.2025.107749","DOIUrl":null,"url":null,"abstract":"<div><div>Explaining decision-making neural network models in deep reinforcement learning (DRL) systems is crucial, albeit challenging. Abstract policy graphs (APGs) emerge as an effective method for elucidating these models. However, constructing highly explainable APGs with high-fidelity is challenging. Through empirical analysis, we glean an insight that a larger cluster size corresponds to an APG with higher fidelity. We present a novel approach called <em>Abstract-Train-Abstract</em> (ATA), building on the integration of two key ideas. <em>Abstraction-based training</em> facilitates the clustering of abstract states, expanding the scope of each cluster. <em>Abstraction-oriented clustering</em> ensures that states within the same cluster correspond to the same action. Identifying the cluster to which a state belongs enhances the accuracy of predicting its associated action. Our experiments show that ATA surpasses the state of the art, achieving up to 26.63% higher fidelity, while still preserving competitive rewards. Additionally, our user study demonstrates that ATA substantially improves the accuracy of user prediction by 35.7% on average.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107749"},"PeriodicalIF":6.0000,"publicationDate":"2025-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S089360802500629X","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Explaining decision-making neural network models in deep reinforcement learning (DRL) systems is crucial, albeit challenging. Abstract policy graphs (APGs) emerge as an effective method for elucidating these models. However, constructing highly explainable APGs with high-fidelity is challenging. Through empirical analysis, we glean an insight that a larger cluster size corresponds to an APG with higher fidelity. We present a novel approach called Abstract-Train-Abstract (ATA), building on the integration of two key ideas. Abstraction-based training facilitates the clustering of abstract states, expanding the scope of each cluster. Abstraction-oriented clustering ensures that states within the same cluster correspond to the same action. Identifying the cluster to which a state belongs enhances the accuracy of predicting its associated action. Our experiments show that ATA surpasses the state of the art, achieving up to 26.63% higher fidelity, while still preserving competitive rewards. Additionally, our user study demonstrates that ATA substantially improves the accuracy of user prediction by 35.7% on average.

查看原文本刊更多论文

一种用于解释友好型深度强化学习的抽象-训练-抽象方法

解释深度强化学习（DRL）系统中的决策神经网络模型至关重要，尽管具有挑战性。抽象策略图（apg）作为一种阐明这些模型的有效方法而出现。然而，构建具有高保真度的高度可解释的apg是具有挑战性的。通过实证分析，我们发现更大的簇大小对应着更高保真度的APG。我们提出了一种新的方法，称为抽象-训练-抽象（ATA），建立在两个关键思想的整合之上。基于抽象的训练有利于抽象状态的聚类，扩大了每个聚类的范围。面向抽象的集群确保同一集群中的状态对应于相同的操作。识别状态所属的集群可以提高预测其相关动作的准确性。我们的实验表明，ATA超越了目前的技术水平，在保持竞争性奖励的同时，实现了高达26.63%的高保真度。此外，我们的用户研究表明，ATA大大提高了用户预测的准确率，平均提高了35.7%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Neural Networks 工程技术-计算机：人工智能

CiteScore

13.90

自引率

7.70%

发文量

425

审稿时长

67 days

期刊介绍： Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.