Faten A. Khalifa, Hatem M. Abdelkader, Asmaa H. Elsaid
{"title":"An analysis of ensemble pruning methods under the explanation of Random Forest","authors":"Faten A. Khalifa, Hatem M. Abdelkader, Asmaa H. Elsaid","doi":"10.1016/j.is.2023.102310","DOIUrl":null,"url":null,"abstract":"<div><p><span><span><span><span>“Black box” models created by modern machine learning techniques are typically hard to interpret. Thus, the necessity of </span>explainable artificial intelligence (XAI) has grown for understanding the rationale behind those models and converting them into white boxes. </span>Random Forest is a black box model essential in various domains due to its flexibility, ease of use, and remarkable </span>predictive performance<span>. One method for explaining a Random Forest is transforming it into a self-explainable Decision Tree using Forest-Based Tree (FBT) algorithm. It basically consists of three main phases, pruning, conjunction set generation, and Decision Tree construction. In this paper, we examine six state-of-the-art pruning approaches and analyze their effect on FBT performance through pruned FBT (PFBT) in order to minimize its </span></span>computational complexity. This would make it appropriate for forests and datasets of any size. They are assessed on 30 datasets, and the results show that UMEP and Hybrid pruning methods can be effectively used in the pruning stage of the PFBT algorithm in terms of pruning time and predictive performance. However, the AUC-Greedy method achieves good performance with small-size datasets.</p></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2023-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Systems","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0306437923001461","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
“Black box” models created by modern machine learning techniques are typically hard to interpret. Thus, the necessity of explainable artificial intelligence (XAI) has grown for understanding the rationale behind those models and converting them into white boxes. Random Forest is a black box model essential in various domains due to its flexibility, ease of use, and remarkable predictive performance. One method for explaining a Random Forest is transforming it into a self-explainable Decision Tree using Forest-Based Tree (FBT) algorithm. It basically consists of three main phases, pruning, conjunction set generation, and Decision Tree construction. In this paper, we examine six state-of-the-art pruning approaches and analyze their effect on FBT performance through pruned FBT (PFBT) in order to minimize its computational complexity. This would make it appropriate for forests and datasets of any size. They are assessed on 30 datasets, and the results show that UMEP and Hybrid pruning methods can be effectively used in the pruning stage of the PFBT algorithm in terms of pruning time and predictive performance. However, the AUC-Greedy method achieves good performance with small-size datasets.
由现代机器学习技术创建的“黑匣子”模型通常很难解释。因此,为了理解这些模型背后的基本原理并将其转换为白盒,可解释的人工智能(XAI)的必要性已经增长。随机森林是一种黑盒模型,由于其灵活性,易用性和卓越的预测性能在各个领域都是必不可少的。一种解释随机森林的方法是利用基于森林的树(Forest- based Tree, FBT)算法将随机森林转化为可自我解释的决策树。它主要包括三个阶段:剪枝、连接集生成和决策树构建。在本文中,我们研究了六种最先进的修剪方法,并通过修剪FBT (PFBT)来分析它们对FBT性能的影响,以最小化其计算复杂度。这将使它适用于任何规模的森林和数据集。在30个数据集上进行了评估,结果表明,从剪枝时间和预测性能方面来看,UMEP和Hybrid剪枝方法可以有效地用于PFBT算法的剪枝阶段。然而,AUC-Greedy方法在小数据集上取得了很好的性能。
期刊介绍:
Information systems are the software and hardware systems that support data-intensive applications. The journal Information Systems publishes articles concerning the design and implementation of languages, data models, process models, algorithms, software and hardware for information systems.
Subject areas include data management issues as presented in the principal international database conferences (e.g., ACM SIGMOD/PODS, VLDB, ICDE and ICDT/EDBT) as well as data-related issues from the fields of data mining/machine learning, information retrieval coordinated with structured data, internet and cloud data management, business process management, web semantics, visual and audio information systems, scientific computing, and data science. Implementation papers having to do with massively parallel data management, fault tolerance in practice, and special purpose hardware for data-intensive systems are also welcome. Manuscripts from application domains, such as urban informatics, social and natural science, and Internet of Things, are also welcome. All papers should highlight innovative solutions to data management problems such as new data models, performance enhancements, and show how those innovations contribute to the goals of the application.