Artificial Intelligence Review最新文献_第3页

Attention-based graph neural networks: a survey 基于注意力的图神经网络研究综述

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-21 DOI: 10.1007/s10462-023-10577-2

Chengcheng Sun, Chenhao Li, Xiang Lin, Tianji Zheng, Fanrong Meng, Xiaobin Rui, Zhixiao Wang

{"title":"Attention-based graph neural networks: a survey","authors":"Chengcheng Sun, Chenhao Li, Xiang Lin, Tianji Zheng, Fanrong Meng, Xiaobin Rui, Zhixiao Wang","doi":"10.1007/s10462-023-10577-2","DOIUrl":"10.1007/s10462-023-10577-2","url":null,"abstract":"<div><p>Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-attention-based-gnns.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"2263 - 2310"},"PeriodicalIF":12.0,"publicationDate":"2023-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48118729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Computational deep air quality prediction techniques: a systematic review 计算深层空气质量预测技术：系统综述

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-16 DOI: 10.1007/s10462-023-10570-9

Manjit Kaur, Dilbag Singh, Mohamed Yaseen Jabarulla, Vijay Kumar, Jusung Kang, Heung-No Lee

{"title":"Computational deep air quality prediction techniques: a systematic review","authors":"Manjit Kaur, Dilbag Singh, Mohamed Yaseen Jabarulla, Vijay Kumar, Jusung Kang, Heung-No Lee","doi":"10.1007/s10462-023-10570-9","DOIUrl":"10.1007/s10462-023-10570-9","url":null,"abstract":"<div><p>The escalating population and rapid industrialization have led to a significant rise in environmental pollution, particularly air pollution. This has detrimental effects on both the environment and human health, resulting in increased morbidity and mortality. As a response to this pressing issue, the development of air quality prediction models has emerged as a critical research area. In this systematic literature review, we focused on reviewing 203 potential articles published between 2017 and May 2023 obtained from major databases. Our review specifically targeted keywords such as air quality prediction, air pollution prediction, and air quality classification. The review addressed five key research questions, including the types of deep learning (DL) models employed, the performance metrics considered, the best-performing models based on quantitative analysis, and the existing challenges and future prospects in the field. Additionally, we highlighted the limitations of current air quality prediction models and proposed various future research directions to foster further advancements in this area.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"2053 - 2098"},"PeriodicalIF":12.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41970502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

A survey on computational metaphor processing techniques: from identification, interpretation, generation to application 计算隐喻处理技术综述:从识别、解释、生成到应用

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-11 DOI: 10.1007/s10462-023-10564-7

Mengshi Ge, Rui Mao, Erik Cambria

引用次数: 4

Robust stability of dynamical neural networks with multiple time delays: a review and new results 多时滞动态神经网络的鲁棒稳定性：综述和新结果

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-10 DOI: 10.1007/s10462-023-10552-x

Ezgi Aktas, Ozlem Faydasicok, Sabri Arik

{"title":"Robust stability of dynamical neural networks with multiple time delays: a review and new results","authors":"Ezgi Aktas, Ozlem Faydasicok, Sabri Arik","doi":"10.1007/s10462-023-10552-x","DOIUrl":"10.1007/s10462-023-10552-x","url":null,"abstract":"<div><p>Robust stability properties of continuous-time dynamical neural networks involving time delay parameters have been extensively studied, and many sufficient criteria for robust stability of various classes of delayed dynamical neural networks have been obtained in the past literature. The class of activation functions and the types of delay terms involved in the mathematical models of dynamical neural networks are two main parameters in the determination of stability conditions for these neural network models. In this article, we will analyse a neural network model of relatively having a more complicated mathematical form where the neural system has the multiple time delay terms and the activation functions satisfy the Lipschitz conditions. By deriving a new and alternative upper bound value for the <span>(l_2)</span>-norm of uncertain intervalised matrices and constructing some various forms of the same type of a Lyapunov functional, this paper will first propose new results on global robust stability of dynamical Hopfield neural networks having multiple time delay terms in the presence of the Lipschitz activation functions. Then, we show that some simple modified changes in robust stability conditions proposed for multiple delayed Hopfield neural network model directly yield robust stability conditions of multiple delayed Cohen-Grossberg neural network model. We will also make a very detailed review of the previously published robust stability research results, which are basically in the nonsingular M-matrix or various algebraic inequalities forms. In particular, the robust stability results proposed in this paper are proved to generalize almost all previously reported robust stability conditions for multiple delayed neural network models. Some concluding remarks and future works regarding robust stability analysis of dynamical neural systems are addressed.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"1647 - 1684"},"PeriodicalIF":12.0,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42593746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A systematic literature review on AutoML for multi-target learning tasks AutoML用于多目标学习任务的系统文献综述

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-10 DOI: 10.1007/s10462-023-10569-2

Aline Marques Del Valle, Rafael Gomes Mantovani, Ricardo Cerri

{"title":"A systematic literature review on AutoML for multi-target learning tasks","authors":"Aline Marques Del Valle, Rafael Gomes Mantovani, Ricardo Cerri","doi":"10.1007/s10462-023-10569-2","DOIUrl":"10.1007/s10462-023-10569-2","url":null,"abstract":"<div><p>Automated machine learning (AutoML) aims to automate machine learning (ML) tasks, eliminating human intervention from the learning process as much as possible. However, most studies on AutoML are related to unique targets. This article aimed to identify and analyze studies on AutoML applied to multi-label classification and multi-target regression through a systematic literature review (SLR). Initially, we defined the research questions, the search string, the data sources for the search, and the inclusion and exclusion criteria. Then, we carried out the study selection process in four steps, with snowballing being the last stage. Altogether 12 studies were selected to compose SLR. All studies automated the task of ML model search of the pipeline, one study automated the task of feature engineering of the pipeline, all were related to Multi-label Classification, and only one addressed multi-target regression. The search space consisted of algorithms/neural operations and hyperparameters, the studies employed optimization algorithms (such as Genetic Algorithms and Hierarchical Task Networks) to produce increasingly better candidate solutions and one metric to assess the quality of candidate solutions. Only two studies employed Transfer Learning to contribute to AutoML. This article reviewed AutoML, multi-label classification, and multi-target regression and, by answering the SLR research questions, showed how current studies address these issues and gave insights into future directions for AutoML and multi-target tasks.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"2013 - 2052"},"PeriodicalIF":12.0,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10462-023-10569-2.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46643629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

RNN-AFOX: adaptive FOX-inspired-based technique for automated tuning of recurrent neural network hyper-parameters RNN-AFOX:基于自适应fox的循环神经网络超参数自动调谐技术

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-09 DOI: 10.1007/s10462-023-10568-3

Hosam ALRahhal, Razan Jamous

{"title":"RNN-AFOX: adaptive FOX-inspired-based technique for automated tuning of recurrent neural network hyper-parameters","authors":"Hosam ALRahhal, Razan Jamous","doi":"10.1007/s10462-023-10568-3","DOIUrl":"10.1007/s10462-023-10568-3","url":null,"abstract":"<div><p>The energy markets, particularly oil and gas, have been significantly affected by the outbreak of the COVID-19 pandemic in terms of price and availability. In addition to the pandemic, the Russia-Ukraine war has contributed to concerns about the reduction in the oil supply. AI techniques are widely employed for prediction oil prices as an alternative to traditional techniques. In this paper, an AI-based optimization model called adaptive fox-inspired optimization (AFOX) model is presented, then recurrent neural network (RNN) is combined with AFOX to form a hybrid model called recurrent neural network with adaptive fox-inspired (RNN-AFOX) model. The proposed model is used to predict Crude Oil Prices. In the proposed model, AFOX is used to find the best hyper-parameters of the RNN and employed these hyper-parameters to build best RNN structure and use it to forecast the closing price of the oil market. The results show that the RNN-AFOX model achieved a high accuracy prediction with very small error and the coefficient of determination (R-squared) equal to 0.99 outperforming the RNN model in terms of accuracy prediction by about 24%, the FOX model by about 20% and the AFOX model by about 14%. Moreover, RNN-AFOX was evaluated under the impact of the COVID-19 pandemic and the Russia-Ukraine war. The results show the efficiency of RNN-AFOX in forecasting the closing prices of oil with high accuracy. In general, the proposed RNN-AFOX model overcomes other studied models in terms of Mean Absolute Percentage Error, Mean Absolute Error, Mean Square Error, Root Mean Square Error, coefficient of determination (R-squared) and consumption time.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"1981 - 2011"},"PeriodicalIF":12.0,"publicationDate":"2023-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43978790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A systematic review of federated learning from clients’ perspective: challenges and solutions 从客户的角度系统回顾联邦学习:挑战和解决方案

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-07 DOI: 10.1007/s10462-023-10563-8

Yashothara Shanmugarasa, Hye-young Paik, Salil S. Kanhere, Liming Zhu

{"title":"A systematic review of federated learning from clients’ perspective: challenges and solutions","authors":"Yashothara Shanmugarasa, Hye-young Paik, Salil S. Kanhere, Liming Zhu","doi":"10.1007/s10462-023-10563-8","DOIUrl":"10.1007/s10462-023-10563-8","url":null,"abstract":"<div><p>Federated learning (FL) is a machine learning approach that decentralizes data and its processing by allowing clients to train intermediate models on their devices with locally stored data. It aims to preserve privacy as only model updates are shared with a central server rather than raw data. In recent years, many reviews have evaluated FL from the system (general challenges) or server’s perspectives, ignoring the importance of clients’ perspectives. Although FL helps users have control over their data, there are many challenges arising from decentralization, specifically from the perspectives of clients who are the main contributors to FL. Therefore, in response to the gap in the literature, this study intends to explore client-side challenges and available solutions by conducting a systematic literature review on 238 primary studies. Further, we analyze if a solution identified for one type of challenge is also applicable to other challenges and if there are impacts to consider. The conclusion of this survey reveals that servers and platforms have to work with clients to address client-side challenges.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"1773 - 1827"},"PeriodicalIF":12.0,"publicationDate":"2023-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10462-023-10563-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42936082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

New staircase sinusoidal voltage synthesizer and optimal interval type-2 fuzzy controller for dynamic voltage restorer to compensate voltage disturbances 新型阶梯正弦电压合成器及补偿电压扰动的动态电压恢复器最优区间2型模糊控制器

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-07 DOI: 10.1007/s10462-023-10572-7

Ali Darvish Falehi, Hossein Torkaman

{"title":"New staircase sinusoidal voltage synthesizer and optimal interval type-2 fuzzy controller for dynamic voltage restorer to compensate voltage disturbances","authors":"Ali Darvish Falehi, Hossein Torkaman","doi":"10.1007/s10462-023-10572-7","DOIUrl":"10.1007/s10462-023-10572-7","url":null,"abstract":"<div><p>In this paper, a new staircase sinusoidal voltage synthesizer based on dc-dc boost converter and dc-ac multilevel inverter is proposed for dynamic voltage restorer to accurately compensate the power quality issues. The dc-dc boost converter which is connected to photovoltaic-based power supply system can provide required energy and high-gain voltage [2*(1−K)<sup>−2</sup>] via tracking the maximum power for the compensation process. The dc-ac multilevel inverter which their dc power sources have followed septenary geometric progression can provide high step staircase sinusoidal voltage [7<sup>(Nsw/6)</sup>] with low switch count. In this regard, the proposed staircase sinusoidal voltage synthesizer can effectively compensate the voltage disturbance conditions. In view of the fact that the compensator control system must properly operate so that the proposed voltage synthesizer reveals its step creation capability, an interval type-2 fuzzy controller is implemented to assist the power electronic part. The controller parameters have been optimally extracted using multi-objective stochastic fractal search algorithm to ensure the control process accuracy. To verify and validate the compensation capability of the proposed compensator, the simulation results have been focused on asymmetrical voltage disturbance conditions. Finally, the comparison and simulation results have strongly confirmed the high step creation and compensation capabilities of the proposed DVR.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"2125 - 2150"},"PeriodicalIF":12.0,"publicationDate":"2023-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43128846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Machine and deep learning for longitudinal biomedical data: a review of methods and applications 纵向生物医学数据的机器和深度学习：方法和应用综述

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-05 DOI: 10.1007/s10462-023-10561-w

Anna Cascarano, Jordi Mur-Petit, Jerónimo Hernández-González, Marina Camacho, Nina de Toro Eadie, Polyxeni Gkontra, Marc Chadeau-Hyam, Jordi Vitrià, Karim Lekadir

{"title":"Machine and deep learning for longitudinal biomedical data: a review of methods and applications","authors":"Anna Cascarano, Jordi Mur-Petit, Jerónimo Hernández-González, Marina Camacho, Nina de Toro Eadie, Polyxeni Gkontra, Marc Chadeau-Hyam, Jordi Vitrià, Karim Lekadir","doi":"10.1007/s10462-023-10561-w","DOIUrl":"10.1007/s10462-023-10561-w","url":null,"abstract":"<div><p>Exploiting existing longitudinal data cohorts can bring enormous benefits to the medical field, as many diseases have a complex and multi-factorial time-course, and start to develop long before symptoms appear. With the increasing healthcare digitisation, the application of machine learning techniques for longitudinal biomedical data may enable the development of new tools for assisting clinicians in their day-to-day medical practice, such as for early diagnosis, risk prediction, treatment planning and prognosis estimation. However, due to the heterogeneity and complexity of time-varying data sets, the development of suitable machine learning models introduces major challenges for data scientists as well as for clinical researchers. This paper provides a comprehensive and critical review of recent developments and applications in machine learning for longitudinal biomedical data. Although the paper provides a discussion of clustering methods, its primary focus is on the prediction of static outcomes, defined as the value of the event of interest at a given instant in time, using longitudinal features, which has emerged as the most commonly employed approach in healthcare applications. First, the main approaches and algorithms for building longitudinal machine learning models are presented in detail, including their technical implementations, strengths and limitations. Subsequently, most recent biomedical and clinical applications are reviewed and discussed, showing promising results in a wide range of medical specialties. Lastly, we discuss current challenges and consider future directions in the field to enhance the development of machine learning tools from longitudinal biomedical data.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"1711 - 1771"},"PeriodicalIF":12.0,"publicationDate":"2023-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10462-023-10561-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49564474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Deep neural network pruning method based on sensitive layers and reinforcement learning 基于敏感层和强化学习的深度神经网络修剪方法

IF 12 2区计算机科学

Artificial Intelligence Review Pub Date : 2023-08-05 DOI: 10.1007/s10462-023-10566-5

Wenchuan Yang, Haoran Yu, Baojiang Cui, Runqi Sui, Tianyu Gu

{"title":"Deep neural network pruning method based on sensitive layers and reinforcement learning","authors":"Wenchuan Yang, Haoran Yu, Baojiang Cui, Runqi Sui, Tianyu Gu","doi":"10.1007/s10462-023-10566-5","DOIUrl":"10.1007/s10462-023-10566-5","url":null,"abstract":"<div><p>It is of great significance to compress neural network models so that they can be deployed on resource-constrained embedded mobile devices. However, due to the lack of theoretical guidance for non-salient network components, existing model compression methods are inefficient and labor-intensive. In this paper, we propose a new pruning method to achieve model compression. By exploring the rank ordering of the feature maps of convolutional layers, we introduce the concept of sensitive layers and treat layers with more low-rank feature maps as sensitive layers. We propose a new algorithm for finding sensitive layers while using reinforcement learning deterministic strategies to automate pruning for insensitive layers. Experimental results show that our method achieves significant improvements over the state-of-the-art in floating-point operations and parameter reduction, with lower precision loss. For example, using ResNet-110 on CIFAR-10 achieves a 62.2% reduction in floating-point operations by removing 63.9% of parameters. When testing ResNet-50 on ImageNet, our method reduces floating-point operations by 53.8% by deleting 39.9% of the parameters.</p></div>","PeriodicalId":8449,"journal":{"name":"Artificial Intelligence Review","volume":"56 2","pages":"1897 - 1917"},"PeriodicalIF":12.0,"publicationDate":"2023-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49270664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1