{"title":"Addressing Machine Learning Problems in the Non-Negative Orthant","authors":"Ioannis Tsingalis;Constantine Kotropoulos","doi":"10.1109/TETCI.2024.3379239","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3379239","url":null,"abstract":"Frequently, equality constraints are imposed on the objective function of machine learning algorithms aiming at increasing their robustness and generalization. In addition, non-negativity constraints imposed on the objective function aim to improve interpretability. This paper proposes a framework that solves problems in the non-negative orthant with additional equality constraints. This framework is characterized by an iteration complexity \u0000<inline-formula><tex-math>${mathcal{O}} {({ln}, {epsilon} ^{{ -varrho }})}$</tex-math></inline-formula>\u0000 with \u0000<inline-formula><tex-math>${epsilon}$</tex-math></inline-formula>\u0000 denoting the accuracy and \u0000<inline-formula><tex-math>${varrho}$</tex-math></inline-formula>\u0000 being the condition number. To avoid “zig-zagging”, a diminishing learning rate is adopted without harming the convergence of the learning procedure. Simple and well-established tools of the theory of Lagrange multipliers for constrained optimization are employed to derive the updating rules and study their convergence properties. To the best of our knowledge, this is the first time these tools are combined in a unified way to derive the proposed optimizer. Its efficiency is demonstrated by conducting classification experiments on well-known datasets, yielding promising results.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"3951-3965"},"PeriodicalIF":5.3,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Generalized Population-Based Training for Hyperparameter Optimization in Reinforcement Learning","authors":"Hui Bai;Ran Cheng","doi":"10.1109/TETCI.2024.3389777","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3389777","url":null,"abstract":"Hyperparameter optimization plays a key role in the machine learning domain. Its significance is especially pronounced in reinforcement learning (RL), where agents continuously interact with and adapt to their environments, requiring dynamic adjustments in their learning trajectories. To cater to this dynamicity, the Population-Based Training (PBT) was introduced, leveraging the collective intelligence of a population of agents learning simultaneously. However, PBT tends to favor high-performing agents, potentially neglecting the explorative potential of agents on the brink of significant advancements. To mitigate the limitations of PBT, we present the Generalized Population-Based Training (GPBT), a refined framework designed for enhanced granularity and flexibility in hyperparameter adaptation. Complementing GPBT, we further introduce Pairwise Learning (PL). Instead of merely focusing on elite agents, PL employs a comprehensive pairwise strategy to identify performance differentials and provide holistic guidance to underperforming agents. By integrating the capabilities of GPBT and PL, our approach significantly improves upon traditional PBT in terms of adaptability and computational efficiency. Rigorous empirical evaluations across a range of RL benchmarks confirm that our approach consistently outperforms not only the conventional PBT but also its Bayesian-optimized variant.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 5","pages":"3450-3462"},"PeriodicalIF":5.3,"publicationDate":"2024-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142368264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Novel Multi-Source Information Fusion Method Based on Dependency Interval","authors":"Weihua Xu;Yufei Lin;Na Wang","doi":"10.1109/TETCI.2024.3370032","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3370032","url":null,"abstract":"With the rapid development of Big Data era, it is necessary to extract necessary information from a large amount of information. Single-source information systems are often affected by extreme values and outliers, so multi-source information systems are more common and data more reasonable, information fusion is a common method to deal with multi-source information system. Compared with single-valued data, interval-valued data can describe the uncertainty and random change of data more effectively. This article proposes a novel interval-valued multi-source information fusion method: A multi-source information fusion method based on dependency interval. This method needs to construct a dependency function, which takes into account the interval length and the number of data points in the interval, so as to make the obtained data more centralized and eliminate the influence of outliers and extreme values. Due to the unfixed boundary of the dependency interval, a median point within the interval is selected as a bridge to simplify the acquisition of the dependency interval. Furthermore, a multi-source information system fusion algorithm based on dependency intervals was proposed, and experiments were conducted on 9 UCI datasets to compare the classification accuracy and quality of the proposed algorithm with traditional information fusion methods. The experimental results show that this method is more effective than the maximum interval method, quartile interval method, and mean interval method, and the validity of the data has been proven through hypothesis testing.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 4","pages":"3180-3194"},"PeriodicalIF":5.3,"publicationDate":"2024-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yang Wang;Zhipeng Lü;Junwen Ding;Zhouxing Su;Rafael Martí
{"title":"A Weighted Vertex Cover-Based Intensification Tabu Search for the Capacitated Dispersion Problem","authors":"Yang Wang;Zhipeng Lü;Junwen Ding;Zhouxing Su;Rafael Martí","doi":"10.1109/TETCI.2024.3389768","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3389768","url":null,"abstract":"The dispersion problem consists of selecting a subset of elements from a data set in order to maximize its diversity, which has many applications in real-world scenarios. For the capacitated dispersion problem (CDP), it seeks for a subset such that the minimum distance among the selected elements is as large as possible while satisfying a demand constraint. In this paper, we propose a weighted vertex cover-based intensification tabu search algorithm (WVC-ITS) for solving this challenging optimization problem. First, it transforms the CDP into a series of decision version subproblems, i.e., the weighted vertex cover problem. Then, it tackles each subproblem with an intensification tabu search-based algorithm. Computational experiments on 100 benchmark instances used in the literature and 20 newly generated challenging instances show that WVC-ITS is highly competitive in terms of both solution quality and computational efficiency. Compared with the state-of-the-art algorithms, WVC-ITS is able to obtain the best results for all the 120 instances within very short computational time and improve the previous best known results for 17 benchmark instances.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"4225-4236"},"PeriodicalIF":5.3,"publicationDate":"2024-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Bilevel Periodically Interactive Evolutionary Algorithm for Personalized Service Customization in Wireless-Powered Cooperative MEC","authors":"Ning Yang;Hai-Lin Liu","doi":"10.1109/TETCI.2024.3386622","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3386622","url":null,"abstract":"This article addresses the pricing scheme in a wireless-powered cooperative mobile edge computing (WP-CoMEC) system, focusing on personalized service customization. Traditional pricing schemes in such systems often assume a passive mode, with the service provider leading, and the device owner following. However, with the rise of personalized requirements, this paper proposes a novel approach where the device owner becomes an active participant in the pricing scheme, leading to personally customized services. The proposed pricing model formulates a bilevel multi-objective optimization problem, considering task offloading, resource allocation, and energy harvesting. This comprehensive approach ensures a more holistic optimization process. To address the computational challenges posed by the bilevel pricing model, this article proposes a bilevel periodically interactive evolutionary algorithm (BL-PIEA), which efficiently handles mixed variables, complex objective conflicts, and the inner nested structure of the bilevel pricing model. The proposed BL-PIEA is tested on ten instances, and the results indicate that BL-PIEA can effectively solve the proposed pricing model, showcasing superior performance in terms of reduced run time and saved evaluation budgets compared to other algorithms. With the proposed bilevel pricing model solved by BL-PIEA, the service provider can make out better pricing schemes that satisfy the device owner's requirements, so as to achieve a good personalized service customization.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"4090-4105"},"PeriodicalIF":5.3,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multilevel Joint Association Networks for Diverse Human Motion Prediction","authors":"Linwei Chen;Wanshu Fan;Xu Gui;Yaqing Hou;Xin Yang;Qiang Zhang;Xiaopeng Wei;Dongsheng Zhou","doi":"10.1109/TETCI.2024.3386840","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3386840","url":null,"abstract":"Predicting accurate and diverse human motion presents a challenging task due to the complexity and uncertainty of future human motion. Existing works have explored sampling techniques and body modeling approaches to enhance diversity while maintaining the accuracy of human motion prediction. However, most of them often fall short in capturing the hierarchical features of the correlations between joints. To address these limitations, we propose in this paper the Multilevel Joint Association Network, a novel deep generative model designed to achieve both diverse and controllable motion prediction by adjusting the manner in which the human body is modeled. Our model incorporates two Graph Convolution Networks (GCNs) to enhance the extraction of features, resulting in more accurate future motion samples. Furthermore, we employ a multi-level Transformer generator that effectively capture the contact information between human joints and the personality characteristics of human joints, enabling the generated future motion samples with high diversity and low error. Extensive experimental results on two challenging datasets Human3.6 M and HumanEva-I, indicate that the proposed method achieves state-of-the-art performance in terms of both diversity and accuracy.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"4165-4178"},"PeriodicalIF":5.3,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691691","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"WFormer: A Transformer-Based Soft Fusion Model for Robust Image Watermarking","authors":"Ting Luo;Jun Wu;Zhouyan He;Haiyong Xu;Gangyi Jiang;Chin-Chen Chang","doi":"10.1109/TETCI.2024.3386916","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3386916","url":null,"abstract":"Most deep neural network (DNN) based image watermarking models often employ the encoder-noise-decoder structure, in which watermark is simply duplicated for expansion and then directly fused with image features to produce the encoded image. However, simple duplication will generate watermark over-redundancies, and the communication between the cover image and watermark in different domains is lacking in image feature extraction and direction fusion, which degrades the watermarking performance. To solve those drawbacks, this paper proposes a Transformer-based soft fusion model for robust image watermarking, namely WFormer. Specifically, to expand watermark effectively, a watermark preprocess module (WPM) is designed with Transformers to extract valid and expanded watermark features by computing its self-attention. Then, to replace direct fusion, a soft fusion module (SFM) is deployed to integrate Transformers into image fusion with watermark by mining their long-range correlations. Precisely, self-attention is computed to extract their own latent features, and meanwhile, cross-attention is learned for bridging their gap to embed watermark effectively. In addition, a feature enhancement module (FEM) builds communication between the cover image and watermark by capturing their cross-feature dependencies, which tunes image features in accordance with watermark features for better fusion. Experimental results show that the proposed WFormer outperforms the existing state-of-the-art watermarking models in terms of invisibility, robustness, and embedding capacity. Furthermore, ablation results prove the effectiveness of the WPM, the FEM, and the SFM.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"4179-4196"},"PeriodicalIF":5.3,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Low-Contrast Medical Image Segmentation via Transformer and Boundary Perception","authors":"Yinglin Zhang;Ruiling Xi;Wei Wang;Heng Li;Lingxi Hu;Huiyan Lin;Dave Towey;Ruibin Bai;Huazhu Fu;Risa Higashita;Jiang Liu","doi":"10.1109/TETCI.2024.3353624","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3353624","url":null,"abstract":"Low-contrast medical image segmentation is a challenging task that requires full use of local details and global context. However, existing convolutional neural networks (CNNs) cannot fully exploit global information due to limited receptive fields and local weight sharing. On the other hand, the transformer effectively establishes long-range dependencies but lacks desirable properties for modeling local details. This paper proposes a Transformer-embedded Boundary perception Network (TBNet) that combines the advantages of transformer and convolution for low-contrast medical image segmentation. Firstly, the transformer-embedded module uses convolution at the low-level layer to model local details and uses the Enhanced TRansformer (ETR) to capture long-range dependencies at the high-level layer. This module can extract robust features with semantic contexts to infer the possible target location and basic structure in low-contrast conditions. Secondly, we utilize the decoupled body-edge branch to promote general feature learning and precept precise boundary locations. The ETR establishes long-range dependencies across the whole feature map range and is enhanced by introducing local information. We implement it in a parallel mode, i.e., the group of self-attention with multi-head captures the global relationship, and the group of convolution retains local details. We compare TBNet with other state-of-the-art (SOTA) methods on the cornea endothelial cell, ciliary body, and kidney segmentation tasks. The TBNet improves segmentation performance, proving its effectiveness and robustness.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 3","pages":"2297-2309"},"PeriodicalIF":5.3,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141094882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Passivity-Based Formation Control for Fractional-Order Multiagent Systems With and Without Communication Delay","authors":"Jin-Liang Wang;Lina Huang;Shun-Yan Ren;Tingwen Huang","doi":"10.1109/TETCI.2024.3386837","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3386837","url":null,"abstract":"Because fractional-order differential equations can more accurately model the dynamics of agents, it is very meaningful to investigate the formation control for fractional-order multi-agent systems (FOMASs). Recently, passivity has been widely utilized to tackle various cooperative control problems for integer-order MASs since passive systems are internally stable. Apparently, it is also advantageous to deal with the formation control problem of FOMASs based on the passivity. In this paper, two types of formation control problems for second-order nonlinear FOMASs are discussed by employing the passivity, that is, the cases without and with communication delay, respectively. On the basis of the devised state feedback and adaptive state feedback controllers, several passivity conditions for single fractional-order agent are derived. Furthermore, by selecting suitable distributed state feedback control strategy and exploiting the passivity of fractional-order agent, two formation criteria for the FOMAS are given. In addition, the above obtained results are further extended to the case where communication delay exists in the control input. Finally, two numerical examples are provided to substantiate the effectiveness of the derived formation criteria.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 6","pages":"4143-4154"},"PeriodicalIF":5.3,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xudong Wang;Xi'ai Chen;Weihong Ren;Zhi Han;Huijie Fan;Yandong Tang;Lianqing Liu
{"title":"Compensation Atmospheric Scattering Model and Two-Branch Network for Single Image Dehazing","authors":"Xudong Wang;Xi'ai Chen;Weihong Ren;Zhi Han;Huijie Fan;Yandong Tang;Lianqing Liu","doi":"10.1109/TETCI.2024.3386838","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3386838","url":null,"abstract":"Most existing dehazing networks rely on synthetic hazy-clear image pairs for training, and thus fail to work well in real-world scenes. In this paper, we deduce a reformulated atmospheric scattering model for a hazy image and propose a novel lightweight two-branch dehazing network. In the model, we use a Transformation Map to represent the dehazing transformation and use a Compensation Map to represent variable illumination compensation. Based on this model, we design a \u0000<underline>T</u>\u0000wo-\u0000<underline>B</u>\u0000ranch \u0000<underline>N</u>\u0000etwork (TBN) to jointly estimate the Transformation Map and Compensation Map. Our TBN is designed with a shared Feature Extraction Module and two Adaptive Weight Modules. The Feature Extraction Module is used to extract shared features from hazy images. The two Adaptive Weight Modules generate two groups of adaptive weighted features for the Transformation Map and Compensation Map, respectively. This design allows for a targeted conversion of features to the Transformation Map and Compensation Map. To further improve the dehazing performance in the real-world, we propose a semi-supervised learning strategy for TBN. Specifically, by performing supervised pre-training based on synthetic image pairs, we propose a Self-Enhancement method to generate pseudo-labels, and then further train our TBN with the pseudo-labels in a semi-supervised way. Extensive experiments demonstrate that the model-based TBN outperforms the state-of-the-art methods on various real-world datasets.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"8 4","pages":"2880-2896"},"PeriodicalIF":5.3,"publicationDate":"2024-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}