arXiv - CS - Neural and Evolutionary Computing最新文献_第8页

Federated Fairness Analytics: Quantifying Fairness in Federated Learning 联合公平分析：量化联合学习中的公平性

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-15 DOI: arxiv-2408.08214

Oscar Dilley, Juan Marcelo Parra-Ullauri, Rasheed Hussain, Dimitra Simeonidou

{"title":"Federated Fairness Analytics: Quantifying Fairness in Federated Learning","authors":"Oscar Dilley, Juan Marcelo Parra-Ullauri, Rasheed Hussain, Dimitra Simeonidou","doi":"arxiv-2408.08214","DOIUrl":"https://doi.org/arxiv-2408.08214","url":null,"abstract":"Federated Learning (FL) is a privacy-enhancing technology for distributed ML.\u0000By training models locally and aggregating updates - a federation learns\u0000together, while bypassing centralised data collection. FL is increasingly\u0000popular in healthcare, finance and personal computing. However, it inherits\u0000fairness challenges from classical ML and introduces new ones, resulting from\u0000differences in data quality, client participation, communication constraints,\u0000aggregation methods and underlying hardware. Fairness remains an unresolved\u0000issue in FL and the community has identified an absence of succinct definitions\u0000and metrics to quantify fairness; to address this, we propose Federated\u0000Fairness Analytics - a methodology for measuring fairness. Our definition of\u0000fairness comprises four notions with novel, corresponding metrics. They are\u0000symptomatically defined and leverage techniques originating from XAI,\u0000cooperative game-theory and networking engineering. We tested a range of\u0000experimental settings, varying the FL approach, ML task and data settings. The\u0000results show that statistical heterogeneity and client participation affect\u0000fairness and fairness conscious approaches such as Ditto and q-FedAvg\u0000marginally improve fairness-performance trade-offs. Using our techniques, FL\u0000practitioners can uncover previously unobtainable insights into their system's\u0000fairness, at differing levels of granularity in order to address fairness\u0000challenges in FL. We have open-sourced our work at:\u0000https://github.com/oscardilley/federated-fairness.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"14 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Universality of Real Minimal Complexity Reservoir 真实最小复杂性水库的普遍性

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-15 DOI: arxiv-2408.08071

Robert Simon Fong, Boyu Li, Peter Tiňo

{"title":"Universality of Real Minimal Complexity Reservoir","authors":"Robert Simon Fong, Boyu Li, Peter Tiňo","doi":"arxiv-2408.08071","DOIUrl":"https://doi.org/arxiv-2408.08071","url":null,"abstract":"Reservoir Computing (RC) models, a subclass of recurrent neural networks, are\u0000distinguished by their fixed, non-trainable input layer and dynamically coupled\u0000reservoir, with only the static readout layer being trained. This design\u0000circumvents the issues associated with backpropagating error signals through\u0000time, thereby enhancing both stability and training efficiency. RC models have\u0000been successfully applied across a broad range of application domains.\u0000Crucially, they have been demonstrated to be universal approximators of\u0000time-invariant dynamic filters with fading memory, under various settings of\u0000approximation norms and input driving sources. Simple Cycle Reservoirs (SCR) represent a specialized class of RC models with\u0000a highly constrained reservoir architecture, characterized by uniform ring\u0000connectivity and binary input-to-reservoir weights with an aperiodic sign\u0000pattern. For linear reservoirs, given the reservoir size, the reservoir\u0000construction has only one degree of freedom -- the reservoir cycle weight. Such\u0000architectures are particularly amenable to hardware implementations without\u0000significant performance degradation in many practical tasks. In this study we\u0000endow these observations with solid theoretical foundations by proving that\u0000SCRs operating in real domain are universal approximators of time-invariant\u0000dynamic filters with fading memory. Our results supplement recent research\u0000showing that SCRs in the complex domain can approximate, to arbitrary\u0000precision, any unrestricted linear reservoir with a non-linear readout. We\u0000furthermore introduce a novel method to drastically reduce the number of SCR\u0000units, making such highly constrained architectures natural candidates for\u0000low-complexity hardware implementations. Our findings are supported by\u0000empirical studies on real-world time series datasets.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"87 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation 通过适应推进尖峰神经网络的时空处理

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-14 DOI: arxiv-2408.07517

Maximilian Baronig, Romain Ferrand, Silvester Sabathiel, Robert Legenstein

{"title":"Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation","authors":"Maximilian Baronig, Romain Ferrand, Silvester Sabathiel, Robert Legenstein","doi":"arxiv-2408.07517","DOIUrl":"https://doi.org/arxiv-2408.07517","url":null,"abstract":"Efficient implementations of spiking neural networks on neuromorphic hardware\u0000promise orders of magnitude less power consumption than their non-spiking\u0000counterparts. The standard neuron model for spike-based computation on such\u0000neuromorphic systems has long been the leaky integrate-and-fire (LIF) neuron.\u0000As a promising advancement, a computationally light augmentation of the LIF\u0000neuron model with an adaptation mechanism experienced a recent upswing in\u0000popularity, caused by demonstrations of its superior performance on\u0000spatio-temporal processing tasks. The root of the superiority of these\u0000so-called adaptive LIF neurons however, is not well understood. In this\u0000article, we thoroughly analyze the dynamical, computational, and learning\u0000properties of adaptive LIF neurons and networks thereof. We find that the\u0000frequently observed stability problems during training of such networks can be\u0000overcome by applying an alternative discretization method that results in\u0000provably better stability properties than the commonly used Euler-Forward\u0000method. With this discretization, we achieved a new state-of-the-art\u0000performance on common event-based benchmark datasets. We also show that the\u0000superiority of networks of adaptive LIF neurons extends to the prediction and\u0000generation of complex time series. Our further analysis of the computational\u0000properties of networks of adaptive LIF neurons shows that they are particularly\u0000well suited to exploit the spatio-temporal structure of input sequences.\u0000Furthermore, these networks are surprisingly robust to shifts of the mean input\u0000strength and input spike rate, even when these shifts were not observed during\u0000training. As a consequence, high-performance networks can be obtained without\u0000any normalization techniques such as batch normalization or batch-normalization\u0000through time.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"44 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Surrogate-Assisted Search with Competitive Knowledge Transfer for Expensive Optimization 针对昂贵优化的具有竞争性知识转移的代理辅助搜索

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-13 DOI: arxiv-2408.07176

Xiaoming Xue, Yao Hu, Liang Feng, Kai Zhang, Linqi Song, Kay Chen Tan

{"title":"Surrogate-Assisted Search with Competitive Knowledge Transfer for Expensive Optimization","authors":"Xiaoming Xue, Yao Hu, Liang Feng, Kai Zhang, Linqi Song, Kay Chen Tan","doi":"arxiv-2408.07176","DOIUrl":"https://doi.org/arxiv-2408.07176","url":null,"abstract":"Expensive optimization problems (EOPs) have attracted increasing research\u0000attention over the decades due to their ubiquity in a variety of practical\u0000applications. Despite many sophisticated surrogate-assisted evolutionary\u0000algorithms (SAEAs) that have been developed for solving such problems, most of\u0000them lack the ability to transfer knowledge from previously-solved tasks and\u0000always start their search from scratch, making them troubled by the notorious\u0000cold-start issue. A few preliminary studies that integrate transfer learning\u0000into SAEAs still face some issues, such as defective similarity quantification\u0000that is prone to underestimate promising knowledge, surrogate-dependency that\u0000makes the transfer methods not coherent with the state-of-the-art in SAEAs,\u0000etc. In light of the above, a plug and play competitive knowledge transfer\u0000method is proposed to boost various SAEAs in this paper. Specifically, both the\u0000optimized solutions from the source tasks and the promising solutions acquired\u0000by the target surrogate are treated as task-solving knowledge, enabling them to\u0000compete with each other to elect the winner for expensive evaluation, thus\u0000boosting the search speed on the target task. Moreover, the lower bound of the\u0000convergence gain brought by the knowledge competition is mathematically\u0000analyzed, which is expected to strengthen the theoretical foundation of\u0000sequential transfer optimization. Experimental studies conducted on a series of\u0000benchmark problems and a practical application from the petroleum industry\u0000verify the efficacy of the proposed method. The source code of the competitive\u0000knowledge transfer is available at https://github.com/XmingHsueh/SAS-CKT.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"23 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188177","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Potential of Combined Learning Strategies to Enhance Energy Efficiency of Spiking Neuromorphic Systems 组合学习策略提高尖峰神经形态系统能效的潜力

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-13 DOI: arxiv-2408.07150

Ali Shiri Sichani, Sai Kankatala

{"title":"The Potential of Combined Learning Strategies to Enhance Energy Efficiency of Spiking Neuromorphic Systems","authors":"Ali Shiri Sichani, Sai Kankatala","doi":"arxiv-2408.07150","DOIUrl":"https://doi.org/arxiv-2408.07150","url":null,"abstract":"Ensuring energy-efficient design in neuromorphic computing systems\u0000necessitates a tailored architecture combined with algorithmic approaches. This\u0000manuscript focuses on enhancing brain-inspired perceptual computing machines\u0000through a novel combined learning approach for Convolutional Spiking Neural\u0000Networks (CSNNs). CSNNs present a promising alternative to traditional\u0000power-intensive and complex machine learning methods like backpropagation,\u0000offering energy-efficient spiking neuron processing inspired by the human\u0000brain. The proposed combined learning method integrates Pair-based Spike\u0000Timing-Dependent Plasticity (PSTDP) and power law-dependent\u0000Spike-timing-dependent plasticity (STDP) to adjust synaptic efficacies,\u0000enabling the utilization of stochastic elements like memristive devices to\u0000enhance energy efficiency and improve perceptual computing accuracy. By\u0000reducing learning parameters while maintaining accuracy, these systems consume\u0000less energy and have reduced area overhead, making them more suitable for\u0000hardware implementation. The research delves into neuromorphic design\u0000architectures, focusing on CSNNs to provide a general framework for\u0000energy-efficient computing hardware. Various CSNN architectures are evaluated\u0000to assess how less trainable parameters can maintain acceptable accuracy in\u0000perceptual computing systems, positioning them as viable candidates for\u0000neuromorphic architecture. Comparisons with previous work validate the\u0000achievements and methodology of the proposed architecture.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"28 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Massive Dimensions Reduction and Hybridization with Meta-heuristics in Deep Learning 深度学习中的大规模维度缩减和与元启发式算法的混合

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-13 DOI: arxiv-2408.07194

Rasa Khosrowshahli, Shahryar Rahnamayan, Beatrice Ombuki-Berman

{"title":"Massive Dimensions Reduction and Hybridization with Meta-heuristics in Deep Learning","authors":"Rasa Khosrowshahli, Shahryar Rahnamayan, Beatrice Ombuki-Berman","doi":"arxiv-2408.07194","DOIUrl":"https://doi.org/arxiv-2408.07194","url":null,"abstract":"Deep learning is mainly based on utilizing gradient-based optimization for\u0000training Deep Neural Network (DNN) models. Although robust and widely used,\u0000gradient-based optimization algorithms are prone to getting stuck in local\u0000minima. In this modern deep learning era, the state-of-the-art DNN models have\u0000millions and billions of parameters, including weights and biases, making them\u0000huge-scale optimization problems in terms of search space. Tuning a huge number\u0000of parameters is a challenging task that causes vanishing/exploding gradients\u0000and overfitting; likewise, utilized loss functions do not exactly represent our\u0000targeted performance metrics. A practical solution to exploring large and\u0000complex solution space is meta-heuristic algorithms. Since DNNs exceed\u0000thousands and millions of parameters, even robust meta-heuristic algorithms,\u0000such as Differential Evolution, struggle to efficiently explore and converge in\u0000such huge-dimensional search spaces, leading to very slow convergence and high\u0000memory demand. To tackle the mentioned curse of dimensionality, the concept of\u0000blocking was recently proposed as a technique that reduces the search space\u0000dimensions by grouping them into blocks. In this study, we aim to introduce\u0000Histogram-based Blocking Differential Evolution (HBDE), a novel approach that\u0000hybridizes gradient-based and gradient-free algorithms to optimize parameters.\u0000Experimental results demonstrated that the HBDE could reduce the parameters in\u0000the ResNet-18 model from 11M to 3K during the training/optimizing phase by\u0000metaheuristics, namely, the proposed HBDE, which outperforms baseline\u0000gradient-based and parent gradient-free DE algorithms evaluated on CIFAR-10 and\u0000CIFAR-100 datasets showcasing its effectiveness with reduced computational\u0000demands for the very first time.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"18 1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142188362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Overcoming the Limitations of Layer Synchronization in Spiking Neural Networks 克服尖峰神经网络中层同步的局限性

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-09 DOI: arxiv-2408.05098

Roel Koopman, Amirreza Yousefzadeh, Mahyar Shahsavari, Guangzhi Tang, Manolis Sifalakis

{"title":"Overcoming the Limitations of Layer Synchronization in Spiking Neural Networks","authors":"Roel Koopman, Amirreza Yousefzadeh, Mahyar Shahsavari, Guangzhi Tang, Manolis Sifalakis","doi":"arxiv-2408.05098","DOIUrl":"https://doi.org/arxiv-2408.05098","url":null,"abstract":"Currently, neural-network processing in machine learning applications relies\u0000on layer synchronization, whereby neurons in a layer aggregate incoming\u0000currents from all neurons in the preceding layer, before evaluating their\u0000activation function. This is practiced even in artificial Spiking Neural\u0000Networks (SNNs), which are touted as consistent with neurobiology, in spite of\u0000processing in the brain being, in fact asynchronous. A truly asynchronous\u0000system however would allow all neurons to evaluate concurrently their threshold\u0000and emit spikes upon receiving any presynaptic current. Omitting layer\u0000synchronization is potentially beneficial, for latency and energy efficiency,\u0000but asynchronous execution of models previously trained with layer\u0000synchronization may entail a mismatch in network dynamics and performance. We\u0000present a study that documents and quantifies this problem in three datasets on\u0000our simulation environment that implements network asynchrony, and we show that\u0000models trained with layer synchronization either perform sub-optimally in\u0000absence of the synchronization, or they will fail to benefit from any energy\u0000and latency reduction, when such a mechanism is in place. We then \"make ends\u0000meet\" and address the problem with unlayered backprop, a novel\u0000backpropagation-based training method, for learning models suitable for\u0000asynchronous processing. We train with it models that use different neuron\u0000execution scheduling strategies, and we show that although their neurons are\u0000more reactive, these models consistently exhibit lower overall spike density\u0000(up to 50%), reach a correct decision faster (up to 2x) without integrating all\u0000spikes, and achieve superior accuracy (up to 10% higher). Our findings suggest\u0000that asynchronous event-based (neuromorphic) AI computing is indeed more\u0000efficient, but we need to seriously rethink how we train our SNN models, to\u0000benefit from it.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"93 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141945505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Neuromorphic Keyword Spotting with Pulse Density Modulation MEMS Microphones 利用脉冲密度调制 MEMS 麦克风进行神经形态关键词搜索

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-09 DOI: arxiv-2408.05156

Sidi Yaya Arnaud Yarga, Sean U. N. Wood

引用次数: 0

Sparse Spiking Neural-like Membrane Systems on Graphics Processing Units 图形处理器上的稀疏尖峰神经膜系统

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-08 DOI: arxiv-2408.04343

Javier Hernández-Tello, Miguel Ángel Martínez-del-Amor, David Orellana-Martín, Francis George C. Cabarle

引用次数: 0

The Ungrounded Alignment Problem 不接地气的对齐问题

arXiv - CS - Neural and Evolutionary Computing Pub Date : 2024-08-08 DOI: arxiv-2408.04242

Marc Pickett, Aakash Kumar Nain, Joseph Modayil, Llion Jones

{"title":"The Ungrounded Alignment Problem","authors":"Marc Pickett, Aakash Kumar Nain, Joseph Modayil, Llion Jones","doi":"arxiv-2408.04242","DOIUrl":"https://doi.org/arxiv-2408.04242","url":null,"abstract":"Modern machine learning systems have demonstrated substantial abilities with\u0000methods that either embrace or ignore human-provided knowledge, but combining\u0000benefits of both styles remains a challenge. One particular challenge involves\u0000designing learning systems that exhibit built-in responses to specific abstract\u0000stimulus patterns, yet are still plastic enough to be agnostic about the\u0000modality and exact form of their inputs. In this paper, we investigate what we\u0000call The Ungrounded Alignment Problem, which asks How can we build in\u0000predefined knowledge in a system where we don't know how a given stimulus will\u0000be grounded? This paper examines a simplified version of the general problem,\u0000where an unsupervised learner is presented with a sequence of images for the\u0000characters in a text corpus, and this learner is later evaluated on its ability\u0000to recognize specific (possibly rare) sequential patterns. Importantly, the\u0000learner is given no labels during learning or evaluation, but must map images\u0000from an unknown font or permutation to its correct class label. That is, at no\u0000point is our learner given labeled images, where an image vector is explicitly\u0000associated with a class label. Despite ample work in unsupervised and\u0000self-supervised loss functions, all current methods require a labeled\u0000fine-tuning phase to map the learned representations to correct classes.\u0000Finding this mapping in the absence of labels may seem a fool's errand, but our\u0000main result resolves this seeming paradox. We show that leveraging only letter\u0000bigram frequencies is sufficient for an unsupervised learner both to reliably\u0000associate images to class labels and to reliably identify trigger words in the\u0000sequence of inputs. More generally, this method suggests an approach for\u0000encoding specific desired innate behaviour in modality-agnostic models.","PeriodicalId":501347,"journal":{"name":"arXiv - CS - Neural and Evolutionary Computing","volume":"111 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141945584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0