Journal of Parallel and Distributed Computing最新文献

{"title":"Large-scale and cooperative graybox parallel optimization on the supercomputer Fugaku","authors":"Lorenzo Canonne ,&nbsp;Bilel Derbel ,&nbsp;Miwako Tsuji ,&nbsp;Mitsuhisa Sato","doi":"10.1016/j.jpdc.2024.104921","DOIUrl":"https://doi.org/10.1016/j.jpdc.2024.104921","url":null,"abstract":"<div><p>We design, develop and analyze parallel variants of a state-of-the-art graybox optimization algorithm, namely <span>Drils</span> (Deterministic Recombination and Iterated Local Search), for attacking large-scale pseudo-boolean optimization problems on top of the large-scale computing facilities offered by the supercomputer Fugaku. We first adopt a Master/Worker design coupled with a fully distributed Island-based model, ending up with a number of hybrid OpenMP/MPI implementations of high-level parallel <span>Drils</span> versions. We show that such a design, although effective, can be substantially improved by enabling a more focused iteration-level cooperation mechanism between the core graybox components of the original serial <span>Drils</span> algorithm. Extensive experiments are conducted in order to provide a systematic analysis of the impact of the designed parallel algorithms on search behavior, and their ability to compute high-quality solutions using increasing number of CPU-cores. Results using up to 1024×12-cores NUMA nodes, and NK-landscapes with up to <span><math><mn>10</mn><mo>,</mo><mn>000</mn></math></span> binary variables are reported, providing evidence on the relative strength of the designed hybrid cooperative graybox parallel search.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104921"},"PeriodicalIF":3.8,"publicationDate":"2024-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141090688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"HBPB, applying reuse distance to improve cache efficiency proactively","authors":"Arthur M. Krause,&nbsp;Paulo C. Santos,&nbsp;Arthur F. Lorenzon,&nbsp;Philippe O.A. Navaux","doi":"10.1016/j.jpdc.2024.104919","DOIUrl":"https://doi.org/10.1016/j.jpdc.2024.104919","url":null,"abstract":"<div><p>Cache memories play a significant role in the performance, area, and energy consumption of modern processors, and this impact is expected to grow as on-die memories become larger. While caches are highly effective for cache-friendly access patterns, they introduce unnecessary delays and energy wastage when they fail to serve the required data. Hence, cache bypassing techniques have been proposed to optimize the latency of cache-unfriendly memory accesses. In this scenario, we discuss <em>HBPB</em>, a history-based preemptive bypassing technique that accelerates cache-unfriendly access through the reduced latency of bypassing the caches. By extensively evaluating different real-world applications and hardware cache configurations, we show that <em>HBPB</em> yields energy reductions of up to 75% and performance improvements of up to 50% compared to a version that does not apply cache bypassing. More importantly, we demonstrate that <em>HBPB</em> does not affect the performance of applications with cache-friendly access patterns.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104919"},"PeriodicalIF":3.8,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141078565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Balancing privacy and performance in federated learning: A systematic literature review on methods and metrics","authors":"Samaneh Mohammadi ,&nbsp;Ali Balador ,&nbsp;Sima Sinaei ,&nbsp;Francesco Flammini","doi":"10.1016/j.jpdc.2024.104918","DOIUrl":"10.1016/j.jpdc.2024.104918","url":null,"abstract":"<div><p>Federated learning (FL) as a novel paradigm in Artificial Intelligence (AI), ensures enhanced privacy by eliminating data centralization and brings learning directly to the edge of the user's device. Nevertheless, new privacy issues have been raised particularly during training and the exchange of parameters between servers and clients. While several privacy-preserving FL solutions have been developed to mitigate potential breaches in FL architectures, their integration poses its own set of challenges. Incorporating these privacy-preserving mechanisms into FL at the edge computing level can increase both communication and computational overheads, which may, in turn, compromise data utility and learning performance metrics. This paper provides a systematic literature review on essential methods and metrics to support the most appropriate trade-offs between FL privacy and other performance-related application requirements such as accuracy, loss, convergence time, utility, communication, and computation overhead. We aim to provide an extensive overview of recent privacy-preserving mechanisms in FL used across various applications, placing a particular focus on quantitative privacy assessment approaches in FL and the necessity of achieving a balance between privacy and the other requirements of real-world FL applications. This review collects, classifies, and discusses relevant papers in a structured manner, emphasizing challenges, open issues, and promising research directions.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"192 ","pages":"Article 104918"},"PeriodicalIF":3.8,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0743731524000820/pdfft?md5=2ee3078ecc6441a5efe38d3a7c047d80&pid=1-s2.0-S0743731524000820-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141033667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Cost-aware quantum-inspired genetic algorithm for workflow scheduling in hybrid clouds","authors":"Mehboob Hussain ,&nbsp;Lian-Fu Wei ,&nbsp;Amir Rehman ,&nbsp;Muqadar Ali ,&nbsp;Syed Muhammad Waqas ,&nbsp;Fakhar Abbas","doi":"10.1016/j.jpdc.2024.104920","DOIUrl":"10.1016/j.jpdc.2024.104920","url":null,"abstract":"<div><p>Cloud computing delivers a desirable environment for users to run their different kinds of applications in a cloud. Numerous of these applications (tasks), such as bioinformatics, astronomy, biodiversity, and image analysis, are deadline-sensitive. Such tasks must be properly allocated to virtual machines (VMs) to avoid deadline violations, and they should reduce their execution time and cost. Due to the contradictory environment, minimizing the application task's completion time and execution cost is extremely difficult. Thus, we propose a Cost-aware Quantum-inspired Genetic Algorithm (CQGA) to minimize the execution time and cost by meeting the deadline constraints. CQGA is motivated by quantum computing and genetic algorithm. It combines quantum operators (measure, interference, and rotation) with genetic operators (selection, crossover, and mutation). Quantum operators are used for better population diversity, quick convergence, time-saving, and robustness. Genetic operators help to produce new individuals, have good fitness values for individuals, and play a significant role in preserving the evolution quality of the population. In addition, CQGA used a quantum bit as a probabilistic representation because it has higher population diversity attributes than other representations. The simulation outcome exhibits that the proposed algorithm can obtain outstanding convergence performance and reduced maximum cost than benchmark algorithms.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104920"},"PeriodicalIF":3.8,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141047734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CCFTL: A novel continuity compressed page-level flash address mapping method for SSDs","authors":"Liangkuan Su ,&nbsp;Mingwei Lin ,&nbsp;Jianpeng Zhang ,&nbsp;Yubiao Pan","doi":"10.1016/j.jpdc.2024.104917","DOIUrl":"10.1016/j.jpdc.2024.104917","url":null,"abstract":"<div><p>Given the distinctive characteristics of flash-based solid-state drives (SSDs), such as out-of-place update scheme, as compared to traditional block storage devices, a flash translation layer (FTL) has been introduced to hide these features. In the FTL, there is an address translation module that implements the conversion from logical addresses to physical addresses. However, existing address mapping algorithms currently fail to fully exploit the mapping information generated by large I/O requests. First, based on this observation, we propose a novel continuity compressed page-level flash address mapping method (CCFTL). This method effectively compresses the mapping relationship between consecutive logical addresses and physical addresses, enabling the storage of more mapping information within the same mapping cache size. Next, we introduce two-level LRU linked list to mitigate the issue of compressed mapping entry splitting that arises from handling write requests. Finally, our experiments show that CCFTL reduced average response times by 52.67%, 16.81%, and 12.71% compared to DFTL, TPFTL, and MFTL, respectively. As the mapping cache size decreases from 2 MB to 1 MB, then further decreases to 256 KB, 128 KB, and eventually down to 64 KB, CCFTL experiences an average decline ratio of less than 3% in average response time, while the other three algorithms show an average decline ratio of 9.51%.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104917"},"PeriodicalIF":3.8,"publicationDate":"2024-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141046274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Federated variational generative learning for heterogeneous data in distributed environments","authors":"Wei Xie,&nbsp;Runqun Xiong,&nbsp;Jinghui Zhang,&nbsp;Jiahui Jin,&nbsp;Junzhou Luo","doi":"10.1016/j.jpdc.2024.104916","DOIUrl":"10.1016/j.jpdc.2024.104916","url":null,"abstract":"<div><p>Distributedly training models across diverse clients with heterogeneous data samples can significantly impact the convergence of federated learning. Various novel federated learning methods address these challenges but often require significant communication resources and local computational capacity, leading to reduced global inference accuracy in scenarios with imbalanced label data distribution and quantity skew. To tackle these challenges, we propose FedVGL, a Federated Variational Generative Learning method that directly trains a local generative model to learn the distribution of local features and improve global target model inference accuracy during aggregation, particularly under conditions of severe data heterogeneity. FedVGL facilitates distributed learning by sharing generators and latent vectors with the global server, aiding in global target model training from mapping local data distribution to the variational latent space for feature reconstruction. Additionally, FedVGL implements anonymization and encryption techniques to bolster privacy during generative model transmission and aggregation. In comparison to vanilla federated learning, FedVGL minimizes communication overhead, demonstrating superior accuracy even with minimal communication rounds. It effectively mitigates model drift in scenarios with heterogeneous data, delivering improved target model training outcomes. Empirical results establish FedVGL's superiority over baseline federated learning methods under severe label imbalance and data skew condition. In a Label-based Dirichlet Distribution setting with <em>α</em>=0.01 and 10 clients using the MNIST dataset, FedVGL achieved an exceptional accuracy over 97% with the VGG-9 target model.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104916"},"PeriodicalIF":3.8,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141036748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Energy-efficient triple modular redundancy scheduling on heterogeneous multi-core real-time systems","authors":"Hongzhi Xu ,&nbsp;Binlian Zhang ,&nbsp;Chen Pan ,&nbsp;Keqin Li","doi":"10.1016/j.jpdc.2024.104915","DOIUrl":"https://doi.org/10.1016/j.jpdc.2024.104915","url":null,"abstract":"<div><p>Triple modular redundancy (TMR) fault tolerance mechanism can provide almost perfect fault-masking, which has the great potential to enhance the reliability of real-time systems. However, multiple copies of a task are executed concurrently, which will lead to a sharp increase in system energy consumption. In this work, the problem of parallel applications using TMR on heterogeneous multi-core platforms to minimize energy consumption is studied. First, the heterogeneous earliest finish time algorithm is improved, and then according to the given application's deadline constraints and reliability requirements, an algorithm to extend the execution time of the copies is designed. Secondly, based on the properties of TMR, an algorithm for minimizing the execution overhead of the third copy (MEOTC) is designed. Finally, considering the actual situation of task execution, an online energy management (OEM) method is proposed. The proposed algorithms were compared with the state-of-the-art AFTSA algorithm, and the results show significant differences in energy consumption. Specifically, for light fault detection, the energy consumption of the MEOTC and OEM algorithms was found to be 80% and 72% respectively, compared with AFTSA. For heavy fault detection, the energy consumption of MEOTC and OEM was measured at 61% and 55% respectively, compared with AFTSA.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104915"},"PeriodicalIF":3.8,"publicationDate":"2024-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140951658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SSI−FL: Self-sovereign identity based privacy-preserving federated learning","authors":"Rakib Ul Haque ,&nbsp;A.S.M. Touhidul Hasan ,&nbsp;Mohammed Ali Mohammed Al-Hababi ,&nbsp;Yuqing Zhang ,&nbsp;Dianxiang Xu","doi":"10.1016/j.jpdc.2024.104907","DOIUrl":"https://doi.org/10.1016/j.jpdc.2024.104907","url":null,"abstract":"<div><p>Traditional federated learning (<span><math><mi>FL</mi></math></span>) raises security and privacy concerns such as identity fraud, data poisoning attacks, membership inference attacks, and model inversion attacks. In the conventional <span><math><mi>FL</mi></math></span>, any entity can falsify its identity and initiate data poisoning attacks. Besides, adversaries (<span><math><mi>AD</mi></math></span>) holding the updated global model parameters can retrieve the plain text of the dataset by initiating membership inference attacks and model inversion attacks. To the best of our knowledge, this is the first work to propose a self-sovereign identity (<span><math><mi>SSI</mi></math></span>) and differential privacy (<span><math><mi>DP</mi></math></span>) based <span><math><mi>FL</mi></math></span> namely <span><math><mi>SSI</mi><mo>−</mo><mi>FL</mi></math></span> for addressing all the above issues. The first step in the <span><math><mi>SSI</mi><mo>−</mo><mi>FL</mi></math></span> framework involves establishing a secure connection based on blockchain-based <span><math><mi>SSI</mi></math></span>. This secure connection protects against unauthorized access attacks of any <span><math><mi>AD</mi></math></span> and ensures the transmitted data's authenticity, integrity, and availability. The second step applies <span><math><mi>DP</mi></math></span> to protect against model inversion attacks and membership inference attacks. The third step focuses on establishing <span><math><mi>FL</mi></math></span> with a novel hybrid deep learning to achieve better scores than conventional methods. The <span><math><mi>SSI</mi><mo>−</mo><mi>FL</mi></math></span> performance analysis is done based on security, formal, scalability, and score analysis. Moreover, the proposed method outperforms all the state-of-art techniques.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104907"},"PeriodicalIF":3.8,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140924508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Novel multi-cluster workflow system to support real-time HPC-enabled epidemic science: Investigating the impact of vaccine acceptance on COVID-19 spread","authors":"Parantapa Bhattacharya ,&nbsp;Dustin Machi ,&nbsp;Jiangzhuo Chen ,&nbsp;Stefan Hoops ,&nbsp;Bryan Lewis ,&nbsp;Henning Mortveit ,&nbsp;Srinivasan Venkatramanan ,&nbsp;Mandy L. Wilson ,&nbsp;Achla Marathe ,&nbsp;Przemyslaw Porebski ,&nbsp;Brian Klahn ,&nbsp;Joseph Outten ,&nbsp;Anil Vullikanti ,&nbsp;Dawen Xie ,&nbsp;Abhijin Adiga ,&nbsp;Shawn Brown ,&nbsp;Christopher Barrett ,&nbsp;Madhav Marathe","doi":"10.1016/j.jpdc.2024.104899","DOIUrl":"https://doi.org/10.1016/j.jpdc.2024.104899","url":null,"abstract":"<div><p>We present MacKenzie, a HPC-driven multi-cluster workflow system that was used repeatedly to configure and execute fine-grained US national-scale epidemic simulation models during the COVID-19 pandemic. Mackenzie supported federal and Virginia policymakers, in real-time, for a large number of “what-if” scenarios during the COVID-19 pandemic, and continues to be used to answer related questions as COVID-19 transitions to the endemic stage of the disease. MacKenzie is a novel HPC meta-scheduler that can execute US-scale simulation models and associated workflows that typically present significant big data challenges. The meta-scheduler optimizes the total execution time of simulations in the workflow, and helps improve overall human productivity.</p><p>As an exemplar of the kind of studies that can be conducted using Mackenzie, we present a modeling study to understand the impact of vaccine-acceptance in controlling the spread of COVID-19 in the US. We use a 288 million node synthetic social contact network (digital twin) spanning all 50 US states plus Washington DC, comprised of 3300 counties, with 12 billion daily interactions. The highly-resolved agent-based model used for the epidemic simulations uses realistic information about disease progression, vaccine uptake, production schedules, acceptance trends, prevalence, and social distancing guidelines. Computational experiments show that, for the simulation workload discussed above, MacKenzie is able to scale up well to 10 K CPU cores.</p><p>Our modeling results show that, when compared to faster and accelerating vaccinations, slower vaccination rates due to vaccine hesitancy cause averted infections to drop from 6.7M to 4.5M, and averted total deaths to drop from 39.4 K to 28.2 K across the US. This occurs despite the fact that the final vaccine coverage is the same in both scenarios. We also find that if vaccine acceptance could be increased by 10% in all states, averted infections could be increased from 4.5M to 4.7M (a 4.4% improvement) and total averted deaths could be increased from 28.2 K to 29.9 K (a 6% improvement) nationwide.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"191 ","pages":"Article 104899"},"PeriodicalIF":3.8,"publicationDate":"2024-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140906213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Front Matter 1 - Full Title Page (regular issues)/Special Issue Title page (special issues)","authors":"","doi":"10.1016/S0743-7315(24)00075-3","DOIUrl":"https://doi.org/10.1016/S0743-7315(24)00075-3","url":null,"abstract":"","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"190 ","pages":"Article 104911"},"PeriodicalIF":3.8,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0743731524000753/pdfft?md5=04032493554b9c9a6c79c75f9a9aab5d&pid=1-s2.0-S0743731524000753-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140822484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}