Journal of Parallel and Distributed Computing最新文献

{"title":"Fault-tolerance in biswapped multiprocessor interconnection networks","authors":"Basem Assiri ,&nbsp;Muhammad Faisal Nadeem ,&nbsp;Waqar Ali ,&nbsp;Ali Ahmad","doi":"10.1016/j.jpdc.2024.105009","DOIUrl":"10.1016/j.jpdc.2024.105009","url":null,"abstract":"<div><div>Interconnection networks play a vital role in connecting many sets of processor memories, known as processing vertices. Recently, multiprocessor interconnection networks have obtained much attention due to their cost-effectiveness and wide applications in parallel multi-processor systems connecting processors and memory modules. A locating set in a computer network is to select certain nodes, called the locating set, whose positions determine the positions of all other nodes in the network. The locating number is defined as the minimum size of the locating set needed to identify all network vertices. Since, if any single node fails within the locating set, that set is no longer able to identify all the nodes within the network, if the remaining nodes in the locating set can still locate all other network nodes, then it is termed a fault-tolerant locating set. The fault tolerance becomes highly essential in multiprocessor networks, wherein every processor is subject to an absolute failure, to guarantee the system is at full capacity in case one or more components fail. In this study, we determine the fault-tolerant locating number of biswapped networks by considering different classes of networks as base clusters.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142592959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Front Matter 1 - Full Title Page (regular issues)/Special Issue Title page (special issues)","authors":"","doi":"10.1016/S0743-7315(24)00167-9","DOIUrl":"10.1016/S0743-7315(24)00167-9","url":null,"abstract":"","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142572923","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Design and experimental evaluation of algorithms for optimizing the throughput of dispersed computing","authors":"Xiangchen Zhao ,&nbsp;Diyi Hu,&nbsp;Bhaskar Krishnamachari","doi":"10.1016/j.jpdc.2024.104999","DOIUrl":"10.1016/j.jpdc.2024.104999","url":null,"abstract":"<div><div>We introduce three optimized scheduling algorithms for dispersed computing and present JupiterTP, a real-world system built on k8s and the prior Jupiter system, enabling end-to-end computation on distributed clusters. Distinguishing itself from traditional throughput optimization approaches that focus on theory and simulations, our work is the first implementation of such an end-to-end system capable of handling arbitrary DAGs across diverse computing networks, including public clouds, IoT systems, and edge networks. Beyond mere scheduling, JupiterTP integrates profilers, execution, and orchestration engines, offering unified interfaces for additional scheduling algorithm integrations. The system's performance is tested on real clusters and real applications, compared to prior work that relied on simulations alone. We make JupiterTP available to the community as open-source software at <span><span>https://github.com/ANRGUSC/JupiterTP</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hands-on parallel & distributed computing with Raspberry Pi devices and clusters","authors":"Elizabeth Shoop ,&nbsp;Suzanne J. Matthews ,&nbsp;Richard Brown ,&nbsp;Joel C. Adams","doi":"10.1016/j.jpdc.2024.104996","DOIUrl":"10.1016/j.jpdc.2024.104996","url":null,"abstract":"<div><div>Parallel and distributed computing (PDC) concepts are now required topics for accredited undergraduate computer science programs. However, introducing PDC into the CS curriculum is challenging for several reasons, including an instructor's lack of PDC knowledge and difficulties in accessing PDC hardware. This paper addresses both of these challenges by presenting free, interactive, web-based PDC teaching modules using inexpensive Raspberry Pi single board computers (SBCs). Our materials include a free disk image that makes it possible for instructors to build Raspberry Pi clusters in minutes and use our software in a variety of curricular contexts. Our multi-year assessment of these materials on students and faculty members indicates that: (i) our materials increased students' confidence regarding important PDC concepts and motivated them to study PDC further; and (ii) our materials increased faculty members' confidence and preparedness in teaching key PDC concepts at their own institutions.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Semi-static conditions in low-latency C++ for high frequency trading: Better than branch prediction hints","authors":"Paul Alexander Bilokon,&nbsp;Maximilian Lucuta,&nbsp;Erez Shermer","doi":"10.1016/j.jpdc.2024.105000","DOIUrl":"10.1016/j.jpdc.2024.105000","url":null,"abstract":"<div><div>Conditional branches pose a challenge for code optimisation, particularly in low latency settings. We present a novel language construct, referred to as a semi-static condition, which enables programmers to dynamically modify the direction of a branch at run-time by modifying the assembly code within the underlying executable. Subsequently, we explore scenarios where the use of semi-static conditions outperforms traditional conditional branching, highlighting their potential applications in real-time machine learning and high-frequency trading (HFT). Throughout the development process, key considerations of performance, portability, syntax, and security were taken into account.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142552468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Locating a black hole in a dynamic ring","authors":"Giuseppe Antonio Di Luna ,&nbsp;Paola Flocchini ,&nbsp;Giuseppe Prencipe ,&nbsp;Nicola Santoro","doi":"10.1016/j.jpdc.2024.104998","DOIUrl":"10.1016/j.jpdc.2024.104998","url":null,"abstract":"<div><div>In networked environments supporting mobile agents, a pressing problem is the presence of network sites harmful for the agents. In this paper we consider the danger posed by a node that destroys any incoming agent without leaving any trace. Such a dangerous node is known in the literature as a <em>black hole</em> (<span>Bh</span>). The problem of a team of system agents determining its location, known as <em>black hole search</em> (<span>Bhs</span> ), has been extensively studied in the literature under a variety of assumptions, both in synchronous and asynchronous settings. The main complexity parameter of <span>Bhs</span> <!-->is the number of system agents (called <em>size</em>) needed to solve the problem; other parameters are the number of moves (called <em>cost</em>) performed by the agents, and the <em>time</em> until termination.</div><div>In the existing literature, with only a couple of exceptions, all results are based on a common assumption that the network is <em>static</em>, i.e. its topology does not change in time. We consider instead the <span>Bhs</span> <!-->when the network is <em>dynamic</em>: the link structure of the graph changes over time. While time-varying graphs have been the focus of intense research in the last two decades, very little is known on the problem of locating the <span>Bh</span> in such networks.</div><div>In this paper, we contribute to fill this research gap by studying <span>Bhs</span> <!-->in <em>dynamic ring</em> networks, focusing on the <em>1-interval connectivity</em> adversarial dynamics. Feasibility and complexity of the problem depend on many factors, specifically on the size <em>n</em> of the ring, whether or not <em>n</em> is known, and the type of inter-agent communication (whiteboards, tokens, face-to-face, visual). In this paper, we provide a <em>complete</em> feasibility characterization presenting size optimal algorithms. Furthermore, we establish lower bounds on the cost and time of size-optimal solutions and show that our algorithms achieve those bounds.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"OASR-WFBP: An overlapping aware start-up sharing gradient merging strategy for efficient communication in distributed deep learning","authors":"Yingjie Song ,&nbsp;Zhuo Tang ,&nbsp;Yaohua Wang ,&nbsp;Xiong Xiao ,&nbsp;Zhizhong Liu ,&nbsp;Jing Xia ,&nbsp;Kenli Li","doi":"10.1016/j.jpdc.2024.104997","DOIUrl":"10.1016/j.jpdc.2024.104997","url":null,"abstract":"<div><div>Wait-Free-Back-Propagation (WFBP) is a practical method for distributed deep-learning, but it suffers from a high communication overhead. To address this issue, the communication overhead can be reduced by overlapping gradient communication and computation, and sharing the startup time among multiple gradient communication phases. However, existing optimizations choose to share the startup time greedily and fail to coordinately exploit the overlapping opportunity between computation and communication. We propose an overlapping aware startup sharing Wait-Free-Back-Propagation (OASR-WFBP). An analytic model is designed to guide the sharing procedure. Evaluations show that OASR-WFBP achieves a 5%-16% optimization in iteration time over the state-of-the-art WFBP algorithm.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"High-speed turbulent flows towards the exascale: STREAmS-2 porting and performance","authors":"Srikanth Sathyanarayana ,&nbsp;Matteo Bernardini ,&nbsp;Davide Modesti ,&nbsp;Sergio Pirozzoli ,&nbsp;Francesco Salvadore","doi":"10.1016/j.jpdc.2024.104993","DOIUrl":"10.1016/j.jpdc.2024.104993","url":null,"abstract":"<div><div>Exascale High Performance Computing (HPC) represents a tremendous opportunity to push the boundaries of Computational Fluid Dynamics (CFD), but despite the consolidated trend towards the use of Graphics Processing Units (GPUs), programmability is still an issue. STREAmS-2 (Bernardini et al. Comput. Phys. Commun. 285 (2023) 108644) is a compressible solver for canonical wall-bounded turbulent flows capable of harvesting the potential of NVIDIA GPUs. Here we extend the already available CUDA Fortran backend with a novel HIP backend targeting AMD GPU architectures. The main implementation strategies are discussed along with a novel Python tool that can generate the HIP and CPU code versions allowing developers to focus their attention only on the CUDA Fortran backend. Single GPU performance is analyzed focusing on NVIDIA A100 and AMD MI250x cards which are currently at the core of several HPC clusters. The gap between peak GPU performance and STREAmS-2 performance is found to be generally smaller for NVIDIA cards. Roofline analysis allows tracing this behavior to unexpectedly different computational intensities of the same kernel using the two cards. Additional single-GPU comparisons are performed to assess the impact of grid size, number of parallelized loops, thread masking and thread divergence. Parallel performance is measured on the two largest EuroHPC pre-exascale systems, LUMI (AMD GPUs) and Leonardo (NVIDIA GPUs). Strong scalability reveals more than 80% efficiency up to 16 nodes for Leonardo and up to 32 for LUMI. Weak scalability shows an impressive efficiency of over 95% up to the maximum number of nodes tested (256 for LUMI and 512 for Leonardo). This analysis shows that STREAmS-2 is the perfect candidate to fully exploit the power of current pre-exascale HPC systems in Europe, allowing users to simulate flows with over a trillion mesh points, thus reducing the gap between the Reynolds numbers achievable in high-fidelity simulations and those of real engineering applications.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A zero-knowledge proof federated learning on DLT for healthcare data","authors":"Lorenzo Petrosino ,&nbsp;Luigi Masi ,&nbsp;Federico D'Antoni ,&nbsp;Mario Merone ,&nbsp;Luca Vollero","doi":"10.1016/j.jpdc.2024.104992","DOIUrl":"10.1016/j.jpdc.2024.104992","url":null,"abstract":"<div><div>With the increasingly widespread adoption of Healthcare 4.0 practices, new challenges have arisen for the utilization of collected sensitive data. On the one hand, these data have immense potential to unlock valuable insights for personalized medicine, early disease detection, and predictive analysis thanks to the use of Artificial Intelligence. On the other hand, ensuring the protection of patient privacy is of paramount importance to maintain trust and uphold ethical practices within the healthcare system. Classical centralized learning approaches do not fit well with the privacy and security requirements imposed by the law and the sensitivity of treated data, which is why decentralized learning approaches are gaining ground. Among these, Federated Learning (FL) stands out as a viable alternative, providing greater security and performance comparable to classic centralized learning approaches. However, there are still various attacks targeting the local parameters or gradients updated by the participants. Therefore, we present our architecture based on the conjunction of Zero-Knowledge Proof, FL, and blockchain that implements also the decentralized identifier standard. The adoption of this architecture can grant the execution, management, supervision, and updating of the FL process, guaranteeing the resilience of the system and the reliability and traceability of exchanged data. In order to test the performance, robustness, and implementation costs of the proposed architecture, we develop a case study on the prediction of blood glucose levels in people with Type-1-diabetes. The results of our analysis show an improved system in terms of balance between performance privacy and security, guaranteeing high levels of verifiability, therefore proving the proposed architecture suitable for most of the FL processes needed in the healthcare field.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards value-sensitive and poisoning-proof model aggregation for federated learning on heterogeneous data","authors":"Hui Zeng ,&nbsp;Tongqing Zhou ,&nbsp;Yeting Guo ,&nbsp;Zhiping Cai ,&nbsp;Fang Liu","doi":"10.1016/j.jpdc.2024.104994","DOIUrl":"10.1016/j.jpdc.2024.104994","url":null,"abstract":"<div><div>Federated Learning (FL) enables collaborative model training without sharing data, but traditional static averaging of local updates leads to poor performance on heterogeneous data. The following remedies, either by scheduling data distribution or mitigating local discrepancies, predominately fail to handle fine-grained heterogeneity (e.g., local imbalanced labels). To commence, we reveal that static averaging leads to the global model suffering from the <em>mean fallacy</em>. That is, the averaging process favors the local model with large parameters numerically rather than knowledge. To tackle this, we introduce FedVSA, a simple-yet-effective model aggregation framework sensitive to heterogeneous local data merits. Specifically, we invent a new global loss function for FL by prioritizing the valuable local updates, facilitating efficient convergence. We deduce a softmax-based aggregation rule and prove its convergence property via rigorous theoretical analysis. Additionally, we expose poisoning threats of model replacement that utilize the <em>mean fallacy</em> for attacks. To mitigate this threat, we propose a two-step mechanism involving auditing historic local training statistics and analyzing the <em>Shapley Value</em>. Through extensive experiments, we show that FedVSA achieves faster convergence (~1.52×) and higher accuracy (~1.6%) compared to the baselines. It also effectively mitigates poisoning attacks by agilely recovering and returning to normal aggregation.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":null,"pages":null},"PeriodicalIF":3.4,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142440998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}