Performance Evaluation最新文献

{"title":"Analysis of a queue-length-dependent vacation queue with bulk service, N-policy, set-up time and cost optimization","authors":"P. Karan,&nbsp;S. Pradhan","doi":"10.1016/j.peva.2024.102459","DOIUrl":"10.1016/j.peva.2024.102459","url":null,"abstract":"<div><div>Due to the extensive applications of bulk service vacation queues in manufacturing industries, inventory systems, wireless sensor networks for deducing energy consumption etc., in this article, we analyze the steady-state behavior of an infinite-buffer group arrival bulk service queue with vacation scenario, set-up time and <span><math><mi>N</mi></math></span>-threshold policy. Here the customers arrive according to the compound Poisson process and the server originates the service process with minimum ‘<span><math><mi>a</mi></math></span>’ customers and can give service to maximum ‘<span><math><mi>b</mi></math></span>’ customers at a time. We adopt batch-size-dependent service time as well as queue-length-dependent vacation duration which improve the system’s performance significantly. The <span><math><mi>N</mi></math></span>-threshold policy is proposed to awaken the server from a vacation/dormant state where the service station starts the set-up procedure after the accumulation of pre-decided ‘<span><math><mi>N</mi></math></span>’ customers. Using the supplementary variable technique, firstly, we derive the set of system equations in the steady-state. After that, we obtain the bivariate probability generating functions (pgfs) of queue content and size of the departing batch, the queue content and type of vacation taken by the server at vacation completion epoch and also the single pgf of queue content at the end of set-up time. We extract the joint distribution from those generating functions using the roots method and derive a simple algebraic relation between the probabilities at departure and arbitrary epoch. We also provide assorted numerical results to validate our proposed methodology and obtained theoretical results. The impact of the system parameters on the performance measures is presented through tables and graphs. Finally, a cost optimization function is provided for the benefit of system designers.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"167 ","pages":"Article 102459"},"PeriodicalIF":1.0,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142723450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"FedCust: Offloading hyperparameter customization for federated learning","authors":"Syed Zawad ,&nbsp;Xiaolong Ma ,&nbsp;Jun Yi ,&nbsp;Cheng Li ,&nbsp;Minjia Zhang ,&nbsp;Lei Yang ,&nbsp;Feng Yan ,&nbsp;Yuxiong He","doi":"10.1016/j.peva.2024.102450","DOIUrl":"10.1016/j.peva.2024.102450","url":null,"abstract":"<div><div>Federated Learning (FL) is a new machine learning paradigm that enables training models collaboratively across clients without sharing private data. In FL, data is non-uniformly distributed among clients (i.e., data heterogeneity) and cannot be redistributed nor monitored like in conventional machine learning due to privacy constraints. Such data heterogeneity and privacy requirements bring new challenges for learning hyperparameter optimization as the training dynamics change across clients even within the same training round and they are difficult to be measured due to privacy. The state-of-the-art in hyperparameter customization can greatly improve FL model accuracy but also incur significant computing overheads and power consumption on client devices, and slowdown the training process. To address the prohibitively expensive cost challenge, we explore the possibility of offloading hyperparameter customization to servers. We propose <em>FedCust</em>, a framework that offloads expensive hyperparameter customization cost from the client devices to the central server without violating privacy constraints. Our key discovery is that it is not necessary to do hyperparameter customization for every client, and clients with similar data heterogeneity can use the same hyperparameters to achieve good training performance. We propose heterogeneity measurement metrics for clustering clients into groups such that clients within the same group share hyperparameters. <em>FedCust</em> uses the proxy data from initial model design to emulate different heterogeneity groups and perform hyperparameter customization on the server side without accessing client data nor information. To make the hyperparameter customization scalable, <em>FedCust</em> further employs a Bayesian-strengthened tuner to significantly accelerates the hyperparameter customization speed. Extensive evaluation demonstrates that <em>FedCust</em> achieves up to 7/2/4/4/6% better accuracy than the widely adopted one-size-fits-all approach on popular FL benchmarks FEMNIST, Shakespeare, Cifar100, Cifar10, and Fashion-MNIST respectively, while being scalable and reducing computation, memory, and energy consumption on the client devices, without compromising privacy constraints.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"167 ","pages":"Article 102450"},"PeriodicalIF":1.0,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142704986","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Trust your local scaler: A continuous, decentralized approach to autoscaling","authors":"Martin Straesser ,&nbsp;Stefan Geissler ,&nbsp;Stanislav Lange ,&nbsp;Lukas Kilian Schumann ,&nbsp;Tobias Hossfeld ,&nbsp;Samuel Kounev","doi":"10.1016/j.peva.2024.102452","DOIUrl":"10.1016/j.peva.2024.102452","url":null,"abstract":"<div><div>Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"167 ","pages":"Article 102452"},"PeriodicalIF":1.0,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142662941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enabling scalable and adaptive machine learning training via serverless computing on public cloud","authors":"Ahsan Ali ,&nbsp;Xiaolong Ma ,&nbsp;Syed Zawad ,&nbsp;Paarijaat Aditya ,&nbsp;Istemi Ekin Akkus ,&nbsp;Ruichuan Chen ,&nbsp;Lei Yang ,&nbsp;Feng Yan","doi":"10.1016/j.peva.2024.102451","DOIUrl":"10.1016/j.peva.2024.102451","url":null,"abstract":"<div><div>In today’s production machine learning (ML) systems, models are continuously trained, improved, and deployed. ML design and training are becoming a continuous workflow of various tasks that have dynamic resource demands. Serverless computing is an emerging cloud paradigm that provides transparent resource management and scaling for users and has the potential to revolutionize the routine of ML design and training. However, hosting modern ML workflows on existing serverless platforms has non-trivial challenges due to their intrinsic design limitations such as stateless nature, limited communication support across function instances, and limited function execution duration. These limitations result in a lack of an overarching view and adaptation mechanism for training dynamics, and an amplification of existing problems in ML workflows.</div><div>To address the above challenges, we propose <span>SMLT</span>, an automated, scalable and adaptive serverless framework on public cloud to enable efficient and user-centric ML design and training. <span>SMLT</span> employs an automated and adaptive scheduling mechanism to dynamically optimize the deployment and resource scaling for ML tasks during training. <span>SMLT</span> further enables user-centric ML workflow execution by supporting user-specified training deadline and budget limit. In addition, by providing an end-to-end design, <span>SMLT</span> solves the intrinsic problems in public cloud serverless platforms such as the communication overhead, limited function execution duration, need for repeated initialization, and also provides explicit fault tolerance for ML training. <span>SMLT</span> is open-sourced and compatible with all major ML frameworks. Our experimental evaluation with large, sophisticated modern ML models demonstrates that <span>SMLT</span> outperforms the state-of-the-art VM-based systems and existing public cloud serverless ML training frameworks in both training speed (up to 8<span><math><mo>×</mo></math></span>) and monetary cost (up to 3<span><math><mo>×</mo></math></span>).</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"167 ","pages":"Article 102451"},"PeriodicalIF":1.0,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142704985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Symbolic state-space exploration meets statistical model checking","authors":"Mathis Niehage,&nbsp;Anne Remke","doi":"10.1016/j.peva.2024.102449","DOIUrl":"10.1016/j.peva.2024.102449","url":null,"abstract":"<div><div>Efficient reachability analysis, as well as statistical model checking have been proposed for the evaluation of Hybrid Petri nets with general transitions (HPnGs). Both have different (dis-)advantages. The performance of statistical simulation suffers in large models and the number of required simulation runs to achieve a relatively small confidence interval increases considerably. The approach introduced for analytical reachability analysis of HPnGs, however, becomes infeasible for a large number of random variables. To overcome these limitations, this paper applies statistical model checking (SMC) for a stochastic variant of the Signal Temporal Logic (STL) to a pre-computed symbolic state-space representation of HPnGs, i.e., the Parametric Location Tree (PLT), which has previously been used for model checking HPnGs. Furthermore, we define how to reduce the PLT for a given <em>state-based</em> and <em>path-based</em> STL property, by introducing a three-valued interpretation of a given STL property for every location of the PLT. This paper applies learning in the presence of nondeterminism and considers four different scheduler classes. The proposed improvement is especially useful if a large number of training runs is necessary to optimize the probability that a given STL property holds. A case study on a water tank model shows the feasibility of the approach, as well as improved computation times, when applying the above-mentioned reduction for varying time bounds. We validate our results with existing analytical and simulation tools, as applicable for different types of schedulers.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"167 ","pages":"Article 102449"},"PeriodicalIF":1.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142662857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Spatial queues with nearest neighbour shifts","authors":"B.R. Vinay Kumar ,&nbsp;Lasse Leskelä","doi":"10.1016/j.peva.2024.102448","DOIUrl":"10.1016/j.peva.2024.102448","url":null,"abstract":"<div><div>This work studies queues in a Euclidean space. Consider <span><math><mi>N</mi></math></span> servers that are distributed uniformly in <span><math><msup><mrow><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow></mrow><mrow><mi>d</mi></mrow></msup></math></span>. Customers arrive at the servers according to independent stationary processes. Upon arrival, they probabilistically decide whether to join the queue they arrived at, or shift to one of the nearest neighbours. Such shifting strategies affect the load on the servers, and may cause some of the servers to become overloaded. We derive a law of large numbers and a central limit theorem for the fraction of overloaded servers in the system as the total number of servers <span><math><mrow><mi>N</mi><mo>→</mo><mi>∞</mi></mrow></math></span>. Additionally, in the one-dimensional case (<span><math><mrow><mi>d</mi><mo>=</mo><mn>1</mn></mrow></math></span>), we evaluate the expected fraction of overloaded servers for any finite <span><math><mi>N</mi></math></span>. Numerical experiments are provided to support our theoretical results. Typical applications of the results include electric vehicles queueing at charging stations, and queues in airports or supermarkets.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102448"},"PeriodicalIF":1.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An experimental study on beamforming architecture and full-duplex wireless across two operational outdoor massive MIMO networks","authors":"Hadi Hosseini ,&nbsp;Ahmed Almutairi ,&nbsp;Syed Muhammad Hashir ,&nbsp;Ehsan Aryafar ,&nbsp;Joseph Camp","doi":"10.1016/j.peva.2024.102447","DOIUrl":"10.1016/j.peva.2024.102447","url":null,"abstract":"<div><div>Full-duplex (FD) wireless communication refers to a communication system in which both ends of a wireless link transmit and receive data simultaneously in the same frequency band. One of the major challenges of FD communication is self-interference (SI), which refers to the interference caused by transmitting elements of a radio to its own receiving elements. Fully digital beamforming is a technique used to conduct beamforming and has been recently repurposed to also reduce SI. However, the cost of fully digital systems dramatically increases with the number of antennas, as each antenna requires an independent Tx-Rx RF chain. Hybrid beamforming systems use a much smaller number of RF chains to feed the same number of antennas, and hence can significantly reduce the deployment cost. In this paper, we aim to quantify the performance gap between these two radio architectures in terms of SI cancellation and system capacity in FD multi-user Multiple Input Multiple Output (MIMO) setups. We first obtained over-the-air channel measurement data on two outdoor massive MIMO deployments over the course of three months. We next study SoftNull and M-HBFD as two state-of-the-art transmit (Tx) beamforming based FD systems, and introduce two new joint transmit-receive (Tx-Rx) beamforming based FD systems named TR-FD<span><math><msup><mrow></mrow><mrow><mn>2</mn></mrow></msup></math></span> and TR-HBFD for fully digital and hybrid radio architectures, respectively. We show that the hybrid beamforming systems can achieve 80%–99% of the fully digital systems capacity, depending on the number of users. Our results show that it is possible to get many benefits associated with fully digital massive MIMO systems with a hybrid beamforming architecture at a fraction of the cost.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102447"},"PeriodicalIF":1.0,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142427974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Probabilistic performance evaluation of the class-A device in LoRaWAN protocol on the MAC layer","authors":"Mi Chen ,&nbsp;Lynda Mokdad ,&nbsp;Jalel Ben-Othman ,&nbsp;Jean-Michel Fourneau","doi":"10.1016/j.peva.2024.102446","DOIUrl":"10.1016/j.peva.2024.102446","url":null,"abstract":"<div><div>LoRaWAN is a network technology that provides a long-range wireless network while maintaining low energy consumption. It adopts the pure Aloha MAC protocol and the duty-cycle limitation at both uplink and downlink on the MAC layer to conserve energy. Additionally, LoRaWAN employs orthogonal parameters to mitigate collisions. However, synchronization in star-of-star topology networks and the complicated collision mechanism make it challenging to conduct a quantitative performance evaluation in LoRaWAN. Our previous work proposes a Probabilistic Timed Automata (PTA) model to represent the uplink transmission in LoRaWAN. It is a mathematical model that presents the nondeterministic and probabilistic choice with time passing. However, this model remains a work in progress. This study extends the PTA model to depict Class-A devices in the LoRaWAN protocol. The complete characteristics of LoRaWAN’s MAC layer, such as duty-cycle limits, bidirectional communication, and confirmed message transmission, are accurately modeled. Furthermore, a comprehensive collision model is integrated into the PTA. Various properties are verified using the probabilistic model checker PRISM, and quantitative properties are calculated under diverse scenarios. This quantitative analysis provides valuable insights into the performance and behavior of LoRaWAN networks under varying conditions.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102446"},"PeriodicalIF":1.0,"publicationDate":"2024-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142314790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimal resource management for multi-access edge computing without using cross-layer communication","authors":"Ankita Koley,&nbsp;Chandramani Singh","doi":"10.1016/j.peva.2024.102445","DOIUrl":"10.1016/j.peva.2024.102445","url":null,"abstract":"<div><p>We consider a Multi-access Edge Computing (MEC) system with a set of users, a base station (BS) with an attached MEC server, and a cloud server. The users can serve the service requests locally or can offload them to the BS which in turn can serve a subset of the offloaded requests at the MEC and can forward the requests to the cloud server. The user devices and the MEC server can be dynamically configured to serve different classes of services. The service requests offloaded to the BS incur offloading costs and those forwarded to the cloud incur additional costs; the costs could represent service charges or delays. Aggregate cost minimization subject to stability warrants optimal service scheduling and offloading at the users and the MEC server, at their application layers, and optimal uplink packet scheduling at the users’ MAC layers. Classical back-pressure (BP) based solutions entail cross-layer message exchange, and hence are not viable. We propose virtual queue-based drift-plus-penalty algorithms that are throughput optimal, and achieve the optimal delay arbitrarily closely but do not require cross-layer communication. We first consider an MEC system without local computation, and subsequently, extend our framework to incorporate local computation also. We demonstrate that the proposed algorithms offer almost the same performance as BP based algorithms. These algorithms contain tuneable parameters that offer a trade off between queue lengths at the users and the BS and the offloading costs.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102445"},"PeriodicalIF":1.0,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142241260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Efficient handling of sporadic messages in FlexRay","authors":"Sunil Kumar P.R. ,&nbsp;Manjunath A.S. ,&nbsp;Vinod V.","doi":"10.1016/j.peva.2024.102444","DOIUrl":"10.1016/j.peva.2024.102444","url":null,"abstract":"<div><p>FlexRay is a high-bandwidth protocol that supports hard-deadline periodic and sporadic traffic in modern in-vehicle communication networks. The dynamic segment of FlexRay is used for transmitting hard deadline sporadic messages. In this paper, we describe an algorithm to minimize the duration of the dynamic segment in a FlexRay cycle, yielding better results than existing algorithms in the literature. The proposed algorithm consists of two phases. In the first phase, we assume that a sporadic message instance contends for service with only one instance of each higher-priority message. The lower bound provided by the first phase serves as the initial guess for the number of mini-slots used in the second phase, where an exact scheduling analysis is performed. In the second phase, a sporadic message may contend for service with multiple instances of each higher-priority message. This two-phase approach is efficient because the first phase has low overhead and its estimate greatly reduces the number of iterations needed in the second phase. We conducted experiments using the dataset provided in the literature as well as the SAE benchmark dataset. The experimental results demonstrate superior bandwidth minimization and computational efficiency compared to other algorithms.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102444"},"PeriodicalIF":1.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}