Performance Evaluation最新文献

{"title":"Performance evaluation of containers for low-latency packet processing in virtualized network environments","authors":"Florian Wiedner,&nbsp;Max Helm,&nbsp;Alexander Daichendt,&nbsp;Jonas Andre,&nbsp;Georg Carle","doi":"10.1016/j.peva.2024.102442","DOIUrl":"10.1016/j.peva.2024.102442","url":null,"abstract":"<div><p>Packet processing in current network scenarios faces complex challenges due to the increasing prevalence of requirements such as low latency, high reliability, and resource sharing. Virtualization is a potential solution to mitigate these challenges by enabling resource sharing and on-demand provisioning; however, ensuring high reliability and ultra-low latency remains a key challenge. Since bare-metal systems are often impractical because of high cost and space usage, and the overhead of virtual machines (VMs) is substantial, we evaluate the utilization of containers as a potential lightweight solution for low-latency packet processing. Herein, we discuss the benefits and drawbacks and encourage container environments in low-latency packet processing when the degree of isolation of customer data is adequate and bare metal systems are unaffordable. Our results demonstrate that containers exhibit similar latency performance with more predictable tail-latency behavior than bare metal packet processing. Moreover, deciding which mainboard architecture to use, especially the cache division, is equally vital as containers are prone to higher latencies on more shared caches between cores especially when other optimizations cannot be used. We show that this has a higher impact on latencies within containers than on bare metal or VMs, resulting in the selection of hardware architectures following optimizations as a critical challenge. Furthermore, the results reveal that the virtualization overhead does not impact tail latencies.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102442"},"PeriodicalIF":1.0,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0166531624000476/pdfft?md5=92c046df1bfad30f8dbdb77dadbb4fd5&pid=1-s2.0-S0166531624000476-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137317","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Security-reliability trade-off analysis for transmit antenna selection in cognitive ambient backscatter communications","authors":"Ahmed N. Elbattrawy ,&nbsp;Ahmed H. Abd El-Malek ,&nbsp;Sherif I. Rabia ,&nbsp;Waheed K. Zahra","doi":"10.1016/j.peva.2024.102441","DOIUrl":"10.1016/j.peva.2024.102441","url":null,"abstract":"<div><p>Massive deployment of IoT devices raises the need for energy-efficient spectrum-efficient low-cost communications. Ambient backscatter communication (AmBC) technology provides a promising solution to achieve that. Moreover, incorporating AmBC with cognitive radio networks (CRNs) achieves better spectrum efficiency; however, this comes with performance drawbacks. In this work, we investigate the security and reliability performance of an underlay CRN with AmBC, where the backscattering device (BD) exploits the radio frequency (RF) signals of the secondary transmitter (ST), and both the ST and the BD share a common receiver. Different from previous work, we consider an ST with multiple antenna. The ST employs a transmit antenna selection (TAS) scheme to enhance the ST performance and overcome the performance degradation caused by the BD interference. TAS exploits multiple antenna diversity with lower hardware complexity and power consumption. Considering the Nakagami-<span><math><mi>m</mi></math></span> fading model, closed-form expressions are derived for the outage probability (OP) and intercept probability (IP) of both the ST and the BD transmissions at the legitimate receiver and the eavesdropper. Moreover, the asymptotic behavior of OPs and IPs is also investigated in the high signal-to-noise ratio regime and the high main-to-eavesdropper ratio regime, respectively. Monte Carlo simulations are performed to validate the derived closed-form expressions. Numerical results show that employing TAS enhances the ST and BD reliability performance by percentages up to 98% and 80%, respectively, at high primary user interference threshold values. Moreover, it results in a better security-reliability trade-off for the ST and the BD.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102441"},"PeriodicalIF":1.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142094687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Lure: A simulator for networks of batteryless intermittent nodes","authors":"Mathew L. Wymore,&nbsp;Rohit Sahu,&nbsp;Thomas Ruminski,&nbsp;Vishal Deep,&nbsp;Morgan Ambourn,&nbsp;Gregory Ling,&nbsp;Vishak Narayanan,&nbsp;William Asiedu,&nbsp;Daji Qiao,&nbsp;Henry Duwe","doi":"10.1016/j.peva.2024.102440","DOIUrl":"10.1016/j.peva.2024.102440","url":null,"abstract":"<div><p>The emerging paradigm of batteryless intermittent sensor networks (BISNs) presents new challenges for researchers of low-power wireless systems and protocols. The nature of these challenges exacerbates the difficulty of evaluating networks of physical sensor nodes, making simulation an even more important component in evaluating performance metrics, such as communication throughput and delay, for BISN designs. To our knowledge, existing simulators and analytical models do not meet the unique needs of BISN research; therefore, we have created a new open-source BISN simulator named <em>Lure</em>. Lure is designed from the ground-up for simulation of batteryless intermittent systems and networks. Written in Python, Lure is powerful, flexible, highly configurable, and supports rapid prototyping of new protocols, systems, and applications, with a low learning curve. In this paper, we present Lure and validate it with experimental data to show that Lure can accurately reflect the reality of BISNs. We then demonstrate the process of applying Lure to research questions in select case studies.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"166 ","pages":"Article 102440"},"PeriodicalIF":1.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0166531624000452/pdfft?md5=1c6343234e3ac7dad5efd12075fa6bfd&pid=1-s2.0-S0166531624000452-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142094686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Inference latency prediction for CNNs on heterogeneous mobile devices and ML frameworks","authors":"Zhuojin Li,&nbsp;Marco Paolieri,&nbsp;Leana Golubchik","doi":"10.1016/j.peva.2024.102429","DOIUrl":"10.1016/j.peva.2024.102429","url":null,"abstract":"<div><p>Due to the proliferation of inference tasks on mobile devices, state-of-the-art neural architectures are typically designed using Neural Architecture Search (NAS) to achieve good tradeoffs between machine learning accuracy and inference latency. While measuring inference latency of a huge set of candidate architectures during NAS is not feasible, latency prediction for mobile devices is challenging, because of hardware heterogeneity, optimizations applied by machine learning frameworks, and diversity of neural architectures. Motivated by these challenges, we first quantitatively assess the characteristics of neural architectures (specifically, convolutional neural networks for image classification), ML frameworks, and mobile devices that have significant effects on inference latency. Based on this assessment, we propose an operation-wise framework which addresses these challenges by developing operation-wise latency predictors and achieves high accuracy in end-to-end latency predictions, as shown by our comprehensive evaluations on multiple mobile devices using multicore CPUs and GPUs. To illustrate that our approach does not require expensive data collection, we also show that accurate predictions can be achieved on real-world neural architectures using only small amounts of profiling data.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102429"},"PeriodicalIF":1.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141714597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Preface: Special issue on ACM/SPEC ICPE 2023","authors":"Antinisca Di Marco (Research Track Co-Chairs) ,&nbsp;Petr Tůma (Research Track Co-Chairs)","doi":"10.1016/j.peva.2024.102430","DOIUrl":"10.1016/j.peva.2024.102430","url":null,"abstract":"","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102430"},"PeriodicalIF":1.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141850193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Retransmission performance in a stochastic geometric cellular network model","authors":"Ingemar Kaj,&nbsp;Taisiia Morozova","doi":"10.1016/j.peva.2024.102428","DOIUrl":"https://doi.org/10.1016/j.peva.2024.102428","url":null,"abstract":"<div><p>Suppose sender–receiver transmission links in a downlink network at a given data rate are subject to fading, path loss, and inter-cell interference, and that transmissions either pass, suffer loss, or incur retransmission delay. We introduce a method to obtain the average activity level of the system required for handling the buffered work and from this derive the resulting coverage probability and key performance measures. The technique involves a family of stationary buffer distributions which is used to solve iteratively a nonlinear balance equation for the unknown busy-link probability and then identify throughput, loss probability, and delay. The results allow for a straightforward numerical investigation of performance indicators, are in special cases explicit and may be easily used to study the trade-off between reliability, latency, and data rate.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102428"},"PeriodicalIF":1.0,"publicationDate":"2024-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0166531624000336/pdfft?md5=97726bced24fd554d6c6c2472331ff66&pid=1-s2.0-S0166531624000336-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141484959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the performance evaluation of distributed join-idle-queue load balancing with and without token withdrawals","authors":"Benny Van Houdt","doi":"10.1016/j.peva.2024.102427","DOIUrl":"https://doi.org/10.1016/j.peva.2024.102427","url":null,"abstract":"<div><p>Distributed Join-Idle-Queue load balancing is known to achieve vanishing waiting times in the large-scale limit provided that the number of dispatchers remains fixed, while the number of servers tends to infinity. When the number of dispatchers <span><math><mi>m</mi></math></span> scales to infinity together with the number of servers <span><math><mi>n</mi></math></span>, such that <span><math><mrow><mi>r</mi><mo>=</mo><mi>n</mi><mo>/</mo><mi>m</mi></mrow></math></span> remains fixed, the large-scale performance of Join-Idle-Queue load balancing is less clear as waiting times no longer vanish.</p><p>In this paper we first discuss some existing mean field models for distributed Join-Idle-Queue load balancing with <span><math><mrow><mi>r</mi><mo>=</mo><mi>n</mi><mo>/</mo><mi>m</mi></mrow></math></span> fixed and explain why the well-known model introduced in Lu et al. (2011) is not exact in the large-scale limit. The inexactness is caused by mixing two variants of distributed Join-Idle-Queue load balancing: a variant with and one without token withdrawals. Next we introduce mean field models for Join-Idle-Queue load balancing with and without token withdrawals, where an idle server places a token at a dispatcher with the shortest among <span><math><mi>d</mi></math></span> randomly chosen dispatchers.</p><p>The introduced mean field models in case of token withdrawals imply that for phase type distributed service times and a total job arrival rate of <span><math><mrow><mi>λ</mi><mi>n</mi><mo>&lt;</mo><mi>n</mi></mrow></math></span>, the response time of a job corresponds to that in a standard M/PH/1 queue with load <span><math><mrow><mi>λ</mi><msub><mrow><mi>q</mi></mrow><mrow><mn>0</mn></mrow></msub></mrow></math></span>. The value of <span><math><msub><mrow><mi>q</mi></mrow><mrow><mn>0</mn></mrow></msub></math></span> can be determined numerically and depends on <span><math><mrow><mi>λ</mi><mo>,</mo><mi>r</mi></mrow></math></span> and <span><math><mi>d</mi></math></span>, but not on the job size distribution (apart from its mean). This simple behavior is lost if token withdrawals do not take place. For the models without withdrawals we develop fast numerical algorithms to determine the performance. We present simulation experiments that suggest that the unique fixed point of the introduced mean field models provides exact results in the large-scale limit.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102427"},"PeriodicalIF":1.0,"publicationDate":"2024-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141542944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Energy performance of off-grid green cellular base stations","authors":"Godlove Suila Kuaban ,&nbsp;Erol Gelenbe ,&nbsp;Tadeusz Czachórski ,&nbsp;Piotr Czekalski ,&nbsp;Valery Nkemeni","doi":"10.1016/j.peva.2024.102426","DOIUrl":"10.1016/j.peva.2024.102426","url":null,"abstract":"<div><p>The most energy-hungry parts of mobile networks are the base station sites, which consume around <span><math><mrow><mn>60</mn><mo>−</mo><mn>80</mn><mtext>%</mtext></mrow></math></span> of their total energy. One of the approaches for relieving this energy pressure on the electricity grid infrastructure and reducing the Operational Expenditures (OPEX) is to power base stations with renewable energy. However, the design of a green mobile network requires the dimensioning of the energy harvesting and storage systems through the estimation of the network’s energy demand. Therefore, this paper develops a diffusion-based modelling framework for solar-powered green off-grid base station sites. We apply this framework to evaluate the energy performance of homogeneous and hybrid energy storage systems supplied by harvested solar energy. We present the complete analysis, with numerical examples, to study the relationship between the design parameters and the energy performance metrics. The numerical computations demonstrate how the proposed framework can be applied to evaluate homogeneous and unconventional hybrid energy storage systems.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102426"},"PeriodicalIF":1.0,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141408839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"User-centric Markov reward model for state-dependent Erlang loss systems","authors":"Tobias Hoßfeld ,&nbsp;Poul E. Heegaard ,&nbsp;Martín Varela ,&nbsp;Michael Jarschel","doi":"10.1016/j.peva.2024.102425","DOIUrl":"10.1016/j.peva.2024.102425","url":null,"abstract":"<div><p>Markov reward models are commonly used in the analysis of systems by integrating a reward rate to each system state. Typically, rewards are defined based on system states and reflect the system’s perspective. From a user’s point of view, it is important to consider the changing system conditions and dynamics while the user consumes a service. The key contributions of this paper are proper definitions for (i) system-centric reward and (ii) user-centric reward of the Erlang loss model M/M/n-0 and M/M(x)/n with state-dependent service rates, as well as (iii) the analysis of the relationships between those metrics. Our key result allows a simple computation of the user-centric rewards. The differences between the system-centric and the user-centric rewards are demonstrated for a real-world cloud gaming use case. To the best of our knowledge, this is the first analysis showing the relationship between user-centric rewards and system-centric rewards. This work gives relevant and important insights in how to integrate the user’s perspective in the analysis of Markov reward models and is a blueprint for the analysis of other services beyond cloud gaming while also considering user engagement.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102425"},"PeriodicalIF":1.0,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0166531624000300/pdfft?md5=02058144737eebae5dcee4c833f88db3&pid=1-s2.0-S0166531624000300-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141397922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A comprehensive exploration of approximate DNN models with a novel floating-point simulation framework","authors":"Myeongjin Kwak,&nbsp;Jeonggeun Kim,&nbsp;Yongtae Kim","doi":"10.1016/j.peva.2024.102423","DOIUrl":"https://doi.org/10.1016/j.peva.2024.102423","url":null,"abstract":"<div><p>This paper introduces <em>TorchAxf</em><span>¹</span>, a framework for fast simulation of diverse approximate deep neural network (DNN) models, including spiking neural networks (SNNs). The proposed framework utilizes various approximate adders and multipliers, supports industrial standard reduced precision floating-point formats, such as <span>bfloat16</span>, and accommodates user-customized precision representations. Leveraging GPU acceleration on the PyTorch framework, <em>TorchAxf</em> accelerates approximate DNN training and inference. In addition, it allows seamless integration of arbitrary approximate arithmetic algorithms with C/C++ behavioral models to emulate approximate DNN hardware accelerators.</p><p>We utilize the proposed <em>TorchAxf</em> framework to assess twelve popular DNN models under approximate multiply-and-accumulate (MAC) operations. Through comprehensive experiments, we determine the suitable degree of floating-point arithmetic approximation for these DNN models without significant accuracy loss and offer the optimal reduced precision formats for each DNN model. Additionally, we demonstrate that approximate-aware re-training can rectify errors and enhance pre-trained DNN models under reduced precision formats. Furthermore, <em>TorchAxf</em>, operating on GPU, remarkably reduces simulation time for complex DNN models using approximate arithmetic by up to 131.38<span><math><mo>×</mo></math></span> compared to the baseline optimized CPU implementation. Finally, we compare the proposed framework with state-of-the-art frameworks to highlight its superiority.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"165 ","pages":"Article 102423"},"PeriodicalIF":2.2,"publicationDate":"2024-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141239841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}