Performance Evaluation最新文献

{"title":"Strategic pricing and ranking in recommendation systems with seller competition","authors":"Tushar Shankar Walunj ,&nbsp;Veeraruna Kavitha ,&nbsp;Jayakrishnan Nair ,&nbsp;Priyank Agarwal","doi":"10.1016/j.peva.2025.102518","DOIUrl":"10.1016/j.peva.2025.102518","url":null,"abstract":"<div><div>We study a recommendation system where sellers compete for visibility by strategically offering commissions to a platform that optimally curates a ranked menu of items and their respective prices for each customer. Customers interact sequentially with the menu following a cascade click model, and their purchase decisions are influenced by price sensitivity and positions of various items in the menu. We model the seller-platform interaction as a Stackelberg game with sellers as leaders and consider two different games depending on whether the prices are set by the platform or prefixed by the sellers.</div><div>It is complicated to find the optimal policy of the platform in complete generality; hence, we solve the problem in an important asymptotic regime. In fact, both the games coincide in this regime, obtained by decreasing the customer exploration rates <span><math><mi>γ</mi></math></span> to zero (in this regime, the customers explore fewer items). Through simulations, we illustrate that the limit game well approximates the original game(s) even for exploration probabilities as high as 0.4 (the differences are around 2.54%). Further, the second game (where the sellers prefix the prices) coincides with the approximate game for all values of <span><math><mi>γ</mi></math></span>.</div><div>The core contribution of this paper lies in characterizing the equilibrium structure of the limit game. We show that when sellers are of different strengths, the standard Nash equilibrium does not exist due to discontinuities in utilities. We instead establish the existence of a novel equilibrium solution, namely ‘<span><math><mi>μ</mi></math></span>-connected equilibrium cycle’ (<span><math><mi>μ</mi></math></span>-EC), which captures oscillatory strategic responses at the equilibrium. Unlike the (pure) Nash equilibrium, which defines a fixed point of mutual best responses, this is a set-valued solution concept of connected components. This novel equilibrium concept identifies a Cartesian product set of connected action profiles in the continuous action space that satisfies four important properties: stability against external deviations, no external chains, instability against internal deviations, and minimality. We extend a recently introduced solution concept <em>equilibrium cycle</em> to include stability against measure-zero violations and avoid some topological difficulties to propose <span><math><mi>μ</mi></math></span>-EC.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102518"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145516997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bayesian optimization for dynamic pricing and learning","authors":"Anush Anand,&nbsp;Pranav Agrawal,&nbsp;Tejas Bodas","doi":"10.1016/j.peva.2025.102519","DOIUrl":"10.1016/j.peva.2025.102519","url":null,"abstract":"<div><div>Dynamic pricing is the practice of adjusting the selling price of a product to maximize a firm’s revenue by responding to market demand. The literature typically distinguishes between two settings: infinite inventory, where the firm has unlimited stock and time to sell, and finite inventory, where both inventory and selling horizon are limited. In both cases, the central challenge lies in the fact that the demand function — how sales respond to price — is unknown and must be learned from data. Traditional approaches often assume a specific parametric form for the demand function, enabling the use of reinforcement learning (RL) to identify near-optimal pricing strategies. However, such assumptions may not hold in real-world scenarios, limiting the applicability of these methods.</div><div>In this work, we propose a Gaussian Process (GP) based nonparametric approach to dynamic pricing that avoids restrictive modeling assumptions. We treat the demand function as a black-box function of the price and develop pricing algorithms based on Bayesian Optimization (BO)—a sample-efficient method for optimizing unknown functions. We present BO-based algorithms tailored for both infinite and finite inventory settings and provide regret guarantees for both regimes, thereby quantifying the learning efficiency of our methods. Through extensive experiments, we demonstrate that our BO-based methods outperform several state-of-the-art RL algorithms in terms of revenue, while requiring fewer assumptions and offering greater robustness. This highlights Bayesian Optimization as a powerful and practical tool for dynamic pricing in complex, uncertain environments.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102519"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145568452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparing approximations in the ASIP tandem queue","authors":"Wesley Geelen ,&nbsp;Maria Vlasiou ,&nbsp;Yaron Yeger","doi":"10.1016/j.peva.2025.102523","DOIUrl":"10.1016/j.peva.2025.102523","url":null,"abstract":"<div><div>The Asymmetric Inclusion Process (ASIP) models unidirectional transport with particle clustering, yet remains analytically intractable for systems beyond small sizes. To address this, we develop two approximation methods: the replica mean-field (RMF) limit, providing a first-order approximation, and the power series algorithm (PSA), a numerical scheme based on traffic intensity expansions. We evaluate these approximations against Monte Carlo simulations for general systems and prior exact results for homogeneous ASIP systems. Both methods yield accurate estimates, with PSA closely matching simulations for both homogeneous and heterogeneous systems, while RMF performing well for early sites but being slightly impacted downstream or as load increases. These approximations offer practical and computationally efficient alternatives to simulation, enabling detailed performance analysis of ASIP tandem queues where exact solutions are unavailable.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102523"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145466620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimizing resource allocation for geographically-distributed inference by large language models","authors":"Tingyang Sun ,&nbsp;Ting He ,&nbsp;Bo Ji ,&nbsp;Parimal Parag","doi":"10.1016/j.peva.2025.102527","DOIUrl":"10.1016/j.peva.2025.102527","url":null,"abstract":"<div><div>Large language models (LLMs) have demonstrated extraordinary performance in many artificial intelligence (AI) tasks but are expensive to use, even after training, due to their requirement of high-end GPUs. Recently, a distributed system called PETALS was developed to lower the barrier for deploying LLMs by splitting the model blocks across multiple servers with low-end GPUs distributed over the Internet, which was much faster than swapping the model parameters between the GPU memory and other cheaper but slower local storage media. However, the performance of such a distributed system critically depends on the resource allocation, and how to do so optimally remains unknown. In this work, we present the first systematic study of the resource allocation problem in distributed LLM inference, with focus on two important decisions: block placement and request routing. Our main results include: (i) experimentally validated performance models that can predict the inference performance under given block placement and request routing decisions, (ii) a formulation of the offline optimization of block placement and request routing as a mixed integer linear programming (MILP) problem together with the NP-hardness proof and a polynomial-complexity algorithm with guaranteed performance, and (iii) an adaptation of the offline algorithm for the online setting with the same performance guarantee under bounded load. Through both experiments and experimentally-validated simulations, we have verified that the proposed solution can substantially reduce the inference time compared to the state-of-the-art solution in diverse settings with geographically-distributed servers. As a byproduct, we have also developed a light-weighted CPU-only simulator capable of predicting the performance of distributed LLM inference on GPU servers, which can evaluate large deployments and facilitate future research for researchers with limited GPU access.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102527"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145517000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Designing asymptotically optimal policies for continuous-time weakly coupled MDPs","authors":"Matthieu Perbal,&nbsp;Balakrishna Prabhu,&nbsp;Ina Maria Verloop","doi":"10.1016/j.peva.2025.102528","DOIUrl":"10.1016/j.peva.2025.102528","url":null,"abstract":"<div><div>We study the continuous-time Weakly Coupled Markov Decision Process (WCMDP), a class of decision problems involving multiple interacting Markov processes (or “arms”) subject to shared resource constraints. We present a general framework for policy design using a combination of an underlying Markov process and a sequence of mappings. Our main theoretical result establishes sufficient conditions on the Markov process and mapping defining the policy, such that it is asymptotically optimal as the number of arms grows.</div><div>We construct both deterministic and randomized policies based on a solution to a linear program (LP). These policies initially assign actions to arms — either proportionally (deterministic) or randomly — based on conditional measures derived from the LP. As this initial allocation may violate feasibility constraints, we introduce a mapping to enforce the resource constraints are satisfied. Finally, we numerically evaluate and compare the performance of our proposed policies, both deterministic and randomized, under different choices of mappings.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102528"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145517052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"User equilibria in heterogeneous discriminatory processor sharing queues","authors":"Dieter Fiems ,&nbsp;Balakrishna J. Prabhu","doi":"10.1016/j.peva.2025.102510","DOIUrl":"10.1016/j.peva.2025.102510","url":null,"abstract":"<div><div>We consider a strategic routing game for a two-class discriminatory processor-sharing queue with an additional cost for joining the premium class. We show that, depending on the specific parameters of the system, various equilibria can coexist, including equilibria where the queueing system is not ergodic for the equilibrium traffic split. We also investigate how the server can select the priority of the classes and the fees charged to the customers to maximise its revenue. We then investigate learning strategies that converge to particular equilibria. Finally, we study how the elasticity of the traffic demand affects the equilibrium solutions.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102510"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145159602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The last, the least, and the urgent: Fluid modeling and performance equivalence for scheduling policies in partial service queues with abandonment","authors":"Andres Ferragut,&nbsp;Diego Goldsztajn,&nbsp;Fernando Paganini","doi":"10.1016/j.peva.2025.102517","DOIUrl":"10.1016/j.peva.2025.102517","url":null,"abstract":"<div><div>In several queueing systems, arriving tasks or customers have both service and timing requirements, the latter expressed as a deadline for the task to be served. These systems with customer abandonment have a long and rich history in queueing theory, and have several applications in task scheduling in computer systems, operations research problems, etc. A common feature in all of these works is that they deal with customers reneging from the system only while in the queue, and not during service. However, in several applications, customers may also leave during service, and the partial work performed by the system during their stay is still useful.</div><div>In this paper we analyze these partial service queues with abandonment in a many-server setting, characterizing the equilibrium performance of several policies in terms of the amount of service attained by tasks. For this purpose, we develop fluid models with two-dimensional independent variables, corresponding to service and sojourn times, which take the form of partial differential equations expressed in weak form. These fluid models allow us to consider general and possibly correlated service and timing requirements, as well as a wide range of service disciplines. In particular, we focus on Earliest-Deadline-First, Least-Attained-Service and Last-Come-First-Served, and establish that all three policies have the same equilibrium performance, even though the latter two do not need any information about deadlines. This striking property means that designers may avoid the difficult job of estimating deadlines without incurring a performance penalty. The fluid model conclusions are validated by extensive numerical experiments.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102517"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145568453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ASIP tandem queues with Lévy input and consumption","authors":"Onno Boxma ,&nbsp;Offer Kella ,&nbsp;Jacques Resing","doi":"10.1016/j.peva.2025.102513","DOIUrl":"10.1016/j.peva.2025.102513","url":null,"abstract":"<div><div>We consider an ASIP (asymmetric inclusion process) tandem queue, in which the first queue receives a fluid input according to a nondecreasing Lévy process. Each queue has a gate that opens after independent, exponentially distributed periods for an infinitesimal amount of time, allowing the queue content to move to the next queue. In addition, again at independent exponentially distributed instants, a fixed fraction of a queue content is removed from the system.</div><div>For this model, restricting ourselves to steady state, we obtain the following results. (i) We derive the buffer content distribution of the first queue. (ii) For the 2-queue model, we obtain relatively simple explicit expressions for the Laplace transform of the joint buffer content in several special cases. (iii) Asymptotic results are obtained for the 2-queue model when the above-mentioned buffer content removal process approaches a shot-noise process. (iv) For the general <span><math><mi>n</mi></math></span>-queue case, we show how all moments of the buffer contents at all queues can be obtained. (v) For the general <span><math><mi>n</mi></math></span>-queue case, we sketch an approximation method that allows one in principle to derive tractable expressions for the Laplace transform of the buffer content at each queue, with exact mean buffer contents at all queues.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102513"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145417138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Switching constrained OCO with predictions and feedback delays","authors":"Weici Pan,&nbsp;Zhenhua Liu","doi":"10.1016/j.peva.2025.102524","DOIUrl":"10.1016/j.peva.2025.102524","url":null,"abstract":"<div><div>We examine Online Convex Optimization (OCO) problems with feedback delay and a strict limit on decision switching, which exists in applications such as smart grid and learning. Existing algorithms developed for traditional OCO struggle in this setting, often violating switching constraints or incurring high regrets, as evidenced by simulations. In this paper, we establish a new algorithm, Follow-the-Maximally-Coupled-Latest-Leader (FMCLL), achieving a near-optimal regret of <span><math><mrow><mi>O</mi><mrow><mo>(</mo><mi>T</mi><mo>/</mo><mi>S</mi><mo>)</mo></mrow></mrow></math></span> for such problems with delayed feedbacks and a bound of <span><math><mrow><mi>O</mi><mrow><mo>(</mo><mi>T</mi><mo>/</mo><mi>S</mi><mo>−</mo><mi>τ</mi><mo>)</mo></mrow></mrow></math></span> for problems with predictions of <span><math><mi>τ</mi></math></span> rounds, even though the player is only allowed to move at most <span><math><mi>S</mi></math></span> times in expectation across <span><math><mi>T</mi></math></span> rounds. FMCLL meets performance bounds in scenarios with delays and predictions by using maximal coupling sampling to inform algorithm design for switching-constrained problems. To better apply our framework to practical applications, we also extend the algorithm and results to the bandit feedback setting. Simulations demonstrate FMCLL’s superiority over traditional Gradient Descent or Follow-the-Leader algorithms, excelling under adversarial or stochastic losses and reducing constraint violations.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102524"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145517001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"LLMEmu: A lightweight performance emulator for high-fidelity distributed LLM training","authors":"Siyuan Yang ,&nbsp;Enda Yu ,&nbsp;Pingjing Lu ,&nbsp;Dezun Dong","doi":"10.1016/j.peva.2025.102526","DOIUrl":"10.1016/j.peva.2025.102526","url":null,"abstract":"<div><div>The prohibitive cost of training trillion-parameter large language models (LLMs) necessitates low-cost emulation tools for distributed system optimization. In modern large-scale clusters, communication often becomes the primary bottleneck to scalability. However, existing emulators, such as vTrain and ASTRA-Sim, overlook dynamic network factors that significantly impact performance at scale, resulting in limited emulation accuracy. This work offers an efficient and reliable tool for training system optimization and parallel strategy exploration, considerably lowering the barrier to large-scale AI research. We present LLMEmu, a distributed training emulator that combines real kernel profiling and actual communication execution. First, computation is profiled through real CUDA kernel traces on GPU nodes to construct an operator-level latency lookup table, enabling GPU-like execution on CPU clusters. Second, inter-node communication is executed using communication library primitives (e.g., AllReduce, Send/Recv), triggered by communication anchors embedded in the execution graph, and implemented using a pluggable communication backend. LLMEmu can seamlessly model hybrid parallelism strategies and supports multiple collective algorithms. Its lightweight design incorporates gradient bucketing with latency reuse to minimize overhead while maintaining extensibility to various network interconnects. The effectiveness of LLMEmu is validated through its performance results, demonstrating an average prediction error of only 2.17% on 24-GPU clusters, which outperforms vTrain by 21.09%, and confirming its scalability in modeling training cost distributions across 128-node CPU emulations under varying network conditions.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102526"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145516998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}