Mathematics of Operations Research最新文献_第4页

Steiner Cut Dominants 斯坦纳切割机

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-27 DOI: 10.1287/moor.2022.0280

Michele Conforti, Volker Kaibel

引用次数: 0

Fast Rates for the Regret of Offline Reinforcement Learning 离线强化学习的快速回归率

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-25 DOI: 10.1287/moor.2021.0167

Yichun Hu, Nathan Kallus, Masatoshi Uehara

{"title":"Fast Rates for the Regret of Offline Reinforcement Learning","authors":"Yichun Hu, Nathan Kallus, Masatoshi Uehara","doi":"10.1287/moor.2021.0167","DOIUrl":"https://doi.org/10.1287/moor.2021.0167","url":null,"abstract":"We study the regret of offline reinforcement learning in an infinite-horizon discounted Markov decision process (MDP). While existing analyses of common approaches, such as fitted Q-iteration (FQI), suggest root-n convergence for regret, empirical behavior exhibits much faster convergence. In this paper, we present a finer regret analysis that exactly characterizes this phenomenon by providing fast rates for the regret convergence. First, we show that given any estimate for the optimal quality function, the regret of the policy it defines converges at a rate given by the exponentiation of the estimate’s pointwise convergence rate, thus speeding up the rate. The level of exponentiation depends on the level of noise in the decision-making problem, rather than the estimation problem. We establish such noise levels for linear and tabular MDPs as examples. Second, we provide new analyses of FQI and Bellman residual minimization to establish the correct pointwise convergence guarantees. As specific cases, our results imply one-over-n rates in linear cases and exponential-in-n rates in tabular cases. We extend our findings to general function approximation by extending our results to regret guarantees based on Lp-convergence rates for estimating the optimal quality function rather than pointwise rates, where L2 guarantees for nonparametric estimation can be ensured under mild conditions.Funding: This work was supported by the Division of Information and Intelligent Systems, National Science Foundation [Grant 1846210].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"47 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Semidefinite Approximations for Bicliques and Bi-Independent Pairs 双桥和双独立对的半无限逼近法

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-13 DOI: 10.1287/moor.2023.0046

Monique Laurent, Sven Polak, Luis Felipe Vargas

{"title":"Semidefinite Approximations for Bicliques and Bi-Independent Pairs","authors":"Monique Laurent, Sven Polak, Luis Felipe Vargas","doi":"10.1287/moor.2023.0046","DOIUrl":"https://doi.org/10.1287/moor.2023.0046","url":null,"abstract":"We investigate some graph parameters dealing with bi-independent pairs (A, B) in a bipartite graph [Formula: see text], that is, pairs (A, B) where [Formula: see text], and [Formula: see text] are independent. These parameters also allow us to study bicliques in general graphs. When maximizing the cardinality [Formula: see text], one finds the stability number [Formula: see text], well-known to be polynomial-time computable. When maximizing the product [Formula: see text], one finds the parameter g(G), shown to be NP-hard by Peeters in 2003, and when maximizing the ratio [Formula: see text], one finds h(G), introduced by Vallentin in 2020 for bounding product-free sets in finite groups. We show that h(G) is an NP-hard parameter and, as a crucial ingredient, that it is NP-complete to decide whether a bipartite graph G has a balanced maximum independent set. These hardness results motivate introducing semidefinite programming (SDP) bounds for g(G), h(G), and [Formula: see text] (the maximum cardinality of a balanced independent set). We show that these bounds can be seen as natural variations of the Lovász ϑ-number, a well-known semidefinite bound on [Formula: see text]. In addition, we formulate closed-form eigenvalue bounds, and we show relationships among them as well as with earlier spectral parameters by Hoffman and Haemers in 2001 and Vallentin in 2020.Funding: This work was supported by H2020 Marie Skłodowska-Curie Actions [Grant 813211 (POEMA)].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"23 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140147775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Marginal Values of a Stochastic Game 随机博弈的边际值

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-12 DOI: 10.1287/moor.2023.0297

Luc Attia, Miquel Oliu-Barton, Raimundo Saona

引用次数: 0

Convergence and Stability of Coupled Belief-Strategy Learning Dynamics in Continuous Games 连续博弈中信念-策略耦合学习动态的收敛性与稳定性

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-12 DOI: 10.1287/moor.2022.0161

Manxi Wu, Saurabh Amin, Asuman Ozdaglar

{"title":"Convergence and Stability of Coupled Belief-Strategy Learning Dynamics in Continuous Games","authors":"Manxi Wu, Saurabh Amin, Asuman Ozdaglar","doi":"10.1287/moor.2022.0161","DOIUrl":"https://doi.org/10.1287/moor.2022.0161","url":null,"abstract":"We propose a learning dynamics to model how strategic agents repeatedly play a continuous game while relying on an information platform to learn an unknown payoff-relevant parameter. In each time step, the platform updates a belief estimate of the parameter based on players’ strategies and realized payoffs using Bayes’ rule. Then, players adopt a generic learning rule to adjust their strategies based on the updated belief. We present results on the convergence of beliefs and strategies and the properties of convergent fixed points of the dynamics. We obtain sufficient and necessary conditions for the existence of globally stable fixed points. We also provide sufficient conditions for the local stability of fixed points. These results provide an approach to analyzing the long-term outcomes that arise from the interplay between Bayesian belief learning and strategy learning in games and enable us to characterize conditions under which learning leads to a complete information equilibrium.Funding: Financial support from the Air Force Office of Scientific Research [Project Building Attack Resilience into Complex Networks], the Simons Institute [research fellowship], and a Michael Hammer Fellowship is gratefully acknowledged.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"72 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140147400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost MDP 风险敏感指数成本 MDP 的策略梯度算法

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-11 DOI: 10.1287/moor.2022.0139

Mehrdad Moharrami, Yashaswini Murthy, Arghyadip Roy, R. Srikant

{"title":"A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost MDP","authors":"Mehrdad Moharrami, Yashaswini Murthy, Arghyadip Roy, R. Srikant","doi":"10.1287/moor.2022.0139","DOIUrl":"https://doi.org/10.1287/moor.2022.0139","url":null,"abstract":"We study the risk-sensitive exponential cost Markov decision process (MDP) formulation and develop a trajectory-based gradient algorithm to find the stationary point of the cost associated with a set of parameterized policies. We derive a formula that can be used to compute the policy gradient from (state, action, cost) information collected from sample paths of the MDP for each fixed parameterized policy. Unlike the traditional average cost problem, standard stochastic approximation theory cannot be used to exploit this formula. To address the issue, we introduce a truncated and smooth version of the risk-sensitive cost and show that this new cost criterion can be used to approximate the risk-sensitive cost and its gradient uniformly under some mild assumptions. We then develop a trajectory-based gradient algorithm to minimize the smooth truncated estimation of the risk-sensitive cost and derive conditions under which a sequence of truncations can be used to solve the original, untruncated cost problem.Funding: This work was supported by the Office of Naval Research Global [Grant N0001419-1-2566], the Division of Computer and Network Systems [Grant 21-06801], the Army Research Office [Grant W911NF-19-1-0379], and the Division of Computing and Communication Foundations [Grants 17-04970 and 19-34986].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"7 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140107747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Parametric Semidefinite Programming: Geometry of the Trajectory of Solutions 参数半无限编程：解的轨迹几何

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-08 DOI: 10.1287/moor.2021.0097

Antonio Bellon, Didier Henrion, Vyacheslav Kungurtsev, Jakub Mareček

引用次数: 0

On the (Im-)Possibility of Representing Probability Distributions as a Difference of I.I.D. Noise Terms 论以 I.I.D. 噪声项之差表示概率分布的（非）可能性

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-03-07 DOI: 10.1287/moor.2023.0081

Christian Ewerhart, Marco Serena

引用次数: 0

Strongly Convergent Homogeneous Approximations to Inhomogeneous Markov Jump Processes and Applications 非均质马尔可夫跳跃过程的强收敛同质逼近及其应用

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-02-22 DOI: 10.1287/moor.2022.0153

Martin Bladt, Oscar Peralta

{"title":"Strongly Convergent Homogeneous Approximations to Inhomogeneous Markov Jump Processes and Applications","authors":"Martin Bladt, Oscar Peralta","doi":"10.1287/moor.2022.0153","DOIUrl":"https://doi.org/10.1287/moor.2022.0153","url":null,"abstract":"The study of time-inhomogeneous Markov jump processes is a traditional topic within probability theory that has recently attracted substantial attention in various applications. However, their flexibility also incurs a substantial mathematical burden which is usually circumvented by using well-known generic distributional approximations or simulations. This article provides a novel approximation method that tailors the dynamics of a time-homogeneous Markov jump process to meet those of its time-inhomogeneous counterpart on an increasingly fine Poisson grid. Strong convergence of the processes in terms of the Skorokhod J1 metric is established, and convergence rates are provided. Under traditional regularity assumptions, distributional convergence is established for unconditional proxies, to the same limit. Special attention is devoted to the case where the target process has one absorbing state and the remaining ones transient, for which the absorption times also converge. Some applications are outlined, such as univariate hazard-rate density estimation, ruin probabilities, and multivariate phase-type density evaluation.Funding: M. Bladt and O. Peralta would like to acknowledge financial support from the Swiss National Science Foundation Project 200021_191984. O. Peralta acknowledges financial support from NSF Award #1653354 and AXA Research Fund Award on “Mitigating risk in the wake of the pandemic”.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"159 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139946213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Risk Sharing with Lambda Value at Risk 用 Lambda 风险值分担风险

IF 1.7 3区数学

Mathematics of Operations Research Pub Date : 2024-02-21 DOI: 10.1287/moor.2023.0246

Peng Liu

{"title":"Risk Sharing with Lambda Value at Risk","authors":"Peng Liu","doi":"10.1287/moor.2023.0246","DOIUrl":"https://doi.org/10.1287/moor.2023.0246","url":null,"abstract":"In this paper, we study the risk-sharing problem among multiple agents using lambda value at risk ([Formula: see text]) as their preferences via the tool of inf-convolution, where [Formula: see text] is an extension of value at risk ([Formula: see text]). We obtain explicit formulas of the inf-convolution of multiple [Formula: see text] with monotone Λ and explicit forms of the corresponding optimal allocations, extending the results of the inf-convolution of [Formula: see text]. It turns out that the inf-convolution of several [Formula: see text] is still a [Formula: see text] under some mild condition. Moreover, we investigate the inf-convolution of one [Formula: see text] and a general monotone risk measure without cash additivity, including [Formula: see text], expected utility, and rank-dependent expected utility as special cases. The expression of the inf-convolution and the explicit forms of the optimal allocation are derived, leading to some partial solution of the risk-sharing problem with multiple [Formula: see text] for general Λ functions. Finally, we discuss the risk-sharing problem with [Formula: see text], another definition of lambda value at risk. We focus on the inf-convolution of [Formula: see text] and a risk measure that is consistent with the second-order stochastic dominance, deriving very different expression of the inf-convolution and the forms of the optimal allocations.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"234 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139923182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0