Mathematics of Operations Research最新文献

{"title":"Zero-One Composite Optimization: Lyapunov Exact Penalty and a Globally Convergent Inexact Augmented Lagrangian Method","authors":"Penghe Zhang, Naihua Xiu, Ziyan Luo","doi":"10.1287/moor.2021.0320","DOIUrl":"https://doi.org/10.1287/moor.2021.0320","url":null,"abstract":"We consider the problem of minimizing the sum of a smooth function and a composition of a zero-one loss function with a linear operator, namely the zero-one composite optimization problem (0/1-COP). It has a vast body of applications, including the support vector machine (SVM), calcium dynamics fitting (CDF), one-bit compressive sensing (1-bCS), and so on. However, it remains challenging to design a globally convergent algorithm for the original model of 0/1-COP because of the nonconvex and discontinuous zero-one loss function. This paper aims to develop an inexact augmented Lagrangian method (IALM), in which the generated whole sequence converges to a local minimizer of 0/1-COP under reasonable assumptions. In the iteration process, IALM performs minimization on a Lyapunov function with an adaptively adjusted multiplier. The involved Lyapunov penalty subproblem is shown to admit the exact penalty theorem for 0/1-COP, provided that the multiplier is optimal in the sense of the proximal-type stationarity. An efficient zero-one Bregman alternating linearized minimization algorithm is also designed to achieve an approximate solution of the underlying subproblem in finite steps. Numerical experiments for handling SVM, CDF, and 1-bCS demonstrate the satisfactory performance of the proposed method in terms of solution accuracy and time efficiency. Funding: This work was supported by the Fundamental Research Funds for the Central Universities [Grant 2022YJS099] and the National Natural Science Foundation of China [Grants 12131004 and 12271022].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"26 5","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138947219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Counting and Enumerating Optimum Cut Sets for Hypergraph k-Partitioning Problems for Fixed k","authors":"Calvin Beideman, Karthekeyan Chandrasekaran, Weihang Wang","doi":"10.1287/moor.2022.0259","DOIUrl":"https://doi.org/10.1287/moor.2022.0259","url":null,"abstract":"We consider the problem of enumerating optimal solutions for two hypergraph k-partitioning problems, namely, Hypergraph-k-Cut and Minmax-Hypergraph-k-Partition. The input in hypergraph k-partitioning problems is a hypergraph [Formula: see text] with positive hyperedge costs along with a fixed positive integer k. The goal is to find a partition of V into k nonempty parts [Formula: see text]—known as a k-partition—so as to minimize an objective of interest. (1) If the objective of interest is the maximum cut value of the parts, then the problem is known as Minmax-Hypergraph-k-Partition. A subset of hyperedges is a minmax-k-cut-set if it is the subset of hyperedges crossing an optimum k-partition for Minmax-Hypergraph-k-Partition. (2) If the objective of interest is the total cost of hyperedges crossing the k-partition, then the problem is known as Hypergraph-k-Cut. A subset of hyperedges is a min-k-cut-set if it is the subset of hyperedges crossing an optimum k-partition for Hypergraph-k-Cut. We give the first polynomial bound on the number of minmax-k-cut-sets and a polynomial-time algorithm to enumerate all of them in hypergraphs for every fixed k. Our technique is strong enough to also enable an [Formula: see text]-time deterministic algorithm to enumerate all min-k-cut-sets in hypergraphs, thus improving on the previously known [Formula: see text]-time deterministic algorithm, in which n is the number of vertices and p is the size of the hypergraph. The correctness analysis of our enumeration approach relies on a structural result that is a strong and unifying generalization of known structural results for Hypergraph-k-Cut and Minmax-Hypergraph-k-Partition. We believe that our structural result is likely to be of independent interest in the theory of hypergraphs (and graphs).Funding: All authors were supported by NSF AF 1814613 and 1907937.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"16 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138630708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Iterates of the Frank–Wolfe Algorithm May Not Converge","authors":"Jérôme Bolte, Cyrille W. Combettes, Edouard Pauwels","doi":"10.1287/moor.2022.0057","DOIUrl":"https://doi.org/10.1287/moor.2022.0057","url":null,"abstract":"The Frank–Wolfe algorithm is a popular method for minimizing a smooth convex function f over a compact convex set [Formula: see text]. Whereas many convergence results have been derived in terms of function values, almost nothing is known about the convergence behavior of the sequence of iterates [Formula: see text]. Under the usual assumptions, we design several counterexamples to the convergence of [Formula: see text], where f is d-time continuously differentiable, [Formula: see text], and [Formula: see text]. Our counterexamples cover the cases of open-loop, closed-loop, and line-search step-size strategies and work for any choice of the linear minimization oracle, thus demonstrating the fundamental pathologies in the convergence behavior of [Formula: see text].Funding: The authors acknowledge the support of the AI Interdisciplinary Institute ANITI funding through the French “Investments for the Future – PIA3” program under the Agence Nationale de la Recherche (ANR) agreement [Grant ANR-19-PI3A0004], the Air Force Office of Scientific Research, Air Force Material Command, U.S. Air Force [Grants FA866-22-1-7012 and ANR MaSDOL 19-CE23-0017-0], ANR Chess [Grant ANR-17-EURE-0010], ANR Regulia, and Centre Lagrange.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"4 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138563425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Convergence Analysis of Accelerated Stochastic Gradient Descent Under the Growth Condition","authors":"You-Lin Chen, Sen Na, Mladen Kolar","doi":"10.1287/moor.2021.0293","DOIUrl":"https://doi.org/10.1287/moor.2021.0293","url":null,"abstract":"We study the convergence of accelerated stochastic gradient descent (SGD) for strongly convex objectives under the growth condition, which states that the variance of stochastic gradient is bounded by a multiplicative part that grows with the full gradient and a constant additive part. Through the lens of the growth condition, we investigate four widely used accelerated methods: Nesterov’s accelerated method (NAM), robust momentum method (RMM), accelerated dual averaging method (DAM+), and implicit DAM+ (iDAM+). Although these methods are known to improve the convergence rate of SGD under the condition that the stochastic gradient has bounded variance, it is not well understood how their convergence rates are affected by the multiplicative noise. In this paper, we show that these methods all converge to a neighborhood of the optimum with accelerated convergence rates (compared with SGD), even under the growth condition. In particular, NAM, RMM, and iDAM+ enjoy acceleration only with a mild multiplicative noise, whereas DAM+ enjoys acceleration, even with a large multiplicative noise. Furthermore, we propose a generic tail-averaged scheme that allows the accelerated rates of DAM+ and iDAM+ to nearly attain the theoretical lower bound (up to a logarithmic factor in the variance term). We conduct numerical experiments to support our theoretical conclusions.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"24 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138580818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Quantitative Convergence for Displacement Monotone Mean Field Games with Controlled Volatility","authors":"Joe Jackson, Ludovic Tangpi","doi":"10.1287/moor.2023.0106","DOIUrl":"https://doi.org/10.1287/moor.2023.0106","url":null,"abstract":"We study the convergence problem for mean field games with common noise and controlled volatility. We adopt the strategy recently put forth by Laurière and the second author, using the maximum principle to recast the convergence problem as a question of “forward-backward propagation of chaos” (i.e., (conditional) propagation of chaos for systems of particles evolving forward and backward in time). Our main results show that displacement monotonicity can be used to obtain this propagation of chaos, which leads to quantitative convergence results for open-loop Nash equilibria for a class of mean field games. Our results seem to be the first (quantitative or qualitative) that apply to games in which the common noise is controlled. The proofs are relatively simple and rely on a well-known technique for proving wellposedness of forward-backward stochastic differential equations, which is combined with displacement monotonicity in a novel way. To demonstrate the flexibility of the approach, we also use the same arguments to obtain convergence results for a class of infinite horizon discounted mean field games.Funding: J. Jackson is supported by the National Science Foundation [Grant DGE1610403]. L. Tangpi is partially supported by the National Science Foundation [Grants DMS-2005832 and DMS-2143861].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"17 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138580879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Linear Program-Based Policies for Restless Bandits: Necessary and Sufficient Conditions for (Exponentially Fast) Asymptotic Optimality","authors":"Nicolas Gast, Bruno Gaujal, Chen Yan","doi":"10.1287/moor.2022.0101","DOIUrl":"https://doi.org/10.1287/moor.2022.0101","url":null,"abstract":"We provide a framework to analyze control policies for the restless Markovian bandit model under both finite and infinite time horizons. We show that when the population of arms goes to infinity, the value of the optimal control policy converges to the solution of a linear program (LP). We provide necessary and sufficient conditions for a generic control policy to be (i) asymptotically optimal, (ii) asymptotically optimal with square root convergence rate, and (iii) asymptotically optimal with exponential rate. We then construct the LP-index policy that is asymptotically optimal with square root convergence rate on all models and with exponential rate if the model is nondegenerate in finite horizon and satisfies a uniform global attractor property in infinite horizon. We next define the LP-update policy, which is essentially a repeated LP-index policy that solves a new LP at each decision epoch. We conclude by providing numerical experiments to compare the efficiency of different LP-based policies.Funding: This work was supported by Agence Nationale de la Recherche [Grant ANR-19-CE23-0015].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"232 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Proximity and Flatness Bounds for Linear Integer Optimization","authors":"Marcel Celaya, Stefan Kuhlmann, Joseph Paat, Robert Weismantel","doi":"10.1287/moor.2022.0335","DOIUrl":"https://doi.org/10.1287/moor.2022.0335","url":null,"abstract":"This paper deals with linear integer optimization. We develop a technique that can be applied to provide improved upper bounds for two important questions in linear integer optimization. Given an optimal vertex solution for the linear relaxation, how far away is the nearest optimal integer solution (if one exists; proximity bounds)? If a polyhedron contains no integer point, what is the smallest number of integer parallel hyperplanes defined by an integral, nonzero, normal vector that intersect the polyhedron (flatness bounds)? This paper presents a link between these two questions by refining a proof technique that has been recently introduced by the authors. A key technical lemma underlying our technique concerns the areas of certain convex polygons in the plane; if a polygon [Formula: see text] satisfies [Formula: see text], where τ denotes [Formula: see text] counterclockwise rotation and [Formula: see text] denotes the polar of K, then the area of [Formula: see text] is at least three.Funding: J. Paat was supported by the Natural Sciences and Engineering Research Council of Canada [Grant RGPIN-2021-02475]. R. Weismantel was supported by the Einstein Stiftung Berlin.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"225 2","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Allocating Indivisible Goods to Strategic Agents: Pure Nash Equilibria and Fairness","authors":"Georgios Amanatidis, Georgios Birmpas, Federico Fusco, Philip Lazos, Stefano Leonardi, Rebecca Reiffenhäuser","doi":"10.1287/moor.2022.0058","DOIUrl":"https://doi.org/10.1287/moor.2022.0058","url":null,"abstract":"We consider the problem of fairly allocating a set of indivisible goods to a set of strategic agents with additive valuation functions. We assume no monetary transfers, and therefore, a mechanism in our setting is an algorithm that takes as input the reported—rather than the true—values of the agents. Our main goal is to explore whether there exist mechanisms that have pure Nash equilibria for every instance and, at the same time, provide fairness guarantees for the allocations that correspond to these equilibria. We focus on two relaxations of envy-freeness, namely, envy-freeness up to one good (EF1) and envy-freeness up to any good (EFX), and we positively answer the preceding question. In particular, we study two algorithms that are known to produce such allocations in the nonstrategic setting: round-robin (EF1 allocations for any number of agents) and a cut-and-choose algorithm of Plaut and Roughgarden (EFX allocations for two agents). For round-robin, we show that all of its pure Nash equilibria induce allocations that are EF1 with respect to the underlying true values, whereas for the algorithm of Plaut and Roughgarden, we show that the corresponding allocations not only are EFX, but also satisfy maximin share fairness, something that is not true for this algorithm in the nonstrategic setting! Further, we show that a weaker version of the latter result holds for any mechanism for two agents that always has pure Nash equilibria, which all induce EFX allocations.Funding: This work was supported by the Horizon 2020 European Research Council Advanced “Algorithmic and Mechanism Design Research in Online Markets” [Grant 788893], the Ministero dell’Università e della Ricerca Research project of national interest (PRIN) “Algorithms, Games, and Digital Markets,” the Future Artificial Intelligence Research project funded by the NextGenerationEU program within the National Recovery and Resilience Plan (PNRR-PE-AI) scheme [M4C2, investment 1.3, line on Artificial Intelligence], the National Recovery and Resilience Plan-Ministero dell’Università e della Ricerca (PNRR-MUR) project IR0000013-SoBigData.it, the Nederlandse Organisatie voor Wetenschappelijk Onderzoek Veni Project [Grant VI.Veni.192.153], and the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program [Grant MIS 5154714].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"228 5","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Cost of Nonconvexity in Deterministic Nonsmooth Optimization","authors":"Siyu Kong, A. S. Lewis","doi":"10.1287/moor.2022.0289","DOIUrl":"https://doi.org/10.1287/moor.2022.0289","url":null,"abstract":"We study the impact of nonconvexity on the complexity of nonsmooth optimization, emphasizing objectives such as piecewise linear functions, which may not be weakly convex. We focus on a dimension-independent analysis, slightly modifying a 2020 black-box algorithm of Zhang-Lin-Jegelka-Sra-Jadbabaie that approximates an ϵ-stationary point of any directionally differentiable Lipschitz objective using [Formula: see text] calls to a specialized subgradient oracle and a randomized line search. Seeking by contrast a deterministic method, we present a simple black-box version that achieves [Formula: see text] for any difference-of-convex objective and [Formula: see text] for the weakly convex case. Our complexity bound depends on a natural nonconvexity modulus that is related, intriguingly, to the negative part of directional second derivatives of the objective, understood in the distributional sense.Funding: This work was supported by the National Science Foundation [Grant DMS-2006990].","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"231 5","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Worst-Case Iteration Bounds for Log Barrier Methods on Problems with Nonconvex Constraints","authors":"Oliver Hinder, Yinyu Ye","doi":"10.1287/moor.2020.0274","DOIUrl":"https://doi.org/10.1287/moor.2020.0274","url":null,"abstract":"Interior point methods (IPMs) that handle nonconvex constraints such as IPOPT, KNITRO and LOQO have had enormous practical success. We consider IPMs in the setting where the objective and constraints are thrice differentiable, and have Lipschitz first and second derivatives on the feasible region. We provide an IPM that, starting from a strictly feasible point, finds a μ-approximate Fritz John point by solving [Formula: see text] trust-region subproblems. For IPMs that handle nonlinear constraints, this result represents the first iteration bound with a polynomial dependence on [Formula: see text]. We also show how to use our method to find scaled-KKT points starting from an infeasible solution and improve on existing complexity bounds.Funding: This work was supported by Air Force Office of Scientific Research [9550-23-1-0242]. A significant portion of this work was done at Stanford where O. Hinder was supported by the PACCAR, Inc., Stanford Graduate Fellowship and the Dantzig-Lieberman fellowship.","PeriodicalId":49852,"journal":{"name":"Mathematics of Operations Research","volume":"231 6","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}