{"title":"Polarized consensus-based dynamics for optimization and sampling","authors":"Leon Bungert, Tim Roith, Philipp Wacker","doi":"10.1007/s10107-024-02095-y","DOIUrl":"https://doi.org/10.1007/s10107-024-02095-y","url":null,"abstract":"<p>In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several global minima or distributions with many modes, respectively. For this, we “polarize” the dynamics with a localizing kernel and the resulting model can be viewed as a bounded confidence model for opinion formation in the presence of common objective. Instead of being attracted to a common weighted mean as in the original consensus-based methods, which prevents the detection of more than one minimum or mode, in our method every particle is attracted to a weighted mean which gives more weight to nearby particles. We prove that in the mean-field regime the polarized CBS dynamics are unbiased for Gaussian targets. We also prove that in the zero temperature limit and for sufficiently well-behaved strongly convex objectives the solution of the Fokker–Planck equation converges in the Wasserstein-2 distance to a Dirac measure at the minimizer. Finally, we propose a computationally more efficient generalization which works with a predefined number of clusters and improves upon our polarized baseline method for high-dimensional optimization.\u0000</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141189696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploiting the polyhedral geometry of stochastic linear bilevel programming","authors":"Gonzalo Muñoz, David Salas, Anton Svensson","doi":"10.1007/s10107-024-02097-w","DOIUrl":"https://doi.org/10.1007/s10107-024-02097-w","url":null,"abstract":"<p>We study linear bilevel programming problems whose lower-level objective is given by a random cost vector with known distribution. We consider the case where this distribution is nonatomic, allowing to reformulate the problem of the leader using the Bayesian approach in the sense of Salas and Svensson (SIAM J Optim 33(3):2311–2340, 2023), with a decision-dependent distribution that concentrates on the vertices of the feasible set of the follower’s problem. We call this a vertex-supported belief. We prove that this formulation is piecewise affine over the so-called chamber complex of the feasible set of the high-point relaxation. We propose two algorithmic approaches to solve general problems enjoying this last property. The first one is based on enumerating the vertices of the chamber complex. This approach is not scalable, but we present it as a computational baseline and for its theoretical interest. The second one is a Monte-Carlo approximation scheme based on the fact that randomly drawn points of the domain lie, with probability 1, in the interior of full-dimensional chambers, where the problem (restricted to this chamber) can be reduced to a linear program. Finally, we evaluate these methods through computational experiments showing both approaches’ advantages and challenges.\u0000</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141171635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amitabh Basu, Hongyi Jiang, Phillip Kerger, Marco Molinaro
{"title":"Information complexity of mixed-integer convex optimization","authors":"Amitabh Basu, Hongyi Jiang, Phillip Kerger, Marco Molinaro","doi":"10.1007/s10107-024-02099-8","DOIUrl":"https://doi.org/10.1007/s10107-024-02099-8","url":null,"abstract":"<p>We investigate the information complexity of mixed-integer convex optimization under different types of oracles. We establish new lower bounds for the standard first-order oracle, improving upon the previous best known lower bound. This leaves only a lower order linear term (in the dimension) as the gap between the lower and upper bounds. This is derived as a corollary of a more fundamental “transfer” result that shows how lower bounds on information complexity of continuous convex optimization under different oracles can be transferred to the mixed-integer setting in a black-box manner. Further, we (to the best of our knowledge) initiate the study of, and obtain the first set of results on, information complexity under oracles that only reveal <i>partial</i> first-order information, e.g., where one can only make a binary query over the function value or subgradient at a given point. We give algorithms for (mixed-integer) convex optimization that work under these less informative oracles. We also give lower bounds showing that, for some of these oracles, every algorithm requires more iterations to achieve a target error compared to when complete first-order information is available. That is, these oracles are provably less informative than full first-order oracles for the purpose of optimization.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141171634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A tight $$(1.5+epsilon )$$ -approximation for unsplittable capacitated vehicle routing on trees","authors":"Claire Mathieu, Hang Zhou","doi":"10.1007/s10107-024-02094-z","DOIUrl":"https://doi.org/10.1007/s10107-024-02094-z","url":null,"abstract":"<p>In the unsplittable capacitated vehicle routing problem (UCVRP) on trees, we are given a rooted tree with edge weights and a subset of vertices of the tree called terminals. Each terminal is associated with a positive demand between 0 and 1. The goal is to find a minimum length collection of tours starting and ending at the root of the tree such that the demand of each terminal is covered by a single tour (i.e., the demand cannot be split), and the total demand of the terminals in each tour does not exceed the capacity of 1.</p><p>For the special case when all terminals have equal demands, a long line of research culminated in a quasi-polynomial time approximation scheme [Jayaprakash and Salavatipour, TALG 2023] and a polynomial time approximation scheme [Mathieu and Zhou, TALG 2023].</p><p>In this work, we study the general case when the terminals have arbitrary demands. Our main contribution is a polynomial time <span>((1.5+epsilon ))</span>-approximation algorithm for the UCVRP on trees. This is the first improvement upon the 2-approximation algorithm more than 30 years ago. Our approximation ratio is essentially best possible, since it is NP-hard to approximate the UCVRP on trees to better than a 1.5 factor.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141153885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ReLU neural networks of polynomial size for exact maximum flow computation","authors":"Christoph Hertrich, Leon Sering","doi":"10.1007/s10107-024-02096-x","DOIUrl":"https://doi.org/10.1007/s10107-024-02096-x","url":null,"abstract":"<p>This paper studies the expressive power of artificial neural networks with rectified linear units. In order to study them as a model of <i>real-valued</i> computation, we introduce the concept of <i>Max-Affine Arithmetic Programs</i> and show equivalence between them and neural networks concerning natural complexity measures. We then use this result to show that two fundamental combinatorial optimization problems can be solved with polynomial-size neural networks. First, we show that for any undirected graph with <i>n</i> nodes, there is a neural network (with fixed weights and biases) of size <span>(mathcal {O}(n^3))</span> that takes the edge weights as input and computes the value of a minimum spanning tree of the graph. Second, we show that for any directed graph with <i>n</i> nodes and <i>m</i> arcs, there is a neural network of size <span>(mathcal {O}(m^2n^2))</span> that takes the arc capacities as input and computes a maximum flow. Our results imply that these two problems can be solved with strongly polynomial time algorithms that solely use affine transformations and maxima computations, but no comparison-based branchings.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141152122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Stable set polytopes with high lift-and-project ranks for the Lovász–Schrijver SDP operator","authors":"Yu Hin Au, Levent Tunçel","doi":"10.1007/s10107-024-02093-0","DOIUrl":"https://doi.org/10.1007/s10107-024-02093-0","url":null,"abstract":"<p>We study the lift-and-project rank of the stable set polytopes of graphs with respect to the Lovász–Schrijver SDP operator <span>({{,textrm{LS},}}_+)</span>. In particular, we focus on a search for relatively small graphs with high <span>({{,textrm{LS},}}_+)</span>-rank (i.e., the least number of iterations of the <span>({{,textrm{LS},}}_+)</span> operator on the fractional stable set polytope to compute the stable set polytope). We provide families of graphs whose <span>({{,textrm{LS},}}_+)</span>-rank is asymptotically a linear function of its number of vertices, which is the best possible up to improvements in the constant factor. This improves upon the previous best result in this direction from 1999, which yielded graphs whose <span>({{,textrm{LS},}}_+)</span>-rank only grew with the square root of the number of vertices.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141152123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Optimal general factor problem and jump system intersection","authors":"Yusuke Kobayashi","doi":"10.1007/s10107-024-02098-9","DOIUrl":"https://doi.org/10.1007/s10107-024-02098-9","url":null,"abstract":"<p>In the optimal general factor problem, given a graph <span>(G=(V, E))</span> and a set <span>(B(v) subseteq {mathbb {Z}})</span> of integers for each <span>(v in V)</span>, we seek for an edge subset <i>F</i> of maximum cardinality subject to <span>(d_F(v) in B(v))</span> for <span>(v in V)</span>, where <span>(d_F(v))</span> denotes the number of edges in <i>F</i> incident to <i>v</i>. A recent crucial work by Dudycz and Paluch shows that this problem can be solved in polynomial time if each <i>B</i>(<i>v</i>) has no gap of length more than one. While their algorithm is very simple, its correctness proof is quite complicated. In this paper, we formulate the optimal general factor problem as the jump system intersection, and reveal when the algorithm by Dudycz and Paluch can be applied to this abstract form of the problem. By using this abstraction, we give another correctness proof of the algorithm, which is simpler than the original one. We also extend our result to the valuated case.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141152157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A characterization of maximal homogeneous-quadratic-free sets","authors":"Gonzalo Muñoz, Joseph Paat, Felipe Serrano","doi":"10.1007/s10107-024-02092-1","DOIUrl":"https://doi.org/10.1007/s10107-024-02092-1","url":null,"abstract":"<p>The intersection cut framework was introduced by Balas in 1971 as a method for generating cutting planes in integer optimization. In this framework, one uses a full-dimensional convex <i>S</i>-free set, where <i>S</i> is the feasible region of the integer program, to derive a cut separating <i>S</i> from a non-integral vertex of a linear relaxation of <i>S</i>. Among all <i>S</i>-free sets, it is the inclusion-wise maximal ones that yield the strongest cuts. Recently, this framework has been extended beyond the integer case in order to obtain cutting planes in non-linear settings. In this work, we consider the specific setting when <i>S</i> is defined by a homogeneous quadratic inequality. In this ‘quadratic-free’ setting, every function <span>(Gamma : D^m rightarrow D^n)</span>, where <span>(D^k)</span> is the unit sphere in <span>(mathbb {R}^k)</span>, generates a representation of a quadratic-free set. While not every <span>(Gamma )</span> generates a maximal quadratic free set, it is the case that every full-dimensional maximal quadratic free set is generated by some <span>(Gamma )</span>. Our main result shows that the corresponding quadratic-free set is full-dimensional and maximal if and only if <span>(Gamma )</span> is non-expansive and satisfies a technical condition. This result yields a broader class of maximal <i>S</i>-free sets than previously known. Our result stems from a new characterization of maximal <i>S</i>-free sets (for general <i>S</i> beyond the quadratic setting) based on sequences that ‘expose’ inequalities defining the <i>S</i>-free set.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141152037","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eranda Çela, Bettina Klinz, Stefan Lendl, Gerhard J. Woeginger, Lasse Wulf
{"title":"A linear time algorithm for linearizing quadratic and higher-order shortest path problems","authors":"Eranda Çela, Bettina Klinz, Stefan Lendl, Gerhard J. Woeginger, Lasse Wulf","doi":"10.1007/s10107-024-02086-z","DOIUrl":"https://doi.org/10.1007/s10107-024-02086-z","url":null,"abstract":"<p>An instance of the NP-hard Quadratic Shortest Path Problem (QSPP) is called linearizable iff it is equivalent to an instance of the classic Shortest Path Problem (SPP) on the same input digraph. The linearization problem for the QSPP (LinQSPP) decides whether a given QSPP instance is linearizable and determines the corresponding SPP instance in the positive case. We provide a novel linear time algorithm for the LinQSPP on acyclic digraphs which runs considerably faster than the previously best algorithm. The algorithm is based on a new insight revealing that the linearizability of the QSPP for acyclic digraphs can be seen as a local property. Our approach extends to the more general higher-order shortest path problem.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140941067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Gaining or losing perspective for convex multivariate functions on box domains","authors":"Luze Xu, Jon Lee","doi":"10.1007/s10107-024-02087-y","DOIUrl":"https://doi.org/10.1007/s10107-024-02087-y","url":null,"abstract":"<p>Mixed-integer nonlinear optimization formulations of the disjunction between the origin and a polytope via a binary indicator variable is broadly used in nonlinear combinatorial optimization for modeling a fixed cost associated with carrying out a group of activities and a convex cost function associated with the levels of the activities. The perspective relaxation of such models is often used to solve to global optimality in a branch-and-bound context, but it typically requires suitable conic solvers and is not compatible with general-purpose NLP software in the presence of other classes of constraints. This motivates the investigation of when simpler but weaker relaxations may be adequate. Comparing the volume (i.e., Lebesgue measure) of the relaxations as a measure of tightness, we lift some of the results related to the simplex case to the box case. In order to compare the volumes of different relaxations in the box case, it is necessary to find an appropriate concave upper bound that preserves the convexity and is minimal, which is more difficult than in the simplex case. To address the challenge beyond the simplex case, the triangulation approach is used.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140941064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}