Large language models discover complementary heuristics for combinatorial optimization

Abstract

Combinatorial optimization (CO) underpins decision-making across science, engineering and industry, especially manufacturing, logistics, healthcare operations and energy management. However, designing effective heuristics for each new problem variant requires deep domain knowledge and months of expert iteration. Directly prompting large language models (LLMs) can generate executable code, but single-pass synthesis of complete heuristics routinely fails on new CO problems. Here we introduce LLM-driven Algorithm Construction via Complementary Evolution (LACE), a framework that decomposes algorithm design into an input schema, output schema, tool library and heuristic portfolio (the $${\mathcal{I}}$$ – $${\mathcal{O}}$$ $${\mathcal{T}}$$ $${\mathcal{H}}$$ interface) and refines the portfolio through time-constrained complementary evolution. The interface specifies a verified problem contract under which the LLM operates, directing model capacity to high-level algorithmic reasoning rather than problem-specific implementation. Complementary evolution then iteratively generates and selects specialist heuristics capable of solving heterogeneous instances within strict runtime budgets. Across 36 classical CO-Bench problems, LACE attains an average score of 0.945, against 0.870 for the strongest existing LLM-based method and 0.571 for direct LLM prompting without any framework, locating the gain in the framework rather than the model. On four structurally new problems, LACE reaches 0.97–0.99, while five existing LLM-based baselines fail to produce any feasible algorithm. These results demonstrate that effective algorithm discovery for such problems can be automated to a substantial degree. Huatian Gong and colleagues developed LACE, a large language model-based framework that designs optimization algorithms. It builds a verified problem contract, then evolves a portfolio of complementary heuristics that together solve problems that no single method can handle.