Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, Arnaud Doucet
{"title":"用于非配对数据转换的薛定谔桥流","authors":"Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, Arnaud Doucet","doi":"arxiv-2409.09347","DOIUrl":null,"url":null,"abstract":"Mass transport problems arise in many areas of machine learning whereby one\nwants to compute a map transporting one distribution to another. Generative\nmodeling techniques like Generative Adversarial Networks (GANs) and Denoising\nDiffusion Models (DDMs) have been successfully adapted to solve such transport\nproblems, resulting in CycleGAN and Bridge Matching respectively. However,\nthese methods do not approximate Optimal Transport (OT) maps, which are known\nto have desirable properties. Existing techniques approximating OT maps for\nhigh-dimensional data-rich problems, such as DDM-based Rectified Flow and\nSchr\\\"odinger Bridge procedures, require fully training a DDM-type model at\neach iteration, or use mini-batch techniques which can introduce significant\nerrors. We propose a novel algorithm to compute the Schr\\\"odinger Bridge, a\ndynamic entropy-regularised version of OT, that eliminates the need to train\nmultiple DDM-like models. This algorithm corresponds to a discretisation of a\nflow of path measures, which we call the Schr\\\"odinger Bridge Flow, whose only\nstationary point is the Schr\\\"odinger Bridge. We demonstrate the performance of\nour algorithm on a variety of unpaired data translation tasks.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"22 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Schrödinger Bridge Flow for Unpaired Data Translation\",\"authors\":\"Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, Arnaud Doucet\",\"doi\":\"arxiv-2409.09347\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Mass transport problems arise in many areas of machine learning whereby one\\nwants to compute a map transporting one distribution to another. Generative\\nmodeling techniques like Generative Adversarial Networks (GANs) and Denoising\\nDiffusion Models (DDMs) have been successfully adapted to solve such transport\\nproblems, resulting in CycleGAN and Bridge Matching respectively. However,\\nthese methods do not approximate Optimal Transport (OT) maps, which are known\\nto have desirable properties. Existing techniques approximating OT maps for\\nhigh-dimensional data-rich problems, such as DDM-based Rectified Flow and\\nSchr\\\\\\\"odinger Bridge procedures, require fully training a DDM-type model at\\neach iteration, or use mini-batch techniques which can introduce significant\\nerrors. We propose a novel algorithm to compute the Schr\\\\\\\"odinger Bridge, a\\ndynamic entropy-regularised version of OT, that eliminates the need to train\\nmultiple DDM-like models. This algorithm corresponds to a discretisation of a\\nflow of path measures, which we call the Schr\\\\\\\"odinger Bridge Flow, whose only\\nstationary point is the Schr\\\\\\\"odinger Bridge. We demonstrate the performance of\\nour algorithm on a variety of unpaired data translation tasks.\",\"PeriodicalId\":501340,\"journal\":{\"name\":\"arXiv - STAT - Machine Learning\",\"volume\":\"22 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - STAT - Machine Learning\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.09347\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - STAT - Machine Learning","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.09347","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
在机器学习的许多领域都会出现大规模传输问题,即人们希望计算将一个分布传输到另一个分布的地图。生成模型技术,如生成对抗网络(GAN)和去噪扩散模型(DDM),已被成功地用于解决此类传输问题,分别产生了循环生成对抗网络(CycleGAN)和桥匹配(Bridge Matching)。然而,这些方法并不能逼近已知具有理想特性的最优传输(OT)图。现有的针对高维数据丰富问题的近似 OT 地图的技术,如基于 DDM 的整流程序和薛定谔桥程序,需要在每次迭代时完全训练一个 DDM 类型的模型,或者使用可能引入重大误差的迷你批处理技术。我们提出了一种计算 Schr\"odinger Bridge 的新算法,即 OT 的动态熵规则化版本,它无需训练多个类似于 DDM 的模型。该算法对应于路径度量流的离散化,我们称之为薛定谔桥流,其唯一的静止点就是薛定谔桥。我们在各种非配对数据转换任务中演示了我们算法的性能。
Schrödinger Bridge Flow for Unpaired Data Translation
Mass transport problems arise in many areas of machine learning whereby one
wants to compute a map transporting one distribution to another. Generative
modeling techniques like Generative Adversarial Networks (GANs) and Denoising
Diffusion Models (DDMs) have been successfully adapted to solve such transport
problems, resulting in CycleGAN and Bridge Matching respectively. However,
these methods do not approximate Optimal Transport (OT) maps, which are known
to have desirable properties. Existing techniques approximating OT maps for
high-dimensional data-rich problems, such as DDM-based Rectified Flow and
Schr\"odinger Bridge procedures, require fully training a DDM-type model at
each iteration, or use mini-batch techniques which can introduce significant
errors. We propose a novel algorithm to compute the Schr\"odinger Bridge, a
dynamic entropy-regularised version of OT, that eliminates the need to train
multiple DDM-like models. This algorithm corresponds to a discretisation of a
flow of path measures, which we call the Schr\"odinger Bridge Flow, whose only
stationary point is the Schr\"odinger Bridge. We demonstrate the performance of
our algorithm on a variety of unpaired data translation tasks.