协方差矩阵自适应进化策略的对角加速

IF 4.6 2区 计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Y. Akimoto;N. Hansen
{"title":"协方差矩阵自适应进化策略的对角加速","authors":"Y. Akimoto;N. Hansen","doi":"10.1162/evco_a_00260","DOIUrl":null,"url":null,"abstract":"<para>We introduce an acceleration for covariance matrix adaptation evolution strategies (CMA-ES) by means of <italic>adaptive diagonal decoding</i> (dd-CMA). This diagonal acceleration endows the default CMA-ES with the advantages of separable CMA-ES without inheriting its drawbacks. Technically, we introduce a diagonal matrix <inline-formula><mml:math><mml:mi>D</mml:mi></mml:math></inline-formula> that expresses coordinate-wise variances of the sampling distribution in <italic>DCD</i> form. The diagonal matrix can learn a rescaling of the problem in the coordinates within a linear number of function evaluations. Diagonal decoding can also exploit separability of the problem, but, crucially, does not compromise the performance on nonseparable problems. The latter is accomplished by modulating the learning rate for the diagonal matrix based on the condition number of the underlying correlation matrix. dd-CMA-ES not only combines the advantages of default and separable CMA-ES, but may achieve overadditive speedup: it improves the performance, and even the scaling, of the better of default and separable CMA-ES on classes of nonseparable test functions that reflect, arguably, a landscape feature commonly observed in practice.</para>\n \n<para>The article makes two further secondary contributions: we introduce two different approaches to guarantee positive definiteness of the covariance matrix with active CMA, which is valuable in particular with large population size; we revise the default parameter setting in CMA-ES, proposing accelerated settings in particular for large dimension.</para>\n \n<para>All our contributions can be viewed as independent improvements of CMA-ES, yet they are also complementary and can be seamlessly combined. In numerical experiments with dd-CMA-ES up to dimension 5120, we observe remarkable improvements over the original covariance matrix adaptation on functions with coordinate-wise ill-conditioning. The improvement is observed also for large population sizes up to about dimension squared.</para>","PeriodicalId":50470,"journal":{"name":"Evolutionary Computation","volume":"28 3","pages":"405-435"},"PeriodicalIF":4.6000,"publicationDate":"2020-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1162/evco_a_00260","citationCount":"40","resultStr":"{\"title\":\"Diagonal Acceleration for Covariance Matrix Adaptation Evolution Strategies\",\"authors\":\"Y. Akimoto;N. Hansen\",\"doi\":\"10.1162/evco_a_00260\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<para>We introduce an acceleration for covariance matrix adaptation evolution strategies (CMA-ES) by means of <italic>adaptive diagonal decoding</i> (dd-CMA). This diagonal acceleration endows the default CMA-ES with the advantages of separable CMA-ES without inheriting its drawbacks. Technically, we introduce a diagonal matrix <inline-formula><mml:math><mml:mi>D</mml:mi></mml:math></inline-formula> that expresses coordinate-wise variances of the sampling distribution in <italic>DCD</i> form. The diagonal matrix can learn a rescaling of the problem in the coordinates within a linear number of function evaluations. Diagonal decoding can also exploit separability of the problem, but, crucially, does not compromise the performance on nonseparable problems. The latter is accomplished by modulating the learning rate for the diagonal matrix based on the condition number of the underlying correlation matrix. dd-CMA-ES not only combines the advantages of default and separable CMA-ES, but may achieve overadditive speedup: it improves the performance, and even the scaling, of the better of default and separable CMA-ES on classes of nonseparable test functions that reflect, arguably, a landscape feature commonly observed in practice.</para>\\n \\n<para>The article makes two further secondary contributions: we introduce two different approaches to guarantee positive definiteness of the covariance matrix with active CMA, which is valuable in particular with large population size; we revise the default parameter setting in CMA-ES, proposing accelerated settings in particular for large dimension.</para>\\n \\n<para>All our contributions can be viewed as independent improvements of CMA-ES, yet they are also complementary and can be seamlessly combined. In numerical experiments with dd-CMA-ES up to dimension 5120, we observe remarkable improvements over the original covariance matrix adaptation on functions with coordinate-wise ill-conditioning. The improvement is observed also for large population sizes up to about dimension squared.</para>\",\"PeriodicalId\":50470,\"journal\":{\"name\":\"Evolutionary Computation\",\"volume\":\"28 3\",\"pages\":\"405-435\"},\"PeriodicalIF\":4.6000,\"publicationDate\":\"2020-09-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://sci-hub-pdf.com/10.1162/evco_a_00260\",\"citationCount\":\"40\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Evolutionary Computation\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/9185171/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/9185171/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 40

摘要

我们介绍了一种通过自适应对角解码(dd-CMA)加速协方差矩阵自适应进化策略(CMA-ES)。这种对角线加速赋予了默认CMA-ES可分离CMA-ES的优点,而没有继承其缺点。从技术上讲,我们引入了一个对角矩阵D,它以DCD的形式表示采样分布的坐标方差。对角矩阵可以在线性函数求值的范围内学习问题在坐标中的重新缩放。对角线解码也可以利用问题的可分性,但至关重要的是,它不会影响不可分离性问题的性能。后者是通过基于底层相关矩阵的条件数调制对角矩阵的学习率来实现的。dd CMA ES不仅结合了默认和可分离CMA-ES的优点,而且可能实现超加性的加速:它提高了默认和可以分离的CMA-ES在不可分离测试函数类上的性能,甚至提高了扩展性,这些测试函数类可以说反映了实践中常见的横向特征。本文进一步做出了两个次要贡献:我们引入了两种不同的方法来保证具有主动CMA的协方差矩阵的正定性,这在大种群规模的情况下是有价值的;我们修改了CMA-ES中的默认参数设置,特别是针对大维度提出了加速设置。我们的所有贡献都可以被视为CMA-ES的独立改进,但它们也是互补的,可以无缝结合。在高达5120维的dd-CMA ES的数值实验中,我们观察到在具有坐标方向不良条件的函数上,与原始协方差矩阵自适应相比有显著改进。对于高达约平方维的大种群规模,也观察到了这种改善。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Diagonal Acceleration for Covariance Matrix Adaptation Evolution Strategies
We introduce an acceleration for covariance matrix adaptation evolution strategies (CMA-ES) by means of adaptive diagonal decoding (dd-CMA). This diagonal acceleration endows the default CMA-ES with the advantages of separable CMA-ES without inheriting its drawbacks. Technically, we introduce a diagonal matrix D that expresses coordinate-wise variances of the sampling distribution in DCD form. The diagonal matrix can learn a rescaling of the problem in the coordinates within a linear number of function evaluations. Diagonal decoding can also exploit separability of the problem, but, crucially, does not compromise the performance on nonseparable problems. The latter is accomplished by modulating the learning rate for the diagonal matrix based on the condition number of the underlying correlation matrix. dd-CMA-ES not only combines the advantages of default and separable CMA-ES, but may achieve overadditive speedup: it improves the performance, and even the scaling, of the better of default and separable CMA-ES on classes of nonseparable test functions that reflect, arguably, a landscape feature commonly observed in practice. The article makes two further secondary contributions: we introduce two different approaches to guarantee positive definiteness of the covariance matrix with active CMA, which is valuable in particular with large population size; we revise the default parameter setting in CMA-ES, proposing accelerated settings in particular for large dimension. All our contributions can be viewed as independent improvements of CMA-ES, yet they are also complementary and can be seamlessly combined. In numerical experiments with dd-CMA-ES up to dimension 5120, we observe remarkable improvements over the original covariance matrix adaptation on functions with coordinate-wise ill-conditioning. The improvement is observed also for large population sizes up to about dimension squared.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Evolutionary Computation
Evolutionary Computation 工程技术-计算机:理论方法
CiteScore
6.40
自引率
1.50%
发文量
20
审稿时长
3 months
期刊介绍: Evolutionary Computation is a leading journal in its field. It provides an international forum for facilitating and enhancing the exchange of information among researchers involved in both the theoretical and practical aspects of computational systems drawing their inspiration from nature, with particular emphasis on evolutionary models of computation such as genetic algorithms, evolutionary strategies, classifier systems, evolutionary programming, and genetic programming. It welcomes articles from related fields such as swarm intelligence (e.g. Ant Colony Optimization and Particle Swarm Optimization), and other nature-inspired computation paradigms (e.g. Artificial Immune Systems). As well as publishing articles describing theoretical and/or experimental work, the journal also welcomes application-focused papers describing breakthrough results in an application domain or methodological papers where the specificities of the real-world problem led to significant algorithmic improvements that could possibly be generalized to other areas.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信