{"title":"基于预训练变压器的约束工程问题快速准确的贝叶斯优化。","authors":"Rosen Ting-Ying Yu, Cyril Picard, Faez Ahmed","doi":"10.1007/s00158-025-03987-z","DOIUrl":null,"url":null,"abstract":"<p><p>Bayesian Optimization (BO) is a foundational strategy in engineering design optimization for efficiently handling black-box functions with many constraints and expensive evaluations. This paper introduces a novel constraint-handling framework for Bayesian Optimization (BO) using Prior-data Fitted Networks (PFNs), a foundation transformer model. Unlike traditional approaches requiring separate Gaussian Process (GP) models for each constraint, our framework leverages PFN's transformer architecture to evaluate objectives and constraints simultaneously in a single forward pass using in-context learning. Through comprehensive benchmarking across 15 test problems spanning synthetic, structural, and engineering design challenges, we demonstrate an order of magnitude speedup while maintaining or improving solution quality compared to conventional GP-based methods with constrained expected improvement (CEI). Our approach particularly excels at engineering problems by rapidly finding feasible, optimal solutions. This benchmark framework for evaluating new BO algorithms in engineering design will be published at https://github.com/rosenyu304/BOEngineeringBenchmark.</p>","PeriodicalId":21994,"journal":{"name":"Structural and Multidisciplinary Optimization","volume":"68 3","pages":"66"},"PeriodicalIF":3.6000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11985669/pdf/","citationCount":"0","resultStr":"{\"title\":\"Fast and accurate Bayesian optimization with pre-trained transformers for constrained engineering problems.\",\"authors\":\"Rosen Ting-Ying Yu, Cyril Picard, Faez Ahmed\",\"doi\":\"10.1007/s00158-025-03987-z\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Bayesian Optimization (BO) is a foundational strategy in engineering design optimization for efficiently handling black-box functions with many constraints and expensive evaluations. This paper introduces a novel constraint-handling framework for Bayesian Optimization (BO) using Prior-data Fitted Networks (PFNs), a foundation transformer model. Unlike traditional approaches requiring separate Gaussian Process (GP) models for each constraint, our framework leverages PFN's transformer architecture to evaluate objectives and constraints simultaneously in a single forward pass using in-context learning. Through comprehensive benchmarking across 15 test problems spanning synthetic, structural, and engineering design challenges, we demonstrate an order of magnitude speedup while maintaining or improving solution quality compared to conventional GP-based methods with constrained expected improvement (CEI). Our approach particularly excels at engineering problems by rapidly finding feasible, optimal solutions. This benchmark framework for evaluating new BO algorithms in engineering design will be published at https://github.com/rosenyu304/BOEngineeringBenchmark.</p>\",\"PeriodicalId\":21994,\"journal\":{\"name\":\"Structural and Multidisciplinary Optimization\",\"volume\":\"68 3\",\"pages\":\"66\"},\"PeriodicalIF\":3.6000,\"publicationDate\":\"2025-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11985669/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Structural and Multidisciplinary Optimization\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1007/s00158-025-03987-z\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/4/10 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Structural and Multidisciplinary Optimization","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1007/s00158-025-03987-z","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/4/10 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
Fast and accurate Bayesian optimization with pre-trained transformers for constrained engineering problems.
Bayesian Optimization (BO) is a foundational strategy in engineering design optimization for efficiently handling black-box functions with many constraints and expensive evaluations. This paper introduces a novel constraint-handling framework for Bayesian Optimization (BO) using Prior-data Fitted Networks (PFNs), a foundation transformer model. Unlike traditional approaches requiring separate Gaussian Process (GP) models for each constraint, our framework leverages PFN's transformer architecture to evaluate objectives and constraints simultaneously in a single forward pass using in-context learning. Through comprehensive benchmarking across 15 test problems spanning synthetic, structural, and engineering design challenges, we demonstrate an order of magnitude speedup while maintaining or improving solution quality compared to conventional GP-based methods with constrained expected improvement (CEI). Our approach particularly excels at engineering problems by rapidly finding feasible, optimal solutions. This benchmark framework for evaluating new BO algorithms in engineering design will be published at https://github.com/rosenyu304/BOEngineeringBenchmark.
期刊介绍:
The journal’s scope ranges from mathematical foundations of the field to algorithm and software development, and from benchmark examples to case studies of practical applications in structural, aero-space, mechanical, civil, chemical, naval and bio-engineering.
Fields such as computer-aided design and manufacturing, uncertainty quantification, artificial intelligence, system identification and modeling, inverse processes, computer simulation, bio-mechanics, bio-medical applications, nano-technology, MEMS, optics, chemical processes, computational biology, meta-modeling, DOE and active control of structures are covered when the topic is closely related to the optimization of structures or fluids.
Structural and Multidisciplinary Optimization publishes original research papers, review articles, industrial applications, brief notes, educational articles, book reviews, conference diary, forum section, discussions on papers, authors´ replies, obituaries, announcements and society news.