ORACLE: Online reinforcement-driven adaptive constraint learning engine for nonlinear optimization

Abstract

Constrained black-box optimization with machine learning models in process systems engineering faces two fundamental challenges: expensive function evaluations and highly constrained feasible spaces requiring extensive upfront sampling. We present ORACLE, to our knowledge the first reinforcement learning-based constrained data-driven optimization algorithm that simultaneously learns feasible regions and optimizes the objective function through unified adaptive learning. Neural network surrogates approximate expensive constraint and objective evaluations, while an adaptive validation mechanism safeguards against classifier false positives by triggering true evaluations when predictions are uncertain or candidate solutions exceed the current optimum. A feasibility classifier provides probability estimates and gradient information to guide a Soft Actor-Critic agent toward feasible regions, with validated samples driving incremental surrogate refinement throughout the optimization process. Applied to a 2-dimensional benchmark, the 5-dimensional G04 function, and a 36-dimensional waterflooding problem with 416 constraints, the framework achieves order-of-magnitude computational savings compared to prior classification-based approaches while guaranteeing feasible solutions.