Physics Informed Piecewise Linear Neural Networks for Process Optimization

Constructing first-principles models is usually a challenging and time-consuming task due to the complexity of real-life processes. On the other hand, data-driven modeling, particularly a neural network model, often suffers from overfitting and lack of useful and high-quality data. At the same time, embedding trained machine learning models directly into the optimization problems has become an effective and state-of-the-art approach for surrogate optimization, whose performance can be improved by physics-informed machine learning. This study proposes using piecewise linear neural network models with physics-informed knowledge for optimization problems with neural network models embedded. In addition to using widely accepted and naturally piecewise linear rectified linear unit (ReLU) activation functions, this study also suggests piecewise linear approximations for the hyperbolic tangent activation function to widen the domain. Optimization of three case studies, a blending process, an industrial distillation column, and a crude oil column are investigated. Physics-informed trained neural network-based optimal results are closer to global optimality for all cases. Finally, associated CPU times for the optimization problems are much shorter than the standard optimization results.