A feature optimized attention transformer with kinetic information capture and weighted robust Z-score for industrial NOx emission forecasting

Abstract

Establishing an accurate and stable NO_x concentration prediction model is of great significance for pollution control in refineries and achieving carbon neutrality goals. The actual industrial denitrification process exhibits high nonlinearity, strong coupling, and multivariable dynamic characteristics, which makes modeling difficult. This paper proposes a novel model based on Transformer designed to uncover the potential dynamic relationships between variables. First, for unstable industrial data, a Weighted Robust Z-score (WRZ) method is employed, which assigns weights to data points and uses weighted median and interquartile range to replace traditional mean and standard deviation for calculating deviations of data points. Second, to address the complex dynamic characteristics of the data, an Enhanced Pooling Feature Module (EPFM) is proposed, combining weighted pooling and average pooling to optimize feature extraction. Embedded in Transformer, EPFM adjusts attention, highlighting key features. Finally, Attention scores visualization explicitly clarifies variable interaction mechanisms, enhancing model interpretability. Experiments on four chemical datasets validated the proposed model's effectiveness. In the catalytic cracking regeneration flue gas denitrification dataset, the proposed method has RMSE values of 0.8 and 0.922, and R² values of 0.993 and 0.956, outperforming others. It offers an effective way to boost industrial denitrification efficiency and reduce NO_x emissions.