Transformer-based sequence-to-sequence soft sensor using missing data in industrial processes

Abstract

Handling missing process data is a basic requirement for soft sensor development in process systems engineering. In strongly dynamic and nonlinear operations, imputers that ignore information carried by key performance indicators often yield biased completions and deteriorate soft sensing performance. To address this issue, a transformer-based sequence-to-sequence model (Trans-S2S) is proposed that performs imputation and prediction jointly within an end-to-end framework. The self-attention mechanism can directly model long-term dependencies at different times and is less sensitive to missing values. A long short-term memory then imputes missing values while capturing short-term nonlinearity. For the prediction, a long short-term memory is used to predict key performance indicators from the completed data. Both modules are trained jointly, allowing the soft sensing task to guide imputation. Across missing rate scenarios, Trans-S2S reduces soft sensing errors by 22.41 %–63.75 % on the penicillin fermentation benchmark and by 4.76 %–30.00 % on the cement production dataset, compared with baseline methods. Explanation analysis on the penicillin case further indicates that the imputations remain context-consistent and agree with known process mechanisms.