Multi-scale trend decomposition mixture of experts and time series retrieval-augmented modeling for erythromycin fermentation process

Abstract

Multivariate time series (MTS) is the primary modality for storing data in real-world and industrial applications. In the context of batch fermentation processes, such data exhibit periodicity and repetition between samples, while demonstrating stage-wise and trending patterns within samples. Effectively leveraging historical production samples to uncover stage-specific characteristics and dynamic distribution patterns is a crucial approach for improving predictive accuracy. This paper proposes an MTS modeling framework that combines Retrieval-Augmented Generation (RAG) and a Mixture of Experts (MoE) model, i.e., Multi-scale Augmented Series Trend Experts with Retrieval, referred to as MASTER. We designed a general temporal feature augmentation method (MTS-RAG) to enhance predictive accuracy by efficiently completing contextual information during the data loading stage using representative historical samples. Additionally, we developed a multi-scale trend decomposition model based on the Kolmogorov-Arnold Network, which enhances both interpretability and predictive performance by independently modeling trend and seasonal components. Inspired by the success of sparse MoE in large language models, we introduce a Time Stage Router that employs temporal position embeddings and sparse gating structures to assist the model in identifying the current fermentation phase, thereby improving its generalization and practicality in multi-stage tasks. On an industrial dataset of erythromycin fermentation processes, MASTER achieved state-of-the-art predictive performance, and ablation studies further validated the effectiveness of its components.