Causal Semantic Alignment for LLM-Based Time Series Forecasting.

Abstract

Recent advances in large language models (LLMs) have opened new possibilities for time series forecasting by enabling alignment between temporal patterns and pretrained word embeddings. However, most LLM-based forecasting methods overlook the heterogeneous nature of time series, where stable semantics and dynamic variations are often entangled. Such entanglement may introduce environmentally induced variations during cross-modal alignment, reducing the stability of semantic representations. To address this issue, we propose a causal variable-level alignment Transformer (CVAformer), a variable-level semantic alignment framework for time series forecasting. CVAformer explicitly decomposes each variable into invariant and dynamic representations before alignment and performs dynamic-factor conditioning to improve alignment robustness. In addition, CVAformer employs NCBlocks to model cross-variable interactions that support semantic alignment and forecasting. Extensive experiments across long-term, short-term, few-shot, and zero-shot forecasting settings demonstrate that CVAformer consistently achieves competitive or state-of-the-art performance on diverse benchmarks. Further analyses verify the effectiveness of variable-level semantic alignment and invariant-dynamic decomposition, providing a promising direction for integrating representation learning and causal reasoning in LLM-based time series forecasting.