A scalable tensor-based MDTW approach for multi-modal time series patterns clustering

Abstract

Multi-modal Time Series (MTS) is a vital ingredient to Predictive Multi-modal Artificial Intelligence (PMAI). MTS systems capture varying temporal modalities and their inherent dependencies for their accurate analytics. However, efficiently exploring these cross-modalities relationships is a challenging research due to their complexity facets and information redundancies. MTS patterns' pairwise similarity measures precede PMAI. Multi-modal Dynamic Time Warping (MDTW) is frequently explored to quantify similar MTS. Yet, it's reliant on the orthogonal conditioned local similarity measures that ignore the contributions of MTS' underlying structural relationships in the warping process and, hence, susceptible to unrealistic matching. This paper addresses the setbacks by recommending a scalable MTS recognition model, named Tensor-Slices Distance (TSD)-based MDTW (TSD-MDTW), that's subsequently advanced to two more distinct models termed Weighted modality and TSD (WmTSD-MDTW) and TSD-Mahalanobis (TSDMaha-MDTW). To quantify an alignment's cost, TSD-MDTW incorporates intrinsic spatial dependencies between modalities' coordinates, while WmTSD-MDTW relaxes information redundancies through weighing modalities based on information richness, whereas TSDMaha-MDTW embodies modalities dependencies and their coordinates' innate spatial dependencies. Besides, it proposes a scalable Tensor-based DTW (TDTW) model that re-formulates MDTW into multiple dimensions that are found paralleling warping processes. Theoretical and empirical experimental results on MTS multi-modal datasets encompassing load patterns and meteorological modalities reveal TDTW's efficiency and proposals' superior performances in terms of cluster compactness and separation over MDTW employing the state-of-the-art local similarity measures.