Novelty Assessment Method for Streaming Samples Based on Slope Entropy

Abstract

During spectral detection operations, when the streaming samples under test differ significantly from the modeling samples, the prediction error of the spectral model tends to increase. In such cases, it becomes necessary to incorporate novel samples into the modeling set for model updating. A common approach to assess sample novelty involves calculating the spectral residual, typically using the Q residual. However, existing methods generally employ a single distance scalar to evaluate the novelty of multidimensional spectra. This approach suffers from the issue that residuals at different positions are aggregated into a single value, which is overly integrative. Consequently, variations at different locations may lead to the same novelty value. Therefore, a window-based method that evaluates data in segments is more reasonable. The proposed method segments the residual dimensionality using a windowing approach, discretizes the residual values, and classifies and counts the patterns of residual variations within each window. The slope entropy derived from the residual is then used to represent sample novelty. The effectiveness of the proposed method is demonstrated through both simulated and real-world data.