Fragmentation of Time Series Is Not an Anomaly, but the Norm

Abstract

Contemporary technologies have simplified and made accessible the collection and processing of data from scientific observations. Current solutions enable rapid preliminary processing of collected data, cleansing it of outliers related to measurement errors and filling gaps in fragmented time series. However, the ease with which this is achieved presents risks of indiscriminate use of such solutions. Consequently, data indicating real physical anomalies may be discarded, and the amalgamation of time series fragments might result in data that does not correspond to the observed processes and phenomena. Such a situation was justified in the past when there was a scarcity of computational resources. Discarding inherently unreliable values and filling gaps simplified and accelerated data analysis. Now, with sufficient computational power available, it is possible to begin searching for patterns in what was previously considered observational error and discarded. Moreover, the volume of accumulated data may allow for the consideration of fragments of time series as parts of a regular process, without filling the gaps with artificial data created based on our assumptions about the nature of the observed processes and phenomena. This raises the question of the necessity to adapt the approaches used in collecting and analyzing observational results to the possibilities afforded by new computational tools.