Synopsis for microbiological data stream analysis

Abstract

This paper derives from the extensive analysis of the results of microbiological laboratory output in a large hospital. In this environment data items are automatically collected by transactions linked to the analysis devices and the data flow rate can be very high. In addition, the time of the data items is extremely relevant, since a variation of the number of positive findings can be generated by dangerous events, such as outbreaks. The typical setting of data mining applications is to process data, in search of some hidden information. Unfortunately in this environment, we have many transactions and everyone of these is a potentially important information. When the data set changes, some kind of re-computation, either from scratch or according to differences only, has to be done. Things change when the change rate increases, so as to make difficult to process changes before new changes arrive. We define this particular situation as a data stream, and we devise a framework allowing efficient analysis of data streams. We propose a base set of techniques, which can be used to analyze data streams in Euclidean space, putting together information processing and statistical techniques. In particular, we are interested in detecting "alarms" in microbiological time series, that is points in time where the detected data differ from the values expected on the basis of past history. We provide also experimental results, based on real and data.