On-the-fly processing of continuous data streams with a pipeline of microprocessors

Abstract

A pipeline of microprocessors which is able to perform substantial on-the-fly transformations with large amounts of data has been developed. The general concept is a pipelined structure for associative processing. This structure is based on the use of a long sequence (pipeline) of identical, relatively simple associative pattern matching/transforming elements. The associative pipeline achieves high performance for relatively small algorithms with a large volume of data and so is well suited for use with very large databases. The effectiveness of the developed pipeline has been analyzed for various database applications. This system can implement basic operations of relational algebra, as well as rather sophisticated filtering functions; in particular, it can be used to control the I/O operations for the purpose of computer security.<>