Parallel information retrieval on a distributed memory multiprocessor system

Abstract

This paper presents a parallel information retrieval (IR) system which provides information for users with precision and speed. For precision, the IR system adopts a full-text search method. To supply a fast information service, parallel processing techniques are used, so multiple queries are processed concurrently and each of these queries is also handled in parallel using multiple processors. For the efficient utilization of multiple processors, we developed a processor allocation method which dynamically assigns various-size processor clusters for incoming queries based on the current workload of the system. The parallel IR model is implemented on a multi-transputer system composed of 16 processors. According to the experiments, a linear speed-up of up to 11.3-fold is obtained, and network and hard disk overheads are negligible in comparison with the response time.