Fiesta: Parallelism for Data Collection and Intelligent Inference in a Distributed Heterogeneous Environment

Abstract

Widely spread data over the World Wide Web is becoming an important resource that can be used in a variety of applications. In this paper, we propose an event notification and intelligent inference system based on information gathered from the profiles of registered users over the Web. Our model uses parallel workers to fetch and incrementally update the changing user data through the Web. The raw data is transformed in a canonical manner for correlations to generate notifications to a user about important events in the life of related people. We make use of asynchronous distributed task queuing, which uses an open source message-oriented middleware as a broker for message passing between our application server and the multiple background workers. This helps achieve a high degree of parallelism and scalability. A value added by this model is the higher quality of user experience provided by timely notifications. The model would also benefit suppliers catering to users' needs for various life events by promoting e-commerce and soon m-commerce. A different set of parallel workers could be conceived to capture the data about suppliers and their offers to carry out matching.