Trigger grouping: a scalable approach to large scale information monitoring

Abstract

Information change monitoring services are becoming increasingly useful as more and more information is published on the Web. A major research challenge is how to make the service scalable to serve millions of monitoring requests. Such services usually use soft triggers to model users' monitoring requests. We have developed an effective trigger grouping scheme to optimize the trigger processing. The main idea behind this scheme is to reduce repeated computation by grouping monitoring requests of similar structures together. In this paper, we evaluate our approach using both measurements on real systems and simulations. The study shows significant performance gains using the trigger grouping approach. Moreover, the gains are critically dependent on group size and group size distribution (e.g., Zipf). We also discuss the benefit, trade-off, and runtime characteristics of the proposed approach.