Selectivity estimation in web query optimization

Abstract

Web Query optimization techniques have not scaled up to the quality of classical database optimizers. The main reason is the lack of availability of meta data statistics from local data sources. This leads to enormous errors in the calculation of optimization parameters such as selectivity of an operator which can degrade the query execution performance and result in bloated response time. In this work, the problem of selectivity estimation is addressed through Histogram construction and Probabilistic selectivity estimation. Both these techniques are robust and scalable to any kind of Web Query Engine. Empirical results also demonstrate the superior quality of these techniques.