An Incremental Algorithm for Estimating Extreme Quantiles

Abstract

Extreme quantile is a very influential and powerful performance measure in high risk environments like financial markets, targeted advertising and high frequency trading. Extreme quantiles are defined as the threshold in the range of the performance values of the system being monitored beyond which the probability is extremely low. Unfortunately, the estimation of extreme quantiles is usually accompanied by high variance. We provide an incremental, single pass and adaptive variance reduction technique to estimate extreme quantiles. We further provide additional theoretical and empirical analysis pertaining to the effectiveness of our approach. Our experiments show considerable performance improvement over other widely popular algorithms.