Instrumenting cloud caches for online workload monitoring: the case of online miss rate curve estimation in memcached

Fast and efficient algorithms to estimate miss rate curves have recently been proposed, yet these have not been incorporated into cloud caches. Numerous applications that could benefit from these techniques are relying on less useful cache metrics or incomplete information. We study how to instrument cloud caches to obtain online miss rate curves (MRCs). Our approach leverages state-of-the-art algorithms and data structures, thus incurring in negligible overhead. We also propose an alternative design that makes it easier to change the MRC estimation algorithm, as well as plug-in other monitoring techniques. We implemented our designs in one of the top cloud caches: Memcached. We show via experimentation, that our implementation is efficient. Finally, we discuss how our solution can be used to improve the management of cloud caches; in particular, our code can be used by caching middleware to auto-adapt to changes in workload and maximize performance.