Chiffchaff: Observability and analytics to achieve high availability

`Anywhere, Anytime and Any Device' is often used to characterize the next generation Internet. Achieving the above in light of the increasing use of the Internet worldwide, especially fueled by mobile Internet usage, and the exponential growth in the number of connected devices is non-trivial. In particular, the three As require development of infrastructure which is highly available, performant and scalable. Additionally, from a corporate standpoint, high efficiency is of utmost importance. To facilitate high availability, deep observability of physical, system and application metrics and analytics support, say for systematic capacity planning, is needed. Although there exist many commercial services to assist observability in the data center, public/ private cloud, they lack analytics support. To this end, we developed a framework at Twitter, called Chiffchaff, to drive capacity planning in light of growing user base. Specifically, the framework provides support for automatic mining of application metrics and subsequent visualization of trends (for example, Week-over-Week (WoW), Month-over-Month (MoM)), data distribution et cetera. Further, the framework enables deep diving into traffic patterns, which can be used to guide load balancing in shared systems. We illustrate the use of Chiffchaff with production traffic.