Rapid Canary Assessment Through Proxying and Two-Stage Load Balancing

Canary releasing is a means to check quality aspects of new software versions in a production environment, keeping risk to a minimum. We propose a novel approach to minimize the interference of canarying with the production system by (1) actively controlling request distribution in order to shorten the time a canary has to be running to achieve representative results and (2) unifying responsibility for that purpose and the collection of performance data about the canary on an ephemeral proxy load balancer. For (1) we contribute a two-stage load balancing approach, that implements weighted round-robin and a bucketing approach for HTTP-based services, that equalizes requests based on request URL. With explicit assumptions about infrastructure capabilities, we also describe the design of a proxy-based canary assessment tool to support (2). Both parts are implemented as a prototype and evaluated regarding feasibility, effectiveness and overhead. While the results cannot represent realistic production scenarios, they strongly indicate the validity of shortening a canary's lifetime by accepting a small overhead in client latency.