Locks, deadlocks and abstractions: experiences with multi-threaded programming at CloudFlare, Inc.

Abstract

At CloudFlare[1, 2], we are about a year into our public release. Over the last six months we’ve seen exponential growth. CloudFlare provides a content delivery network currently serving over ten billion page views/month to over 200 million unique visitors. During July 2011 approximately ten percent of all people on the Internet visited a CloudFlare powered site at least once. Figure 1 shows monthly page views served by CloudFlare over the past year. We run a highly customized software stack on a limited number of powerful physical servers deployed in twelve data centers on three continents. The upshot of all of this is that we’ve been forced to rapidly code, and re-code, to take full advantage of 24 plus cores per machine. This experience report is a very brief survey of the programming models and debugging methodology CloudFlare uses. We first describe two ways in which CloudFlare deals with concurrency issues. We then compare bugs and features in two applications which are representative of the above paradigms.