Cost to serve of large scale online systems

Abstract

Online systems typically provide a variety of different service offerings. For example, an internet search engine provides the service of searching web pages, videos, images, news, maps etc. Each offering can utilize different physical and/or virtual systems, networks, data centers, and so forth. Thus, a request to search videos may use some, but not all, of the resources used by a request to search images. Also, each video query will not use the same number of resources due to caching and ranking algorithms. Due to this it can become extremely difficult to ascertain the Cost to Serve (CTS) of an offering. CTS is required to understand cost of the product offerings for request per second (RPS), create rate card for partner deals, target efficiency areas and decide ROI of services. In this paper, we define the CTS methodology for Bing. In this methodology, CTS is calculated by determining operational RPS of each platform in Bing and the average number of times a type of request touches those platforms. Prior to this work, CTS was calculated by manually tagging capacity used by each offering and number of observed queries. The methodology described here can be applied to any other large scale online distributed system.