Modeling the performance of Ultra-Large-Scale systems using layered simulations

The backbone of cloud computing platforms like Amazon S3 and Salesforce is formed by Ultra-Large-Scale (ULS) systems, i.e., complex, globally distributed infrastructure consisting of heterogeneous sets of software and hardware nodes. To ensure that a ULS system can scale to handle increasing service demand, it is important to understand the system's performance behaviour, for example to pro-actively plan for hardware upgrades. A good performance model should address concerns from all stakeholders at the level appropriate to their knowledge, interest, and experience. However, this is not straightforward, since stakeholders of ULS systems have a wide range of backgrounds and concerns: software developers are more interested in the performance of individual software components in the system, whereas managers are concerned about the performance of the entire system in different configurations. In this paper, we adapt the “4+1 View” model for software architecture to performance analysis models by building simulation models with multiple layers of abstraction. As a proof-of-concept, we conducted case studies on an open source RSS (Really Simple Syndication) Cloud system that actively delivers notifications of newly published content to subscribers, and on a hypothetical, industry-inspired performance monitor for ULS systems. We show that our layered simulation models are effective in identifying performance bottlenecks and optimal system configurations, balancing across performance objectives.