CARVE: A Cognitive Agent for Resource Value Estimation

Recently, industry has begun investigating and moving towards utility computing, where computational resources (processing, memory and I/O) are availably on demand at a market cost. On-demand access to computational resources enables fine-grained resource allocation for web-based applications, e.g., the possibility of provisioning for a minimum workload while allowing the rental of additional resources for unexpected workload changes. However, renting additional resources relies on the ability to quickly and accurately estimate the value of the resource. This paper introduces CARVE: a cognitive agent for resource value estimation. CARVE is a machine-learning based approach that learns to predict the change in system value of having more or less system resources. Using only low-level statistics and with no custom instrumentation of the operating system or middleware, CARVE is able to make informed decisions about the return on investment of physical memory when implemented and evaluated on a partitioned system running a multi-partition, multi-process distributed benchmark. We show that CARVE is competitive with static choices of computing resources over a variety of test workloads and also has the ability to outperform all static configurations.