Exploiting procedure level locality to reduce instruction cache misses

High instruction fetch bandwidth is essential for high performance in today's wide-issue out-of-order processors. Instruction caches must provide a low miss rate as well as low latency. We introduce procedure level relocation, a class of dynamic feedback-directed optimizations that substantially reduce the instruction cache miss rate by exploiting the temporal locality of procedure usage. Based on the observation that half of all procedures executed are at most 128 bytes in length, we present a small procedure cache, a small and fast explicitly managed memory for storing small procedures. We show that procedure level relocation into a small procedure cache reduces the instruction cache miss rate by an average of 15%.