A scalable architecture for ordered irregular parallelism

We present a new parallel architecture that exploits ordered irregular parallelism, which is abundant but hard to mine with current software and hardware techniques. In this architecture, called Swarm, programs consist of short tasks, as small as tens of instructions each, with programmer-specified order constraints. Swarm executes tasks speculatively and out of order, and efficiently speculates thousands of tasks ahead of the earliest active task to uncover enough parallelism. Furthermore, Swarm sends task to run close to their data whenever possible, reducing data movement. We contribute several new techniques that allow Swarm to scale to large core counts and speculation windows, including a new execution model, speculation-aware hardware task management, selective aborts, and scalable ordered task commits.