Barrier Synchronization vs. Voltage Noise: A Quantitative Analysis*

Synchronization is the key to guaranteeing correct execution of parallel programs at any scale. Barriers represent heavily used synchronization primitives which prevent parallel tasks from proceeding to subsequent stages of computation before all tasks are done with previous stages. Accordingly, all tasks wait at a barrier until the slowest tasks finish, at which point all tasks can proceed to the next stage of computation. This usually translates into an abrupt change in the activity, i.e., current demand from the power delivery network, and if not orchestrated properly, can easily lead to voltage emergencies. In this study we characterize the impact of different barrier structures on voltage noise.