SAP: Shared-Aware Prefetching for Reducing Inter-Chiplet Data Access Latency

Abstract

In multi-chiplet systems, inter-chiplet shared-data transfers pose a significant bottleneck, prolonging the critical paths of memory accesses. In inter-chiplet coherence traffic, since each chiplet often needs to wait reactively for data from remote chiplets, proactive data-fetching mechanisms such as prefetching are essential to anticipate inter-chiplet data accesses and mitigate latency. Nevertheless, traditional prefetchers are inadequate for explicitly handling inter-chiplet shared-data transfers, overlooking potential prefetching opportunities. To overcome this limitation, we propose SAP, a shared-aware prefetching mechanism that minimizes inter-chiplet data access latency. By transforming the IO chiplet into an active prefetching agent, SAP proactively fetches inter-chiplet shared data before demand requests arrive, utilizing a sharing table to track recent shared-data events and a prefetch agent to initiate inter-chiplet data transfers early. Our experiments on a chiplet-based system demonstrate that SAP improves system throughput by 13.44% and reduces execution time by 12.33% compared to the prior design.