Co-DIMM: Inter-Socket Data Sharing via a Common DIMM Channel

Abstract

To improve computing density, modern datacenters widely deploy server chassis with several processor sockets integrated as independent nodes. Distributed applications processing enormous datasets on such systems require frequent inter-node communication. Data sharing among distributed on-board socket nodes in the same server chassis via commodity networking and inter-socket connection technologies is inefficient, though. To address this problem, we propose inter-socket data sharing via normal memory access instructions. Co-DIMM eliminates the overheads of protocol-stack processing and data movement through the network. Instead of sharing data through centralized shared memory based on NUMA inter-socket connections, DDR switches allow Co-DIMM ownership to be changed dynamically to support asynchronous producer-consumer data sharing. We implement Co-DIMM in a custom in-house FPGA-based platform to generate preliminary results showing that data-sharing latency between two sockets is as low as 1.33μs. We present potential Co-DIMM usage scenarios and discuss implementation challenges.