A latency-optimized hybrid network for clustering FPGAs (abstract only)

The data-intensive applications that will shape computing in the coming decades require scalable architectures that incorporate scalable data and compute resources and can support unstructured (e.g., logs) and semi-structured (e.g., large graph, XML) data sets. To explore the suitability of FPGAs for these computations, we are constructing an FPGA-based system with a memory capacity of 512 GB from a collection of 32 Virtex-5 FPGAs spread across 8 enclosures. This poster describes the system's interconnect that combines inter-enclosure high-speed serial links and wide, single-ended intra-enclosure on-board traces with a network topology that optimizes for latency and bandwidth for small packets. The network uses a multi-level radix-12 router optimized for the asymmetry between the inter- and intra-enclosure links. The system has a peak theoretical bisection bandwidth of 247.2 Gb/s and a total switching capacity of 2.13 Tb/s. Under random traffic, the network sustains an aggregate throughput of 354.3 Gb/s. The channel transceivers and router consume 22% of the FPGAs' Block RAMs and 33% of their FPGA Slices.