DIDA: Distributed In-Network Intelligent Data Plane for Machine Learning Applications

Abstract

Recent advances in network switch designs have enabled machine learning inference directly within the switch at line speed. However, hardware constraints limit switches capabilities of tracking stateful features essential for accurate inference, as the demand for these features grows rapidly with line rates. To address this, we propose DIDA, a distributed in-network machine learning approach. In DIDA, feature extraction occurs at the host, features are transmitted via in-band telemetry, and inference is performed on the switches. In this paper, we evaluate the effectiveness and efficiency of this architecture. We examine its impact on network bandwidth, CPU and memory usage at the host, and its robustness across different feature sets and deep neural network classifications.