Research of Key Technologies of Distributed Stream Processing Based on FaaS

Abstract

Serverless computing has emerged as a promising paradigm for cloud-based stream processing applications characterized by fluctuating workloads and latency sensitivity. While existing Function-as-a-Service (FaaS) implementations primarily focus on homogeneous CPU/memory resource scaling, they fail to address the challenges of heterogeneous resource management and coordinated elasticity in distributed stream processing. This study proposes HFaaS, a novel serverless framework that integrates dataflow programming with heterogeneous resource orchestration for stream processing applications. The key innovations include: (1) a dataflow-oriented function composition model enabling dynamic scaling of individual processing stages through peer-to-point communication mechanisms, (2) a fine-grained GPU resource allocation strategy achieving 15% + utilization improvement through device sharing and elastic scaling capabilities, and (3) a locality-aware scheduling algorithm optimizing task placement based on data proximity and heterogeneous resource availability. Experimental results demonstrate that HFaaS effectively coordinates multi-stage function scaling while maintaining sub-second latency guarantees. The proposed resource allocation strategy improves GPU utilization by 15.2% compared to conventional static allocation approaches, with network overhead reduced by 31.6% through data-local scheduling. This work bridges the gap between serverless architectures and modern stream processing requirements, providing a unified platform for building resource-efficient, latency-sensitive distributed applications in heterogeneous cloud environments.