Scalable High-Performance Architecture for Evolving Recommender System

Recommender systems are expected to scale to the requirement of the large number of recommendations made to the customers and to keep the latency of recommendations within a stringent limit. Such requirements make architecting a recommender system a challenge. This challenge is exacerbated when different ML/DL models are employed simultaneously. This paper presents how we accelerated a recommender system that contained a state-of-the-art Graph neural network (GNN) based DL model and a dot product-based ML model. The ML model was used offline, where its recommendations were cached, and the GNN-based model provided recommendations in real time. The merging of offline results with the results provided by the real-time session-based recommendation model again posed a challenge for latency. We could reduce the model's recommendation latency from 1.5 seconds to under 65 milliseconds with careful re-architecting. We also improved the throughput from 1 recommendation per second to 1500 recommendations per second on a VM with 16-core CPU and 64 GB RAM.