Memory-Based Data Management for Large-Scale Distributed Rendering

Abstract

With the increasing requirements of large-scale animation rendering, I/O congestion has become one of the main bottlenecks to constrain the whole performance seriously. What's worse, frequent rendering data access in remote storages brings a heavy burden of storage and delay of data access. Distributed Memory caching System for Rendering (RenDMS) is proposed to alleviate I/O congestion and improve data access performance. Rendering data is accessed from memories but not remote storages to reduce the overhead of data access and transmission. The concept of rendering unit is put forward to cluster rendering nodes, and MPI-based RPC, two-level data management, and dynamic adaptive placement are proposed to make rendering performance much better. Experiments have demonstrated the effectiveness of RenDMS. Compared with direct remote disk access, parallel rendering with RenDMS can not only shorten data access time more than 40%, but also speedup the execution of rendering applications efficiently.