Database caching for job-level computing

Abstract

This paper improves upon Job-Level (JL) computing, a general distributed computing approach. In JL computing, a client maintains the overall search tree and parcels the overall search into coarse-grained jobs, which are then each calculated by pre-existing game-playing programs. In order to support large-scale problems such as solving 7×7 killall-Go, or building opening books for 9×9 Go or Connect6, JL computing is modified so that the entire search tree is stored in a database, as opposed to simply being stored in the client process' memory. However, the time cost of accessing this database becomes a bottleneck on performance when using a large number of computing resources. This paper proposes a cache mechanism for JL search trees. Instead of the previous approach, where the entire search tree is stored in the database, we maintain parts of the search tree in the memory of the client process to reduce the number of database accesses. Our method significantly improves the performance of job operations. Assuming that each job requires 30 seconds on average, the JL application with this cache mechanism can allow for the use of 5036 distributed computing resources in parallel without database accesses becoming the performance bottleneck.