An improved replacement strategy for function caching

Function caching is the technique of remembering previous function calls and avoiding the cost of recomputing them. Function caching provides a simple way of implementing dynamic programming algorithms and can provide a facility for incremental computation. Previous discussions of function caching have generally relied on the user to purge items from the function cache or have proposed a strategy such as least-recently-used without any analysis of the appropriateness of that strategy. We describe a formal model that allows us to describe the potential of a function cache and use that model to develop a practical cache replacement strategy that performs substantially better than currently used strategies. Benchmarks show that in use in an incremental theorem prover, our caching strategy produces almost a factor of four improvement in running time over a system running without function caching and almost a factor of two improvement in running time over a system using a standard cache replacement strategy.