Optimizing a search-based code recommendation system

Search-based code recommendation systems with a large-scale code repository can provide the programmers example code snippets that teach them not only names in application programming interface of libraries and frameworks, but also practical usages consisting of multiple steps. However, it is not easy to optimize such systems because usefulness of recommended code is indirect and hard to be measured. We propose a method that mechanically evaluates usefulness for our recommendation system called Selene. By using the proposed method, we adjusted several search and user-interface parameters in Selene for better recall factor, and also learned characteristics of those parameters.