Mining relevant solutions for programming tasks from search engine results

Abstract

Official documentation of software development technologies, for example, APIs, may not be sufficient for all developer needs, so searching on the Internet is a usual practice. Nonetheless, finding useful information may be challenging because the best solutions are not always among the first ranked pages. Developers need to read and discard irrelevant pages, that is, those without code examples or those that have content with little focus on the desired solution. This work aims at proposing an approach to mine relevant solutions for programming tasks from search engine results by removing irrelevant pages. The authors evaluated the top-20 pages returned by the Google search engine, for 10 different queries, and observed that only 31% of the evaluated pages are relevant to developers. Then, the authors proposed and evaluated three different approaches to mine the relevant pages returned by the search engine. Google's search engine has been used as a baseline, and authors’ results have shown that it returns a reasonable number of irrelevant pages for developers, and the authors could establish an effective approach to remove irrelevant pages, suggesting that developers could benefit from a customised web search filter for development content.