(Automated) Literature Analysis - Threats and Experiences

Abstract

The number of scientific publications is increasing each year, specifically in the field of computer science. In order to condense existing knowledge, evidence-based software engineering is concerned with systematic literature reviews, surveys, and other kinds of literature analysis. These methods are used to summarize the evidence on empirical studies – or approaches in general – and to identify gaps for new research opportunities. However, executing systematic review processes requires a considerable amount of time and effort. Consequently, researchers have proposed several semi-automated approaches to support and facilitate different steps of such methods. With our current research, we aim to assist researchers to efficiently and effectively execute different steps, namely the search for and selection of primary studies. In this paper, we report several issues we identified during our research that threaten any kind of literature analysis and hamper suitable tool support. We further recommend solutions to mitigate these threats. Overall, our goal is to raise researchers’ and publishers’ awareness regarding several potential threats on literature analysis, to support software engineers in designing suitable tools for research, and to encourage the research community to solve these threats.