Quality assessment of systematic reviews in software engineering: a tertiary study

Abstract

Context: The quality of an Systematic Literature Review (SLR) is as good as the quality of the reviewed papers. Hence, it is vital to rigorously assess the papers included in an SLR. There has been no tertiary study aimed at reporting the state of the practice of quality assessment used in SLRs in Software Engineering (SE). Objective: We aimed to study the practices of quality assessment of the papers included in SLRs in SE. Method: We conducted a tertiary study of the SLRs that have performed quality assessment of the reviewed papers. Results: We identified and analyzed different aspects of the quality assessment of the papers included in 127 SLRs. Conclusion: Researchers use a variety of strategies for quality assessment of the papers reviewed, but report little about the justification for the used criteria. The focus is creditability but not relevance aspect of the papers. Appropriate guidelines are required for devising quality assessment strategies.