A scoping review of engineering education systematic reviews

Abstract

Background

Systematic review or systematic literature review (SLR) methodologies are a powerful tool for evidence-based decision making. The method originated in the medical sciences but has since been adopted by other disciplines, including engineering education (EE).

Purpose

We aimed to answer two research questions: (i) To what extent is the SLR research method being applied in EE? (ii) How closely are SLRs published in EE following established reporting guidelines for the methodology?

Scope/Method

We searched Inspec, Compendex, and ERIC for engineering-related SLRs and meta-analyses (MAs). We included English language papers that contained an explicit SLR search, or where it appeared the methodology was intended by the author(s). We completed a data extraction process for 21 descriptive and quality-related items, including engineering discipline, which allowed us to identify the EE studies analyzed in this article.

Results

This sub-analysis presents the results of 276 EE-related reviews. We found the use of SLR/MA methods is growing in EE, with 93% of papers published during 2015–2022. However, we found that authors are not generally following established guidelines for reporting their methods and findings.

Conclusions

Not following the best practices for conducting and reporting SLRs can result in the presentation of incorrect summaries and analyses due to missed evidence. Including search experts (e.g., librarians) trained in conducting SLRs can improve review quality. There is also an opportunity for EE-related publishers to recruit experts trained in conducting SLRs as peer reviewers to participate in evaluating submitted reviews.