On the Reliability and Validity of Digital Competence Evaluation in Higher Education

Abstract

This paper aims to provide an overview of the reliability and validity of the instruments used to measure digital competence in higher education. Three research questions were addressed about: 1) the demographic and methodological characteristics of the literature, 2) typical assessments and reporting practices, and 3) the association between studies’ characteristics and their reporting practices. Through a systematic mapping study, 60 primary studies published in the period 2018-2020 were analyzed and classified. The results indicated that most of the studies originated in Europe and Asia; they focused on university students from Social Sciences; they administered instruments composed of cognitive items for diagnosing digital competence; they were published in journals not indexed by the Journal Citation Report; and about 50% did not report on the reliability and validity of the instrument used. Studies published in low-quality journals or focused on administration only are more susceptible to not reporting these psychometric assessments.