Development and validation of a virtual learning management system usability questionnaire: A case study.

Abstract

Background: The post-pandemic era has seen a surge in the popularity of Virtual Learning Management Systems (VLMS). However, there is a noticeable lack of tools to measure the usability of these systems. As technology evolves, user needs change, necessitating updated tools for system evaluation.

Objective: This study aims to develop and validate a VLMS usability questionnaire, specifically designed to assess the usability of a university learning management system.

Methods: The VLMS usability tool was systematically developed based on relevant domains identified in existing literature and expert opinions. It was then tested for face validity, content validity, and reliability. In a case study, the tool was distributed among 200 students from a Medical Sciences university who had used the Navid VLMS system.

Results: Semi-structured interviews with experts were analyzed using directed content analysis, resulting in 21 items categorized into four domains: effectiveness, reliability, learnability, and security. The content validity index and ratio were 0.939 and 0.976, respectively. The Intra Class Correlation (ICC) estimates for each section of the questionnaire ranged from 0.8-0.9, indicating high reliability. Cronbach's alpha was 0.97, suggesting excellent internal consistency. The case study results showed that the Navid platform achieved an average usability score of 70.36, with a standard deviation of 10.6, indicating moderate to high usability.

Conclusions: The VLMS usability tool is a valid and reliable instrument for assessing the usability of the Navid learning management system. It can be used to improve the usability of the Navid system and serve as a benchmark for assessing the usability of other similar VLMSs.