Developing an Evaluation System for Quality of Health Educational Short Videos on Social Media (LassVQ) Using Nominal Group Technique and Analytic Hierarchy Process: Qualitative Study.

Background: With the increasing use of social media platforms for health communication, the quality of health educational short videos (HESVs) has become a key concern. However, no standardized framework exists to evaluate the quality of health videos on social media, highlighting the need for a comprehensive evaluation system.

Objective: This study aimed to develop a valid and structured evaluation tool for assessing the quality of HESVs on social media.

Methods: The initial evaluation indicators obtained from the literature review and brainstorming undertaken in the study group were provided to the nominal group reference Lasswell's 5W communication model, and 2 rounds of nominal group technique (NGT) were carried out to screen, add, revise, and adjust indicators, and reach a consensus of evaluation system. The indicators were then ranked based on their significance, as scored by the experts using the analytic hierarchy process. The content validity was assessed by experts who rated the relevance of each indicator on a 4-point Likert scale.

Results: The primary indicators include communicator, communication content, communication channel, and communication effect, along with 13 secondary indicators and 34 tertiary indicators. In total, 11 experts were enrolled in the NGT, 45% (5/11) of experts had a doctoral degree, and 80% (9/11) of them were ranked as an associate professor or professor. The average values of the expert judgment coefficient and authority coefficient were 0.93 (SD 0.08) and 0.85 (SD 0.10), respectively. In round 1 of NGT, the "communication target" of 5 primary indicators, 7 of 20 secondary indicators, and 66 of 94 tertiary indicators did not reach a consensus, and therefore, they were not deleted and proceeded to the next round of NGT. In round 2 of NGT, 1 primary indicator, 7 secondary indicators, and 59 tertiary indicators were deleted based on the consensus criteria. Among primary indicators, communication content was found to be the most influential, accounting for 45.68%. Among secondary indicators, credibility, scientificity, availability, and social attention were the most influential indicators, with priorities of 56.67%, 24.26%, 74.62%, and 39.89% in their respective categories. Among tertiary indicators, "become a hot search recommended by the platform" was the most influential indicator with a weight of 0.07. The content validity of all the evaluation indicators was 0.73-1.0, and the scale-level content validity index (average) was 0.87 (SD 0.15), which was indicated as acceptable.

Conclusions: The evaluation system for the quality of HESVs on social media (LassVQ; the Lasswell's Video Quality scale) was developed, and its validity was acceptable. The proposed evaluation system can be used in conjunction with qualitative methods to gain a holistic perspective on the multidimensional quality of HESVs on social media.