Assessment of the reliability and quality of pancreatic cancer related short videos on mainstream platforms: cross-sectional study.

Abstract

Background: Despite its high fatality rate, pancreatic cancer remains largely overlooked by the public. The rise of short-form video platforms has made them hubs for health-related content, yet the quality and reliability of this information are often in doubt.

Objective: This study is poised to scrutinize the quality and trustworthiness of videos pertaining to pancreatic cancer across these digital landscapes.

Methods: We analyzed the content and publishers of such videos on TikTok, Bilibili, and Kwai using the Global Quality Scale (GQS), modified DISCERN (mDISCERN), and Medical Quality Video Evaluation Tool (MQ-VET). We also correlated the findings with video rankings and compared the quality between the Chinese and USA platforms in 2023 and 2024.

Results: In 2023, 300 videos were analyzed with median scores indicating medium quality but low reliability, the median GQS, mDISCERN and MQ-VET scores were 2, 2 and 45, respectively. The short videos created by medical practitioners demonstrated significantly higher median scores compared to those by non-medical practitioners in GQS scores (3 [IQR, 2-4] vs. 2 [IQR, 2-3]; P < 0.001), mDISCERN scores (2 [IQR, 2-3] vs. 2 [IQR, 1-2]; P < 0.001), and MQ-VET scores (46 [IQR, 40-52] vs. 37 [IQR, 30-45.5]; P < 0.001). mDISCERN scores showed superiority in treatment-related (3 [IQR, 2-3]), prevention-related (3 [IQR, 2.75-3]), and disease-related videos (including anatomical, pathologic, epidemiologic, and basic research related to pancreatic cancer)(2 [IQR, 2-3]) compared to News and Reports (2 [IQR, 1-2]) and invalid information content (1 [IQR, 1-1]; P < 0.001). TikTok had significantly higher mDISCERN scores (2 [IQR, 2-3] vs Bilibili: 2 [IQR, 1.75-3]; P = 0.024) and MQ-VET scores (47 [IQR, 43-53.5] vs Kwai: 44.5 [IQR, 38.25-49.75]; P = 0.033) for medical professional videos. Video quality showed a weak correlation with rankings. And the GQS scores of short videos in China in 2024 decreased compared with that in 2023 (2 [IQR, 2-3] vs 3 [IQR, 2-4]; P = 0.009). Additionally, in 2024, both medical and non-medical practitioners' videos on the Chinese TikTok platform exhibited lower quality and reliability compared to their counterparts in the USA.

Conclusion: Pancreatic cancer-related short videos are of medium quality and low reliability, particularly on Chinese platforms. Videos from medical professionals are more trustworthy. There is a need for better curation and algorithms to ensure accurate health information dissemination and to enhance public understanding and management of pancreatic cancer.