Formalistic data and code availability policy in high-profile medical journals and pervasive policy-practice gaps in published articles: A meta-research study.

Background: Poor data and code (DAC) sharing undermines open science principles. This study evaluates the stringency of DAC availability policies in high-profile medical journals and identifies policy-practice gaps (PPG) in published articles.

Methods: DAC availability policies of 931 Q1 medical journals (Clarivate JCR 2021) were evaluated, with PPGs quantified across 3,191 articles from The BMJ, JAMA, NEJM, and The Lancet.

Results: Only 9.1% (85/931) of journals mandated DAC sharing and availability statements, with 70.6% of these lacking mechanisms to verify authenticity, and 61.2% allowing publication despite invalid sharing. Secondary analysis revealed a disproportionate distribution of policies across subspecialties, with 18.6% (11/59) of subspecialties having >20% journals with mandated policies. Journal impact factors exhibited positive correlations with the stringency of availability statement policies (ρ = 0.20, p < 0.001) but not with sharing policies (ρ = 0.01, p = 0.737). Among the 3,191 articles, PPGs were observed in over 90% of cases. Specifically, 33.7% lacked DAC availability statements, 23.3% refused sharing (58.4% of which without justification in public statements), and 13.5% declared public sharing, with 39.0% being unreachable. Finally, only 0.5% achieved full computational reproducibility.

Conclusions: Formalistic policies and prevalent PPGs undermine DAC transparency, necessitating a supportive publication ecosystem that empowers authors to uphold scientific responsibility and integrity.