Publication of observational studies making claims of causation over time

To examine methodology characteristics over time and investigate research impact before and after the start of the COVID-19 era, we analyzed original articles published in The New England Journal of Medicine between October 26, 2017 and August 27, 2022. April 1, 2020 was used as the defining date dividing before and after the COVID-19 era. Out of 1051 original articles, 515 (49 %) were before and 536 (51 %) were after the COVID-19 era. Two independent reviewers categorized and reconciled methodology into groups: “randomized trial” (715 articles), “uncontrolled experimental study” (128), “descriptive observational study” (168), and “observational study making a causal claim” (40). We extracted subsequent citations and Altmetric data for each article to assess impact.

The median number of social media shares was 2272 (IQR: 743–7821) for observational studies making a causal conclusion, compared to 306 (IQR: 70–606) for randomized trials (p-value=<0.001). The median Altmetric score for randomized COVID-19 trials (2421, IQR: 1063–3920) was not significantly different than that of COVID-19 observational studies making a causal claim (2583, IQR: 1513–6197, p-value = 0.42), but it was significantly lower than descriptive observational COVID-19 studies (4093, IQR: 2545–6823, p-value = 0.04).

We conclude that there has been a steady increase in the number and percentage of observational studies that make causal conclusions about the efficacy of an intervention. Research concerning COVID-19, regardless of methodology, has seen a sharp rise in dissemination as measured through Altmetric's social media score and subsequent citations.