Assessment of Topics Published in Leading Medical Journals Using Natural Language Processing

Abstract

Introduction: Topic detection can be used to identify trends in literature, providing valuable insight into the direction of the field. We developed a natural language processing (NLP) based method to identify topics from given abstracts and assessed the main topics of published articles by top medical journals in the last three years. Methods: This study utilized a two-part methodology to extract and classify original articles published by four non-specialized medical journals; Lancet, New England Journal of Medicine, Journal of the American Medical Association, and British Medical Journal. The first part employed bibliometric data collection to search for original articles published between 2020 and 2022. The second part used an NLP approach based on the BERTopic model to classify the articles included into separate topics. Results: The model was able to classify 1,540 articles out of the included 2,081 (79.42%) into 39 different topics in 11 fields. COVID-19-related and cancer treatment-related articles constituted approximately 25% and 7% of all published papers during 2020-2022 respectively. The study found that each of the included general medical journal tended to focus on certain topics more than others. Conclusion: We identified a new methodology that can identify topics discussed in medical literature from abstracts as an input. We also demonstrated the potential of this methodology for analyzing trends in medical literature more efficiently and effectively. This study's methodology can be replicated on a larger scale with more papers, more journals, and over a longer period, highlighting the importance of further research using NLP models.