An open-set semi-supervised multi-task learning framework for context classification in biomedical texts

Abstract

Objective

In biomedical research, knowledge about the relationships between entities, including genes, proteins, and drugs, is vital for elucidating complex biological processes and intracellular pathway mechanisms. While natural language processing (NLP) methods have shown great success in biomedical relation extraction (RE), extracted relations often lack contextual information such as cell type, cell line, and intracellular location. Previous studies treated this problem as a post hoc context-relation association task, limited by the absence of a golden standard corpus and prone to error propagation. To address these challenges, we propose CELESTA (Context Extraction through LEarning with Semi-supervised multi-Task Architecture), an open-set semi-supervised multi-task learning (OSSL-MTL) framework for biomedical context classification.

Methods

We designed a multi-task learning (MTL) architecture that integrates with the semi-supervised learning (SSL) strategies to leverage unlabeled data containing both in-distribution (ID) and out-of-distribution (OOD) examples. We created a large-scale dataset consisting of five context classification tasks by curating two large Biological Expression Language (BEL) corpora and annotating them with our new entity span annotation method. Additionally, we developed an OOD detector to distinguish between ID and OOD instances within the unlabeled data and applied data augmentation with an external database to enrich our dataset.

Results

Extensive experiments show that our framework significantly improves context classification performance. Our best OSSL-MTL models achieve F1 scores of 77.75% and 82.87% on location and disease classification tasks, and the SSL-MTL models without OOD detection perform best for cell line and cell type classification. The OOD detection experiment confirms that the OOD detector effectively identifies unknown categories while maintaining ID accuracy. Qualitative analysis shows improved extraction of implicit contexts compared to baseline models.

Conclusion

Our analysis demonstrates the effectiveness of the framework CELESTA in improving context classification and extracting contextual information with high accuracy. The newly created dataset and code are publicly available on GitHub (https://github.com/pitt-miskov-zivanov-lab/CELESTA).