Context-Aware Contrastive Representation Learning for Zero-Shot Biomedical Text Classification.

Abstract

Biomedical text classification refers to the task of annotating a biomedical text with its relevant labels from a candidate label set. Most of the existing approach operate in a fully supervised setting and thus heavily rely on human-annotated training data which is both labor-intensive and monetarily expensive. To address this, we propose to formulate biomedical text classification under the zero-shot learning (ZSL) paradigm that does not require any labeled training data and only relies on label surface names for training and inference. Specifically, we propose a new context-aware contrastive learning technique for ZSL that fully exploits the context information present in the biomedical text to generate semantically enriched feature representations needed for accurate zero-shot biomedical text classification. Unlike existing contrastive learning approaches that typically employ random text segmentation strategies to generate contrastive pairs, our approach utilizes the context information inherently present in biomedical text to generate semantically meaningful contrastive pairs. Extensive experiments on the largest available biomedical corpus validates the effectiveness of the proposed approach.