A Model for Zero-shot Text Multi-labeling Using Semantics-based Labels

Abstract

We introduce a transformer-based method to associate relevant tags to text passages or blocks such as categories to pages of a website, marking sections in an article, or social postings subject tagging. In contrast with traditional multi-label formulations, the proposed approach uses semantic definitions of the tags available during training, and the model outputs a binary prediction of whether the described category applies to a document or not. The transformer-based model learns the semantics of the definition of a tag, and therefore works for tags not seen during training. Performance on domain-specific datasets can be further improved via transfer learning after fine-tuning with relatively little additional labeled data required.