Contrastive Pre-training for Imbalanced Corporate Credit Ratings

Abstract

The corporate credit rating reflects the level of corporate credit and plays a crucial role in modern financial risk control. But real-world credit rating data usually shows long-tail distributions, which means a heavy class imbalanced problem challenging the corporate credit rating system greatly. To tackle that, inspired by the recent advances of pre-train techniques in self-supervised representation learning, we propose a novel framework named Contrastive Pre-training for Corporate Credit Rating (CP4CCR), which utilizes the self-supervision for getting over the class imbalance. Specifically, we propose to, in the first phase, exert contrastive self-supervised pre-training without label information, which aims to learn a better class-agnostic initialization. Furthermore, two self-supervised tasks are developed within CP4CCR: (i) Feature Masking (FM) and (ii) Feature Swapping(FS). In the second phase, we can train any standard corporate credit rating model initialized by the pre-trained network. Extensive experiments conducted on the real public-listed corporate rating dataset, prove that CP4CCR can improve the performance of standard corporate credit rating models, especially for the class with few samples.