Hybrid Recommender System for Detection of Rare Cases Applied to Pulsar Candidate Selection

Abstract

Detection of extremely rare cases is a challenging problem for most machine learning algorithms, especially if class overlapping is present. In this paper we propose a hybrid recommender system that uses a target rare case to state users' requirements and ranks the candidates using a similarity function which is calculated as a weighted sum of individual feature similarities. Specifically, the weight of each feature is computed as a product of its association with the class label and the outlyingness of its value. We apply this hybrid recommender system on the radio pulsar candidate selection problem, for detection of two different types of rare cases: low signal-to-noise (S/N) pulsars and Fast Radio Bursts (FRBs). Our results show that the proposed approach successfully detects both low S/N pulsars and FRBs. When there is class overlapping, as in case of low S/N pulsars, treating rare feature values as outliers and increasing their weights in the similarity function improve the detection performance. For FRBs, which compared to the low S/N pulsars are relatively more distinguishable from the non-astrophysical signals, uniform weighting outperformed the feature-weighting methods. The proposed hybrid recommender system can be used in other application domains that share similar requirements such as high recall and face similar challenges such as class imbalance and class overlapping.