Till now, deep learning–based intelligent diagnosis models combined with multisource information have become popular, but tough issues like multisource feature extraction and information redundancy may sacrifice the models’ representational power and result in a degraded performance. Aiming at the above problems, this paper proposed a novel model called multisource heterogeneous information selective fusion network (MHI-SFN) for rolling bearing fault diagnosis. In MHI-SFN, multisource heterogeneous signals were stacked together and directly fed into model and grouped convolution was adopted to replace standard convolution throughout the structure, enabling kernels to firstly focus on the feature extraction of every individual signal and then perform efficient feature fusion work as needed. Then, selective kernel modules were designed to adaptively assign suitable kernel sizes and selectively fuse the valuable information between different scales of feature map from different signal sources. Lastly, channel attention was introduced to adaptively alleviate the information correlation and redundancy between the extracted features. Compared with other multisource information–based methods, MHI-SFH automatically solves the multisource feature fusion and information redundancy problems with its specially designed structure, avoiding complicated hand-crafted signal processing steps and achieving a powerful end-to-end intelligent fault diagnosis. The proposed method was experimentally verified on two rolling bearing datasets, and the results proved the feasibility and superiority of the MHI-SFN model.