Deep learning has been demonstrated effective in many neuroimaging applications. However, in many scenarios, the number of imaging sequences capturing information related to small vessel disease lesions is insufficient to support data-driven techniques. Additionally, cohort-based studies may not always have the optimal or essential imaging sequences for accurate lesion detection. Therefore, it is necessary to determine which imaging sequences are crucial for precise detection. This study introduces a deep learning framework to detect enlarged perivascular spaces (ePVS) and aims to find the optimal combination of MRI sequences for deep learning-based quantification. We implemented an effective lightweight U-Net adapted for ePVS detection and comprehensively investigated different combinations of information from SWI, FLAIR, T1-weighted (T1w), and T2-weighted (T2w) MRI sequences. The experimental results showed that T2w MRI is the most important for accurate ePVS detection, and the incorporation of SWI, FLAIR and T1w MRI in the deep neural network had minor improvements in accuracy and resulted in the highest sensitivity and precision (sensitivity = 0.82, precision = 0.83). The proposed method achieved comparable accuracy at a minimal time cost compared to manual reading. The proposed automated pipeline enables robust and time-efficient readings of ePVS from MR scans and demonstrates the importance of T2w MRI for ePVS detection and the potential benefits of using multimodal images. Furthermore, the model provides whole-brain maps of ePVS, enabling a better understanding of their clinical correlates compared to the clinical rating methods within only a couple of brain regions.