Multimodal cross-domain contrastive learning: A self-supervised generative and geometric framework for visual perception

Self-Supervised Contrastive Representation Learning (SSCRL) has gained significant attention for its ability to learn meaningful representations from unlabeled data by leveraging contrastive learning principles. However, existing SSCRL approaches struggle with effectively handling heterogeneous data formats, particularly discrete and binary representations, limiting adaptability across multiple domains. This limitation hinders the generalization of learned representations, especially in applications requiring structured feature encoding and robust cross-domain adaptability. To address this, we propose the Modular QCB Learner, a novel algorithm designed to enhance representation learning for heterogeneous data types. This framework builds upon SSCRL by incorporating a Real Non-Volume Preserving transformation to optimize continuous representations, ensuring alignment with a Gaussian distribution. For discrete representation learning, vector quantization is utilized along with a Poisson distribution, while binary representations are modeled through nonlinear transformations and the Bernoulli distribution. Multi-Domain Mixture Optimization (MiDO) is introduced to facilitate joint optimization of different representation types by integrating multiple loss functions. To evaluate effectiveness, synthetic data generation is performed on extracted representations and compared with baselines. Experiments on CIFAR-10 confirm the Modular QCB Learner improves representation quality, demonstrating robustness across diverse data domains with applications in synthetic data generation, anomaly detection and multimodal learning.