Task-oriented semantic communication with foundation models

Abstract

In the future development direction of the sixth generation (6G) mobile communication, several communication models are proposed to face the growing challenges of the task. The rapid development of artificial intelligence (AI) foundation models provides significant support for efficient and intelligent communication interactions. In this paper, we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models. First, we segment the image by using task prompts based on the segment anything model (SAM) and contrastive language-image pretraining (CLIP). Meanwhile, we adopt Bezier curve to enhance the mask to improve the segmentation accuracy. Second, we have differentiated semantic compression and transmission approaches for segmented content. Third, we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users' specific task requirements. Finally, the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.