Xijun Wang, Dongshan Ye, Chenyuan Feng, Howard H. Yang, Xiang Chen, Tony Q. S. Quek
{"title":"利用 GenAI 进行可信的图像语义交流:可解释性、可控性和效率","authors":"Xijun Wang, Dongshan Ye, Chenyuan Feng, Howard H. Yang, Xiang Chen, Tony Q. S. Quek","doi":"arxiv-2408.03806","DOIUrl":null,"url":null,"abstract":"Image semantic communication (ISC) has garnered significant attention for its\npotential to achieve high efficiency in visual content transmission. However,\nexisting ISC systems based on joint source-channel coding face challenges in\ninterpretability, operability, and compatibility. To address these limitations,\nwe propose a novel trustworthy ISC framework. This approach leverages text\nextraction and segmentation mapping techniques to convert images into\nexplainable semantics, while employing Generative Artificial Intelligence\n(GenAI) for multiple downstream inference tasks. We also introduce a multi-rate\nISC transmission protocol that dynamically adapts to both the received\nexplainable semantic content and specific task requirements at the receiver.\nSimulation results demonstrate that our framework achieves explainable\nlearning, decoupled training, and compatible transmission in various\napplication scenarios. Finally, some intriguing research directions and\napplication scenarios are identified.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"2 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency\",\"authors\":\"Xijun Wang, Dongshan Ye, Chenyuan Feng, Howard H. Yang, Xiang Chen, Tony Q. S. Quek\",\"doi\":\"arxiv-2408.03806\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Image semantic communication (ISC) has garnered significant attention for its\\npotential to achieve high efficiency in visual content transmission. However,\\nexisting ISC systems based on joint source-channel coding face challenges in\\ninterpretability, operability, and compatibility. To address these limitations,\\nwe propose a novel trustworthy ISC framework. This approach leverages text\\nextraction and segmentation mapping techniques to convert images into\\nexplainable semantics, while employing Generative Artificial Intelligence\\n(GenAI) for multiple downstream inference tasks. We also introduce a multi-rate\\nISC transmission protocol that dynamically adapts to both the received\\nexplainable semantic content and specific task requirements at the receiver.\\nSimulation results demonstrate that our framework achieves explainable\\nlearning, decoupled training, and compatible transmission in various\\napplication scenarios. Finally, some intriguing research directions and\\napplication scenarios are identified.\",\"PeriodicalId\":501082,\"journal\":{\"name\":\"arXiv - MATH - Information Theory\",\"volume\":\"2 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - MATH - Information Theory\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2408.03806\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - MATH - Information Theory","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.03806","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency
Image semantic communication (ISC) has garnered significant attention for its
potential to achieve high efficiency in visual content transmission. However,
existing ISC systems based on joint source-channel coding face challenges in
interpretability, operability, and compatibility. To address these limitations,
we propose a novel trustworthy ISC framework. This approach leverages text
extraction and segmentation mapping techniques to convert images into
explainable semantics, while employing Generative Artificial Intelligence
(GenAI) for multiple downstream inference tasks. We also introduce a multi-rate
ISC transmission protocol that dynamically adapts to both the received
explainable semantic content and specific task requirements at the receiver.
Simulation results demonstrate that our framework achieves explainable
learning, decoupled training, and compatible transmission in various
application scenarios. Finally, some intriguing research directions and
application scenarios are identified.