{"title":"利用连体 ViT 网络解耦前景和背景,实现弱监督语义分割","authors":"","doi":"10.1016/j.neucom.2024.128540","DOIUrl":null,"url":null,"abstract":"<div><p>Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods.</p></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5000,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation\",\"authors\":\"\",\"doi\":\"10.1016/j.neucom.2024.128540\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods.</p></div>\",\"PeriodicalId\":19268,\"journal\":{\"name\":\"Neurocomputing\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.5000,\"publicationDate\":\"2024-09-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neurocomputing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0925231224013110\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurocomputing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0925231224013110","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation
Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods.
期刊介绍:
Neurocomputing publishes articles describing recent fundamental contributions in the field of neurocomputing. Neurocomputing theory, practice and applications are the essential topics being covered.