Deep Interactive Thin Object Selection

J. Liew, Scott D. Cohen, Brian L. Price, Long Mai, Jiashi Feng
{"title":"Deep Interactive Thin Object Selection","authors":"J. Liew, Scott D. Cohen, Brian L. Price, Long Mai, Jiashi Feng","doi":"10.1109/WACV48630.2021.00035","DOIUrl":null,"url":null,"abstract":"Existing deep learning based interactive segmentation methods have achieved remarkable performance with only a few user clicks, e.g. DEXTR [32] attaining 91.5% IoU on PASCAL VOC with only four extreme clicks. However, we observe even the state-of-the-art methods would often struggle in cases of objects to be segmented with elongated thin structures (e.g. bug legs and bicycle spokes). We investigate such failures, and find the critical reasons behind are two-fold: 1) lack of appropriate training dataset; and 2) extremely imbalanced distribution w.r.t. number of pixels belonging to thin and non-thin regions. Targeted at these challenges, we collect a large-scale dataset specifically for segmentation of thin elongated objects, named ThinObject-5K. Also, we present a novel integrative thin object segmentation network consisting of three streams. Among them, the high-resolution edge stream aims at preserving fine-grained details including elongated thin parts; the fixed-resolution context stream focuses on capturing semantic contexts. The two streams’ outputs are then amalgamated in the fusion stream to complement each other for help producing a refined segmentation output with sharper predictions around thin parts. Extensive experimental results well demonstrate the effectiveness of our proposed solution on segmenting thin objects, surpassing the baseline by ~ 30% IoUthin despite using only four clicks. Codes and dataset are available at https://github.com/liewjunhao/thin-object-selection.","PeriodicalId":236300,"journal":{"name":"2021 IEEE Winter Conference on Applications of Computer Vision (WACV)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"15","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 IEEE Winter Conference on Applications of Computer Vision (WACV)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/WACV48630.2021.00035","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 15

Abstract

Existing deep learning based interactive segmentation methods have achieved remarkable performance with only a few user clicks, e.g. DEXTR [32] attaining 91.5% IoU on PASCAL VOC with only four extreme clicks. However, we observe even the state-of-the-art methods would often struggle in cases of objects to be segmented with elongated thin structures (e.g. bug legs and bicycle spokes). We investigate such failures, and find the critical reasons behind are two-fold: 1) lack of appropriate training dataset; and 2) extremely imbalanced distribution w.r.t. number of pixels belonging to thin and non-thin regions. Targeted at these challenges, we collect a large-scale dataset specifically for segmentation of thin elongated objects, named ThinObject-5K. Also, we present a novel integrative thin object segmentation network consisting of three streams. Among them, the high-resolution edge stream aims at preserving fine-grained details including elongated thin parts; the fixed-resolution context stream focuses on capturing semantic contexts. The two streams’ outputs are then amalgamated in the fusion stream to complement each other for help producing a refined segmentation output with sharper predictions around thin parts. Extensive experimental results well demonstrate the effectiveness of our proposed solution on segmenting thin objects, surpassing the baseline by ~ 30% IoUthin despite using only four clicks. Codes and dataset are available at https://github.com/liewjunhao/thin-object-selection.
深度交互瘦对象选择
现有的基于深度学习的交互式分割方法在用户点击很少的情况下就取得了显著的性能,例如DEXTR b[32]在PASCAL VOC上获得了91.5%的IoU,只需要4次极端点击。然而,我们观察到,即使是最先进的方法,在用细长结构(例如虫子腿和自行车辐条)分割物体的情况下,也经常会遇到困难。我们对这些失败进行了调查,发现背后的关键原因有两个方面:1)缺乏适当的训练数据集;2)薄区和非薄区像素的W.R.T.分布极不平衡。针对这些挑战,我们收集了一个专门用于细细长物体分割的大规模数据集,名为ThinObject-5K。此外,我们还提出了一种新的由三个流组成的综合瘦目标分割网络。其中,高分辨率边缘流旨在保留细粒度细节,包括拉长的薄部件;固定分辨率上下文流侧重于捕获语义上下文。然后将两个流的输出合并到融合流中以相互补充,以帮助生成精细的分割输出,并对薄部分进行更清晰的预测。大量的实验结果很好地证明了我们提出的解决方案在分割薄物体方面的有效性,尽管只使用了4次点击,但仍比基线高出了30%。代码和数据集可在https://github.com/liewjunhao/thin-object-selection上获得。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信