Learning Positional Priors for Pretraining 2D Pose Estimators

Kun Zhang, Ping Yao, Rui Wu, Chuanguang Yang, Ding Li, Min Du, Kai Deng, Renbiao Liu, Tianyao Zheng
{"title":"Learning Positional Priors for Pretraining 2D Pose Estimators","authors":"Kun Zhang, Ping Yao, Rui Wu, Chuanguang Yang, Ding Li, Min Du, Kai Deng, Renbiao Liu, Tianyao Zheng","doi":"10.1145/3475723.3484252","DOIUrl":null,"url":null,"abstract":"The target of 2D human pose estimation is to locate the keypoints of body parts from 2D images. State-of-the-art methods for pose estimation usually construct pixel-wise heatmaps from keypoints as labels for learning neural networks, which are usually initialized randomly or using classification models on large dataset, such as ImageNet, for their backbones. According to statistical data, there are strong positional priors for human keypoints, which are highly dependent on their relationship between image patches. To learn positional priors for pretraining pose estimators, we propose Heatmap-Style Jigsaw Puzzles (HSJP) problem as self-supervised pretext task, whose target is to predict the location of each patch from an image composed of shuffled patches. During pretraining, we only use person images in MS-COCO, rather than introducing extra large dataset like ImageNet. A heatmap-style label for patch location is designed and our learning process is in a non-contrastive way. The weights learned by HSJP pretext task are utilised as backbones of 2D human pose estimators, which are then finetuned on MS-COCO human keypoints dataset. With two popular and strong 2D human pose estimators, HRNet and SimpleBaseline, we evaluate mAP score on both MS-COCO validation and test-dev datasets. Our experiments show that downstream pose estimators with our self-supervised pretraining obtain much better performance than those trained from scratch, and are comparable to those using ImageNet classification models as their initial backbones.","PeriodicalId":209434,"journal":{"name":"Proceedings of the 2nd International Workshop on Human-centric Multimedia Analysis","volume":"62 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 2nd International Workshop on Human-centric Multimedia Analysis","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3475723.3484252","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2

Abstract

The target of 2D human pose estimation is to locate the keypoints of body parts from 2D images. State-of-the-art methods for pose estimation usually construct pixel-wise heatmaps from keypoints as labels for learning neural networks, which are usually initialized randomly or using classification models on large dataset, such as ImageNet, for their backbones. According to statistical data, there are strong positional priors for human keypoints, which are highly dependent on their relationship between image patches. To learn positional priors for pretraining pose estimators, we propose Heatmap-Style Jigsaw Puzzles (HSJP) problem as self-supervised pretext task, whose target is to predict the location of each patch from an image composed of shuffled patches. During pretraining, we only use person images in MS-COCO, rather than introducing extra large dataset like ImageNet. A heatmap-style label for patch location is designed and our learning process is in a non-contrastive way. The weights learned by HSJP pretext task are utilised as backbones of 2D human pose estimators, which are then finetuned on MS-COCO human keypoints dataset. With two popular and strong 2D human pose estimators, HRNet and SimpleBaseline, we evaluate mAP score on both MS-COCO validation and test-dev datasets. Our experiments show that downstream pose estimators with our self-supervised pretraining obtain much better performance than those trained from scratch, and are comparable to those using ImageNet classification models as their initial backbones.
学习预训练2D姿态估计器的位置先验
二维人体姿态估计的目标是从二维图像中定位人体部位的关键点。最先进的姿态估计方法通常从关键点构建逐像素热图作为学习神经网络的标签,神经网络通常随机初始化或使用大型数据集(如ImageNet)上的分类模型作为其主干。统计数据表明,人类关键点存在很强的位置先验,高度依赖于图像斑块之间的关系。为了学习预训练姿态估计器的位置先验,我们提出了热图式拼图(Heatmap-Style Jigsaw Puzzles, HSJP)问题作为自监督的借口任务,其目标是从由洗牌后的小块组成的图像中预测每个小块的位置。在预训练过程中,我们只使用MS-COCO中的人物图像,而没有像ImageNet那样引入超大的数据集。我们设计了一个热图风格的标签用于patch的位置,并且我们的学习过程是非对比的。利用HSJP借口任务学习到的权重作为2D人体姿态估计器的骨架,然后在MS-COCO人体关键点数据集上进行微调。使用两个流行且强大的2D人体姿势估计器HRNet和SimpleBaseline,我们在MS-COCO验证和测试开发数据集上评估mAP分数。我们的实验表明,使用我们的自监督预训练的下游姿态估计器比从头训练的姿态估计器获得了更好的性能,并且与使用ImageNet分类模型作为初始骨干的姿态估计器相当。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书