{"title":"基于自监督学习的测试时间自适应注视估计","authors":"Pengwei Yin;Jingjing Wang;Xiaojun Wu","doi":"10.1109/TCE.2024.3523486","DOIUrl":null,"url":null,"abstract":"Gaze estimation plays a significant role in consumer electronics, particularly in the realm of user interface and interactive technology. While existing methods rely on either few-shot adaptation requiring annotated samples or unsupervised domain adaptation necessitating source domain data, these approaches face limitations due to the high cost of annotation and data privacy concerns. This paper addresses this critical gap by introducing a novel test-time adaptation framework for gaze estimation that operates without the need for source domain data or annotated samples for adaptation. Here, we present a dual-objective training strategy that combines supervised and self-supervised learning on the source domain, with a particular focus on a face and eye reconstruction task designed to enhance the learning of head pose and eye direction features crucial for gaze estimation. At test time, our model undergoes adaptation solely through fine-tuning with the self-supervised objective, optimizing the model’s ability to estimate gaze in new, unseen scenarios. Our extensive experiments on benchmarks validate the effectiveness of our approach, demonstrating improved generalization capabilities without the dependency on expensive annotations or sensitive source domain data.","PeriodicalId":13208,"journal":{"name":"IEEE Transactions on Consumer Electronics","volume":"71 1","pages":"75-89"},"PeriodicalIF":4.3000,"publicationDate":"2024-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Test-Time Adaptation With Self-Supervised Learning for Gaze Estimation\",\"authors\":\"Pengwei Yin;Jingjing Wang;Xiaojun Wu\",\"doi\":\"10.1109/TCE.2024.3523486\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Gaze estimation plays a significant role in consumer electronics, particularly in the realm of user interface and interactive technology. While existing methods rely on either few-shot adaptation requiring annotated samples or unsupervised domain adaptation necessitating source domain data, these approaches face limitations due to the high cost of annotation and data privacy concerns. This paper addresses this critical gap by introducing a novel test-time adaptation framework for gaze estimation that operates without the need for source domain data or annotated samples for adaptation. Here, we present a dual-objective training strategy that combines supervised and self-supervised learning on the source domain, with a particular focus on a face and eye reconstruction task designed to enhance the learning of head pose and eye direction features crucial for gaze estimation. At test time, our model undergoes adaptation solely through fine-tuning with the self-supervised objective, optimizing the model’s ability to estimate gaze in new, unseen scenarios. Our extensive experiments on benchmarks validate the effectiveness of our approach, demonstrating improved generalization capabilities without the dependency on expensive annotations or sensitive source domain data.\",\"PeriodicalId\":13208,\"journal\":{\"name\":\"IEEE Transactions on Consumer Electronics\",\"volume\":\"71 1\",\"pages\":\"75-89\"},\"PeriodicalIF\":4.3000,\"publicationDate\":\"2024-12-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Consumer Electronics\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10817535/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Consumer Electronics","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10817535/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Test-Time Adaptation With Self-Supervised Learning for Gaze Estimation
Gaze estimation plays a significant role in consumer electronics, particularly in the realm of user interface and interactive technology. While existing methods rely on either few-shot adaptation requiring annotated samples or unsupervised domain adaptation necessitating source domain data, these approaches face limitations due to the high cost of annotation and data privacy concerns. This paper addresses this critical gap by introducing a novel test-time adaptation framework for gaze estimation that operates without the need for source domain data or annotated samples for adaptation. Here, we present a dual-objective training strategy that combines supervised and self-supervised learning on the source domain, with a particular focus on a face and eye reconstruction task designed to enhance the learning of head pose and eye direction features crucial for gaze estimation. At test time, our model undergoes adaptation solely through fine-tuning with the self-supervised objective, optimizing the model’s ability to estimate gaze in new, unseen scenarios. Our extensive experiments on benchmarks validate the effectiveness of our approach, demonstrating improved generalization capabilities without the dependency on expensive annotations or sensitive source domain data.
期刊介绍:
The main focus for the IEEE Transactions on Consumer Electronics is the engineering and research aspects of the theory, design, construction, manufacture or end use of mass market electronics, systems, software and services for consumers.