{"title":"SFDA-rPPG:具有时空一致性的无源域自适应远程生理测量技术","authors":"Yiping Xie, Zitong Yu, Bingjie Wu, Weicheng Xie, Linlin Shen","doi":"arxiv-2409.12040","DOIUrl":null,"url":null,"abstract":"Remote Photoplethysmography (rPPG) is a non-contact method that uses facial\nvideo to predict changes in blood volume, enabling physiological metrics\nmeasurement. Traditional rPPG models often struggle with poor generalization\ncapacity in unseen domains. Current solutions to this problem is to improve its\ngeneralization in the target domain through Domain Generalization (DG) or\nDomain Adaptation (DA). However, both traditional methods require access to\nboth source domain data and target domain data, which cannot be implemented in\nscenarios with limited access to source data, and another issue is the privacy\nof accessing source domain data. In this paper, we propose the first\nSource-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which\novercomes these limitations by enabling effective domain adaptation without\naccess to source domain data. Our framework incorporates a Three-Branch\nSpatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency\nacross domains. Furthermore, we propose a new rPPG distribution alignment loss\nbased on the Frequency-domain Wasserstein Distance (FWD), which leverages\noptimal transport to align power spectrum distributions across domains\neffectively and further enforces the alignment of the three branches. Extensive\ncross-domain experiments and ablation studies demonstrate the effectiveness of\nour proposed method in source-free domain adaptation settings. Our findings\nhighlight the significant contribution of the proposed FWD loss for\ndistributional alignment, providing a valuable reference for future research\nand applications. The source code is available at\nhttps://github.com/XieYiping66/SFDA-rPPG","PeriodicalId":501130,"journal":{"name":"arXiv - CS - Computer Vision and Pattern Recognition","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency\",\"authors\":\"Yiping Xie, Zitong Yu, Bingjie Wu, Weicheng Xie, Linlin Shen\",\"doi\":\"arxiv-2409.12040\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Remote Photoplethysmography (rPPG) is a non-contact method that uses facial\\nvideo to predict changes in blood volume, enabling physiological metrics\\nmeasurement. Traditional rPPG models often struggle with poor generalization\\ncapacity in unseen domains. Current solutions to this problem is to improve its\\ngeneralization in the target domain through Domain Generalization (DG) or\\nDomain Adaptation (DA). However, both traditional methods require access to\\nboth source domain data and target domain data, which cannot be implemented in\\nscenarios with limited access to source data, and another issue is the privacy\\nof accessing source domain data. In this paper, we propose the first\\nSource-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which\\novercomes these limitations by enabling effective domain adaptation without\\naccess to source domain data. Our framework incorporates a Three-Branch\\nSpatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency\\nacross domains. Furthermore, we propose a new rPPG distribution alignment loss\\nbased on the Frequency-domain Wasserstein Distance (FWD), which leverages\\noptimal transport to align power spectrum distributions across domains\\neffectively and further enforces the alignment of the three branches. Extensive\\ncross-domain experiments and ablation studies demonstrate the effectiveness of\\nour proposed method in source-free domain adaptation settings. Our findings\\nhighlight the significant contribution of the proposed FWD loss for\\ndistributional alignment, providing a valuable reference for future research\\nand applications. The source code is available at\\nhttps://github.com/XieYiping66/SFDA-rPPG\",\"PeriodicalId\":501130,\"journal\":{\"name\":\"arXiv - CS - Computer Vision and Pattern Recognition\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Computer Vision and Pattern Recognition\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.12040\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Computer Vision and Pattern Recognition","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.12040","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency
Remote Photoplethysmography (rPPG) is a non-contact method that uses facial
video to predict changes in blood volume, enabling physiological metrics
measurement. Traditional rPPG models often struggle with poor generalization
capacity in unseen domains. Current solutions to this problem is to improve its
generalization in the target domain through Domain Generalization (DG) or
Domain Adaptation (DA). However, both traditional methods require access to
both source domain data and target domain data, which cannot be implemented in
scenarios with limited access to source data, and another issue is the privacy
of accessing source domain data. In this paper, we propose the first
Source-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which
overcomes these limitations by enabling effective domain adaptation without
access to source domain data. Our framework incorporates a Three-Branch
Spatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency
across domains. Furthermore, we propose a new rPPG distribution alignment loss
based on the Frequency-domain Wasserstein Distance (FWD), which leverages
optimal transport to align power spectrum distributions across domains
effectively and further enforces the alignment of the three branches. Extensive
cross-domain experiments and ablation studies demonstrate the effectiveness of
our proposed method in source-free domain adaptation settings. Our findings
highlight the significant contribution of the proposed FWD loss for
distributional alignment, providing a valuable reference for future research
and applications. The source code is available at
https://github.com/XieYiping66/SFDA-rPPG