... International Conference on Learning Representations最新文献

{"title":"Fast Value Tracking for Deep Reinforcement Learning.","authors":"Frank Shih, Faming Liang","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Reinforcement learning (RL) tackles sequential decision-making problems by creating agents that interacts with their environment. However, existing algorithms often view these problem as static, focusing on point estimates for model parameters to maximize expected rewards, neglecting the stochastic dynamics of agent-environment interactions and the critical role of uncertainty quantification. Our research leverages the Kalman filtering paradigm to introduce a novel and scalable sampling algorithm called Langevinized Kalman Temporal-Difference (LKTD) for deep reinforcement learning. This algorithm, grounded in Stochastic Gradient Markov Chain Monte Carlo (SGMCMC), efficiently draws samples from the posterior distribution of deep neural network parameters. Under mild conditions, we prove that the posterior samples generated by the LKTD algorithm converge to a stationary distribution. This convergence not only enables us to quantify uncertainties associated with the value function and model parameters but also allows us to monitor these uncertainties during policy updates throughout the training phase. The LKTD algorithm paves the way for more robust and adaptable reinforcement learning approaches.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2024 ","pages":"23393-23417"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12918776/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147274230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Causal-StoNet: Causal Inference for High-Dimensional Complex Data.","authors":"Yaxin Fang, Faming Liang","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>With the advancement of data science, the collection of increasingly complex datasets has become commonplace. In such datasets, the data dimension can be extremely high, and the underlying data generation process can be unknown and highly nonlinear. As a result, the task of making causal inference with high-dimensional complex data has become a fundamental problem in many disciplines, such as medicine, econometrics, and social science. However, the existing methods for causal inference are frequently developed under the assumption that the data dimension is low or that the underlying data generation process is linear or approximately linear. To address these challenges, this paper proposes a novel causal inference approach for dealing with high-dimensional complex data. The proposed approach is based on deep learning techniques, including sparse deep learning theory and stochastic neural networks, that have been developed in recent literature. By using these techniques, the proposed approach can address both the high dimensionality and unknown data generation process in a coherent way. Furthermore, the proposed approach can also be used when missing values are present in the datasets. Extensive numerical studies indicate that the proposed approach outperforms existing ones.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2024 ","pages":"25328-25351"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12918773/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147274239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What's in a Prior? Learned Proximal Networks for Inverse Problems.","authors":"Zhenghan Fang, Sam Buchanan, Jeremias Sulam","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Proximal operators are ubiquitous in inverse problems, commonly appearing as part of algorithmic strategies to regularize problems that are otherwise ill-posed. Modern deep learning models have been brought to bear for these tasks too, as in the framework of plug-and-play or deep unrolling, where they loosely resemble proximal operators. Yet, something essential is lost in employing these purely data-driven approaches: there is no guarantee that a general deep network represents the proximal operator of any function, nor is there any characterization of the function for which the network might provide some approximate proximal. This not only makes guaranteeing convergence of iterative schemes challenging but, more fundamentally, complicates the analysis of what has been learned by these networks about their training data. Herein we provide a framework to develop <i>learned proximal networks</i> (LPN), prove that they provide exact proximal operators for a data-driven nonconvex regularizer, and show how a new training strategy, dubbed <i>proximal matching</i>, provably promotes the recovery of the log-prior of the true data distribution. Such LPN provide general, unsupervised, expressive proximal operators that can be used for general inverse problems with convergence guarantees. We illustrate our results in a series of cases of increasing complexity, demonstrating that these models not only result in state-of-the-art performance, but provide a window into the resulting priors learned from data.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2024 ","pages":"15652-15692"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12930532/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147314785","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimal Sketching for Residual Error Estimation for Matrix and Vector Norms.","authors":"Yi Li, Honghao Lin, David P Woodruff","doi":"","DOIUrl":"","url":null,"abstract":"&lt;p&gt;&lt;p&gt;We study the problem of residual error estimation for matrix and vector norms using a linear sketch. Such estimates can be used, for example, to quickly assess how useful a more expensive low-rank approximation computation will be. The matrix case concerns the Frobenius norm and the task is to approximate the &lt;math&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/math&gt; -residual &lt;math&gt; &lt;msub&gt; &lt;mrow&gt; &lt;mfenced&gt;&lt;mrow&gt;&lt;mi&gt;A&lt;/mi&gt; &lt;mo&gt;-&lt;/mo&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;A&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/mrow&gt; &lt;/mfenced&gt; &lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;F&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; of the input matrix &lt;math&gt;&lt;mi&gt;A&lt;/mi&gt;&lt;/math&gt; within a &lt;math&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;mn&gt;1&lt;/mn&gt; &lt;mo&gt;+&lt;/mo&gt; &lt;mi&gt;ε&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; -factor, where &lt;math&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;A&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; is the optimal rank- &lt;math&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/math&gt; approximation. We provide a tight bound of &lt;math&gt;&lt;mi&gt;Θ&lt;/mi&gt; &lt;mfenced&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;2&lt;/mn&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;/&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;ε&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;4&lt;/mn&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/mfenced&gt; &lt;/math&gt; on the size of bilinear sketches, which have the form of a matrix product &lt;math&gt;&lt;mi&gt;S&lt;/mi&gt; &lt;mi&gt;A&lt;/mi&gt; &lt;mi&gt;T&lt;/mi&gt;&lt;/math&gt; . This improves the previous &lt;math&gt;&lt;mi&gt;O&lt;/mi&gt; &lt;mfenced&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;2&lt;/mn&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;/&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;ε&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;6&lt;/mn&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/mfenced&gt; &lt;/math&gt; upper bound in (Andoni et al. SODA 2013) and gives the first non-trivial lower bound, to the best of our knowledge. In our algorithm, our sketching matrices &lt;math&gt;&lt;mi&gt;S&lt;/mi&gt;&lt;/math&gt; and &lt;math&gt;&lt;mi&gt;T&lt;/mi&gt;&lt;/math&gt; can both be sparse matrices, allowing for a very fast update time. We demonstrate that this gives a substantial advantage empirically, for roughly the same sketch size and accuracy as in previous work. For the vector case, we consider the &lt;math&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mo&gt;ℓ&lt;/mo&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; -norm for &lt;math&gt;&lt;mi&gt;p&lt;/mi&gt; &lt;mo&gt;&gt;&lt;/mo&gt; &lt;mn&gt;2&lt;/mn&gt;&lt;/math&gt; , where the task is to approximate the &lt;math&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/math&gt; -residual &lt;math&gt; &lt;msub&gt; &lt;mrow&gt; &lt;mfenced&gt;&lt;mrow&gt;&lt;mi&gt;x&lt;/mi&gt; &lt;mo&gt;-&lt;/mo&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;x&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/mrow&gt; &lt;/mfenced&gt; &lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; up to a constant factor, where &lt;math&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;x&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; is the optimal &lt;math&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/math&gt; -sparse approximation to &lt;math&gt;&lt;mi&gt;x&lt;/mi&gt;&lt;/math&gt; . Such vector norms are frequently studied in the data stream literature and are useful for finding frequent items or so-called heavy hitters. We establish an upper bound of &lt;math&gt;&lt;mi&gt;O&lt;/mi&gt; &lt;mfenced&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;2&lt;/mn&gt; &lt;mo&gt;/&lt;/mo&gt; &lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;n&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mn&gt;1&lt;/mn&gt; &lt;mo&gt;-&lt;/mo&gt; &lt;mn&gt;2&lt;/mn&gt; &lt;mo&gt;/&lt;/mo&gt; &lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/mfenced&gt; &lt;/math&gt; poly &lt;math&gt; &lt;mfenced&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;mtext&gt;log&lt;/mtext&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;n&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;/mfenced&gt; &lt;/math&gt; for constant &lt;math&gt;&lt;mi&gt;ε&lt;/mi&gt;&lt;/math","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2024 ","pages":"54610-54620"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12962743/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147380508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Safe and Robust Watermark Injection with a Single OoD Image.","authors":"Shuyang Yu, Junyuan Hong, Haobo Zhang, Haotao Wang, Zhangyang Wang, Jiayu Zhou","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Training a high-performance deep neural network requires large amounts of data and computational resources. Protecting the intellectual property (IP) and commercial ownership of a deep model is challenging yet increasingly crucial. A major stream of watermarking strategies implants verifiable backdoor triggers by poisoning training samples, but these are often unrealistic due to data privacy and safety concerns and are vulnerable to minor model changes such as fine-tuning. To overcome these challenges, we propose a safe and robust backdoor-based watermark injection technique that leverages the diverse knowledge from a single out-of-distribution (OoD) image, which serves as a secret key for IP verification. The independence of training data makes it agnostic to third-party promises of IP security. We induce robustness via random perturbation of model parameters during watermark injection to defend against common watermark removal attacks, including fine-tuning, pruning, and model extraction. Our experimental results demonstrate that the proposed watermarking approach is not only time- and sample-efficient without training data, but also robust against the watermark removal attacks above. Codes are available: https://github.com/illidanlab/Single_oodwatermark.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2024 ","pages":"42500-42511"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12970704/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147439114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Graph Domain Adaptation via Theory-Grounded Spectral Regularization.","authors":"Yuning You, Tianlong Chen, Zhangyang Wang, Yang Shen","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Transfer learning on graphs drawn from varied distributions (domains) is in great demand across many applications. Emerging methods attempt to learn domain-invariant representations using graph neural networks (GNNs), yet the empirical performances vary and the theoretical foundation is limited. This paper aims at designing theory-grounded algorithms for graph domain adaptation (GDA). (<b>i</b>) As the first attempt, we derive a model-based GDA bound closely related to two GNN spectral properties: <i>spectral smoothness (SS)</i> and <i>maximum frequency response (MFR)</i>. This is achieved by cross-pollinating between the OT-based (optimal transport) DA and graph filter theories. (<b>ii</b>) Inspired by the theoretical results, we propose algorithms regularizing spectral properties of SS and MFR to improve GNN transferability. We further extend the GDA theory into the more challenging scenario of conditional shift, where spectral regularization still applies. (<b>iii</b>) More importantly, our analyses of the theory reveal which regularization would improve performance of what transfer learning scenario, (<b>iv</b>) with numerical agreement with extensive real-world experiments: SS and MFR regularizations bring more benefits to the scenarios of <i>node transfer</i> and <i>link transfer</i>, respectively. In a nut-shell, our study paves the way toward explicitly constructing and training GNNs that can capture more transferable representations across graph domains. Codes are released at https://github.com/Shen-Lab/GDA-SpecReg.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2023 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13339966/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148407926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MECTA: Memory-Economic Continual Test-Time Model Adaptation.","authors":"Junyuan Hong, Lingjuan Lyu, Jiayu Zhou, Michael Spranger","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Continual Test-time Adaptation (CTA) is a promising art to secure accuracy gains in continually-changing environments. The state-of-the-art adaptations improve out-of-distribution model accuracy via computation-efficient online test-time gradient descents but meanwhile cost about times of memory versus the inference, even if only a small portion of parameters are updated. Such high memory consumption of CTA substantially impedes wide applications of advanced CTA on memory-constrained devices. In this paper, we provide a novel solution, dubbed MECTA, to drastically improve the memory efficiency of gradient-based CTA. Our profiling shows that the major memory overhead comes from the intermediate cache for back-propagation, which scales by the batch size, channel, and layer number. Therefore, we propose to reduce batch sizes, adopt an adaptive normalization layer to maintain stable and accurate predictions, and stop the back-propagation caching heuristically. On the other hand, we prune the networks to reduce the computation and memory overheads in optimization and recover the parameters afterward to avoid forgetting. The proposed MECTA is efficient and can be seamlessly plugged into state-of-the-art CTA algorithms at negligible overhead on computation and memory. On three datasets, CIFAR10, CIFAR100, and ImageNet, MECTA improves the accuracy by at least 6% with constrained memory and significantly reduces the memory costs of ResNet50 on ImageNet by at least 70% with comparable accuracy. Our codes can be accessed at https://github.com/SonyAI/MECTA.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2023 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13526513/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148869382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"GRACE-C: Generalized Rate Agnostic Causal Estimation via Constraints.","authors":"Mohammadsajad Abavisani, David Danks, Sergey M Plis","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Graphical structures estimated by causal learning algorithms from time series data can provide misleading causal information if the causal timescale of the generating process fails to match the measurement timescale of the data. Existing algorithms provide limited resources to respond to this challenge, and so researchers must either use models that they know are likely misleading, or else forego causal learning entirely. Existing methods face up-to-four distinct shortfalls, as they might <i>a</i>) require that the difference between causal and measurement timescales is known; <i>b</i>) only handle very small number of random variables when the timescale difference is unknown; <i>c</i>) only apply to pairs of variables; or <i>d</i>) be unable to find a solution given statistical noise in the data. This paper addresses these challenges. Our approach combines constraint programming with both theoretical insights into the problem structure and prior information about admissible causal interactions to achieve multiple orders of magnitude in speed-up. The resulting system maintains theoretical guarantees while scaling to significantly larger sets of random variables (> 100) without knowledge of timescale differences. This method is also robust to edge misidentification and can use parametric connection strengths, while optionally finding the optimal solution among many possible ones.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2023 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13596948/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149027503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fairness and Accuracy Under Domain Generalization.","authors":"Thai-Hoang Pham, Xueru Zhang, Ping Zhang","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>As machine learning (ML) algorithms are increasingly used in high-stakes applications, concerns have arisen that they may be biased against certain social groups. Although many approaches have been proposed to make ML models fair, they typically rely on the assumption that data distributions in training and deployment are identical. Unfortunately, this is commonly violated in practice and a model that is fair during training may lead to an unexpected outcome during its deployment. Although the problem of designing robust ML models under dataset shifts has been widely studied, most existing works focus only on the transfer of accuracy. In this paper, we study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from <i>never-before-seen</i> domains. We first develop theoretical bounds on the unfairness and expected loss at deployment, and then derive sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning. Guided by this, we design a learning algorithm such that fair ML models learned with training data still have high fairness and accuracy when deployment environments change. Experiments on real-world data validate the proposed algorithm. Model implementation is available at https://github.com/pth1993/FATDM.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2023 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13532314/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148876197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Domain Generalization via Heckman-Type Selection Models.","authors":"Hyungu Kahng, Hyungrok Do, Judy Zhong","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The domain generalization (DG) setup considers the problem where models are trained on data sampled from multiple domains and evaluated on test domains unseen during training. In this paper, we formulate DG as a sample selection problem where each domain is sampled from a common underlying population through non-random sampling probabilities that correlate with both the features and the outcome. Under this setting, the fundamental <i>iid</i> assumption of the empirical risk minimization (ERM) is violated, so it often performs worse on test domains whose non-random sampling probabilities differ from the domains in the training dataset. We propose a Selection-Guided DG (SGDG) framework to learn the selection probability of each domain and the joint distribution of the outcome and domain selection variables. The proposed SGDG is domain generalizable as it intends to minimize the risk under the population distribution. We theoretically prove that, under certain regular conditions, SGDG can achieve smaller risk than ERM. Furthermore, we present a class of parametric SGDG (HeckmanDG) estimators applicable to continuous, binary, and multinomial outcomes. We also demonstrate its efficacy empirically through simulations and experiments on a set of benchmark datasets comparing with other well-known DG methods.</p>","PeriodicalId":521258,"journal":{"name":"... International Conference on Learning Representations","volume":"2023 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13684989/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149248475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}