Proceedings of machine learning research最新文献_第8页

The Unintended Consequences of Discount Regularization: Improving Regularization in Certainty Equivalence Reinforcement Learning 折扣正则化的意外后果:改进确定性等价强化学习中的正则化

Proceedings of machine learning research Pub Date : 2023-06-20 DOI: 10.48550/arXiv.2306.11208

Sarah Rathnam, S. Parbhoo, Weiwei Pan, Susan A. Murphy, F. Doshi-Velez

{"title":"The Unintended Consequences of Discount Regularization: Improving Regularization in Certainty Equivalence Reinforcement Learning","authors":"Sarah Rathnam, S. Parbhoo, Weiwei Pan, Susan A. Murphy, F. Doshi-Velez","doi":"10.48550/arXiv.2306.11208","DOIUrl":"https://doi.org/10.48550/arXiv.2306.11208","url":null,"abstract":"Discount regularization, using a shorter planning horizon when calculating the optimal policy, is a popular choice to restrict planning to a less complex set of policies when estimating an MDP from sparse or noisy data (Jiang et al., 2015). It is commonly understood that discount regularization functions by de-emphasizing or ignoring delayed effects. In this paper, we reveal an alternate view of discount regularization that exposes unintended consequences. We demonstrate that planning under a lower discount factor produces an identical optimal policy to planning using any prior on the transition matrix that has the same distribution for all states and actions. In fact, it functions like a prior with stronger regularization on state-action pairs with more transition data. This leads to poor performance when the transition matrix is estimated from data sets with uneven amounts of data across state-action pairs. Our equivalence theorem leads to an explicit formula to set regularization parameters locally for individual state-action pairs rather than globally. We demonstrate the failures of discount regularization and how we remedy them using our state-action-specific method across simple empirical examples as well as a medical cancer simulator.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"202 1","pages":"28746-28767"},"PeriodicalIF":0.0,"publicationDate":"2023-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47482371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

On Distribution Dependent Sub-Logarithmic Query Time of Learned Indexing 学习索引的分布相关次对数查询时间

Proceedings of machine learning research Pub Date : 2023-06-19 DOI: 10.48550/arXiv.2306.10651

Sepanta Zeighami, C. Shahabi

{"title":"On Distribution Dependent Sub-Logarithmic Query Time of Learned Indexing","authors":"Sepanta Zeighami, C. Shahabi","doi":"10.48550/arXiv.2306.10651","DOIUrl":"https://doi.org/10.48550/arXiv.2306.10651","url":null,"abstract":"A fundamental problem in data management is to find the elements in an array that match a query. Recently, learned indexes are being extensively used to solve this problem, where they learn a model to predict the location of the items in the array. They are empirically shown to outperform non-learned methods (e.g., B-trees or binary search that answer queries in O(logn) time) by orders of magnitude. However, success of learned indexes has not been theoretically justified. Only existing attempt shows the same query time of O(logn), but with a constant factor improvement in space complexity over non-learned methods, under some assumptions on data distribution. In this paper, we significantly strengthen this result, showing that under mild assumptions on data distribution, and the same space complexity as non-learned methods, learned indexes can answer queries in O(loglogn) expected query time. We also show that allowing for slightly larger but still near-linear space overhead, a learned index can achieve O(1) expected query time. Our results theoretically prove learned indexes are orders of magnitude faster than non-learned methods, theoretically grounding their empirical success.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"261 1","pages":"40669-40680"},"PeriodicalIF":0.0,"publicationDate":"2023-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79545250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Assessing the Impact of Context Inference Error and Partial Observability on RL Methods for Just-In-Time Adaptive Interventions 评估上下文推断误差和部分可观察性对实时自适应干预RL方法的影响

Proceedings of machine learning research Pub Date : 2023-05-17 DOI: 10.48550/arXiv.2305.09913

Karine Karine, P. Klasnja, Susan A. Murphy, Benjamin M Marlin

引用次数: 0

DIET: Conditional independence testing with marginal dependence measures of residual information. DIET：利用残差信息的边际依赖性测量进行条件独立性检验。

Proceedings of machine learning research Pub Date : 2023-04-01

Mukund Sudarshan, Aahlad Puli, Wesley Tansey, Rajesh Ranganath

{"title":"DIET: Conditional independence testing with marginal dependence measures of residual information.","authors":"Mukund Sudarshan, Aahlad Puli, Wesley Tansey, Rajesh Ranganath","doi":"","DOIUrl":"","url":null,"abstract":"Conditional randomization tests (CRTs) assess whether a variable <math><mi>x</mi></math> is predictive of another variable <math><mi>y</mi></math>, having observed covariates <math><mrow><mi>z</mi></mrow></math>. CRTs require fitting a large number of predictive models, which is often computationally intractable. Existing solutions to reduce the cost of CRTs typically split the dataset into a train and test portion, or rely on heuristics for interactions, both of which lead to a loss in power. We propose the decoupled independence test (DIET), an algorithm that avoids both of these issues by leveraging marginal independence statistics to test conditional independence relationships. DIET tests the marginal independence of two random variables: <math><mrow><msub><mi>F</mi><mrow><mi>x</mi><mo>∣</mo><mi>z</mi></mrow></msub><mo>(</mo><mi>x</mi><mo>∣</mo><mi>z</mi><mo>)</mo></mrow></math> and <math><mrow><msub><mi>F</mi><mrow><mi>y</mi><mo>∣</mo><mi>z</mi></mrow></msub><mo>(</mo><mi>y</mi><mo>∣</mo><mi>z</mi><mo>)</mo></mrow></math> where <math><mrow><msub><mi>F</mi><mrow><mo>⋅</mo><mo>∣</mo><mi>z</mi></mrow></msub><mo>(</mo><mo>⋅</mo><mo>∣</mo><mi>z</mi><mo>)</mo></mrow></math> is a conditional cumulative distribution function (CDF) for the distribution <math><mrow><mi>p</mi><mo>(</mo><mo>⋅</mo><mo>∣</mo><mi>z</mi><mo>)</mo></mrow></math>. These variables are termed \"information residuals.\" We give sufficient conditions for DIET to achieve finite sample type-1 error control and power greater than the type-1 error rate. We then prove that when using the mutual information between the information residuals as a test statistic, DIET yields the most powerful conditionally valid test. Finally, we show DIET achieves higher power than other tractable CRTs on several synthetic and real benchmarks.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"206 ","pages":"10343-10367"},"PeriodicalIF":0.0,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10484293/pdf/nihms-1899844.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10577745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE. 协变量知情表示学习预防iVAE后塌陷。

Proceedings of machine learning research Pub Date : 2023-04-01

Young-Geun Kim, Ying Liu, Xue-Xin Wei

{"title":"Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE.","authors":"Young-Geun Kim, Ying Liu, Xue-Xin Wei","doi":"","DOIUrl":"","url":null,"abstract":"The recently proposed identifiable variational autoencoder (iVAE) framework provides a promising approach for learning latent independent components (ICs). iVAEs use auxiliary covariates to build an identifiable generation structure from covariates to ICs to observations, and the posterior network approximates ICs given observations and covariates. Though the identifiability is appealing, we show that iVAEs could have local minimum solution where observations and the approximated ICs are independent given covariates.-a phenomenon we referred to as the posterior collapse problem of iVAEs. To overcome this problem, we develop a new approach, covariate-informed iVAE (CI-iVAE) by considering a mixture of encoder and posterior distributions in the objective function. In doing so, the objective function prevents the posterior collapse, resulting latent representations that contain more information of the observations. Furthermore, CI-iVAE extends the original iVAE objective function to a larger class and finds the optimal one among them, thus having tighter evidence lower bounds than the original iVAE. Experiments on simulation datasets, EMNIST, Fashion-MNIST, and a large-scale brain imaging dataset demonstrate the effectiveness of our new method.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"206 ","pages":"2641-2660"},"PeriodicalIF":0.0,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10226378/pdf/nihms-1902106.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9908011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation. 不要上当：标签泄漏在解释方法及其定量评价的重要性。

Proceedings of machine learning research Pub Date : 2023-04-01

Neil Jethani, Adriel Saporta, Rajesh Ranganath

引用次数: 0

Continuous-Time Decision Transformer for Healthcare Applications. 用于医疗保健应用的连续时间决策变压器。

Proceedings of machine learning research Pub Date : 2023-04-01

Zhiyue Zhang, Hongyuan Mei, Yanxun Xu

{"title":"Continuous-Time Decision Transformer for Healthcare Applications.","authors":"Zhiyue Zhang, Hongyuan Mei, Yanxun Xu","doi":"","DOIUrl":"","url":null,"abstract":"Offline reinforcement learning (RL) is a promising approach for training intelligent medical agents to learn treatment policies and assist decision making in many healthcare applications, such as scheduling clinical visits and assigning dosages for patients with chronic conditions. In this paper, we investigate the potential usefulness of Decision Transformer (Chen et al., 2021)-a new offline RL paradigm-in medical domains where decision making in continuous time is desired. As Decision Transformer only handles discrete-time (or turn-based) sequential decision making scenarios, we generalize it to Continuous-Time Decision Transformer that not only considers the past clinical measurements and treatments but also the timings of previous visits, and learns to suggest the timings of future visits as well as the treatment plan at each visit. Extensive experiments on synthetic datasets and simulators motivated by real-world medical applications demonstrate that Continuous-Time Decision Transformer is able to outperform competitors and has clinical utility in terms of improving patients' health and prolonging their survival by learning high-performance policies from logged data generated using policies of different levels of quality.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"206 ","pages":"6245-6262"},"PeriodicalIF":0.0,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10907982/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140023967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Causal Learning through Deliberate Undersampling. 通过故意减少取样进行因果学习

Proceedings of machine learning research Pub Date : 2023-04-01

Kseniya Solovyeva, David Danks, Mohammadsajad Abavisani, Sergey Plis

引用次数: 0

Causal isotonic calibration for heterogeneous treatment effects 异质治疗效果的因果等渗校准

Proceedings of machine learning research Pub Date : 2023-02-27 DOI: 10.48550/arXiv.2302.14011

L. Laan, Ernesto Ulloa-P'erez, M. Carone, Alexander Luedtke

引用次数: 0

Fair admission risk prediction with proportional multicalibration. 比例多重校准的公平入场风险预测。

Proceedings of machine learning research Pub Date : 2023-01-01

William G La Cava, Elle Lett, Guangya Wan

{"title":"Fair admission risk prediction with proportional multicalibration.","authors":"William G La Cava, Elle Lett, Guangya Wan","doi":"","DOIUrl":"","url":null,"abstract":"Fair calibration is a widely desirable fairness criteria in risk prediction contexts. One way to measure and achieve fair calibration is with multicalibration. Multicalibration constrains calibration error among flexibly-defined subpopulations while maintaining overall calibration. However, multicalibrated models can exhibit a higher percent calibration error among groups with lower base rates than groups with higher base rates. As a result, it is possible for a decision-maker to learn to trust or distrust model predictions for specific groups. To alleviate this, we propose proportional multicalibration, a criteria that constrains the percent calibration error among groups and within prediction bins. We prove that satisfying proportional multicalibration bounds a model's multicalibration as well its differential calibration, a fairness criteria that directly measures how closely a model approximates sufficiency. Therefore, proportionally calibrated models limit the ability of decision makers to distinguish between model performance on different patient groups, which may make the models more trustworthy in practice. We provide an efficient algorithm for post-processing risk prediction models for proportional multicalibration and evaluate it empirically. We conduct simulation studies and investigate a real-world application of PMC-postprocessing to prediction of emergency department patient admissions. We observe that proportional multicalibration is a promising criteria for controlling simultaneous measures of calibration fairness of a model over intersectional groups with virtually no cost in terms of classification performance.","PeriodicalId":74504,"journal":{"name":"Proceedings of machine learning research","volume":"209 ","pages":"350-378"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10417639/pdf/nihms-1917236.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10008223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0