Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning最新文献_第4页

DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models 基于gan的真实世界超分辨率模型的伪影检测和删除

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-07-05 DOI: 10.48550/arXiv.2307.02457

Liangbin Xie, Xintao Wang, Xiangyu Chen, Gengyan Li, Ying Shan, Jiantao Zhou, Chao Dong

{"title":"DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models","authors":"Liangbin Xie, Xintao Wang, Xiangyu Chen, Gengyan Li, Ying Shan, Jiantao Zhou, Chao Dong","doi":"10.48550/arXiv.2307.02457","DOIUrl":"https://doi.org/10.48550/arXiv.2307.02457","url":null,"abstract":"Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. However, it is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts, especially in practical scenarios. Previous works typically suppress artifacts with an extra loss penalty in the training phase. They only work for in-distribution artifact types generated during training. When applied in real-world scenarios, we observe that those improved methods still generate obviously annoying artifacts during inference. In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths. We then develop a novel method, namely, DeSRA, to Detect and then Delete those SR Artifacts in practice. Specifically, we propose to measure a relative local variance distance from MSE-SR results and GAN-SR results, and locate the problematic areas based on the above distance and semantic-aware thresholds. After detecting the artifact regions, we develop a finetune procedure to improve GAN-based SR models with a few samples, so that they can deal with similar types of artifacts in more unseen real data. Equipped with our DeSRA, we can successfully eliminate artifacts from inference and improve the ability of SR models to be applied in real-world scenarios. The code will be available at https://github.com/TencentARC/DeSRA.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"580 1","pages":"38204-38226"},"PeriodicalIF":0.0,"publicationDate":"2023-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77365873","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Trading-Off Payments and Accuracy in Online Classification with Paid Stochastic Experts 付费随机专家在线分类的权衡报酬与准确性

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-07-03 DOI: 10.48550/arXiv.2307.00836

Dirk van der Hoeven, Ciara Pike-Burke, Haotian Qiu, N. Cesa-Bianchi

引用次数: 0

Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers 用一种新的双指数移动平均线双向观察自适应和非自适应动量优化器

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-07-02 DOI: 10.48550/arXiv.2307.00631

Yineng Chen, Z. Li, Lefei Zhang, Bo Du, Hai Zhao

{"title":"Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers","authors":"Yineng Chen, Z. Li, Lefei Zhang, Bo Du, Hai Zhao","doi":"10.48550/arXiv.2307.00631","DOIUrl":"https://doi.org/10.48550/arXiv.2307.00631","url":null,"abstract":"Optimizer is an essential component for the success of deep learning, which guides the neural network to update the parameters according to the loss on the training set. SGD and Adam are two classical and effective optimizers on which researchers have proposed many variants, such as SGDM and RAdam. In this paper, we innovatively combine the backward-looking and forward-looking aspects of the optimizer algorithm and propose a novel textsc{Admeta} (textbf{A} textbf{D}ouble exponential textbf{M}oving averagtextbf{E} textbf{T}o textbf{A}daptive and non-adaptive momentum) optimizer framework. For backward-looking part, we propose a DEMA variant scheme, which is motivated by a metric in the stock market, to replace the common exponential moving average scheme. While in the forward-looking part, we present a dynamic lookahead strategy which asymptotically approaches a set value, maintaining its speed at early stage and high convergence performance at final stage. Based on this idea, we provide two optimizer implementations, textsc{AdmetaR} and textsc{AdmetaS}, the former based on RAdam and the latter based on SGDM. Through extensive experiments on diverse tasks, we find that the proposed textsc{Admeta} optimizer outperforms our base optimizers and shows advantages over recently proposed competitive optimizers. We also provide theoretical proof of these two algorithms, which verifies the convergence of our proposed textsc{Admeta}.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"14 1","pages":"4764-4803"},"PeriodicalIF":0.0,"publicationDate":"2023-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87958071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. 将黑箱分割成可解释模型的混合物：路线、解释、重复。

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-07-01

Shantanu Ghosh, Ke Yu, Forough Arabshahi, Kayhan Batmanghelich

{"title":"Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat.","authors":"Shantanu Ghosh, Ke Yu, Forough Arabshahi, Kayhan Batmanghelich","doi":"","DOIUrl":"","url":null,"abstract":"ML model design either starts with an interpretable model or a Blackbox and explains it post hoc. Blackbox models are flexible but difficult to explain, while interpretable models are inherently explainable. Yet, interpretable models require extensive ML knowledge and tend to be less flexible and underperforming than their Blackbox variants. This paper aims to blur the distinction between a post hoc explanation of a Blackbox and constructing interpretable models. Beginning with a Blackbox, we iteratively carve out a mixture of interpretable experts (MoIE) and a residual network. Each interpretable model specializes in a subset of samples and explains them using First Order Logic (FOL), providing basic reasoning on concepts from the Blackbox. We route the remaining samples through a flexible residual. We repeat the method on the residual network until all the interpretable models explain the desired proportion of data. Our extensive experiments show that our route, interpret, and repeat approach (1) identifies a diverse set of instance-specific concepts with high concept completeness via MoIE without compromising in performance, (2) identifies the relatively \"harder\" samples to explain via residuals, (3) outperforms the interpretable by-design models by significant margins during test-time interventions, and (4) fixes the shortcut learned by the original Blackbox. The code for MoIE is publicly available at: https://github.com/batmanlab/ICML-2023-Route-interpret-repeat.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"202 ","pages":"11360-11397"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10500943/pdf/nihms-1915804.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10305812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat 划分和征服黑箱到可解释模型的混合:路线，解释，重复

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-07-01 DOI: 10.48550/arXiv.2307.05350

Shantanu Ghosh, K. Yu, Forough Arabshahi, K. Batmanghelich

{"title":"Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat","authors":"Shantanu Ghosh, K. Yu, Forough Arabshahi, K. Batmanghelich","doi":"10.48550/arXiv.2307.05350","DOIUrl":"https://doi.org/10.48550/arXiv.2307.05350","url":null,"abstract":"ML model design either starts with an interpretable model or a Blackbox and explains it post hoc. Blackbox models are flexible but difficult to explain, while interpretable models are inherently explainable. Yet, interpretable models require extensive ML knowledge and tend to be less flexible and underperforming than their Blackbox variants. This paper aims to blur the distinction between a post hoc explanation of a Blackbox and constructing interpretable models. Beginning with a Blackbox, we iteratively carve out a mixture of interpretable experts (MoIE) and a residual network. Each interpretable model specializes in a subset of samples and explains them using First Order Logic (FOL), providing basic reasoning on concepts from the Blackbox. We route the remaining samples through a flexible residual. We repeat the method on the residual network until all the interpretable models explain the desired proportion of data. Our extensive experiments show that our route, interpret, and repeat approach (1) identifies a diverse set of instance-specific concepts with high concept completeness via MoIE without compromising in performance, (2) identifies the relatively \"harder\" samples to explain via residuals, (3) outperforms the interpretable by-design models by significant margins during test-time interventions, and (4) fixes the shortcut learned by the original Blackbox. The code for MoIE is publicly available at: https://github.com/batmanlab/ICML-2023-Route-interpret-repeat.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"61 1","pages":"11360-11397"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78317504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Geometric Autoencoders - What You See is What You Decode 几何自动编码器-你看到的就是你解码的

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-06-30 DOI: 10.48550/arXiv.2306.17638

Philipp Nazari, Sebastian Damrich, F. Hamprecht

引用次数: 1

Hierarchical Neural Coding for Controllable CAD Model Generation 层次神经编码在可控CAD模型生成中的应用

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-06-30 DOI: 10.48550/arXiv.2307.00149

Xiang Xu, P. Jayaraman, J. Lambourne, Karl D. D. Willis, Yasutaka Furukawa

引用次数: 2

Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations 神经元真的崩溃了吗?神经表征中的细粒度结构

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-06-29 DOI: 10.48550/arXiv.2306.17105

Yongyi Yang, J. Steinhardt, Wei Hu

{"title":"Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations","authors":"Yongyi Yang, J. Steinhardt, Wei Hu","doi":"10.48550/arXiv.2306.17105","DOIUrl":"https://doi.org/10.48550/arXiv.2306.17105","url":null,"abstract":"Recent work has observed an intriguing ''Neural Collapse'' phenomenon in well-trained neural networks, where the last-layer representations of training samples with the same label collapse into each other. This appears to suggest that the last-layer representations are completely determined by the labels, and do not depend on the intrinsic structure of input distribution. We provide evidence that this is not a complete description, and that the apparent collapse hides important fine-grained structure in the representations. Specifically, even when representations apparently collapse, the small amount of remaining variation can still faithfully and accurately captures the intrinsic structure of input distribution. As an example, if we train on CIFAR-10 using only 5 coarse-grained labels (by combining two classes into one super-class) until convergence, we can reconstruct the original 10-class labels from the learned representations via unsupervised clustering. The reconstructed labels achieve $93%$ accuracy on the CIFAR-10 test set, nearly matching the normal CIFAR-10 accuracy for the same architecture. We also provide an initial theoretical result showing the fine-grained representation structure in a simplified synthetic setting. Our results show concretely how the structure of input data can play a significant role in determining the fine-grained structure of neural representations, going beyond what Neural Collapse predicts.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"140 1","pages":"39453-39487"},"PeriodicalIF":0.0,"publicationDate":"2023-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77767147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

DUET: 2D Structured and Approximately Equivariant Representations 二维结构和近似等变表示

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-06-28 DOI: 10.48550/arXiv.2306.16058

Xavier Suau, Federico Danieli, Thomas Anderson Keller, Arno Blaas, Chen Huang, Jason Ramapuram, Dan Busbridge, L. Zappella

{"title":"DUET: 2D Structured and Approximately Equivariant Representations","authors":"Xavier Suau, Federico Danieli, Thomas Anderson Keller, Arno Blaas, Chen Huang, Jason Ramapuram, Dan Busbridge, L. Zappella","doi":"10.48550/arXiv.2306.16058","DOIUrl":"https://doi.org/10.48550/arXiv.2306.16058","url":null,"abstract":"Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.","PeriodicalId":74529,"journal":{"name":"Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning","volume":"21 1","pages":"32749-32769"},"PeriodicalIF":0.0,"publicationDate":"2023-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84980102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Curious Replay for Model-based Adaptation 基于模型的适应的好奇重放

Proceedings of the ... International Conference on Machine Learning. International Conference on Machine Learning Pub Date : 2023-06-28 DOI: 10.48550/arXiv.2306.15934

Isaac Kauvar, Christopher Doyle, Linqi Zhou, N. Haber

引用次数: 2