Lin Kong, Xiaying Bai, Yang Hu, Fanhua Shang, Yuanyuan Liu, Hongying Liu
{"title":"A Stochastic Variance Reduced Extragradient Method for Sparse Machine Learning Problems","authors":"Lin Kong, Xiaying Bai, Yang Hu, Fanhua Shang, Yuanyuan Liu, Hongying Liu","doi":"10.1109/ICDMW.2019.00033","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00033","url":null,"abstract":"This paper mainly considers the optimization problem of minimizing the average of a number of smooth convex functions and a general convex function capable of proximal mapping. In particular, we consider non-smooth functions, and assume that the smooth convex function is strongly convex, which often occur in data mining and deep learning. For non-smooth large-scale high-dimensional machine learning problems, this paper introduces both the variance reduction technique in SVRG and extragradient descent, and proposes an efficient proximal stochastic variance reduced extragradient algorithm, called EProx-SVRG. From our theoretical results, we know that in the expected form, EProx-SVRG converges to the optimum at a linear convergence rate for strongly convex problems, and the convergence guarantee for non-strongly convex problems is also provided. Moreover, EProx-SVRG achieves the same oracle complexity as Prox-SVRG. Finally, our experimental results show that our algorithm can obtain an improved performance compared with Prox-SVRG and Katyusha.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"20 10","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120812102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Takehiro Tezuka, Lihua Wang, Takuya Hayashi, S. Ozawa
{"title":"A Fast Privacy-Preserving Multi-Layer Perceptron Using Ring-LWE-Based Homomorphic Encryption","authors":"Takehiro Tezuka, Lihua Wang, Takuya Hayashi, S. Ozawa","doi":"10.1109/ICDMW.2019.00014","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00014","url":null,"abstract":"Concerns about leaking privacy from data have been preventing from making good use of so-called big data, while privacy-preserving data analysis would still be a promising research direction. In this paper, we propose Privacy-Preserving Multi-Layer Perceptron (PP-MLP) that can compute the prediction real-time using Ring-LWE-based homomorphic encryption. We implement the proposed PP-MLP in the form of a two-party model consisting of client and server. The former encrypts input data and receives a classification result from a server, and the latter performs prediction over encrypted data. This scheme enables a client to acquire prediction without revealing actual data contents against a server. The proposed PP-MLP can make a fast prediction that requires up to 80 msec per input without a significant drop in classification accuracy compared to the convention multi-layer perceptron for plaintexts.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"15 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120990057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep Transfer Learning With Field-Based Measurements for Large Area Classification","authors":"Zachary L. Langford, J. Kumar, F. Hoffman","doi":"10.1109/ICDMW.2019.00047","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00047","url":null,"abstract":"Classification of Electro-Optical (EO) datasets using Deep Neural Networks (DNNs) has lead to high-performance supervised learning algorithms. However, DNNs require a large amount of labeled data for training which often have limited availability in ecological studies. Up-to-date classification of vegetation in sensitive Arctic ecosystems continue to be a challenge. In an ecosystem undergoing rapid change, capturing the dynamics of vegetation requires the existing maps of vegetation (such as Alaska Existing Vegetation Type (AKEVT), circa 2000) to be updated based on frequent field based observation of vegetation. A method is needed to transfer the knowledge gained from field-collected observations to a larger area using DNNs. Transfer learning is a machine learning technique where a model trained on one task is re-purposed on another similar task. This paper seeks to train DNNs using field-collected observations and apply transfer learning to apply the knowledge gained to update existing vegetation map at larger scale. We test two DNN methods, (1) a deep multilayer perceptron (MLP) model and (2) siamese MLP network that uses a structure to rank similarity between inputs and can be used for training datasets with few samples and show good performance with limited datasets (e.g. few-shot learning). The results show ~90% accuracy (using the field observations for evaluation) when transfer learning is applied to a siamese network, compared to ~45% accurate when a MLP is trained on the AKEVT and evaluated on the field observations. The approach show promise for improving and update the existing vegetation maps over large areas using limited field-based observations.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"74 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127347344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Efficient Bayesian Method for Advancing the Application of Deep Learning in Earth Science","authors":"D. Lu, Siyan Liu, D. Ricciuto","doi":"10.1109/ICDMW.2019.00048","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00048","url":null,"abstract":"Despite their wide and successful applications, deep learning (DL) models are prone to overfitting for small training datasets, produce a poor predictive performance for uncertain data, and provide point estimations without any indication of the accuracy and credibility. These limitations of the deterministic DL models hinder their effective application in Earth system science where the labelled data are sparse, noisy and incomplete with large uncertainty and where the predictive uncertainty quantification is needed for scientific understanding and policy decision making. Integration of Bayesian inference into DL models adds an estimate of uncertainty and regularization in the predictions. However, traditional Bayesian methods are computationally unaffordable and inflexible for high-dimensional problems, which limits their application to DL systems that typically have millions of model parameters. In this effort, we propose an efficient and general-purpose Bayesian inference method to advance DL model optimization and uncertainty quantification, so as to facilitate the adoption of DL in Earth sciences. In a demonstration, we integrate the proposed Bayesian method with a feedforward neural network (NN) to build a fast-to-evaluate surrogate of the complex Energy Exascale Earth System Land Model for efficient modeling. The formulated Bayesian NN, using a small number of training data, produces an accurate prediction with high credibility, whereas with the same small training size, the deterministic NN cannot yield a reasonable estimation and does not provide confidence information. The proposed Bayesian method is computationally efficient and flexible, capable of integration with diverse network variants such as convolutional NNs and recurrent NNs to advance the application of DL in Earth sciences.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114400956","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Video Saliency Detection with Gated CNN and Residual Architecture","authors":"Hang Yu, Derong Chen, Jiulu Gong","doi":"10.1109/ICDMW.2019.00131","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00131","url":null,"abstract":"This paper proposes a novel gated recurrently end-to-end model of video salient object detection task to enhance both the performances and efficiency. A context aware aggregation module is first introduced for spatial features learning, these spatial features are then fused directly with a squeeze and excitation block. A two stage gated recurrent module is follow for temporal feature learning and shortcut connections between spatial and temporal features are add to alleviate the degradation problem. Our model has real-time speed of 25 fps on a single GPU which is similar to the approaches for static image salient object detection. We evaluate the proposed method on DAVIS and FBMS dataset. Experimental results show that comparing with other state-of-art saliency detectors, our method is more effective and efficient.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"137 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124598522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Content-Based Network Influence Probabilities: Extraction and Application","authors":"Alvis Logins, Panagiotis Karras","doi":"10.1109/ICDMW.2019.00020","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00020","url":null,"abstract":"The diffusion of information and influence in networks shapes ideas, habits, and behaviors. Several diffusion control methods have been proposed to harness, maximize, limit, or direct such diffusion processes in desirable ways. State-of-the-art algorithms for prescriptive fine-grained diffusion control rely on simple models, most prominently the Independent Cascade (IC) model, rather than on advanced machine learning approaches. The simplicity of such models can be an advantage. Yet, to exploit this advantage, one needs not only well-designed algorithms, but also a powerful model-training framework that yields well-informed models. Unfortunately, much research effort has been devoted to algorithm design, while the development of techniques for informing the underlying model has been largely neglected. We propose a new content analysis workflow that derives realistic IC model parameters for diffusion control algorithms. We rely on a log of user text messages to derive a measure of similarity among those messages, and therefrom calculate the probability that one node influences another. We evaluate our model in terms of its predictive power and apply it to two representative diffusion control problems under the IC model. Our results showcase the capacity of our methods to make correct predictions, and provide the first, to our knowledge, study of diffusion control problems with a real-world probability model.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"83 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126699237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Saideep, R. U. Kiran, K. Zettsu, Philippe Fournier-Viger, M. Kitsuregawa, P. K. Reddy
{"title":"Discovering Periodic Patterns in Irregular Time Series","authors":"C. Saideep, R. U. Kiran, K. Zettsu, Philippe Fournier-Viger, M. Kitsuregawa, P. K. Reddy","doi":"10.1109/ICDMW.2019.00147","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00147","url":null,"abstract":"Finding (partial) periodic patterns in time series data is a challenging problem of great importance in many applications. Due to computational reasons, most previous studies in this area have focused on the efficient discovery of periodic patterns in regular time series data. Unfortunately, these studies have limited applicability because real-world data naturally exists as an irregular time series. This paper proposes a more flexible model of periodic pattern that may be present in irregular time series. Two measures, period and period-support, were employed to determine the interestingness of a pattern in a series. The former measure captures the inter-arrival times of a pattern in a series, while the latter captures the number of periodic occurrences of a pattern in a series. A novel tree structure, called Periodic Pattern tree (PP-tree), has been introduced to record the irregular occurrences of items within the series. A pattern-growth algorithm has also been presented to find all periodic patterns from PP-tree. Experimental results demonstrate that the proposed model can find useful information, and the algorithm is efficient.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123975444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"User and Topic Hybrid Context Embedding for Finance-Related Text Data Mining","authors":"Chenyu Wang, Zhongchen Miao, Yuefeng Lin, Jian Gao","doi":"10.1109/ICDMW.2019.00112","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00112","url":null,"abstract":"As large amounts of finance-related microblogs, posted by both professional and amateur investors, are mingling in online social networks that could bring impact to real financial markets, it's an essential task to analyze financial texts posted by different users. Besides that, as barely nobody could be an all-round expert in various financial products such as stocks, futures and bonds, topic preferences of different users should also be considered in these mining tasks. That is to say, financial text contents should be analyzed in both user and topic level. In recent advances of deep learning technology, learning the \"embedding\" representation of contents is a state-of-art method in financial text mining over hand-crafted feature extraction techniques. Along this line, in this paper we propose a hybrid embedding method that both the \"embedding\" representation of users and the \"embedding\" representation of financial topics are learned from the posted finance-related contents in online social networks. Following the commonly used methods in embedding-related researches, we apply extensive experiments with real-world datasets to show the effectiveness of our hybrid user and topic embedding (UTE) approach, in both intrinsic and extrinsic ways. It's shown that our approach can intrinsically distinguish social network users with their learned embeddings, so they can be grouped into human-explainable clusters. Moreover, we also propose a case study on sentiment analysis by applying our user and topic hybrid embedding with a deep contextual neural network architecture. The results prove that our approach outperforms other baseline methods in financial-related sentiment analysis, and hopefully for other downstream text mining tasks as well.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124215388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Few-Shot Learning Based on Attention Relation Compare Network","authors":"Xian-Qin Ma, Chongchong Yu, Xin Yang, Xiuxin Chen","doi":"10.1109/ICDMW.2019.00100","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00100","url":null,"abstract":"In the field of computer vision, great progress has been made in the classification of few-shot images recently. However, challenges in improving classification accuracy and training stability remain. To address this issue, a novel meta learning method aiming at few-shot was proposed in this paper, which is a relationship comparison network based on attention mechanism. The network is an end-to-end network, consisting of three folds: feature coding, feature combination, and relationship coding. Firstly, the attention mechanism is introduced into the feature coding module and spectral normalization is used to normalize the weight parameters of the convolutional layer of the module, resulting in better extraction of key feature information from few-shot images and enhanced training stability of the attention mechanism comparison network. Secondly, in order to compare the feature information of the sample set and the query set, the feature information of the extracted sample set and the query set is combined into a new feature information in the feature combination module. Besides, the relationship coding module can learn a depth measurement method to calculate the relationship score of combined feature information and realize the classification of few-shot images. The performance of Attention Relation Compare Network Meta-Learning was evaluated on few-shot data set, namely the Omniglot and Mini ImageNet. It was indicated that our method excelled other few-shot learning method in accuracy and efficiency and improve the training stability to some extent . Experiments show that our method produces very competitive results when compared to the state-of-the-art method, and our model also exhibits the training stability.","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116326970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating Performance of Deep Neural Networks for Crop Classification and Complex Facility Recognition","authors":"K. Gadiraju, B. Ramachandra, Ranga Raju Vatsavai","doi":"10.1109/ICDMW.2019.00087","DOIUrl":"https://doi.org/10.1109/ICDMW.2019.00087","url":null,"abstract":"Automatic identification of different crops purely using satellite imagery is an active area of research which helps the government in accurately estimating the yields of different crops, thereby helping them prepare proper action plans to ensure that the supply of food matches the demand. With rapid rise in the world population, this is an essential factor. On the other hand, detecting different types of complex structures such as coal plants, gas plants, nuclear power plants and airports is another emerging area of research. Detecting nuclear power plants is essential from a national security perspective, for monitoring nuclear proliferation. With the availability of powerful computing resources, deep neural networks have increasingly become the de facto standard in several image classification challenges. In this paper, we evaluate the performance of several well-known deep neural networks (Convolutional Neural Networks) in performing the aforementioned two tasks - crop classification and complex facility recognition. Based on our experiments, we will evaluate how state-of-the-art networks will perform in performing tasks such as crop classification and complex facility recognition","PeriodicalId":258707,"journal":{"name":"2019 International Conference on Data Mining Workshops (ICDMW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130858444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}