{"title":"Automatic feature engineering from very high dimensional event logs using deep neural networks","authors":"Kai Hu, Joey Wang, Yong Liu, Datong Chen","doi":"10.1145/3326937.3341262","DOIUrl":"https://doi.org/10.1145/3326937.3341262","url":null,"abstract":"As communication networks have grown, event logs have increased in both size and complexity at a very fast rate. Thus, mining event logs has become very challenging due to their high variety and volume. The traditional solution to model raw event logs is to transform the raw logs into features with fewer dimensions through manual feature engineering. However, feature engineering is very time-consuming, and its quality is highly dependent on data scientists' domain knowledge. Furthermore, repeatedly preprocessing event logs significantly delays the scoring process, which must scan all items in the logs. In this paper, we present our recent study on mining high-dimensional event logs using deep neural networks. We propose a Midway Neural Network (MNN) to avoid both manual feature engineering and the re-preprocessing of event logs. MNN embeds an input feature vector from a particular time window into a dense representation and memorizes these midway representations for incremental training and prediction. The experimental results demonstrated that the proposed method minimized human intervention, decreased the time for training and scoring, and decreased the memory and storage costs while maintaining a similar modeling performance compared to traditional solutions. We hope that our insights and knowledge can inspire colleagues who are working on similar problems.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128482610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Pairwise multi-layer nets for learning distributed representation of multi-field categorical data","authors":"Ying Wen, Tianyao Chen, Jun Wang, Weinan Zhang","doi":"10.1145/3326937.3341251","DOIUrl":"https://doi.org/10.1145/3326937.3341251","url":null,"abstract":"This paper presents a method of pairwise multi-layer networks for multi-field categorical data, which widely exists with various applications such as web search, recommender systems, social link prediction, and computational advertising. The success of non-linear models, e.g., factorization machines, boosted trees, has proved the potential of exploring the interactions among inter-field discrete categories. Inspired by Word2Vec, the distributed representation for natural language, we propose a PMLN (Pairwise Multi-Layer Nets) model to learn the distributed representation for multi-field categorical data. In PMLN, a low-dimensional continuous vector is automatically learned for each category in each field. The interactions among inter-field categories are explored by different neural gates and the most informative ones are selected by pooling layers. Such combined categories can be further explored by performing more gate interactions with another category and then selected by additional pooling operations. In our experiments, with the exploration of the interactions between pairwise categories over layers, the model outperforms state-of-the-art models in a supervised learning task, i.e., ad click prediction, while capturing the most significant interactions from the data in an unsupervised fashion.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"2016 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121335448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A unified neural network approach to e-commerce relevance learning","authors":"Yunjiang Jiang, Yue Shang, Rui Li, Wen-Yun Yang, Guoyu Tang, Chaoyi Ma, Yun Xiao, Eric Zhao","doi":"10.1145/3326937.3341259","DOIUrl":"https://doi.org/10.1145/3326937.3341259","url":null,"abstract":"Result relevance scoring is critical to e-commerce search user experience. Traditional information retrieval methods focus on keyword matching and hand-crafted or counting-based numeric features, with limited understanding of item semantic relevance. We describe a highly-scalable feed-forward neural model to provide relevance score for (query, item) pairs, using only user query and item title as features, and both user click feedback as well as limited human ratings as labels. Several general enhancements were applied to further optimize eval/test metrics, including Siamese pairwise architecture, random batch negative co-training, and point-wise fine-tuning. We found significant improvement over GBDT baseline as well as several off-the-shelf deep-learning baselines on an independently constructed ratings dataset. The GBDT model relies on 10 times more features. We also present metrics for select subset combinations of techniques mentioned above.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"100 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122958515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Collaborative filtering via learning characteristics of neighborhood based on convolutional neural networks","authors":"Yugang Jia, Xin Wang, Jinting Zhang","doi":"10.1145/3326937.3341250","DOIUrl":"https://doi.org/10.1145/3326937.3341250","url":null,"abstract":"Collaborative filtering (CF) is an extensively studied topic in Recommender System. Recent approaches use the statistical framework based on local Taylor approximations to unify both user based and item based CF algorithms and improve the performance of estimating unknown ratings. In this paper, we propose a new Machine Learning approach based on Convolutional Neural Networks to exploit complex latent user-item relations, using features extracted from the neighborhood of unknown rating via local approximations. Experimental results on two benchmark data sets demonstrate the effectiveness of the proposed approach via comparing to state-of-the-art methods.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131180612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yanru Qu, Ting Bai, Weinan Zhang, J. Nie, Jian Tang
{"title":"An end-to-end neighborhood-based interaction model for knowledge-enhanced recommendation","authors":"Yanru Qu, Ting Bai, Weinan Zhang, J. Nie, Jian Tang","doi":"10.1145/3326937.3341257","DOIUrl":"https://doi.org/10.1145/3326937.3341257","url":null,"abstract":"This paper studies graph-based recommendation, where an interaction graph is built from historical responses and is leveraged to alleviate data sparsity and cold start problems. We reveal an early summarization problem in previous graph-based models, and propose Neighborhood Interaction (NI) model to capture each neighbor pair (between user-side and item-side) distinctively. NI model is more expressive and captures more complicated structural patterns behind user-item interactions. To enrich the neighborhood information, we also introduce Graph Neural Networks (GNNs) and Knowledge Graphs (KGs) to NI, resulting an end-to-end model, namely Knowledge-enhanced Neighborhood Interaction (KNI). Our experiments on 4 real world datasets show that, compared with state-of-the-art feature-based, meta path-based, and KG-based recommendation models, KNI achieves superior performance in click-through rate prediction (1.1%-8.4% absolute AUC improvements) and outperforms by a wide margin in top-N recommendation.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121627529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hai-Xia Luo, Shichuan Ma, Anandaraj Selvaraj, Yu Sun
{"title":"Learning job representation using directed graph embedding","authors":"Hai-Xia Luo, Shichuan Ma, Anandaraj Selvaraj, Yu Sun","doi":"10.1145/3326937.3341263","DOIUrl":"https://doi.org/10.1145/3326937.3341263","url":null,"abstract":"In recent years, embedding technologies have gained popularity in many areas of machine learning, such as NLP, computer vision, information retrieval, etc.. In this paper, we propose a latent representation of job positions consisting of job title and company pairs, which can capture not only similarity relations but also ordering relations among job positions. We first construct a directed graph of job positions from the user's job transition history in the resume data, then we train the job position embedding using an asymmetric relation preserving graph embedding algorithm. Experimental results on a career move prediction task using real-world data set demonstrated that the proposed embedding solution can outperform state-of-the-art embedding methods.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128911867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xuyang Wu, Xinyang Gao, Weinan Zhang, Rui Luo, Jun Wang
{"title":"Learning over categorical data using counting features: with an application on click-through rate estimation","authors":"Xuyang Wu, Xinyang Gao, Weinan Zhang, Rui Luo, Jun Wang","doi":"10.1145/3326937.3341260","DOIUrl":"https://doi.org/10.1145/3326937.3341260","url":null,"abstract":"Input data for many machine learning applications are often categorical and contain multiple fields. A common feature representation for such categorical data is one-hot encoding, which expresses data instances as high-dimensional sparse binary vectors. Given this encoding machine learning models, such as logistic regression or boosted trees, are trained. However, the following problems occur when dealing with large-scale data sets: (i) The binary feature space is sparse yet extremely large, which can require a large amount of computational resources. (ii) Models based on such a feature representation will typically need to be re-trained in order to keep up-to-date with any changes in the data distribution. (iii) The one-hot feature representation provides little generalisation ability. In this paper, we propose counting features, a novel statistics-based feature engineering paradigm, to address the above problems. Mathematically, we show a deterministic relationship between the optimal regression parameters of counting features and one-hot binary features. Then, in the context of click-through rate estimation in online advertising, we demonstrate that counting features indeed bring better generalisation ability. Our experiments on real-world large-scale datasets demonstrate that, despite their compressed nature, the proposed counting feature engineering outperforms the one-hot binary encoded features in various cases such as cold start training and cross-campaigns training.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131895884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Biye Jiang, Chao Deng, H. Yi, Zelin Hu, Guorui Zhou, Yang Zheng, Sui Huang, Xinyang Guo, Dongyue Wang, Yue Song, Liqin Zhao, Zhi Wang, P. Sun, Yu Zhang, Di Zhang, Jinhui Li, Jian Xu, Xiaoqiang Zhu, Kun Gai
{"title":"XDL","authors":"Biye Jiang, Chao Deng, H. Yi, Zelin Hu, Guorui Zhou, Yang Zheng, Sui Huang, Xinyang Guo, Dongyue Wang, Yue Song, Liqin Zhao, Zhi Wang, P. Sun, Yu Zhang, Di Zhang, Jinhui Li, Jian Xu, Xiaoqiang Zhu, Kun Gai","doi":"10.1145/3326937.3341255","DOIUrl":"https://doi.org/10.1145/3326937.3341255","url":null,"abstract":"With the rapid growth of data and computing power, deep learning based approaches have become the main solution for many artificial intelligence problems such as image classification, speech recognition and computer vision. Several excellent deep learning (DL) frameworks including Tensorflow, MxNet and PyTorch have been made open-sourced, further accelerating the advance of the community. However, existing DL frameworks are not designed for applications involving high-dimensional sparse data, which exists widely in many successful online businesses such as search engine, recommender systems and online advertising. In these industrial scenarios, deep models are typically trained on large scale datasets with up to billions of sparse features and hundreds of billions of samples, bringing great challenges to DL framework. In this paper, we introduce a high-performance, large-scale and distributed DL framework named XDL which provides an elegant solution to fill the gap between general design of existing DL frameworks and industrial requirements arising from high-dimensional sparse data. Since 2016, XDL has been successfully deployed in Alibaba, serving many productions such as online advertising and recommender system. Running on hundreds of GPU cards in parallel, XDL can train deep models with tens of billions parameters within only several hours. Besides its excellent performance and flexibility, XDL is also friendly to developers. Algorithm scientists in Alibaba can develop and deploy new deep models with only several lines of simple codes. The XDL API and a reference implementation were released as an open-source package under the Apache 2.0 license in December, 2018 and are available at https://github.com/alibaba/xdeeplearning.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"123 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122315751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hui Ye, Xiaopeng Ma, Qingfeng Pan, Huaqiang Fang, Hang Xiang, Tongzhen Shao
{"title":"An adaptive approach for anomaly detector selection and fine-tuning in time series","authors":"Hui Ye, Xiaopeng Ma, Qingfeng Pan, Huaqiang Fang, Hang Xiang, Tongzhen Shao","doi":"10.1145/3326937.3341253","DOIUrl":"https://doi.org/10.1145/3326937.3341253","url":null,"abstract":"The anomaly detection of time series is a hotspot of time series data mining. The own characteristics of different anomaly detectors determine the abnormal data that they are good at. There is no detector can be optimizing in all types of anomalies. Moreover, it still has difficulties in industrial production due to problems such as a single detector can't be optimized at different time windows of the same time series. This paper proposes an adaptive model based on time series characteristics and selecting appropriate detector and run-time parameters for anomaly detection, which is called ATSDLN(Adaptive Time Series Detector Learning Network). We take the time series as the input of the model, and learn the time series representation through FCN. In order to realize the adaptive selection of detectors and run-time parameters according to the input time series, the outputs of FCN are the inputs of two sub-networks: the detector selection network and the run-time parameters selection network. In addition, the way that the variable layer width design of the parameter selection sub-network and the introduction of transfer learning make the model be with more expandability. Through experiments, it is found that ATSDLN can select appropriate anomaly detector and run-time parameters, and have strong expandability, which can quickly transfer. We investigate the performance of ATSDLN in public data sets, our methods outperform other methods in most cases with higher effect and better adaptation. We also show experimental results on public data sets to demonstrate how model structure and transfer learning affect the effectiveness.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114423325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zheng Gao, Lin Guo, Chi Ma, Xiao Ma, Kai Sun, Hang Xiang, Xiaoqiang Zhu, Hongsong Li, Xiaozhong Liu
{"title":"AMAD: adversarial multiscale anomaly detection on high-dimensional and time-evolving categorical data","authors":"Zheng Gao, Lin Guo, Chi Ma, Xiao Ma, Kai Sun, Hang Xiang, Xiaoqiang Zhu, Hongsong Li, Xiaozhong Liu","doi":"10.1145/3326937.3341256","DOIUrl":"https://doi.org/10.1145/3326937.3341256","url":null,"abstract":"Anomaly detection is facing with emerging challenges in many important industry domains, such as cyber security and online recommendation and advertising. The recent trend in these areas calls for anomaly detection on time-evolving data with high-dimensional categorical features without labeled samples. Also, there is an increasing demand for identifying and monitoring irregular patterns at multiple resolutions. In this work, we propose a unified end-to-end approach to solve these challenges by combining the advantages of Adversarial Autoencoder and Recurrent Neural Network. The model learns data representations cross different scales with attention mechanisms, on which an enhanced two-resolution anomaly detector is developed for both instances and data blocks. Extensive experiments are performed over three types of datasets to demonstrate the efficacy of our method and its superiority over the state-of-art approaches.","PeriodicalId":125289,"journal":{"name":"Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data","volume":"111 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117238928","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}