Approximate Deep Network Embedding for Mining Large-Scale Graphs

Abstract

In the Big Data era, we are witnessing the flood of big graph data in terms of volume, variety, and velocity. The use of online social media and online shopping sites has provided access to a huge volume of interactions among information entities and objects. Scaling compute-intensive graph analysis applications on huge graphs with millions or billions of vertices and edges is widely recognized as a challenging big data research problem. A lot of research efforts in network embedding aim to learn low-dimensional representation of big graphs, enable easy integration with existing graph mining algorithms, and thus allow to achieve acceptable quality of big graph analysis on network embedding results with superior efficiency and scalability. However, how to enhance network embedding itself in terms of both efficiency and scalability is still an open problem. We are still short of efficient and scalable network embedding techniques to scale themselves on big graphs with millions or billions of vertices and edges, with the awareness of the intrinsic global and local characteristics of graph data. Most network embedding techniques exploit shallow-structured architectures, and thus lead to sub-optimal network representations. We also see lots of potential to utilize approximation theories and deep learning techniques to elevate both efficiency and scalability. In order to promote big network embedding from theoretical points of view, by representing graph data in deep learning architectures, we develop a suite of competitive learning-based approximate deep network embedding techniques that are able to leverage both efficiency and scalability of network embedding while preserving the computational utility with three major components. First, we propose a dynamic competitive learning-based algorithm to combine global network embedding and local network embedding into a unified model to utilize the advantages of both techniques. Second, we develop a network embedding-based algorithm with the optimization of competitive learning to tightly integrate vertex clustering and edge clustering by mutually enhancing each other. Third but last, we explore the opportunities of competitive learning and ranking for the optimal top-K neuron selection in the learning process of deep network embedding, in order to achieve a good balance between effectiveness and efficiency. The approximate deep network embedding approaches allow the deep learning model themselves to deactivate those insignificant neurons in the hidden layers through competitive learning, and thus reduce the computational cost of the feedforward pass and the back propagation.